analitics

Pages

Monday, December 23, 2019

Python 3.7.5 : About Django REST framework.

First, let's activate my Python virtual environment:
[mythcat@desk django]$ source env/bin/activate
I update my django version 3.0 up to 3.0.1.
(env) [mythcat@desk django]$ pip3 install --upgrade django --user
Collecting django
...
      Successfully uninstalled Django-3.0
Successfully installed django-3.0.1
The next step comes with installation of Python modules for Django and Django REST:
(env) [mythcat@desk django]$ pip3 install djangorestframework --user
Collecting djangorestframework
...
Installing collected packages: djangorestframework
Successfully installed djangorestframework-3.11.0
Into my folder mysite I run this commands
(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ python3 manage.py makemigrations
No changes detected
(env) [mythcat@desk mysite]$ python3 manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions, test001
Running migrations:
  No migrations to apply.
To pass information over to an HTTP GET request, the information object must be translated into valid response data.
The Django implements serializers for this.
Serializers provide deserialization, allowing parsed data to be converted back into complex types and allow complex data such as querysets and model instances to be converted to native Python datatypes that can then be easily rendered into JSON, XML or other content types.
Let's create the mysite/serializers.py:
(env) [mythcat@desk mysite]$ cd mysite/
(env) [mythcat@desk mysite]$ vim serializers.py 
The code for this python script is this:
from django.contrib.auth.models import User, Group
from rest_framework import serializers

class UserSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = User
        fields = ['url', 'username', 'email', 'groups']

class GroupSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = Group
        fields = ['url', 'name']
The next changes will be on urls.py and views.py.
My urls.py file from the Django-chart project is this:
from django.contrib import admin
from django.urls import path
from test001.views import home_page
from test001.views import Test001ChartView
#
from django.urls import include, path
from rest_framework import routers
from test001 import views

router = routers.DefaultRouter()
router.register(r'users', views.UserViewSet)
router.register(r'groups', views.GroupViewSet)

app_name = 'test001'
urlpatterns = [
    path('admin/', admin.site.urls),
    #path('', home_page, name ='home'),
    path('', Test001ChartView.as_view(), name = 'home'), 
    # Use automatic URL routing
    # Can also include login URLs for the browsable API
    path('', include(router.urls)),
    path('api-auth/', include('rest_framework.urls', namespace='rest_framework'))
]
My views.py file is this:
from django.http import HttpResponse
from django.shortcuts import render
# snippet 
from django.shortcuts import get_object_or_404
# for chart 
from django.views.generic import TemplateView
from .models import Test001, Snippet
#
#def home_page(request):
#    return HttpResponse('Home page!')

# django framework
from django.contrib.auth.models import User, Group
from rest_framework import viewsets
from mysite.serializers import UserSerializer, GroupSerializer

def home_page(request):
    return render(request, 'test001/home.html',{
        'name':'CGF',
        'html_items': ['a','b','c','d','e']
    })


# define view for chart 
class Test001ChartView(TemplateView):
    template_name = 'test001/chart.html'

    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        context["qs"] = Test001.objects.all()
        return context


def snippet_detail(request, id):
    snippet = get_object_or_404(Snippet, id=id)
    return render(request, 'test001/snippets_detail.html', {'snippet': snippet})

class UserViewSet(viewsets.ModelViewSet):
    """
    API endpoint  allows users to be viewed or edited.
    """
    queryset = User.objects.all().order_by('-date_joined')
    serializer_class = UserSerializer

class GroupViewSet(viewsets.ModelViewSet):
    """
    API endpoint  allows groups to be viewed or edited.
    """
    queryset = Group.objects.all()
    serializer_class = GroupSerializer
The settings module for this my project is stored in mysite/settings.py and I add this:
INSTALLED_APPS = [
    ...
    'rest_framework',
]
I run the django project and works well:
python3 manage.py runserver
The http://127.0.0.1:8000/users/ page come with this output:


Sunday, December 22, 2019

Python 3.7.5 : The new PyQt5 released.

The PyQt v5.14.0 has been released with support for Qt v5.14.0.
[mythcat@desk ~]$ pip3 install --upgrade PyQt5 --user
Collecting PyQt5
...
Installing collected packages: PyQt5-sip, PyQt5
Successfully installed PyQt5-5.14.0 PyQt5-sip-12.7.0
Let's see how can see the version:
[mythcat@desk ~]$ python3 
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5.Qt import PYQT_VERSION_STR
>>> print("PyQt version:", PYQT_VERSION_STR)
PyQt version: 5.14.0


Saturday, December 21, 2019

Python 3.7.5 : Simple web search with google python package.

This is a simple search on the web with python google package.
[mythcat@desk ~]$ pip3 install google --user
Collecting google
...
Installing collected packages: google
Successfully installed google-2.0.3
This is a simple example for search on web with this words: protv news 2019.
From the python package, I need to import just the search and used it.
The python package need a variable string named query.
The search wants to know the words from query and arguments, see the help:
Help on function search in module googlesearch:

search(query, tld='com', lang='en', tbs='0', safe='off', num=10, start=0, stop=None, 
domains=None, pause=2.0, tpe='', country='', extra_params=None, user_agent=None)
    Search the given query string using Google.
    
    :param str query: Query string. Must NOT be url-encoded.
    :param str tld: Top level domain.
    :param str lang: Language.
    :param str tbs: Time limits (i.e "qdr:h" => last hour,
        "qdr:d" => last 24 hours, "qdr:m" => last month).
    :param str safe: Safe search.
    :param int num: Number of results per page.
    :param int start: First result to retrieve.
    :param int stop: Last result to retrieve.
        Use None to keep searching forever.
    :param list domains: A list of web domains to constrain
        the search.
    :param float pause: Lapse to wait between HTTP requests.
        A lapse too long will make the search slow, but a lapse too short may
        cause Google to block your IP. Your mileage may vary!
    :param str tpe: Search type (images, videos, news, shopping, books, apps)
        Use the following values {videos: 'vid', images: 'isch',
        news: 'nws', shopping: 'shop', books: 'bks', applications: 'app'}
    :param str country: Country or region to focus the search on. Similar to
        changing the TLD, but does not yield exactly the same results.
        Only Google knows why...
    :param dict extra_params: A dictionary of extra HTTP GET
        parameters, which must be URL encoded. For example if you don't want
        Google to filter similar results you can set the extra_params to
        {'filter': '0'} which will append '&filter=0' to every query.
    :param str user_agent: User agent for the HTTP requests.
        Use None for the default.
    
    :rtype: generator of str
    :return: Generator (iterator) that yields found URLs.
        If the stop parameter is None the iterator will loop forever.
This is the script:
[mythcat@desk ~]$ python3
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from googlesearch import search
>>> query = "protv news 2019"
>>> my_results_list = []
>>> for url in search(query,        
...     tld = 'com',  
...     lang = 'en',       
...     num = 10,     
...     start = 0,    
...     stop = None,  
...     pause = 2.0,):
...     my_results_list.append(url)
...     print(url)
... 
https://stirileprotv.ro/protvnews/
https://stirileprotv.ro/
https://stirileprotv.ro/superbun/protv-news.html
https://www.facebook.com/ProTvNews/
https://www.youtube.com/playlist?list=PLCJaU-QvLGR_FSZw6yeqBJHDe9LgFDRdy
https://www.youtube.com/watch?v=HaiQtDlaNic
https://www.youtube.com/watch?v=hxMEgAANSl4
https://www.youtube.com/channel/UCbbDChpDluLkdnH8QMwN6qA
https://www.youtube.com/watch?v=5zuN9uWFcTE
https://protvplus.ro/tv-live/1-pro-tv
https://en.wikipedia.org/wiki/Pro_TV
https://pro-tv.com/news/page/2/
https://m.youtube.cat/channel/UCbbDChpDluLkdnH8QMwN6qA
...

Sunday, December 15, 2019

Python 3.7.5 : Simple intro in CSRF.

CSRF or Cross-Site Request Forgery is a technique used by cyber-criminals to force users into executing unwanted actions on a web application.
To protect against web form CSRF attacks, it's isn't sufficient for web applications to trust authenticated users, must be equipped with a unique identifier called a CSRF token similar to a session identifier.
Django 3.0 can be used with CSRF, see the documentation page.
The CSRF process is a simple one in Django framework.
However, it's highly recommended to use the CsrfViewMiddleware instead.
To activate the django.middleware.csrf.CsrfViewMiddleware in the settings.py file.
Basically, you can use the decorator method in the view.py file.
For example, see documentation:
@csrf_protect
However csrf_protect will check only POST requests.
This annotation CSRF tells Django I'm not handling CSRF properly and don't fail this.
If I don't use this annotation then the CSRF is on if the post request is on by default and I will receive a 403 error.
As the developer using templates you don't have to know anything about that and you can use this in your template HTML5 file.
{% csrf_token %}
I can use the token with my code if I want to send it to the website.
from django.middleware.csrf import get_token
Then this can be used in view.py with a hidden form field with the name csrfmiddlewaretoken present in all outgoing POST forms.
You can used in any forms tag area like: input ...

def my_csrf_form(request):
    response = """ ... type = "hidden" name = "csrfmiddlewaretoken" value = "__token__" ... """
This can be used with:
token = get_token(request)
response = response.replace('__token__', html.escape(token))
response += dumpdata('POST', request.POST)
return HttpResponse(response)
This will get a token for the current request and fill into value = "__token__".
Using my old example from GitHub.
First, I start the project:

[mythcat@desk projects]$ source django/env/bin/activate
(env) [mythcat@desk projects]$ ls
cache  django  kaggle  logs  OSMnx  pygal_ex  SantaClaus.py
(env) [mythcat@desk projects]$ cd django/
(env) [mythcat@desk django]$ ls
env  mysite  venv
(env) [mythcat@desk django]$ cd mysite/
I change the chart.html file with this before javascript script:
... {% csrf_token %} ... 
You can use it in javascript - jQuery like this, if you want:
var csrftoken = jQuery("[name=csrfmiddlewaretoken]").val();
The CSRF protection on Django on my form can bu used in view.py:
...
# use csrf_protect
from django.views.decorators.csrf import csrf_protect
from django.utils.decorators import method_decorator
...
# define view for chart 
class Test001ChartView(TemplateView):
    csrf_protected_method = method_decorator(csrf_protect)
I test it and works well.
The documentation tells us:
The CSRF protection cannot protect against man-in-the-middle attacks, so use HTTPS with HTTP Strict Transport Security. It also assumes validation of the HOST header and that there aren’t any cross-site scripting vulnerabilities on your site (because XSS vulnerabilities already let an attacker do anything a CSRF vulnerability allows and much worse).

Saturday, December 14, 2019

Python 3.7.5 : Django admin shell by Grzegorz Tężycki.

Today I tested another python package for Django named django-admin-shell.
This package created by Grzegorz Tężycki can be found on GitHub and come with the intro:
Django application can execute python code in your project’s environment on django admin site. You can use similar as python manage shell without reloading the environment.
[mythcat@desk ~]$ cd projects/
[mythcat@desk projects]$ cd django/
...
[mythcat@desk projects]$ source django/env/bin/activate
(env) [mythcat@desk projects]$ 

(env) [mythcat@desk projects]$ pip3 install django-admin-shell --user
Collecting django-admin-shell
...
Installing collected packages: django-admin-shell
Successfully installed django-admin-shell-0.1 
First, I test my project to see if this working.
(env) [mythcat@desk projects]$ cd django/
(env) [mythcat@desk django]$ ls
env  mysite  venv
(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ ls
db.sqlite3  manage.py  mysite  test001
(env) [mythcat@desk mysite]$ python3 manage.py runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
December 14, 2019 - 18:23:47
Django version 3.0, using settings 'mysite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C. 
You need to add this package into the settings.py file:
INSTALLED_APPS = [
    ...
    'django_admin_shell',
    ...
]
I add this package into my url.py file:
...
from django.conf.urls import include, url
...
urlpatterns = [
...
    # admin shell 
    url(r'^admin/shell/', include('django_admin_shell.urls')),
]
After that, you can start the project and see if this working: http://127.0.0.1:8000/admin/shell/
This screenshot shows me is working well:

Wednesday, December 11, 2019

Python 3.7.5 : The Pygal python package.

Today's tutorial aims to get data from a URL and display it with the Pygal python package.
I believe that global warming is a very important topic for human evolution.
You can read more about this topic on this website.
About this python package you can learn more at the official website.
[mythcat@desk ~]$ pip3 install Pygal --user
Collecting Pygal
...
Installing collected packages: Pygal
Successfully installed Pygal-2.4.0
Let's test this python package with this script:
import pygal
import pandas as pd
import numpy as np
URL = 'https://data.giss.nasa.gov/gistemp/graphs/graph_data/
Global_Mean_Estimates_based_on_Land_and_Ocean_Data/graph.txt'

df = pd.read_fwf(
    URL,
    skiprows=(0,1,2,4),
    index_col=0,
)
print(df.head)

year_data = df['No_Smoothing']
print(year_data.head)
one_chart = pygal.Bar()
one_chart.title = "Land-Ocean Temperature Index (C)"
one_chart.add("",year_data)
one_chart.render_in_browser()
The result is this output:

Monday, December 9, 2019

Python 3.7.5 : The OSMnx python package.

About this python package named OSMnx, you can read on GitHub.
OSMnx is a Python package that lets you download spatial geometries and model, project, visualize, and analyze street networks from OpenStreetMap's APIs. Users can download and model walkable, drivable, or bikable urban networks with a single line of Python code, and then easily analyze and visualize them ...
You cannot install osmnx directly in Fedora 31 Linux distro because of the dependency with python packages.
[mythcat@desk projects]$ pip3 install osmnx --user
Installation depends by rtree python packages:
[root@desk projects]# dnf install python3-rtree.noarch
...
Installed:
  python3-rtree-0.9.1-1.fc31.noarch 
spatialindex-1.9.3-1.fc31.x86_64                             

Complete!
Now you can install with pip3 tool if you use the python3.
[mythcat@desk projects]$ pip3 install osmnx --user
Collecting osmnx
...
Installing collected packages: pyparsing, kiwisolver, cycler, matplotlib, descartes, networkx, Shapely,
 pyproj, click, click-plugins, cligj, attrs, munch, fiona, geopandas, osmnx
Successfully installed Shapely-1.6.4.post2 attrs-19.3.0 click-7.0 click-plugins-1.1.1 cligj-0.5.0 
cycler-0.10.0 descartes-1.1.0 fiona-1.8.13 geopandas-0.6.2 kiwisolver-1.1.0 matplotlib-3.1.2 
munch-2.5.0 networkx-2.4 osmnx-0.11 pyparsing-2.4.5 pyproj-2.4.2.post1
Let's test the default example:
[mythcat@desk projects]$ python3
Python 3.7.5 (default, Oct 17 2019, 12:16:48) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import osmnx as ox
>>> ox.config(log_file=True, log_console=True, use_cache=True)
Configured osmnx
>>> my_loc  = ox.graph_from_address('Fălticeni, Suceava, 725200, Romania', network_type= 'all', 
retain_all =True)
Geocoded "Fa?lticeni, Suceava, 725200, Romania" to (47.4597637, 26.30255)
Projected the GeoDataFrame "geometry to project" to UTM-35 in 0.01 seconds
Projected the GeoDataFrame "geometry to project" to default_crs in 0.01 seconds
Created bounding box 1000 meters in each direction from (47.4597637, 26.30255): 47.46876106367192,
47.45076632191398,26.31581675508982,26.289283302659218
Projected the GeoDataFrame "geometry to project" to UTM-35 in 0.01 seconds
Projected the GeoDataFrame "geometry to project" to default_crs in 0.01 seconds
Projected the GeoDataFrame "geometry to project" to UTM-35 in 0.01 seconds
Projected the GeoDataFrame "geometry to project" to default_crs in 0.01 seconds
Requesting network data within bounding box from API in 1 request(s)
Retrieved response from cache file "cache/eb15fb167cf173e622992dbc2b7b7c77.json" for URL 
"http://overpass-api.de/api/interpreter?data=%5Bout%3Ajson%5D%5Btimeout%3A180%5D%3B%28way
%5B%22highway%22%5D%5B%22area%22%21~%22yes%22%5D%5B%22highway%22%21~%22proposed%7Cconstruction
%7Cabandoned%7Cplatform%7Craceway%22%5D%5B%22service%22%21~%22private%22%5D%5B%22access%22%21
~%22private%22%5D%2847.446267%2C26.282649%2C47.473260%2C26.322452%29%3B%3E%3B%29%3Bout%3B"
Got all network data within bounding box from API in 1 request(s) and 0.01 seconds
Creating networkx graph from downloaded OSM data...
Created graph with 2,003 nodes and 3,930 edges in 0.08 seconds
Added edge lengths to graph in 0.06 seconds
Truncated graph by bounding box in 0.05 seconds
Begin topologically simplifying the graph...
Identified 339 edge endpoints in 0.03 seconds
Constructed all paths to simplify in 0.01 seconds
Simplified graph (from 2,003 to 339 nodes and from 3,930 to 803 edges) in 0.22 seconds
Truncated graph by bounding box in 0.01 seconds
Got the counts of undirected street segments incident to each node (before 
removing peripheral edges) in 0.07 seconds
graph_from_bbox() returning graph with 235 nodes and 556 edges
graph_from_point() returning graph with 235 nodes and 556 edges
graph_from_address() returning graph with 235 nodes and 556 edges
>>> my_proj = ox.project_graph(my_loc)
Created a GeoDataFrame from graph in 0.05 seconds
Projected the GeoDataFrame "unnamed_nodes" to UTM-35 in 0.03 seconds
Projected the GeoDataFrame "unnamed_edges" to UTM-35 in 0.10 seconds
Extracted projected node geometries from GeoDataFrame in 0.01 seconds
Rebuilt projected graph in 0.94 seconds
>>> fig, ax = ox.plot_graph(my_proj)
Begin plotting the graph...
Created GeoDataFrame "unnamed_UTM_edges" from graph in 0.07 seconds
Drew the graph edges in 0.04 seconds
This python module comes with many features.
You can select a polygon area and show the map.
This is the result of source code from this tutorial:

Sunday, December 8, 2019

Python 3.7.5 : Starting with kaggle platform.

Kaggle is the world's largest community of data scientists and the platform is the fastest way to get started on a new data science project.
A good choice to use Kaggle is this feature: Kaggle provides free access to NVidia K80 GPUs in kernels.
The tutorial for today is about kaggle and is new for me because I hear about this opportunity last year.
This platform that hosts data science and machine learning competitions can give the people a good area for development.
The official blog can tell you more about how can this platform works.
The kaggle A.P.I. can be found at GitHub.
Let's start the tutorial with the pip3 install tool:
[mythcat@desk kaggle]$ pip3 install kaggle --upgrade --user
Collecting kaggle
...
[mythcat@desk kaggle]$ mkdir ~/.kaggle/
[mythcat@desk kaggle]$ mv kaggle.json ~/.kaggle/kaggle.json
[mythcat@desk kaggle]$ kaggle 
Warning: Your Kaggle API key is readable by other users on this system! To fix this, you can
 run 'chmod 600 /home/mythcat/.kaggle/kaggle.json'
usage: kaggle [-h] [-v] {competitions,c,datasets,d,kernels,k,config} ...
kaggle: error: the following arguments are required: command
[mythcat@desk kaggle]$ chmod 600 /home/mythcat/.kaggle/kaggle.json
You can use the kaggle command to get informations from kaggle platform:
[mythcat@desk kaggle]$ kaggle competitions list 
ref                                            deadline             category            reward  teamCount
  userHasEntered  
---------------------------------------------  -------------------  ---------------  ---------  ---------
  --------------  
digit-recognizer                               2030-01-01 00:00:00  Getting Started  Knowledge       2305
           False  
titanic                                        2030-01-01 00:00:00  Getting Started  Knowledge      17135
           False  
house-prices-advanced-regression-techniques    2030-01-01 00:00:00  Getting Started  Knowledge       5532
           False  
imagenet-object-localization-challenge         2029-12-31 07:00:00  Research         Knowledge         57
           False  
google-quest-challenge                         2020-02-10 23:59:00  Featured           $25,000        410
           False  
tensorflow2-question-answering                 2020-01-22 23:59:00  Featured           $50,000        810
           False  
data-science-bowl-2019                         2020-01-22 23:59:00  Featured          $160,000       1762
           False  
pku-autonomous-driving   
...
The commands for this platform can be seen at GitHub:
kaggle competitions {list, files, download, submit, submissions, leaderboard}
kaggle datasets {list, files, download, create, version, init}
kaggle kernels {list, init, push, pull, output, status}
kaggle config {view, set, unset}
The kaggle platform use an online tool with notebooks similar with Jupyter notebook for process coding in Kernels.
The process of using models need the datasets and you can use the kaggle datasets.
You can use the New Notebook button from that page to start using the datasets.
The datasets can be load from kaggle or can be uploaded.
The next window let you to Select new notebook settings.
I used Python with a notebook and I got the default source code:
# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load in 

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)

# Input data files are available in the "../input/" directory.
# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input
 directory

import os
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

# Any results you write to the current directory are saved as output.
Now, I can read the dataset shown in the right the Data with input
(read-only data)
and output from kaggle with pandas module:
data = pd.read_csv("../input/lego-database/colors.csv")
data.head()
The Commit button let you to save your work for later.
You can see my online test I created with dataset Lego and python on my kaggle page.

Saturday, December 7, 2019

Python 3.7.5 : Using the django with javascript.

The Django framework can work great with javascript.
I start this project a few days ago.
You can start a simple Django project, see my old tutorials.
I used a template for main page in my template folder named chart.html.
The chart.min.js is set on base.html.
The links work from the Django framework to HTML5 templates with javascript.
The full project can be found on my GitHub project.
See the screen output:

Thursday, December 5, 2019

Python 3.7.5 : About PEP 8.

Today is December 5, 2019, and this is the reason I wrote about PEP 8.
The official webpage can be found at this webpage.
PEP 8 recommends using 4 spaces to show indentation and tabs should only be used to maintain consistency in the code.
The Python Library is conservative and 79 characters are the maximum required line limit as suggested by PEP 8.
The main goal is to avoid line wrapping.
Indentation allows you to differentiate between multiple lines of code and a single line of code that spans multiple lines.
The closing braces with the first white-space character of the last line.
The use of blank lines can greatly improve the readability of your code.
This rule allows the reader to understand the separation of the sections of code and the relation between them.
TypeNaming ConventionsExamples
VariableUsing short names with CapWords.T, AnyString, My_First_Variable
FunctionUsing a lowercase word or words with underscores to improve readability.function, my_first_function
ClassUsing CapWords and do not use underscores between words.Student, MyFirstClass
MethodUsing lowercase words separated by underscores.Student_method, method
ConstantsUsing all capital letters with underscores separating wordsTOTAL, MY_CONSTANT, MAX_FLOW
ExceptionsUsing CapWords without underscores.IndexError, NameError
ModuleUsing short lower-case letters using underscores.module.py, my_first_module.py
PackageUsing short lowercase words and underscores are discouraged.package, my_first_package
About comments, PEP 8 basically recommends using block comments for general-purpose coding.
Document strings or docstrings start at the first line of any function, class, file, module or method.
These type of comments are enclosed between single quotations ( ''') or double quotations ( """ ).
To improving the readability of expressions and statements use whitespace for any character or sequence of characters.
We need to avoiding whitespaces:
  • inside parentheses, brackets, or braces;
  • before a comma, a semicolon, or a colon;
  • before open parenthesis that initiates the argument list of a function call;
  • before an open bracket that begins an index or a slice;
  • between a trailing comma and a closing parenthesis;
  • to adjust assignment operators;
PEP 8 recommends using a clear development for implementation of source code, for example PEP recommends using .startswith() and .endswith() instead of list slicing.
You can check whether your code actually complies with the rules and regulations of PEP 8 or not.
The linters is a program that analyzes your code and detects program errors, syntax errors, bugs, and structural problems.
An autoformatted tool will format your code to adapt to PEP 8 automatically.
Both tools used to implement and check PEP 8 compliance.
To know more about PEP 8 and its book of guidelines, you can refer to pep8.org.
You can check it online at this webpage.

Tuesday, December 3, 2019

Python 3.7.5 : The new Django version 3.0 .

On December 2, 2019, comes with Django 3.0 Released.
This new release comes with many changes and features, let's see :
  • Django 3.0 supports new versions of Python 3.6, 3.7, and 3.8 (the old Django 2.2.x series is the last to support Python 3.5);
  • Django now officially supports MariaDB 10.1 and higher;
  • Django is fully async-capable by providing support for running as an ASGI application;
  • The exclusion class constraints on PostgreSQL;
  • the output BooleanField may now be used directly in QuerySet filters;
  • you can have custom enumeration types
  • cache features: add_never_cache_headers() and never_cache() now add the private directive to Cache-Control headers;
  • file storage, forms, internationalization, and logging features;
  • the management commands are new options;
  • the Requests and Responses, Models and Security has many changes and new features;
  • the Tests and test cases have a good implementation;
  • new minor features for django.contrib.admin, django.contrib.auth, django.contrib.gis, django.contrib.postgres, django.contrib.sessions, django.contrib.syndication and more
All these features can be read at the official webpage.

Friday, November 29, 2019

Python 3.7.5 : Script install and import python packages.

This script will try to import Python packages from a list.
If these packages are not installed then they will be installed on the system.
import sys
import subprocess

if __name__ == '__main__': 
    def ModuleInstall(package_name):
        try:
            subprocess.check_call(['python3', '-m', 'pip3', 'install', package_name, "--user"])
        except:
            subprocess.check_call(['python', '-m', 'pip', 'install', package_name, "--user"])
    def ModuleLoaded(package_name):
        __import__(package_name)
        print ("Successfully imported ", package_name, '.')

    required_libs = [
    "optparse",
    "socket",
    "select",
    "threading",
    "time",
    "re",
    "ssl",
    "textwrap",
    "sys",
    "random",
    "traceback",
    "binascii",
    "struct",
    "inspect"]
    for lib_name in required_libs:
        try:
            ModuleLoaded(lib_name)
        except:
            ModuleInstall(lib_name)

Monday, November 25, 2019

Python 3.7.5 : Python and SQL, a fast approach.

Today he had to solve a task that required declaring a series of consecutive days to read from several SQL tables.
As you know, this task would require a complex query to include statements.
I solved it quite simply by using a python script that would automatically generate the days and then concatenate them into a query.
It may not be the best solution but it is very fast and does not require very advanced SQL knowledge.
Let's see the source code:
from datetime import date, timedelta

start_date = date(2019, 01, 1)
end_date = date(2019, 11, 24)
delta = timedelta(days=1)
q1 = str("SELECT DATE_FORMAT(updated, '%Y-%m-%d') AS day, intern, COUNT(*) FROM `intern")
q2 = str("` WHERE intern = 0 GROUP BY intern, day UNION ALL ")
while start_date <= end_date:
    print (q1+start_date.strftime("%Y_%m_%d")+q2)
    start_date += delta
The result querry will be like this:
SELECT DATE_FORMAT(updated, "%Y-%m-%d") AS day, intern, COUNT(*) FROM `intern2019_01_01` GROUP BY intern, 
day UNION ALL 
SELECT DATE_FORMAT(updated, "%Y-%m-%d") AS day, intern, COUNT(*) FROM `intern2019_01_02` GROUP BY intern,
 day UNION ALL
...
SELECT DATE_FORMAT(updated, "%Y-%m-%d") AS day, intern, COUNT(*) FROM `intern2019_11_24` GROUP BY intern,
 day UNION ALL
The last step, I remove the UNION ALL from the last SELECT and I used with phpmyadmin.