python-catalin

Sunday, February 16, 2020

Python 3.7.5 : The httpx python package.

Today I will present a new python packet that can help you in developing web applications.
This is the next generation HTTP client for Python and is named httpx.
This python package comes with a nice logo: a butterfly.
The official webpage can be found at this webpage.
The development team come with this intro:
HTTPX is a fully featured HTTP client for Python 3, which provides sync and async APIs, and support for both HTTP/1.1 and HTTP/2.
I install it on my Fedora 31 distro with the pip3 tool.

[mythcat@desk ~]$ pip3 install httpx --user
...
Successfully installed h11-0.9.0 h2-3.2.0 hpack-3.0.0 hstspreload-2020.2.15 httpx-0.11.1 
hyperframe-5.2.0 rfc3986-1.3.2 sniffio-1.1.0

With this python package, you can build a simple application with https API, migrate an application that uses web requests to make HTTP call.
use it as a test client for your web project, build a web spider and much more.
All HTTP methods get, post, patch, put, delete are implemented as coroutines in httpx python package and supports HTTP/2.
Let's start with few lines of source code:

[mythcat@desk ~]$ python3 
Python 3.7.6 (default, Jan 30 2020, 09:44:41) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import httpx
>>> out = httpx.get('https://www.google.com')
>>> out.status_code
200
>>> out.headers['content-type']
'text/html; charset=ISO-8859-1'
>>> out.text[:76]
'<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang='
>>> out.http_version
'HTTP/1.1'
>>> with httpx.Client() as client:
...     out_client = client.get('https://www.google.com')
... 
>>> out_client.http_version
'HTTP/1.1'

You can use asynchronous call to bost your python application using httpx python package.
I will make a tutorial about asyncio in the future.
For example, the website named example.com can return json data and using the next source code I can get it using the asyncio package.

import httpx
import asyncio
from typing import Dict
async def get_web() -> Dict:
    resp = await httpx.get("https://example.com/")
    if resp.status_code == httpx.codes.OK:
        return resp.json()
if __name__ == '__main__':   
    comments = asyncio.run(get_web())

Use async and await when you have to execute HTTP calls with httpx because is fully Requests compatible async HTTP Client.
The HTTPX comes with a lot of features, see the official GitHub webpage.

Saturday, February 15, 2020

Python 3.7.5 : Use Brython in web development to avoid javascript.

The tutorial for today is about how can avoid the javascript and use python script in webdevelopment using the Brython.
Brython's goal is to replace Javascript with Python, as the scripting language for web browsers. see the official webpage.
It is necessary to include brython.js and to run the brython() function upon page load using the onload attribute of the BODY tag.
You can use python language in the HTML file or you can write it in a separate file, and to load it using the src attribute of the script tag:

<html>
<head>
<script src="/brython.js"></script>
</head>
<body onload="brython()">
<script type="text/python" src="test.py"></script>
<input id="zone" autocomplete="off">
<button id="mybutton">click!</button>
</body>
</html>

Let's see one simple example with one edit and button bind

<html>
<head>
<script src="/brython.js"></script>
</head>
<body onload="brython()">
<script type="text/python">
from browser import document, alert
def echo(ev):
 alert(document["zone"].value)
document["mybutton"].bind("click", echo)
</script>
<input id="zone">
<button id="mybutton">click !</button>
</body>
</html>

When I click on the button, the onclick event calls and run the echo() function and gets the value of the INPUT element, through its id named zone and show the message.
Is fast and works great with javascripts libraries, see the example with THREE demo.

from browser import document, window

THREE = window.THREE

camera = THREE.PerspectiveCamera.new(75, 1, 1, 10000)
camera.position.z = 1000
scene = THREE.Scene.new()
geometry = THREE.CubeGeometry.new(200, 200, 200)
material = THREE.MeshBasicMaterial.new({"color": "#ff0000", "wireframe": True})
mesh = THREE.Mesh.new(geometry, material)
scene.add(mesh)

renderer = THREE.WebGLRenderer.new()
renderer.setSize(444, 444)

document <= renderer.domElement
renderer.render(scene, camera)

def animate(i):
    # note: three.js includes requestAnimationFrame shim
    window.requestAnimationFrame(animate)

    mesh.rotation.x += 0.01
    mesh.rotation.y += 0.02

    renderer.render(scene, camera)   

animate(0)

Tuesday, February 11, 2020

Python 3.7.5 : Using the hug framework - part 001.

Today I will come with another tutorial series about the hug framework.
The hug framework is on the top 3 performing web frameworks for Python and comes with the cleanest way to create HTTP REST APIs on Python 3.
The official webpage can be found hug web page with a good area for learn.
Let's install this python package.

[mythcat@desk projects]$ mkdir hug_001
[mythcat@desk projects]$ cd hug_001/
[mythcat@desk hug_001]$ pip3 install hug --user
...
Successfully installed hug-2.6.1

The Hug is a framework that allows you to expose a single code in several ways:

a local package;
as an A.P.I.;
as a C.L.I.;

Let's create an app.py python script to see all of these options:

import hug

# local pkg 
@hug.local()
def get_products(product:hug.types.text):
        """Get product name"""
        return {"product":product.upper()}

# API
@hug.get()

# CLI
@hug.cli()

To test it I used this source code:

import hug

# local pkg 
@hug.get()
@hug.local()
def get_products(product:hug.types.text):
        """Get product name"""
        return {"product":product.upper()}

The rerver can be run with this command:

[mythcat@desk hug_001]$ hug -f app.py

/#######################################################################\
          `.----``..-------..``.----.
         :/:::::--:---------:--::::://.
        .+::::----##/-/oo+:-##----:::://
        `//::-------/oosoo-------::://.       ##    ##  ##    ##    #####
          .-:------./++o/o-.------::-`   ```  ##    ##  ##    ##  ##
             `----.-./+o+:..----.     `.:///. ########  ##    ## ##
   ```        `----.-::::::------  `.-:::://. ##    ##  ##    ## ##   ####
  ://::--.``` -:``...-----...` `:--::::::-.`  ##    ##  ##   ##   ##    ##
  :/:::::::::-:-     `````      .:::::-.`     ##    ##    ####     ######
   ``.--:::::::.                .:::.`
         ``..::.                .::         EMBRACE THE APIs OF THE FUTURE
             ::-                .:-
             -::`               ::-                   VERSION 2.6.1
             `::-              -::`
              -::-`           -::-
\########################################################################/

 Copyright (C) 2016 Timothy Edmund Crosley
 Under the MIT License


Serving on :8000...

OPen the browser with the http://localhost:8000/ or http://localhost:8000/products:
If you want to use the CLI then you need to make these changes in the source code:

import hug
# cli 
@hug.cli()
# local pkg 
@hug.get('/products')
@hug.local()
def get_products(product:hug.types.text):
        """Get product name"""
        return {"product":product.upper()}
# the main CLI
if __name__ == '__main__':
        get_products.interface.cli()

I can see the help area from python:

[mythcat@desk hug_001]$ hug -f app.py -c help
app

Available Commands:

 - get_products: Get product name

The hug help show the all arguments for use:

[mythcat@desk hug_001]$ hug --help
usage: hug [-h] [-v] [-f FILE] [-m MODULE] [-ho HOST] [-p PORT] [-n] [-ma]
           [-i INTERVAL] [-c COMMAND] [-s]

Hug API Development Server

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -f FILE, --file FILE  file
  -m MODULE, --module MODULE
                        module
  -ho HOST, --host HOST
                        host
  -p PORT, --port PORT  A whole number
  -n, --no_404_documentation
                        Providing any value will set this to true
  -ma, --manual_reload  Providing any value will set this to true
  -i INTERVAL, --interval INTERVAL
                        A whole number
  -c COMMAND, --command COMMAND
                        command
  -s, --silent          Providing any value will set this to true

Wednesday, January 22, 2020

Python 3.7.5 : Django security issues - part 003.

Let's update this subject today with another new tutorial.
In the last tutorial about Django security I wrote about python package named django-axes.
First, let's fix an old issue about a URL pattern that matches anything and expects an integer that generates errors like:

...
  File "/home/mythcat/.local/lib/python3.7/site-packages/django/db/models/fields/__init__.py", line 1772, 
in get_prep_value
    ) from e
ValueError: Field 'id' expected a number but got 'favicon.ico'.
[22/Jan/2020 21:50:06] "GET /favicon.ico/ HTTP/1.1" 500 130547

Now, let's start my project:

[mythcat@desk ~]$ cd projects/
[mythcat@desk projects]$ cd django/
[mythcat@desk django]$ source env/bin/activate

Create a new folder named static in the test001 folder and add a icon file named favicon.ico.

(env) [mythcat@desk django]$ cd mysite/test001/
(env) [mythcat@desk test001]$ mkdir static

In the settings.py file you need to have this source code:


STATIC_URL = '/static/'

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, "static"),
]

Change in the urls.py this line of source code to fix the error:

path('<int:author_id>/',views.index_next, name = 'index_next'),

Let's run the Django project server with:

(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ python3 manage.py runserver

I login into my admin area with user catalin and password adminadmin.
If you try to login with a bad password then the account is locked by django-axes python package.
Use this command to reset all lockouts and access records.

(env) [mythcat@desk mysite]$ python3 manage.py axes_reset
No attempts found.

Into admin area you can see the AXES area with Access attempts and Access logs.
Axes listens to the following signals from django.contrib.auth.signals to log access attempts.
In this case Axes lockout responses on failed user authentication attempts from login views.
The Access logs shows access log, see examples:

Jan. 22, 2020, 8:46 p.m.	-	127.0.0.1	catalin	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36	/admin/login/
Jan. 21, 2020, 6:42 p.m.	Jan. 22, 2020, 8:45 p.m.	127.0.0.1	catalin	Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36	/admin/login/

You can set the axes into settings.py file , see this link.

Tuesday, January 21, 2020

Python 3.7.5 : Use Django Formsets.

Django Formsets manage the complexity of multiple copies of a form in a view.
This simplifies the task of creating a formset for a form that handles multiple instances of a model.
Let's start with my old project:

[mythcat@desk ~]$ cd projects/
[mythcat@desk projects]$ cd django/
[mythcat@desk django]$ source env/bin/activate

Into models.py I add these classes:

#create Inline Form with book and author
class Author(models.Model):
    author_book = models.CharField(max_length = 100)
    def __str__(self):
        return self.author_book

class Book(models.Model):
    book_name = models.CharField(max_length = 100)
    author_book_name = models.ForeignKey(Author,on_delete=models.CASCADE)
    def __str__(self):
        return self.book_name

(env) [mythcat@desk mysite]$ python3 manage.py makemigrations
Migrations for 'test001':
  test001/migrations/0004_author_book.py
    - Create model Author
    - Create model Book
(env) [mythcat@desk mysite]$ python3 manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, axes, contenttypes, sessions, test001
Running migrations:
  Applying test001.0004_author_book... OK

Add the classes to admin.py:

from .models import Author, Book
...
admin.site.register(Author)
admin.site.register(Book)

Now you can log in into the admin area and add authors and then add books.

Add the source code to views.py:

...
# add to views.py Author and Book 
from .models import Author, Book
# Author , Book redirect 
from django.shortcuts import redirect
...
# author and book source code 
def index_next(request, author_id):
    author = Author.objects.get(pk=author_id)
    BookFormset = inlineformset_factory(Author,Book, fields=('book_name',))
    if request.method == 'POST':
        formset = BookFormset(request.POST,instance = author)
        if formset.is_valid():
            formset.save()
            return redirect('index_next',author_id = author_id)
    formset = BookFormset(instance = author)
    return render(request, 'index_next.html', {'formset': formset})
...

Let's fix the URL for the next step.
Add the source code to urls.py:

...
# add index_next to urls.py 
from test001.views import index_next
...
urlpatterns = [
    ...
    path('&lt author_id &gt',views.index_next, name = 'index_next'),
    ...
    ]

Add index_next.html file into the template folder and into this file write HTML5 with a form and one submit button.
In the form tag add this:

{{ formset.as_p }}

Run the runserver command:

(env) [mythcat@desk mysite]$ python3 manage.py runserver

Use this http://127.0.0.1:8000/1 to see the first on the database shown on the browser.
You can customize the output of inline form, see source code:

...
BookFormset = inlineformset_factory(Author,Book, fields=('book_name',), can_delete=False, extra=1)
...

See the full project on my GitHub account at django_chart project repo.

Monday, January 20, 2020

Python 3.7.5 : Django security issues - part 002.

The project can be found at this Github project.
Let's start with my default project and activate the env:

[mythcat@desk ~]$ cd projects/
[mythcat@desk projects]$ cd django/
[mythcat@desk django]$ source env/bin/activate

Let's install this python module:

(env) [mythcat@desk django]$ pip3 install django-axes --user

Make these changes into settings.py:

(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ ls
db.sqlite3  manage.py  mysite  test001
(env) [mythcat@desk mysite]$ cd mysite/
(env) [mythcat@desk mysite]$ vim settings.py

Into your settings.py add axes:

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'test001',
    'rest_framework',
    'axes'
]

Add this source code in settings.py:

AUTHENTICATION_BACKENDS = [
    # AxesBackend should be the first backend in the AUTHENTICATION_BACKENDS list.
    'axes.backends.AxesBackend',

    # Django ModelBackend is the default authentication backend.
    'django.contrib.auth.backends.ModelBackend',
]

Add axes.middleware.AxesMiddleware into MIDDLEWARE area:

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'axes.middleware.AxesMiddleware',
]

Check the configuration with this command:

(env) [mythcat@desk mysite]$ cd ..
(env) [mythcat@desk mysite]$ python manage.py check
System check identified no issues (0 silenced).

Use this command to sync the database:

(env) [mythcat@desk mysite]$ python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, axes, contenttypes, sessions, test001
Running migrations:
  Applying axes.0001_initial... OK
  Applying axes.0002_auto_20151217_2044... OK
  Applying axes.0003_auto_20160322_0929... OK
  Applying axes.0004_auto_20181024_1538... OK
  Applying axes.0005_remove_accessattempt_trusted... OK
  Applying axes.0006_remove_accesslog_trusted... OK

Once Axes is is installed and configured, you can login and logout of your application via the django.contrib.auth views.
This python package can be integrated with some popular 3rd party packages such as Django Allauth, Django REST Framework, and other tools.
I will come with additional information about this python package in the future.

Saturday, January 18, 2020

Python 3.7.5 : Django security issues - part 001.

Django like any website development and framework implementation requires security settings and configurations.
Today I will present some aspects of this topic and then I will come back with other information.
1. First, check your security vulnerabilities by the following command:

[mythcat@desk django]$ source env/bin/activate
(env) [mythcat@desk django]$ cd mysite
(env) [mythcat@desk mysite]$ python3 manage.py check --deploy
...
  File "/home/mythcat/projects/django/mysite/mysite/settings.py", line 14
    <<<<<<< HEAD

This shows us the bad changes in source code, is added by GitHub features.
Let's run it again:

(env) [mythcat@desk mysite]$ python3 manage.py check --deploy
System check identified some issues:

WARNINGS:
?: (security.W004) You have not set a value for the SECURE_HSTS_SECONDS setting.

If your entire site is served only over SSL, you may want to consider setting a value and

enabling HTTP Strict Transport Security. Be sure to read the documentation first; enabling

 HSTS carelessly can cause serious, irreversible problems.
?: (security.W008) Your SECURE_SSL_REDIRECT setting is not set to True. Unless your site

should be available over both SSL and non-SSL connections, you may want to either set this

setting True or configure a load balancer or reverse-proxy server to redirect all connections to HTTPS.
?: (security.W012) SESSION_COOKIE_SECURE is not set to True. Using a secure-only session cookie makes

it more difficult for network traffic sniffers to hijack user sessions.
?: (security.W016) You have 'django.middleware.csrf.CsrfViewMiddleware' in your MIDDLEWARE, but you have

 not set CSRF_COOKIE_SECURE to True. Using a secure-only CSRF cookie makes it more difficult for network

 traffic sniffers to steal the CSRF token.
?: (security.W018) You should not have DEBUG set to True in deployment.
?: (security.W020) ALLOWED_HOSTS must not be empty in deployment.
?: (security.W022) You have not set the SECURE_REFERRER_POLICY setting. Without this, your site

will not send a Referrer-Policy header. You should consider enabling this header to protect user privacy.

System check identified 7 issues (0 silenced).

This output show us the security warnning problems.
2. Use the Observatory by Mozilla site to scan the security status of your Django website.
3. Django has built-in security against most forms of CSRF threats, but The CSRF protection cannot protect against man-in-the-middle attacks.
Use HTTPS with HTTP Strict Transport Security by add these lines in your settings.py file.

CSRF_COOKIE_SECURE = True #to avoid transmitting the CSRF cookie over HTTP accidentally.
SESSION_COOKIE_SECURE = True #to avoid transmitting the session cookie over HTTP accidentally.

4. A Cross-site Scripting (XSS) allows an attacker to inject a script into the content of a website or application.
In your settings.py use this:

django.middleware.security.SecurityMiddleware
...
SECURE_BROWSER_XSS_FILTER = True
SECURE_CONTENT_TYPE_NOSNIFF = True

5. You can create fake admin login page using django-admin-honeypot to attempt unauthorized access.
6. Use SSL Redirect on your settings.py file.

SECURE_SSL_REDIRECT = True

7. Add Content Security Policy (CSP) to your Django website with the installed django-csp, add following lines to your settings.py file:

# Content Security Policy
CSP_DEFAULT_SRC = ("'none'", )
CSP_STYLE_SRC = ("'self'", )
CSP_SCRIPT_SRC = ("'self'", )
CSP_IMG_SRC = ("'self'", )
CSP_FONT_SRC = ("'self'", )
# Google Tag Manager or Google Analytics should be allowed in your CSP policy. 
CSP_DEFAULT_SRC = ("'none'", )
CSP_STYLE_SRC = ("'self'", "fonts.googleapis.com", "'sha256-/3kWSXHts8LrwfemLzY9W0tOv5I4eLIhrf0pT8cU0WI='")
CSP_SCRIPT_SRC = ("'self'", "ajax.googleapis.com", "www.googletagmanager.com", "www.google-analytics.com")
CSP_IMG_SRC = ("'self'", "data:", "www.googletagmanager.com", "www.google-analytics.com")
CSP_FONT_SRC = ("'self'", "fonts.gstatic.com")
CSP_CONNECT_SRC = ("'self'", )
CSP_OBJECT_SRC = ("'none'", )
CSP_BASE_URI = ("'none'", )
CSP_FRAME_ANCESTORS = ("'none'", )
CSP_FORM_ACTION = ("'self'", )
CSP_INCLUDE_NONCE_IN = ('script-src',)

The HTTP Strict Transport Security can be set into your settings.py file:

SECURE_HSTS_SECONDS = 86400  # 1 day
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True

8. Use records login attempts to your Django powered site and prevents attackers from brute forcing using the django-axes.
This tutorial solves most of the security issues of any website built with Django and not just related to this framework.

Saturday, January 11, 2020

Python 3.7.5 : About asterisk operators in Python.

The asterisk known as the star operator is used in Python with more than one meaning attached to it.
Today I will show you some simple examples of how can be used.
Let's start with these issues.
You can merge two or more dictionaries by unpacking them in a new one:

>>> a = {'u': 1}
>>> b = {'v': 2}
>>> ab = {**a, **b, 'c':'d'}
>>> ab
{'u': 1, 'v': 2, 'c': 'd'}

Create multiple assignments:

>>> *x, y = 1, 2, 3, 4
>>> x
[1, 2, 3]

You can split into lists and generate sets:

>>> *a, = "Python"
>>> a
['P', 'y', 't', 'h', 'o', 'n']
>>> print(a)
['P', 'y', 't', 'h', 'o', 'n']
>>> *a,
('P', 'y', 't', 'h', 'o', 'n')
>>> zeros = (0,) * 8
>>> print(zeros)
(0, 0, 0, 0, 0, 0, 0, 0)

For numeric data types is used as multiplication operator:

>>> 2 * 3
6

You can use in mathematical function as an exponent:

>>> 3**2
9

You can create a sequences of strings using it like a repetition operator:

>>> t = 'Bye!'
>>> t * 4 
'Bye!Bye!Bye!Bye!'
>>> p = [0,1]
>>> p * 4 
[0, 1, 0, 1, 0, 1, 0, 1]
>>> r = (0,1)
>>> r * 4
(0, 1, 0, 1, 0, 1, 0, 1)

You can used into definition with arguments: *args and **kwargs:

def set_zero(*args):
    result = 0

The *args will give you all function parameters as a tuple.
The **kwargs will give you all keyword arguments except for those corresponding to a formal parameter as a dictionary.
See the PEP 3102 about Keyword-Only Arguments.

Thursday, January 9, 2020

Python 3.7.5 : The this python package.

The this python package is simple to use:

[mythcat@desk ~]$ python3 
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Monday, January 6, 2020

Python 3.7.5 : Post class and migration process.

Today I will solve some issues with the Django framework like:

create a new class for posts;
explain how the migration process works.
use the database with Django shell;

Let's activate the environment:

[mythcat@desk django]$ source env/bin/activate

I used my old project django-chart, see my old tutorials.
Let's add some source code to the models.py to create a class for the post into my website:

...
# Post data 
from django.utils import timezone
from django.contrib.auth.models import User
...
#create a post
class Post(models.Model):
  title = models.CharField(max_length = 100)
  content = models.TextField()
  date_posted = models.DateTimeField(default = timezone.now)
  #if the user is deleted their posts is deleted 
  author = models.ForeignKey(User, on_delete = models.CASCADE)

Let's use the migration process to update the database:

(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ python3 manage.py makemigrations
Migrations for 'test001':
  test001/migrations/0003_post.py
  - Create model Post

Before to run the migrate command I will show what happened with this migration output from SQL view.
This is great to solve issues and see the exact SQL code generated.
I have in the output of command makemigrations the application name test001 and the migration number 0003.
I will use all of this information output to show how you can see the SQL code:

(env) [mythcat@desk mysite]$ python3 manage.py sqlmigrate test001 0003
BEGIN;
--
-- Create model Post
--
CREATE TABLE "test001_post" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, 
"title" varchar(100) NOT NULL, "content" text NOT NULL, "date_posted" datetime NOT NULL, 
"author_id" integer NOT NULL REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED);
CREATE INDEX "test001_post_author_id_fed29ee6" ON "test001_post" ("author_id");
COMMIT;

This print SQL code that it's going to run when I will use the next command:

(env) [mythcat@desk mysite]$ python3 manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions, test001
Running migrations:
  Applying test001.0003_post... OK

The migration process solves issues linked to changes in the database.
Let's use the Django shell to see the User changes into the database:

(env) [mythcat@desk mysite]$ python3 manage.py shell
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from test001.models import Post
>>> from django.contrib.auth.models import User
>>> User.objects.all()
...
>>> User.objects.first()
...
>>> User.objects.filter(username='catalin')
...
>>> User.objects.filter(username='catalin').first()
...
>>> user = User.objects.filter(username='catalin').first()
>>> user
...
>>> user.id
1
>>> user.pk
1
>>> Post.objects.all()
...

Let's add a post to the database and show it:

>>> post_1=Post(title='First title', content='The first content!',author=user)
>>> post_1.save()
>>> Post.objects.all()
...

I can see the posts and information about the user with the Django shell.

>>> post = Post.objects.first()
>>> post.content
'The first content!'
>>> post.date_posted
...
>>> post.author
...
>>> post.author.email
...
>>> user.post_set.all()
...

You need to add to views.py the new model changes:

...
from .models import Post
...
def posts(request):
  context = {
  'posts':Post.objects.all()
  }
  return render(request, 'test001/posts.html', context)
...

Into the templates folder, I created the file named posts.html to load the data to HTML5.

{% extends 'base.html' %}
{% block content %}
  {% for post in posts %}
  <article>
  <div>{{ post.title }}</div>
  <div>{{ post.content }}</div>
  <div>{{ post.author }}</div>
  <div>{{ post.date_posted|date:"F d, Y" }}</div>
  </article>
  {% endfor %}
{% endblock content %}

To see these changes into the website I add the route to this HTML5 file posts.html into urls.py file:

app_name = 'test001'
urlpatterns = [
...
    path('posts/',posts, name = 'posts'),
...
]

Now I can run the server and see the output from the database into posts URL.

(env) [mythcat@desk mysite]$ python3 manage.py runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
January 06, 2020 - 17:52:29
Django version 3.0.1, using settings 'mysite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
...

This is the output of my running server...

Python 3.7.5 : Set json with settings in Django project.

[mythcat@desk django]$ source env/bin/activate
(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ ls
db.sqlite3  manage.py  mysite  test001
(env) [mythcat@desk mysite]$ pwd
/home/mythcat/projects/django/mysite

Create a file named config.json in the folder django:

(env) [mythcat@desk mysite]$ vim /home/mythcat/projects/django/config.json

Open your settings.py file from your Django project and copy your secret key from this file to config.json, see:

{ 
  "SECRET_KEY":"your_secret_key_from_settings.py"
}

Change your settings.py file with these changes:

...
import json 
...
with open('/home/mythcat/projects/django/config.json') as config_file:
    config = json.load(config_file)
...
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = config['SECRET_KEY']

This allows us to get the SECRET_KEY from the config.json file.
We can add many variables into file config.json, like EMAIL_USER and EMAIL_PASS.
You need to have a validate JSON file and the settings.py will change from this:

EMAIL_HOST_USER = os.environ.get('EMAIL_USER')
...

to this source code for EMAIL_USER variable:

EMAIL_HOST_USER = config.get('EMAIL_USER')
...

Sunday, January 5, 2020

Python 3.7.5 : Testing cryptography python package - part 001.

There are many python packets that present themselves as useful encryption and decryption solutions. I recommend before you test them, use them and spend time with them to focus on the correct study of cryptology because many disadvantages and problems can arise in the correct and safe writing of the source code.
Today I will show you a simple example with cryptography python package.
Let's install this with pip3 tool:

[mythcat@desk projects]$ pip3 install cryptography --user

You can read more about this python package on the documentation website.
Let's try a simple method of encryption and decryption.
I will show you how to solve this issue with the class cryptography.hazmat.primitives.ciphers.aead.AESGCM:
The AES-GCM construction is composed of the AES block cipher utilizing Galois Counter Mode (GCM).
The GCM (Galois Counter Mode) is a mode of operation for block ciphers.
An AEAD (authenticated encryption with additional data) mode is a type
of block cipher mode that simultaneously encrypts the message as well as authenticating it.
The Value of AESGCM key must be 128, 192, or 256 bits, see the size of the key:

[mythcat@desk projects]$ python3
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from cryptography.hazmat.primitives.ciphers.aead import AESGCM
>>> aad = None
>>> key = 'catafestcatafestcatafestcatafest'
>>> aesgcm = AESGCM(key.encode('utf-8'))
>>> nonce = '12345678'
>>> data_binary = b'Hello world!'
>>> 
>>> cipher_text = aesgcm.encrypt(nonce.encode('utf-8'), data_binary, aad)
>>> print(cipher_text)
b'0\xab\xd2!mXe\xc3/\xdb\x15\xcaoT\x0f\x1d\xbb&\xc4\x92\xdf\\ZTMD\xa2\x9f'
>>> data_text = aesgcm.decrypt(nonce.encode('utf-8'), cipher_text, aad)
>>> print(data_text)
b'Hello world!'

Saturday, January 4, 2020

Python 3.7.5 : Testing the PyMongo package - part 001.

MongoDB and PyMongo are not my priorities for the new year 2020 but because they are quite evolved I thought to start presenting it within my free time.
The PyMongo python package is a Python distribution containing tools for working with MongoDB.
The full documentation can be found on this webpage.
You can see my tutorial about how to install the MongoDB into Fedora 31 on this webpage.
I used that webpage install to test this python package, see the result:

[mythcat@desk mongo_test]$ mongo
...
MongoDB server version: 4.2.2-rc1
> use admin 
switched to db admin
> show dbs
> db.auth('admin', 'admin')
1
> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB

Let's install it with pip3 tool:

[mythcat@desk projects]$ pip3 install pymongo --user
Collecting pymongo
...
Successfully installed pymongo-3.10.0

Let's start with a simple example:

[mythcat@desk projects]$ mkdir mongo_test
[mythcat@desk projects]$ cd mongo_test/
[mythcat@desk mongo_test]$ vim mongo001.py

If you have already created the admin user, to run the next spython script you need to change the role like this:

> use admin;
switched to db admin
> db.grantRolesToUser('admin', [{ role: 'root', db: 'admin' }])

The script show you how to use a simple connection to the MongoDB:

import pymongo
from pymongo import MongoClient, errors

MONGO_URI = 'mongodb://admin:admin767779@127.0.0.1:27017/admin'
client = pymongo.MongoClient(MONGO_URI)
print("Server info : ")
print(client.server_info())
print("Databases : " + str(client.list_database_names()))
print("Connect to : admin database!")
db = client['admin']
db2 = client.config
print(db)
print(db2)
print("Collection admin : ")
collection = db['admin']
collection2 = db.config
print(collection)
print(collection2)
print("try call find_one method")
try:
    one_doc= collection2.find_one()
    print ("find_one():", one_doc)
except errors.ServerSelectionTimeoutError as err:
    print ("find_one() ERROR:", err)

print("Client close!")
client.close()

The output is this:

[mythcat@desk mongo_test]$ python3 mongo001.py 
Server info : 
{'version': '4.2.2-rc1', 'gitVersion': 'a0bbbff6ada159e19298d37946ac8dc4b497eadf', 'modules': [],
 'allocator': 'tcmalloc', 'javascriptEngine': 'mozjs', 'sysInfo': 'deprecated', 'versionArray': [4, 2, 2, -49],
 'openssl': {'running': 'OpenSSL 1.1.1d FIPS  10 Sep 2019', 'compiled': 'OpenSSL 1.1.1 FIPS  11 Sep 2018'},
 'buildEnvironment': {'distmod': 'rhel80', 'distarch': 'x86_64', 'cc': '/opt/mongodbtoolchain/v3/bin/gcc:
 gcc (GCC) 8.2.0', 'ccflags': '-fno-omit-frame-pointer -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare
 -Wno-unknown-pragmas -Winvalid-pch -Werror -O2 -Wno-unused-local-typedefs -Wno-unused-function
 -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable
 -Wno-missing-braces -fstack-protector-strong -fno-builtin-memcmp', 'cxx': '/opt/mongodbtoolchain/v3/bin/g++:
 g++ (GCC) 8.2.0', 'cxxflags': '-Woverloaded-virtual -Wno-maybe-uninitialized -fsized-deallocation -std=c++17',
 'linkflags': '-pthread -Wl,-z,now -rdynamic -Wl,--fatal-warnings -fstack-protector-strong -fuse-ld=gold -Wl,
--build-id -Wl,--hash-style=gnu -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro', 'target_arch':
 'x86_64', 'target_os': 'linux'}, 'bits': 64, 'debug': False, 'maxBsonObjectSize': 16777216, 'storageEngines':
 ['biggie', 'devnull', 'ephemeralForTest', 'wiredTiger'], 'ok': 1.0}
Databases : ['admin', 'config', 'local']
Connect to : admin database!
Database(MongoClient(host=['127.0.0.1:27017'], document_class=dict, tz_aware=False, connect=True), 'admin')
Database(MongoClient(host=['127.0.0.1:27017'], document_class=dict, tz_aware=False, connect=True), 'config')
Collection admin : 
Collection(Database(MongoClient(host=['127.0.0.1:27017'], document_class=dict, tz_aware=False, connect=True),
 'admin'), 'admin')
Collection(Database(MongoClient(host=['127.0.0.1:27017'], document_class=dict, tz_aware=False, connect=True),
 'admin'), 'config')
try call find_one method
find_one(): None
Client close!

Thursday, January 2, 2020

Python 3.7.5 : Testing the Falcon framework - part 001.

I start the new year with this python framework named Falcon.
The Falcon is a low-level, high-performance Python framework for building HTTP APIs, app backends, and higher-level frameworks.
The main reason was the speed of this python framework, see this article about falcon benchmark.
You can see is more faster like Flask and Django.
The instalation is easy with pip tool, you can read also the documenation webpage:

[mythcat@desk projects]$ mkdir falcon_test
[mythcat@desk projects]$ cd falcon_test/
[mythcat@desk falcon_test]$ pip3 install falcon --user
Collecting falcon
...
Successfully installed falcon-2.0.0

Falcon also fully supports CPython 2.7 and 3.5+.
If you want to install the latest beta or release candidate use:

[mythcat@desk falcon_test]$ pip3 install --pre falcon --user

The Falcon framework is easy to use.
First, I created a folder named test001 for my falcon application script named app.py:

[mythcat@desk falcon_test]$ mkdir test001
[mythcat@desk falcon_test]$ cd test001/
[mythcat@desk test001]$ vim app.py

In this python script I used python packages json and falcon with a class named req_class:

import json
import falcon

class req_class:
    def on_get(self,req,resp):
        print("on_get class")
my_falcon_api = falcon.API()
my_falcon_api.add_route('/test',req_class())

The url route is set to test.
To test the falcon framework we need the Gunicorn python package.
The Gunicorn is working on my Fedora Linux distro and I'm not sure if working Gunicorn on Windows.
You can try on Windows O.S. the waitress python package.

[mythcat@desk falcon_test]$ pip3 install gunicorn --user
Collecting gunicorn
...
Successfully installed gunicorn-20.0.4

To test this simple Falcon application use this command line where the app python script and my falcon variable A.P.I. named my_falcon_api is used.

[mythcat@desk test001]$ gunicorn app:my_falcon_api
[2020-01-02 18:57:48 +0200] [4401] [INFO] Starting gunicorn 20.0.4
[2020-01-02 18:57:48 +0200] [4401] [INFO] Listening at: http://127.0.0.1:8000 (4401)
[2020-01-02 18:57:48 +0200] [4401] [INFO] Using worker: sync
[2020-01-02 18:57:48 +0200] [4404] [INFO] Booting worker with pid: 4404
on_get class
on_get class

Open in the browser this URL with the route I set: http://127.0.0.1:8000/test.
You will don't see anything in the browser but will see the python output of the print function for my request.

Tuesday, December 31, 2019

News : The Python 2.7 no longer support from Python team.

The 1st of January 2020 will mark the sunset of Python 2.7.
It’s clear that Python 3 is more popular these days.
You can learn more about the popularity of both on Google Trends.
Python 3.0 was released in December 2008.
The main goal was to fix problems existing in Python 2.
Since the 1st January 2020, Python 2 will no longer receive any support whatsoever from the core Python team.
Migrating to Python 3 is recommended, including some of the top libraries, such as machine learning.

Saturday, December 28, 2019

Python 3.7.5 : Fix to python language the GitHub project.

I created a GitHub project with Django and I saw is detect like tcl programming language:

You need to create a file named .gitattributes in the root folder of my repository.
Use this source code to tell GitHub is a python project:

* linguist-vendored
*.py linguist-vendored=false

Now the project will be target with python language.

Tuesday, December 24, 2019

Python 3.7.5 : Is Django the best web framework?

This is the question for today in order to lineup the Django features with any web framework from my point of view.
Let's start with a brief introduction to this framework:
Django was created in the fall of 2003, when the web programmers at the Lawrence Journal-World newspaper, Adrian Holovaty and Simon Willison, began using Python to build applications. Jacob Kaplan-Moss was hired early in Django’s development shortly before Simon Willison's internship ended.[16] It was released publicly under a BSD license in July 2005. The framework was named after guitarist Django Reinhardt.[17], see wikipedia.
The Python which is a high-level programming language interpreted with general-purpose and together with the Django web framework creates a solution in fulfilling the objectives of web programming.
The problem of most of those who do not know closely the programming of this framework remains unknown and can be difficult to understand only from examples and tutorials.
Here are some of the difficulties that I personally encountered and had to solve them step by step.

understand how to use the link system between the python files created by Django;
how to use the templates and how to update them with the Django specific syntax;
understanding the way of displaying and resolving specific errors in Django;
using and setting the settings.py file;
understanding of how the web framework interacts with web technologies;

After solving these problems you will see the true power of this framework:

the development is easier with good and lower development costs and so are the additions and upgrades;
security is very good, see security documentation and deployment checklist;
is an open-source framework and updated by developers who use it;
most used and crowd tested and used to develop DropBox, Quora, Google, and Reddit;
comes with extensive documentation;
a large and community;

The official page of this web framework can be found on this webpage.

Monday, December 23, 2019

Python 3.7.5 : About Django REST framework.

First, let's activate my Python virtual environment:

[mythcat@desk django]$ source env/bin/activate

I update my django version 3.0 up to 3.0.1.

(env) [mythcat@desk django]$ pip3 install --upgrade django --user
Collecting django
...
      Successfully uninstalled Django-3.0
Successfully installed django-3.0.1

The next step comes with installation of Python modules for Django and Django REST:

(env) [mythcat@desk django]$ pip3 install djangorestframework --user
Collecting djangorestframework
...
Installing collected packages: djangorestframework
Successfully installed djangorestframework-3.11.0

Into my folder mysite I run this commands

(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ python3 manage.py makemigrations
No changes detected
(env) [mythcat@desk mysite]$ python3 manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions, test001
Running migrations:
  No migrations to apply.

To pass information over to an HTTP GET request, the information object must be translated into valid response data.
The Django implements serializers for this.
Serializers provide deserialization, allowing parsed data to be converted back into complex types and allow complex data such as querysets and model instances to be converted to native Python datatypes that can then be easily rendered into JSON, XML or other content types.
Let's create the mysite/serializers.py:

(env) [mythcat@desk mysite]$ cd mysite/
(env) [mythcat@desk mysite]$ vim serializers.py

The code for this python script is this:

from django.contrib.auth.models import User, Group
from rest_framework import serializers

class UserSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = User
        fields = ['url', 'username', 'email', 'groups']

class GroupSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = Group
        fields = ['url', 'name']

The next changes will be on urls.py and views.py.
My urls.py file from the Django-chart project is this:

from django.contrib import admin
from django.urls import path
from test001.views import home_page
from test001.views import Test001ChartView
#
from django.urls import include, path
from rest_framework import routers
from test001 import views

router = routers.DefaultRouter()
router.register(r'users', views.UserViewSet)
router.register(r'groups', views.GroupViewSet)

app_name = 'test001'
urlpatterns = [
    path('admin/', admin.site.urls),
    #path('', home_page, name ='home'),
    path('', Test001ChartView.as_view(), name = 'home'), 
    # Use automatic URL routing
    # Can also include login URLs for the browsable API
    path('', include(router.urls)),
    path('api-auth/', include('rest_framework.urls', namespace='rest_framework'))
]

My views.py file is this:

from django.http import HttpResponse
from django.shortcuts import render
# snippet 
from django.shortcuts import get_object_or_404
# for chart 
from django.views.generic import TemplateView
from .models import Test001, Snippet
#
#def home_page(request):
#    return HttpResponse('Home page!')

# django framework
from django.contrib.auth.models import User, Group
from rest_framework import viewsets
from mysite.serializers import UserSerializer, GroupSerializer

def home_page(request):
    return render(request, 'test001/home.html',{
        'name':'CGF',
        'html_items': ['a','b','c','d','e']
    })


# define view for chart 
class Test001ChartView(TemplateView):
    template_name = 'test001/chart.html'

    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        context["qs"] = Test001.objects.all()
        return context


def snippet_detail(request, id):
    snippet = get_object_or_404(Snippet, id=id)
    return render(request, 'test001/snippets_detail.html', {'snippet': snippet})

class UserViewSet(viewsets.ModelViewSet):
    """
    API endpoint  allows users to be viewed or edited.
    """
    queryset = User.objects.all().order_by('-date_joined')
    serializer_class = UserSerializer

class GroupViewSet(viewsets.ModelViewSet):
    """
    API endpoint  allows groups to be viewed or edited.
    """
    queryset = Group.objects.all()
    serializer_class = GroupSerializer

The settings module for this my project is stored in mysite/settings.py and I add this:

INSTALLED_APPS = [
    ...
    'rest_framework',
]

I run the django project and works well:

python3 manage.py runserver

The http://127.0.0.1:8000/users/ page come with this output:

Sunday, December 22, 2019

Python 3.7.5 : The new PyQt5 released.

The PyQt v5.14.0 has been released with support for Qt v5.14.0.

[mythcat@desk ~]$ pip3 install --upgrade PyQt5 --user
Collecting PyQt5
...
Installing collected packages: PyQt5-sip, PyQt5
Successfully installed PyQt5-5.14.0 PyQt5-sip-12.7.0

Let's see how can see the version:

[mythcat@desk ~]$ python3 
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5.Qt import PYQT_VERSION_STR
>>> print("PyQt version:", PYQT_VERSION_STR)
PyQt version: 5.14.0

Saturday, December 21, 2019

Python 3.7.5 : Simple web search with google python package.

This is a simple search on the web with python google package.

[mythcat@desk ~]$ pip3 install google --user
Collecting google
...
Installing collected packages: google
Successfully installed google-2.0.3

This is a simple example for search on web with this words: protv news 2019.
From the python package, I need to import just the search and used it.
The python package need a variable string named query.
The search wants to know the words from query and arguments, see the help:

Help on function search in module googlesearch:

search(query, tld='com', lang='en', tbs='0', safe='off', num=10, start=0, stop=None,

domains=None, pause=2.0, tpe='', country='', extra_params=None, user_agent=None)
    Search the given query string using Google.
    
    :param str query: Query string. Must NOT be url-encoded.
    :param str tld: Top level domain.
    :param str lang: Language.
    :param str tbs: Time limits (i.e "qdr:h" => last hour,
        "qdr:d" => last 24 hours, "qdr:m" => last month).
    :param str safe: Safe search.
    :param int num: Number of results per page.
    :param int start: First result to retrieve.
    :param int stop: Last result to retrieve.
        Use None to keep searching forever.
    :param list domains: A list of web domains to constrain
        the search.
    :param float pause: Lapse to wait between HTTP requests.
        A lapse too long will make the search slow, but a lapse too short may
        cause Google to block your IP. Your mileage may vary!
    :param str tpe: Search type (images, videos, news, shopping, books, apps)
        Use the following values {videos: 'vid', images: 'isch',
        news: 'nws', shopping: 'shop', books: 'bks', applications: 'app'}
    :param str country: Country or region to focus the search on. Similar to
        changing the TLD, but does not yield exactly the same results.
        Only Google knows why...
    :param dict extra_params: A dictionary of extra HTTP GET
        parameters, which must be URL encoded. For example if you don't want
        Google to filter similar results you can set the extra_params to
        {'filter': '0'} which will append '&filter=0' to every query.
    :param str user_agent: User agent for the HTTP requests.
        Use None for the default.
    
    :rtype: generator of str
    :return: Generator (iterator) that yields found URLs.
        If the stop parameter is None the iterator will loop forever.

This is the script:

[mythcat@desk ~]$ python3
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from googlesearch import search
>>> query = "protv news 2019"
>>> my_results_list = []
>>> for url in search(query,        
...     tld = 'com',  
...     lang = 'en',       
...     num = 10,     
...     start = 0,    
...     stop = None,  
...     pause = 2.0,):
...     my_results_list.append(url)
...     print(url)
... 
https://stirileprotv.ro/protvnews/
https://stirileprotv.ro/
https://stirileprotv.ro/superbun/protv-news.html
https://www.facebook.com/ProTvNews/
https://www.youtube.com/playlist?list=PLCJaU-QvLGR_FSZw6yeqBJHDe9LgFDRdy
https://www.youtube.com/watch?v=HaiQtDlaNic
https://www.youtube.com/watch?v=hxMEgAANSl4
https://www.youtube.com/channel/UCbbDChpDluLkdnH8QMwN6qA
https://www.youtube.com/watch?v=5zuN9uWFcTE
https://protvplus.ro/tv-live/1-pro-tv
https://en.wikipedia.org/wiki/Pro_TV
https://pro-tv.com/news/page/2/
https://m.youtube.cat/channel/UCbbDChpDluLkdnH8QMwN6qA
...

Sunday, December 15, 2019

Python 3.7.5 : Simple intro in CSRF.

CSRF or Cross-Site Request Forgery is a technique used by cyber-criminals to force users into executing unwanted actions on a web application.
To protect against web form CSRF attacks, it's isn't sufficient for web applications to trust authenticated users, must be equipped with a unique identifier called a CSRF token similar to a session identifier.
Django 3.0 can be used with CSRF, see the documentation page.
The CSRF process is a simple one in Django framework.
However, it's highly recommended to use the CsrfViewMiddleware instead.
To activate the django.middleware.csrf.CsrfViewMiddleware in the settings.py file.
Basically, you can use the decorator method in the view.py file.
For example, see documentation:

@csrf_protect

However csrf_protect will check only POST requests.
This annotation CSRF tells Django I'm not handling CSRF properly and don't fail this.
If I don't use this annotation then the CSRF is on if the post request is on by default and I will receive a 403 error.
As the developer using templates you don't have to know anything about that and you can use this in your template HTML5 file.

{% csrf_token %}

I can use the token with my code if I want to send it to the website.

from django.middleware.csrf import get_token

Then this can be used in view.py with a hidden form field with the name csrfmiddlewaretoken present in all outgoing POST forms.
You can used in any forms tag area like: input ...


def my_csrf_form(request):
    response = """ ... type = "hidden" name = "csrfmiddlewaretoken" value = "__token__" ... """

This can be used with:

token = get_token(request)
response = response.replace('__token__', html.escape(token))
response += dumpdata('POST', request.POST)
return HttpResponse(response)

This will get a token for the current request and fill into value = "__token__".
Using my old example from GitHub.
First, I start the project:


[mythcat@desk projects]$ source django/env/bin/activate
(env) [mythcat@desk projects]$ ls
cache  django  kaggle  logs  OSMnx  pygal_ex  SantaClaus.py
(env) [mythcat@desk projects]$ cd django/
(env) [mythcat@desk django]$ ls
env  mysite  venv
(env) [mythcat@desk django]$ cd mysite/

I change the chart.html file with this before javascript script:

... {% csrf_token %} ...

You can use it in javascript - jQuery like this, if you want:

var csrftoken = jQuery("[name=csrfmiddlewaretoken]").val();

The CSRF protection on Django on my form can bu used in view.py:

...
# use csrf_protect
from django.views.decorators.csrf import csrf_protect
from django.utils.decorators import method_decorator
...
# define view for chart 
class Test001ChartView(TemplateView):
    csrf_protected_method = method_decorator(csrf_protect)

I test it and works well.
The documentation tells us:
The CSRF protection cannot protect against man-in-the-middle attacks, so use HTTPS with HTTP Strict Transport Security. It also assumes validation of the HOST header and that there aren’t any cross-site scripting vulnerabilities on your site (because XSS vulnerabilities already let an attacker do anything a CSRF vulnerability allows and much worse).

Saturday, December 14, 2019

Python 3.7.5 : Django admin shell by Grzegorz Tężycki.

Today I tested another python package for Django named django-admin-shell.
This package created by Grzegorz Tężycki can be found on GitHub and come with the intro:
Django application can execute python code in your project’s environment on django admin site. You can use similar as python manage shell without reloading the environment.

[mythcat@desk ~]$ cd projects/
[mythcat@desk projects]$ cd django/
...
[mythcat@desk projects]$ source django/env/bin/activate
(env) [mythcat@desk projects]$

(env) [mythcat@desk projects]$ pip3 install django-admin-shell --user
Collecting django-admin-shell
...
Installing collected packages: django-admin-shell
Successfully installed django-admin-shell-0.1

First, I test my project to see if this working.

(env) [mythcat@desk projects]$ cd django/
(env) [mythcat@desk django]$ ls
env  mysite  venv
(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ ls
db.sqlite3  manage.py  mysite  test001
(env) [mythcat@desk mysite]$ python3 manage.py runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
December 14, 2019 - 18:23:47
Django version 3.0, using settings 'mysite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

You need to add this package into the settings.py file:

INSTALLED_APPS = [
    ...
    'django_admin_shell',
    ...
]

I add this package into my url.py file:

...
from django.conf.urls import include, url
...
urlpatterns = [
...
    # admin shell 
    url(r'^admin/shell/', include('django_admin_shell.urls')),
]

After that, you can start the project and see if this working: http://127.0.0.1:8000/admin/shell/
This screenshot shows me is working well:

Wednesday, December 11, 2019

Python 3.7.5 : The Pygal python package.

Today's tutorial aims to get data from a URL and display it with the Pygal python package.
I believe that global warming is a very important topic for human evolution.
You can read more about this topic on this website.
About this python package you can learn more at the official website.

[mythcat@desk ~]$ pip3 install Pygal --user
Collecting Pygal
...
Installing collected packages: Pygal
Successfully installed Pygal-2.4.0

Let's test this python package with this script:

import pygal
import pandas as pd
import numpy as np
URL = 'https://data.giss.nasa.gov/gistemp/graphs/graph_data/
Global_Mean_Estimates_based_on_Land_and_Ocean_Data/graph.txt'

df = pd.read_fwf(
    URL,
    skiprows=(0,1,2,4),
    index_col=0,
)
print(df.head)

year_data = df['No_Smoothing']
print(year_data.head)
one_chart = pygal.Bar()
one_chart.title = "Land-Ocean Temperature Index (C)"
one_chart.add("",year_data)
one_chart.render_in_browser()

The result is this output:

Monday, December 9, 2019

Python 3.7.5 : The OSMnx python package.

About this python package named OSMnx, you can read on GitHub.
OSMnx is a Python package that lets you download spatial geometries and model, project, visualize, and analyze street networks from OpenStreetMap's APIs. Users can download and model walkable, drivable, or bikable urban networks with a single line of Python code, and then easily analyze and visualize them ...
You cannot install osmnx directly in Fedora 31 Linux distro because of the dependency with python packages.

[mythcat@desk projects]$ pip3 install osmnx --user

Installation depends by rtree python packages:

[root@desk projects]# dnf install python3-rtree.noarch
...
Installed:
  python3-rtree-0.9.1-1.fc31.noarch

spatialindex-1.9.3-1.fc31.x86_64                             

Complete!

Now you can install with pip3 tool if you use the python3.

[mythcat@desk projects]$ pip3 install osmnx --user
Collecting osmnx
...
Installing collected packages: pyparsing, kiwisolver, cycler, matplotlib, descartes, networkx, Shapely,

 pyproj, click, click-plugins, cligj, attrs, munch, fiona, geopandas, osmnx
Successfully installed Shapely-1.6.4.post2 attrs-19.3.0 click-7.0 click-plugins-1.1.1 cligj-0.5.0

cycler-0.10.0 descartes-1.1.0 fiona-1.8.13 geopandas-0.6.2 kiwisolver-1.1.0 matplotlib-3.1.2

munch-2.5.0 networkx-2.4 osmnx-0.11 pyparsing-2.4.5 pyproj-2.4.2.post1

Let's test the default example:

[mythcat@desk projects]$ python3
Python 3.7.5 (default, Oct 17 2019, 12:16:48) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import osmnx as ox
>>> ox.config(log_file=True, log_console=True, use_cache=True)
Configured osmnx
>>> my_loc  = ox.graph_from_address('Fălticeni, Suceava, 725200, Romania', network_type= 'all',

retain_all =True)
Geocoded "Fa?lticeni, Suceava, 725200, Romania" to (47.4597637, 26.30255)
Projected the GeoDataFrame "geometry to project" to UTM-35 in 0.01 seconds
Projected the GeoDataFrame "geometry to project" to default_crs in 0.01 seconds
Created bounding box 1000 meters in each direction from (47.4597637, 26.30255): 47.46876106367192,

47.45076632191398,26.31581675508982,26.289283302659218
Projected the GeoDataFrame "geometry to project" to UTM-35 in 0.01 seconds
Projected the GeoDataFrame "geometry to project" to default_crs in 0.01 seconds
Projected the GeoDataFrame "geometry to project" to UTM-35 in 0.01 seconds
Projected the GeoDataFrame "geometry to project" to default_crs in 0.01 seconds
Requesting network data within bounding box from API in 1 request(s)
Retrieved response from cache file "cache/eb15fb167cf173e622992dbc2b7b7c77.json" for URL

"http://overpass-api.de/api/interpreter?data=%5Bout%3Ajson%5D%5Btimeout%3A180%5D%3B%28way

%5B%22highway%22%5D%5B%22area%22%21~%22yes%22%5D%5B%22highway%22%21~%22proposed%7Cconstruction

%7Cabandoned%7Cplatform%7Craceway%22%5D%5B%22service%22%21~%22private%22%5D%5B%22access%22%21

~%22private%22%5D%2847.446267%2C26.282649%2C47.473260%2C26.322452%29%3B%3E%3B%29%3Bout%3B"
Got all network data within bounding box from API in 1 request(s) and 0.01 seconds
Creating networkx graph from downloaded OSM data...
Created graph with 2,003 nodes and 3,930 edges in 0.08 seconds
Added edge lengths to graph in 0.06 seconds
Truncated graph by bounding box in 0.05 seconds
Begin topologically simplifying the graph...
Identified 339 edge endpoints in 0.03 seconds
Constructed all paths to simplify in 0.01 seconds
Simplified graph (from 2,003 to 339 nodes and from 3,930 to 803 edges) in 0.22 seconds
Truncated graph by bounding box in 0.01 seconds
Got the counts of undirected street segments incident to each node (before

removing peripheral edges) in 0.07 seconds
graph_from_bbox() returning graph with 235 nodes and 556 edges
graph_from_point() returning graph with 235 nodes and 556 edges
graph_from_address() returning graph with 235 nodes and 556 edges
>>> my_proj = ox.project_graph(my_loc)
Created a GeoDataFrame from graph in 0.05 seconds
Projected the GeoDataFrame "unnamed_nodes" to UTM-35 in 0.03 seconds
Projected the GeoDataFrame "unnamed_edges" to UTM-35 in 0.10 seconds
Extracted projected node geometries from GeoDataFrame in 0.01 seconds
Rebuilt projected graph in 0.94 seconds
>>> fig, ax = ox.plot_graph(my_proj)
Begin plotting the graph...
Created GeoDataFrame "unnamed_UTM_edges" from graph in 0.07 seconds
Drew the graph edges in 0.04 seconds

This python module comes with many features.
You can select a polygon area and show the map.
This is the result of source code from this tutorial:

Sunday, December 8, 2019

Python 3.7.5 : Starting with kaggle platform.

Kaggle is the world's largest community of data scientists and the platform is the fastest way to get started on a new data science project.
A good choice to use Kaggle is this feature: Kaggle provides free access to NVidia K80 GPUs in kernels.
The tutorial for today is about kaggle and is new for me because I hear about this opportunity last year.
This platform that hosts data science and machine learning competitions can give the people a good area for development.
The official blog can tell you more about how can this platform works.
The kaggle A.P.I. can be found at GitHub.
Let's start the tutorial with the pip3 install tool:

[mythcat@desk kaggle]$ pip3 install kaggle --upgrade --user
Collecting kaggle
...
[mythcat@desk kaggle]$ mkdir ~/.kaggle/
[mythcat@desk kaggle]$ mv kaggle.json ~/.kaggle/kaggle.json
[mythcat@desk kaggle]$ kaggle 
Warning: Your Kaggle API key is readable by other users on this system! To fix this, you can

 run 'chmod 600 /home/mythcat/.kaggle/kaggle.json'
usage: kaggle [-h] [-v] {competitions,c,datasets,d,kernels,k,config} ...
kaggle: error: the following arguments are required: command
[mythcat@desk kaggle]$ chmod 600 /home/mythcat/.kaggle/kaggle.json

You can use the kaggle command to get informations from kaggle platform:

[mythcat@desk kaggle]$ kaggle competitions list 
ref                                            deadline             category            reward  teamCount
  userHasEntered  
---------------------------------------------  -------------------  ---------------  ---------  ---------
  --------------  
digit-recognizer                               2030-01-01 00:00:00  Getting Started  Knowledge       2305
           False  
titanic                                        2030-01-01 00:00:00  Getting Started  Knowledge      17135
           False  
house-prices-advanced-regression-techniques    2030-01-01 00:00:00  Getting Started  Knowledge       5532
           False  
imagenet-object-localization-challenge         2029-12-31 07:00:00  Research         Knowledge         57
           False  
google-quest-challenge                         2020-02-10 23:59:00  Featured           $25,000        410
           False  
tensorflow2-question-answering                 2020-01-22 23:59:00  Featured           $50,000        810
           False  
data-science-bowl-2019                         2020-01-22 23:59:00  Featured          $160,000       1762
           False  
pku-autonomous-driving   
...

The commands for this platform can be seen at GitHub:

kaggle competitions {list, files, download, submit, submissions, leaderboard}
kaggle datasets {list, files, download, create, version, init}
kaggle kernels {list, init, push, pull, output, status}
kaggle config {view, set, unset}

The kaggle platform use an online tool with notebooks similar with Jupyter notebook for process coding in Kernels.
The process of using models need the datasets and you can use the kaggle datasets.
You can use the New Notebook button from that page to start using the datasets.
The datasets can be load from kaggle or can be uploaded.
The next window let you to Select new notebook settings.
I used Python with a notebook and I got the default source code:

# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load in 

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)

# Input data files are available in the "../input/" directory.
# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input

 directory

import os
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

# Any results you write to the current directory are saved as output.

Now, I can read the dataset shown in the right the Data with input
(read-only data) and output from kaggle with pandas module:

data = pd.read_csv("../input/lego-database/colors.csv")
data.head()

The Commit button let you to save your work for later.
You can see my online test I created with dataset Lego and python on my kaggle page.

Saturday, December 7, 2019

Python 3.7.5 : Using the django with javascript.

The Django framework can work great with javascript.
I start this project a few days ago.
You can start a simple Django project, see my old tutorials.
I used a template for main page in my template folder named chart.html.
The chart.min.js is set on base.html.
The links work from the Django framework to HTML5 templates with javascript.
The full project can be found on my GitHub project.
See the screen output:

Thursday, December 5, 2019

Python 3.7.5 : About PEP 8.

Today is December 5, 2019, and this is the reason I wrote about PEP 8.
The official webpage can be found at this webpage.
PEP 8 recommends using 4 spaces to show indentation and tabs should only be used to maintain consistency in the code.
The Python Library is conservative and 79 characters are the maximum required line limit as suggested by PEP 8.
The main goal is to avoid line wrapping.
Indentation allows you to differentiate between multiple lines of code and a single line of code that spans multiple lines.
The closing braces with the first white-space character of the last line.
The use of blank lines can greatly improve the readability of your code.
This rule allows the reader to understand the separation of the sections of code and the relation between them.

Type	Naming Conventions	Examples
Variable	Using short names with CapWords.	T, AnyString, My_First_Variable
Function	Using a lowercase word or words with underscores to improve readability.	function, my_first_function
Class	Using CapWords and do not use underscores between words.	Student, MyFirstClass
Method	Using lowercase words separated by underscores.	Student_method, method
Constants	Using all capital letters with underscores separating words	TOTAL, MY_CONSTANT, MAX_FLOW
Exceptions	Using CapWords without underscores.	IndexError, NameError
Module	Using short lower-case letters using underscores.	module.py, my_first_module.py
Package	Using short lowercase words and underscores are discouraged.	package, my_first_package

About comments, PEP 8 basically recommends using block comments for general-purpose coding.
Document strings or docstrings start at the first line of any function, class, file, module or method.
These type of comments are enclosed between single quotations ( ''') or double quotations ( """ ).
To improving the readability of expressions and statements use whitespace for any character or sequence of characters.
We need to avoiding whitespaces:

inside parentheses, brackets, or braces;
before a comma, a semicolon, or a colon;
before open parenthesis that initiates the argument list of a function call;
before an open bracket that begins an index or a slice;
between a trailing comma and a closing parenthesis;
to adjust assignment operators;

PEP 8 recommends using a clear development for implementation of source code, for example PEP recommends using .startswith() and .endswith() instead of list slicing.
You can check whether your code actually complies with the rules and regulations of PEP 8 or not.
The linters is a program that analyzes your code and detects program errors, syntax errors, bugs, and structural problems.
An autoformatted tool will format your code to adapt to PEP 8 automatically.
Both tools used to implement and check PEP 8 compliance.
To know more about PEP 8 and its book of guidelines, you can refer to pep8.org.
You can check it online at this webpage.

Tuesday, December 3, 2019

Python 3.7.5 : The new Django version 3.0 .

On December 2, 2019, comes with Django 3.0 Released.
This new release comes with many changes and features, let's see :

Django 3.0 supports new versions of Python 3.6, 3.7, and 3.8 (the old Django 2.2.x series is the last to support Python 3.5);
Django now officially supports MariaDB 10.1 and higher;
Django is fully async-capable by providing support for running as an ASGI application;
The exclusion class constraints on PostgreSQL;
the output BooleanField may now be used directly in QuerySet filters;
you can have custom enumeration types
cache features: add_never_cache_headers() and never_cache() now add the private directive to Cache-Control headers;
file storage, forms, internationalization, and logging features;
the management commands are new options;
the Requests and Responses, Models and Security has many changes and new features;
the Tests and test cases have a good implementation;
new minor features for django.contrib.admin, django.contrib.auth, django.contrib.gis, django.contrib.postgres, django.contrib.sessions, django.contrib.syndication and more

All these features can be read at the official webpage.

Friday, November 29, 2019

Python 3.7.5 : Script install and import python packages.

This script will try to import Python packages from a list.
If these packages are not installed then they will be installed on the system.

import sys
import subprocess

if __name__ == '__main__': 
    def ModuleInstall(package_name):
        try:
            subprocess.check_call(['python3', '-m', 'pip3', 'install', package_name, "--user"])
        except:
            subprocess.check_call(['python', '-m', 'pip', 'install', package_name, "--user"])
    def ModuleLoaded(package_name):
        __import__(package_name)
        print ("Successfully imported ", package_name, '.')

    required_libs = [
    "optparse",
    "socket",
    "select",
    "threading",
    "time",
    "re",
    "ssl",
    "textwrap",
    "sys",
    "random",
    "traceback",
    "binascii",
    "struct",
    "inspect"]
    for lib_name in required_libs:
        try:
            ModuleLoaded(lib_name)
        except:
            ModuleInstall(lib_name)

Monday, November 25, 2019

Python 3.7.5 : Python and SQL, a fast approach.

Today he had to solve a task that required declaring a series of consecutive days to read from several SQL tables.
As you know, this task would require a complex query to include statements.
I solved it quite simply by using a python script that would automatically generate the days and then concatenate them into a query.
It may not be the best solution but it is very fast and does not require very advanced SQL knowledge.
Let's see the source code:

from datetime import date, timedelta

start_date = date(2019, 01, 1)
end_date = date(2019, 11, 24)
delta = timedelta(days=1)
q1 = str("SELECT DATE_FORMAT(updated, '%Y-%m-%d') AS day, intern, COUNT(*) FROM `intern")
q2 = str("` WHERE intern = 0 GROUP BY intern, day UNION ALL ")
while start_date <= end_date:
    print (q1+start_date.strftime("%Y_%m_%d")+q2)
    start_date += delta

The result querry will be like this:

SELECT DATE_FORMAT(updated, "%Y-%m-%d") AS day, intern, COUNT(*) FROM `intern2019_01_01` GROUP BY intern, 
day UNION ALL 
SELECT DATE_FORMAT(updated, "%Y-%m-%d") AS day, intern, COUNT(*) FROM `intern2019_01_02` GROUP BY intern,
 day UNION ALL
...
SELECT DATE_FORMAT(updated, "%Y-%m-%d") AS day, intern, COUNT(*) FROM `intern2019_11_24` GROUP BY intern,
 day UNION ALL

The last step, I remove the UNION ALL from the last SELECT and I used with phpmyadmin.

Saturday, November 23, 2019

Python 3.7.5 : Create GUI with npyscreen.

This python module solves the issue of creating easy GUI in the terminal.
The development team tells us:
Npyscreen is a python widget library and application framework for programming terminal or console applications.
The development of this python module is similar to the PyQt python module.
The npyscreen comes with many widgets and easy development.
The GitHub repo for this python module comes with many examples.
The documentation of this python module can be found at this

[mythcat@desk ~]$ pip3 install npyscreen --user
Collecting npyscreen
...
Installing collected packages: npyscreen
Successfully installed npyscreen-4.10.5

Let's see one simple example:

#!/usr/bin/env python3
import npyscreen

class ProductForm(npyscreen.Form):
     def create(self):
        self.product_name = self.add(npyscreen.TitleText, name='Name')
        self.category = self.add(npyscreen.TitleSelectOne, scroll_exit=True,\
     max_height=3, name='Category', values = ['Products 1', 'Products 2', 'Products 3'])
        self.myDate = self.add(npyscreen.TitleDateCombo, name='Date stocking')

class MyTest(npyscreen.NPSAppManaged):
    def onStart(self):
        self.addForm('MAIN', ProductForm, name='New product')

if __name__ == '__main__':
    TestApp = MyTest().run()

The result of this python script can be seen on the next image.

Tuesday, November 19, 2019

Python 3.7.5 : Display a file in the hexadecimal and binary output.

This is an example with a few python3 modules that display a file in the hexadecimal and binary output:

import sys
import os.path
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("FILE", help="the file that you wish to dump to hexadecimal", type=str)
parser.add_argument("-b", "--binary", help="display bytes in binary format instead of hexadecimal")
args = parser.parse_args()

try:
    with open(args.FILE, "rb") as f:
        n = 0
        b = f.read(16)
        while b:
            if not args.binary:
                s1 = " ".join([format(i,'02x') for i in b])
                s1 = s1[0:23] + " " + s1[23:]
                width = 48
            else:
                s1 = " ".join([format(i,'08b') for i in b])
                s1 = s1[0:71] + " " + s1[71:]
                width = 144
            s2 = "".join([chr(i) if 32 <= i <= 127 else "." for i in b])
            print(format(n * 16,'08x'), format(s1), format(s2))
            n += 1
            b = f.read(16)

    print(format(os.path.getsize(args.FILE),'08x'))

except Exception as e:
    print(__file__, ": ", type(e).__name__, " - ", e, sep="", file=sys.stderr)

The hexadecimal output of the sec.png file:

[mythcat@desk test]$ python3 hex.py sec.png 
00000000 89 50 4e 47 0d 0a 1a 0a  00 00 00 0d 49 48 44 52 .PNG........IHDR
00000010 00 00 01 0b 00 00 00 bd  08 03 00 00 00 93 6f a8 ..............o.
00000020 bc 00 00 00 5d 50 4c 54  45 ff ff ff ab ab ab fc ....]PLTE.......
00000030 fc fc a0 a0 a0 f5 f5 f5  a6 a6 a6 f9 f9 f9 9e 9e ................
00000040 9e f1 f1 f1 fa fa fa ea  ea ea ad ad ad a4 a4 a4 ................
00000050 d0 d0 d0 f6 f6 f6 e5 e5  e5 bc bc bc c3 c3 c3 97 ................
00000060 97 97 b4 b4 b4 d8 d8 d8  bf bf bf cb cb cb db db ................
00000070 db e1 e1 e1 93 93 93 7f  7f 7f 8d 8d 8d 85 85 85 .............
00000080 79 79 79 70 70 70 d9 a6  b1 29 00 00 08 fa 49 44 yyyppp...)....ID
00000090 41 54 78 9c e5 9d 89 92  a3 2a 14 40 45 45 09 a8 ATx......*.@EE..
000000a0 80 18 34 d3 3d ef ff 3f  f3 69 92 4e 67 71 45 90 ..4.=..?.i.NgqE.
000000b0 65 4e 55 4f a5 7b 52 c6  20 dc 9d 4b 14 f9 46 f7 eNUO.{R. ..K..F.
000000c0 87 36 79 8d 6d df 86 75  28 03 e4 4f c4 cb 28 ca .6y.m..u(..O..(.
000000d0 63 db f7 62 97 36 91 8f  e9 80 72 ce 6c de 8b 65 c..b.6....r.l..e
000000e0 c8 f9 f9 b7 14 53 68 eb  4e ac 73 a1 ef 7f 91 d2 .....Sh.N.s....
000000f0 c6 7d 6c 04 71 91 e9 be  66 f2 31 14 51 04 dd 97 .}l.q...f.1.Q...
00000100 a0 9c d0 8e e8 be 66 33  f6 d7 5a f3 a7 68 a7 e4 ......f3..Z..h..
00000110 91 f6 f9 8b c1 e8 9f 45  aa f5 53 34 93 45 71 7e .......E..S4.Eq~
00000120 7d 21 4a 9d 97 9d 98 00  ba 67 9f 4e b2 3a 61 a2 }!J......g.N.:a.
00000130 bb bd 96 10 f5 ff e2 b2  c3 d5 ee eb b6 13 b3 4c ...............L
00000140 a0 dd 97 36 06 96 51 c7  7f 7e 41 02 00 c8 da 86 ...6..Q.~A.....
00000150 13 00 bb 7d d7 9d 94 0b  f9 88 44 75 03 fc f5 f6 ...}......Du....
00000160 9c d2 d3 fd 45 d2 ff 34  cf 0f 17 89 75 aa 26 46 ....E..4....u.&F
00000170 b4 2d b9 98 fc 7f a0 75  29 ea 43 4e df b2 68 a2 .-....u).CN..h.
...

The binnary output of sec.png file:

[mythcat@desk test]$ python3 hex.py -b FILE sec.png 
00000000 10001001 01010000 01001110 01000111 00001101 00001010 00011010 00001010  
00000000 00000000 00000000 00001101 01001001 01001000 01000100 01010010 .PNG........IHDR
00000010 00000000 00000000 00000001 00001011 00000000 00000000 00000000 10111101
  00001000 00000011 00000000 00000000 00000000 10010011 01101111 10101000 ..............o.
00000020 10111100 00000000 00000000 00000000 01011101 01010000 01001100 01010100
  01000101 11111111 11111111 11111111 10101011 10101011 10101011 11111100 ....]PLTE.......
00000030 11111100 11111100 10100000 10100000 10100000 11110101 11110101 11110101
  10100110 10100110 10100110 11111001 11111001 11111001 10011110 10011110 ................
00000040 10011110 11110001 11110001 11110001 11111010 11111010 11111010 11101010
  11101010 11101010 10101101 10101101 10101101 10100100 10100100 10100100 ................
00000050 11010000 11010000 11010000 11110110 11110110 11110110 11100101 11100101
  11100101 10111100 10111100 10111100 11000011 11000011 11000011 10010111 ................
00000060 10010111 10010111 10110100 10110100 10110100 11011000 11011000 11011000
  10111111 10111111 10111111 11001011 11001011 11001011 11011011 11011011 ................
00000070 11011011 11100001 11100001 11100001 10010011 10010011 10010011 01111111
  01111111 01111111 10001101 10001101 10001101 10000101 10000101 10000101 .............
00000080 01111001 01111001 01111001 01110000 01110000 01110000 11011001 10100110
  10110001 00101001 00000000 00000000 00001000 11111010 01001001 01000100 yyyppp...)....ID
00000090 01000001 01010100 01111000 10011100 11100101 10011101 10001001 10010010
  10100011 00101010 00010100 01000000 01000101 01000101 00001001 10101000 ATx......*.@EE..
...

Friday, November 15, 2019

Python 3.7.5 : About PEP 8016.

Let's start with PEP 8012 which proposes a new model of Python governance based on consensus and voting, without the role of a centralized singular leader or a governing council and was rejected.
The PEP 8015 formalize the current organization of the Python community and proposes changes.
This PEP 8015 was rejected by a core developer vote described in PEP 8001 on Monday, December 17, 2018.
The next PEP 8016 proposes a model of Python governance based around a steering council.
All candidate PEPs are listed in PEP 8000 and consists of all PEPs numbered in the 801X range.
The PEP 8001 refers to the voting process and choose which governance PEP should be implemented by the Python project.
All these PEP's 2012, 2013, 2014, 2015 are rejected.
The last one PEP 2016 named The Steering Council Model was accepted.
The PEP 2016 comes with these new rules:
This PEP proposes a model of Python governance based around a steering council. The council has broad authority, which they seek to exercise as rarely as possible; instead, they use this power to establish standard processes, like those proposed in the other 801x-series PEPs. This follows the general philosophy that it's better to split up large changes into a series of small changes that can be reviewed independently: instead of trying to do everything in one PEP, we focus on providing a minimal-but-solid foundation for further governance decisions.
You can read more about this PEP on this webpage.

Tuesday, November 5, 2019

Python 3.7.5 : About PEP 3107.

The PEP 3107 introduces a syntax for adding arbitrary metadata annotations to Python functions.
The function annotations refer to syntax parameters with an expression.

def my_function(x: expression, y: expression = 5):
...

For example:

>>> def show(myvar:np.float64):
...     print(type(myvar))
...     print(myvar)
... 
>>> show(1.1)

1.1
>>> def files(filename: str, dot='.') -> list:
...     print(filename)
...     print(type(filename))
... 
>>> files('file.txt')
file.txt

>>> print(files.__annotations__)
{'filename': , 'return': }
>>> print(show.__annotations__)
{'myvar': }
...

You can see the annotation syntax with a dictionary called __annotations__ as an attribute on your functions.
This lets you rewrite Python 3 code with function annotations to be compatible with both Python 3 and Python 2.
Type hints are a specialization of function annotations, and they can also work side by side with other function annotations.
Annotations have no standard meaning or semantics.
There are several benefits to the annotations:

if you rename an argument, the documentation docstring version may be out of date and is easier to see if an argument is not documented;
is no need to come up with a special format of argument because the annotations attribute provides a direct, standard mechanism of access;

Let's see one example using type aliases:


>>> Temperature = float
>>> def forecast(local_temperature: Temperature) -> str:
...     print(local_temperature)
... 
>>> forecast(13.1)
13.1
...

I can create multiple annotations:


>>> def div(a: dict(type=float, help='the dividend'), b: dict(type=float, help='this <> 0)') ) -> 
dict(type=float, help='the result of dividing a by b'):
...     return a / b
... 
>>> div(3,4)
0.75
...

Annotations for excess parameters like *args and **kwargs, allow arbitrary number of arguments to be passed in a function call.
See example with my_func:

def my_func(*args: expression, *kwargs: expression):
...

Annotations combine well with decorators to provide input to a decorator, and decorator-generated wrappers are a good place to put code that gives meaning to annotations, but this is another issue.

Monday, November 4, 2019

Python 3.7.5 : About PEP 506.

Today I did a python evaluation and saw that there are many new aspects that should be kept in mind for a programmer.
So I decided to recall some necessary elements of PEP.
First, PEP stands for Python Enhancement Proposal.
A PEP is a design document providing information to the Python community, or describing a new feature for Python or its processes or environment.
My list will not follow a particular order and I will start with PEP 506.
This PEP 506 proposes the addition of a module for common security-related functions such as generating tokens to the Python standard library.
Python 3.6 added a new module called secrets that is designed to provide an obvious way to reliably generate cryptographically strong pseudo-random values suitable for managing secrets, such as account authentication, tokens, and similar.
Python’s random module was never designed for cryptographic but you can try to use it with urandom function:

[mythcat@desk ~]$ python3
Python 3.7.5 (default, Oct 17 2019, 12:09:47) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.urandom(8)

This module named secrets will contain a set of ready-to-use functions for dealing with anything which should remain secret (passwords, tokens, etc.).

>>> import secrets
>>> import string
>>> alphabet = string.ascii_letters + string.digits
>>> password = ''.join(secrets.choice(alphabet) for i in range(20)) 
>>> print(alphabet, password)
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 mwTKhSxGGBMU3voOV1Kf

The secrets module also provides several methods of generating tokens, see example:
As bytes, secrets.token_bytes;

>>> secrets.token_bytes()
b'I\xf9a\xd1j\xc6\xc9\xa0qV\x82\x07x\xc6\xe9\xbb\xd7<\xfb\xb2?\xe1\x94\xe9\xce\xbc\xaaF\xfc7\xfc='
>>> secrets.token_bytes(8)
b'\rl\xb1\xb9\x04i]d'
>>> secrets.token_bytes(16)
b'B!:G\x1c\xdd.\xacC\x7f\x95)\x1f^\xec\xb2'
>>> secrets.token_bytes(32)
b'\xfa\xa9\xff\x91y\x9e+z9\x88K\x95\xa8\xb0\x06\xc2b:\xf5]\xcf^%~\x0cJ\xdd\x80\xa2\xa0\xdc\xaa'
>>> secrets.token_bytes(64)
b"\xe4(\x80d7c6\\\xb2\xd5\xcb\x92\x8a'\x82\xcb\xfd\xcc\x9a\x8a\xd9jt\x84s\xb0\x8f]\x8cS\xdcP\n\xef\x14\xf6\
xe0+0\xaf\xcfL\xd3\xd0\xfe\x04\x98k\xc38\xf6\xad.~\xd1\xca\xd6\xc9\xf9\xbf\xff8O\xad"

As text, using hexadecimal digits, secrets.token_hex;

>>> secrets.token_hex()
'5a2eb8a0a89ecaf5a64e57215f359012eaaf8a3db51bd1ea171e922a24935183'
>>> secrets.token_hex(8)
'79e7582b72711af7'
>>> secrets.token_hex(16)
'9b274380935ae169ebd41159f7b85cf6'
>>> secrets.token_hex(32)
'0a2e5fde42c6578c3ba36501b69a9339e838d44c3240999a83d349d266bcb164'
>>> secrets.token_hex(64)
'fbd9ab627e9fe6c2b6d715b1438205321ac9139f5089fe6ca4ffece79aa0c08aa84a26fdbb984dc48a0489e1692b19d3f5fe40116be
60f1a1d7d61739718befe'

As text, using URL-safe base-64 encoding, secrets.token_urlsafe.

>>> secrets.token_urlsafe()
'L06rX6fIk1n-gpcLbsHq_w5SgkqgGcvnkjBRcOZqgXs'
>>> secrets.token_urlsafe(8)
'lhOw5llcgsQ'
>>> secrets.token_urlsafe(16)
'A493DgcDMiNx8WjlRswxBA'
>>> secrets.token_urlsafe(32)
'HSb5dqkaPrqFcdsQFYW5N_Fxb_Hxn0ESsT4VMfJcLYY'
>>> secrets.token_urlsafe(64)
'FKPC0LU7Sc_dsxm7m-VMA-vTEKgJeNcD2zpjKBEg0oLZlPBVVM0O5Vztp0ySLifyifok5009LByQUc5z8thCWQ'

analitics

Pages