analitics

Pages

Monday, January 6, 2020

Python 3.7.5 : Post class and migration process.

Today I will solve some issues with the Django framework like:
  • create a new class for posts;
  • explain how the migration process works.
  • use the database with Django shell;
Let's activate the environment:
[mythcat@desk django]$ source env/bin/activate
I used my old project django-chart, see my old tutorials.
Let's add some source code to the models.py to create a class for the post into my website:
...
# Post data 
from django.utils import timezone
from django.contrib.auth.models import User
...
#create a post
class Post(models.Model):
  title = models.CharField(max_length = 100)
  content = models.TextField()
  date_posted = models.DateTimeField(default = timezone.now)
  #if the user is deleted their posts is deleted 
  author = models.ForeignKey(User, on_delete = models.CASCADE)
Let's use the migration process to update the database:
(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ python3 manage.py makemigrations
Migrations for 'test001':
  test001/migrations/0003_post.py
  - Create model Post
Before to run the migrate command I will show what happened with this migration output from SQL view.
This is great to solve issues and see the exact SQL code generated.
I have in the output of command makemigrations the application name test001 and the migration number 0003.
I will use all of this information output to show how you can see the SQL code:
(env) [mythcat@desk mysite]$ python3 manage.py sqlmigrate test001 0003
BEGIN;
--
-- Create model Post
--
CREATE TABLE "test001_post" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, 
"title" varchar(100) NOT NULL, "content" text NOT NULL, "date_posted" datetime NOT NULL, 
"author_id" integer NOT NULL REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED);
CREATE INDEX "test001_post_author_id_fed29ee6" ON "test001_post" ("author_id");
COMMIT;
This print SQL code that it's going to run when I will use the next command:
(env) [mythcat@desk mysite]$ python3 manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions, test001
Running migrations:
  Applying test001.0003_post... OK
The migration process solves issues linked to changes in the database.
Let's use the Django shell to see the User changes into the database:
(env) [mythcat@desk mysite]$ python3 manage.py shell
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from test001.models import Post
>>> from django.contrib.auth.models import User
>>> User.objects.all()
...
>>> User.objects.first()
...
>>> User.objects.filter(username='catalin')
...
>>> User.objects.filter(username='catalin').first()
...
>>> user = User.objects.filter(username='catalin').first()
>>> user
...
>>> user.id
1
>>> user.pk
1
>>> Post.objects.all()
...
Let's add a post to the database and show it:
>>> post_1=Post(title='First title', content='The first content!',author=user)
>>> post_1.save()
>>> Post.objects.all()
...
I can see the posts and information about the user with the Django shell.
>>> post = Post.objects.first()
>>> post.content
'The first content!'
>>> post.date_posted
...
>>> post.author
...
>>> post.author.email
...
>>> user.post_set.all()
...
You need to add to views.py the new model changes:
...
from .models import Post
...
def posts(request):
  context = {
  'posts':Post.objects.all()
  }
  return render(request, 'test001/posts.html', context)
...
Into the templates folder, I created the file named posts.html to load the data to HTML5.
{% extends 'base.html' %}
{% block content %}
  {% for post in posts %}
  <article>
  <div>{{ post.title }}</div>
  <div>{{ post.content }}</div>
  <div>{{ post.author }}</div>
  <div>{{ post.date_posted|date:"F d, Y" }}</div>
  </article>
  {% endfor %}
{% endblock content %}
To see these changes into the website I add the route to this HTML5 file posts.html into urls.py file:
app_name = 'test001'
urlpatterns = [
...
    path('posts/',posts, name = 'posts'),
...
]
Now I can run the server and see the output from the database into posts URL.
(env) [mythcat@desk mysite]$ python3 manage.py runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
January 06, 2020 - 17:52:29
Django version 3.0.1, using settings 'mysite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
...
This is the output of my running server...

Python 3.7.5 : Set json with settings in Django project.

[mythcat@desk django]$ source env/bin/activate
(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ ls
db.sqlite3  manage.py  mysite  test001
(env) [mythcat@desk mysite]$ pwd
/home/mythcat/projects/django/mysite
Create a file named config.json in the folder django:
(env) [mythcat@desk mysite]$ vim /home/mythcat/projects/django/config.json
Open your settings.py file from your Django project and copy your secret key from this file to config.json, see:
{ 
  "SECRET_KEY":"your_secret_key_from_settings.py"
}
Change your settings.py file with these changes:
...
import json 
...
with open('/home/mythcat/projects/django/config.json') as config_file:
    config = json.load(config_file)
...
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = config['SECRET_KEY']
This allows us to get the SECRET_KEY from the config.json file.
We can add many variables into file config.json, like EMAIL_USER and EMAIL_PASS.
You need to have a validate JSON file and the settings.py will change from this:
EMAIL_HOST_USER = os.environ.get('EMAIL_USER')
...
to this source code for EMAIL_USER variable:
EMAIL_HOST_USER = config.get('EMAIL_USER')
...

Sunday, January 5, 2020

Python 3.7.5 : Testing cryptography python package - part 001.

There are many python packets that present themselves as useful encryption and decryption solutions. I recommend before you test them, use them and spend time with them to focus on the correct study of cryptology because many disadvantages and problems can arise in the correct and safe writing of the source code.
Today I will show you a simple example with cryptography python package.
Let's install this with pip3 tool:
[mythcat@desk projects]$ pip3 install cryptography --user
You can read more about this python package on the documentation website.
Let's try a simple method of encryption and decryption.
I will show you how to solve this issue with the class cryptography.hazmat.primitives.ciphers.aead.AESGCM:
The AES-GCM construction is composed of the AES block cipher utilizing Galois Counter Mode (GCM).
The GCM (Galois Counter Mode) is a mode of operation for block ciphers.
An AEAD (authenticated encryption with additional data) mode is a type
of block cipher mode that simultaneously encrypts the message as well as authenticating it.

The Value of AESGCM key must be 128, 192, or 256 bits, see the size of the key:
[mythcat@desk projects]$ python3
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from cryptography.hazmat.primitives.ciphers.aead import AESGCM
>>> aad = None
>>> key = 'catafestcatafestcatafestcatafest'
>>> aesgcm = AESGCM(key.encode('utf-8'))
>>> nonce = '12345678'
>>> data_binary = b'Hello world!'
>>> 
>>> cipher_text = aesgcm.encrypt(nonce.encode('utf-8'), data_binary, aad)
>>> print(cipher_text)
b'0\xab\xd2!mXe\xc3/\xdb\x15\xcaoT\x0f\x1d\xbb&\xc4\x92\xdf\\ZTMD\xa2\x9f'
>>> data_text = aesgcm.decrypt(nonce.encode('utf-8'), cipher_text, aad)
>>> print(data_text)
b'Hello world!'

Saturday, January 4, 2020

Python 3.7.5 : Testing the PyMongo package - part 001.

MongoDB and PyMongo are not my priorities for the new year 2020 but because they are quite evolved I thought to start presenting it within my free time.
The PyMongo python package is a Python distribution containing tools for working with MongoDB.
The full documentation can be found on this webpage.
You can see my tutorial about how to install the MongoDB into Fedora 31 on this webpage.
I used that webpage install to test this python package, see the result:
[mythcat@desk mongo_test]$ mongo
...
MongoDB server version: 4.2.2-rc1
> use admin 
switched to db admin
> show dbs
> db.auth('admin', 'admin')
1
> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB
Let's install it with pip3 tool:
[mythcat@desk projects]$ pip3 install pymongo --user
Collecting pymongo
...
Successfully installed pymongo-3.10.0
Let's start with a simple example:
[mythcat@desk projects]$ mkdir mongo_test
[mythcat@desk projects]$ cd mongo_test/
[mythcat@desk mongo_test]$ vim mongo001.py
If you have already created the admin user, to run the next spython script you need to change the role like this:
> use admin;
switched to db admin
> db.grantRolesToUser('admin', [{ role: 'root', db: 'admin' }])
The script show you how to use a simple connection to the MongoDB:
import pymongo
from pymongo import MongoClient, errors

MONGO_URI = 'mongodb://admin:admin767779@127.0.0.1:27017/admin'
client = pymongo.MongoClient(MONGO_URI)
print("Server info : ")
print(client.server_info())
print("Databases : " + str(client.list_database_names()))
print("Connect to : admin database!")
db = client['admin']
db2 = client.config
print(db)
print(db2)
print("Collection admin : ")
collection = db['admin']
collection2 = db.config
print(collection)
print(collection2)
print("try call find_one method")
try:
    one_doc= collection2.find_one()
    print ("find_one():", one_doc)
except errors.ServerSelectionTimeoutError as err:
    print ("find_one() ERROR:", err)

print("Client close!")
client.close()
The output is this:
[mythcat@desk mongo_test]$ python3 mongo001.py 
Server info : 
{'version': '4.2.2-rc1', 'gitVersion': 'a0bbbff6ada159e19298d37946ac8dc4b497eadf', 'modules': [],
 'allocator': 'tcmalloc', 'javascriptEngine': 'mozjs', 'sysInfo': 'deprecated', 'versionArray': [4, 2, 2, -49],
 'openssl': {'running': 'OpenSSL 1.1.1d FIPS  10 Sep 2019', 'compiled': 'OpenSSL 1.1.1 FIPS  11 Sep 2018'},
 'buildEnvironment': {'distmod': 'rhel80', 'distarch': 'x86_64', 'cc': '/opt/mongodbtoolchain/v3/bin/gcc:
 gcc (GCC) 8.2.0', 'ccflags': '-fno-omit-frame-pointer -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare
 -Wno-unknown-pragmas -Winvalid-pch -Werror -O2 -Wno-unused-local-typedefs -Wno-unused-function
 -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable
 -Wno-missing-braces -fstack-protector-strong -fno-builtin-memcmp', 'cxx': '/opt/mongodbtoolchain/v3/bin/g++:
 g++ (GCC) 8.2.0', 'cxxflags': '-Woverloaded-virtual -Wno-maybe-uninitialized -fsized-deallocation -std=c++17',
 'linkflags': '-pthread -Wl,-z,now -rdynamic -Wl,--fatal-warnings -fstack-protector-strong -fuse-ld=gold -Wl,
--build-id -Wl,--hash-style=gnu -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro', 'target_arch':
 'x86_64', 'target_os': 'linux'}, 'bits': 64, 'debug': False, 'maxBsonObjectSize': 16777216, 'storageEngines':
 ['biggie', 'devnull', 'ephemeralForTest', 'wiredTiger'], 'ok': 1.0}
Databases : ['admin', 'config', 'local']
Connect to : admin database!
Database(MongoClient(host=['127.0.0.1:27017'], document_class=dict, tz_aware=False, connect=True), 'admin')
Database(MongoClient(host=['127.0.0.1:27017'], document_class=dict, tz_aware=False, connect=True), 'config')
Collection admin : 
Collection(Database(MongoClient(host=['127.0.0.1:27017'], document_class=dict, tz_aware=False, connect=True),
 'admin'), 'admin')
Collection(Database(MongoClient(host=['127.0.0.1:27017'], document_class=dict, tz_aware=False, connect=True),
 'admin'), 'config')
try call find_one method
find_one(): None
Client close!


Thursday, January 2, 2020

Python 3.7.5 : Testing the Falcon framework - part 001.

I start the new year with this python framework named Falcon.
The Falcon is a low-level, high-performance Python framework for building HTTP APIs, app backends, and higher-level frameworks.
The main reason was the speed of this python framework, see this article about falcon benchmark.
You can see is more faster like Flask and Django.
The instalation is easy with pip tool, you can read also the documenation webpage:
[mythcat@desk projects]$ mkdir falcon_test
[mythcat@desk projects]$ cd falcon_test/
[mythcat@desk falcon_test]$ pip3 install falcon --user
Collecting falcon
...
Successfully installed falcon-2.0.0
Falcon also fully supports CPython 2.7 and 3.5+.
If you want to install the latest beta or release candidate use:
[mythcat@desk falcon_test]$ pip3 install --pre falcon --user
The Falcon framework is easy to use.
First, I created a folder named test001 for my falcon application script named app.py:
[mythcat@desk falcon_test]$ mkdir test001
[mythcat@desk falcon_test]$ cd test001/
[mythcat@desk test001]$ vim app.py
In this python script I used python packages json and falcon with a class named req_class:
import json
import falcon

class req_class:
    def on_get(self,req,resp):
        print("on_get class")
my_falcon_api = falcon.API()
my_falcon_api.add_route('/test',req_class())
The url route is set to test.
To test the falcon framework we need the Gunicorn python package.
The Gunicorn is working on my Fedora Linux distro and I'm not sure if working Gunicorn on Windows.
You can try on Windows O.S. the waitress python package.
[mythcat@desk falcon_test]$ pip3 install gunicorn --user
Collecting gunicorn
...
Successfully installed gunicorn-20.0.4
To test this simple Falcon application use this command line where the app python script and my falcon variable A.P.I. named my_falcon_api is used.
[mythcat@desk test001]$ gunicorn app:my_falcon_api
[2020-01-02 18:57:48 +0200] [4401] [INFO] Starting gunicorn 20.0.4
[2020-01-02 18:57:48 +0200] [4401] [INFO] Listening at: http://127.0.0.1:8000 (4401)
[2020-01-02 18:57:48 +0200] [4401] [INFO] Using worker: sync
[2020-01-02 18:57:48 +0200] [4404] [INFO] Booting worker with pid: 4404
on_get class
on_get class
Open in the browser this URL with the route I set: http://127.0.0.1:8000/test.
You will don't see anything in the browser but will see the python output of the print function for my request.

Tuesday, December 31, 2019

News : The Python 2.7 no longer support from Python team.

The 1st of January 2020 will mark the sunset of Python 2.7.
It’s clear that Python 3 is more popular these days.
You can learn more about the popularity of both on Google Trends.
Python 3.0 was released in December 2008.
The main goal was to fix problems existing in Python 2.
Since the 1st January 2020, Python 2 will no longer receive any support whatsoever from the core Python team.
Migrating to Python 3 is recommended, including some of the top libraries, such as machine learning.

Saturday, December 28, 2019

Python 3.7.5 : Fix to python language the GitHub project.

I created a GitHub project with Django and I saw is detect like tcl programming language:

You need to create a file named .gitattributes in the root folder of my repository.
Use this source code to tell GitHub is a python project:
* linguist-vendored
*.py linguist-vendored=false
Now the project will be target with python language.

Tuesday, December 24, 2019

Python 3.7.5 : Is Django the best web framework?

This is the question for today in order to lineup the Django features with any web framework from my point of view.
Let's start with a brief introduction to this framework:
Django was created in the fall of 2003, when the web programmers at the Lawrence Journal-World newspaper, Adrian Holovaty and Simon Willison, began using Python to build applications. Jacob Kaplan-Moss was hired early in Django’s development shortly before Simon Willison's internship ended.[16] It was released publicly under a BSD license in July 2005. The framework was named after guitarist Django Reinhardt.[17], see wikipedia.
The Python which is a high-level programming language interpreted with general-purpose and together with the Django web framework creates a solution in fulfilling the objectives of web programming.
The problem of most of those who do not know closely the programming of this framework remains unknown and can be difficult to understand only from examples and tutorials.
Here are some of the difficulties that I personally encountered and had to solve them step by step.
  1. understand how to use the link system between the python files created by Django;
  2. how to use the templates and how to update them with the Django specific syntax;
  3. understanding the way of displaying and resolving specific errors in Django;
  4. using and setting the settings.py file;
  5. understanding of how the web framework interacts with web technologies;
After solving these problems you will see the true power of this framework:
  1. the development is easier with good and lower development costs and so are the additions and upgrades;
  2. security is very good, see security documentation and deployment checklist;
  3. is an open-source framework and updated by developers who use it;
  4. most used and crowd tested and used to develop DropBox, Quora, Google, and Reddit;
  5. comes with extensive documentation;
  6. a large and community;
The official page of this web framework can be found on this webpage.

Monday, December 23, 2019

Python 3.7.5 : About Django REST framework.

First, let's activate my Python virtual environment:
[mythcat@desk django]$ source env/bin/activate
I update my django version 3.0 up to 3.0.1.
(env) [mythcat@desk django]$ pip3 install --upgrade django --user
Collecting django
...
      Successfully uninstalled Django-3.0
Successfully installed django-3.0.1
The next step comes with installation of Python modules for Django and Django REST:
(env) [mythcat@desk django]$ pip3 install djangorestframework --user
Collecting djangorestframework
...
Installing collected packages: djangorestframework
Successfully installed djangorestframework-3.11.0
Into my folder mysite I run this commands
(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ python3 manage.py makemigrations
No changes detected
(env) [mythcat@desk mysite]$ python3 manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions, test001
Running migrations:
  No migrations to apply.
To pass information over to an HTTP GET request, the information object must be translated into valid response data.
The Django implements serializers for this.
Serializers provide deserialization, allowing parsed data to be converted back into complex types and allow complex data such as querysets and model instances to be converted to native Python datatypes that can then be easily rendered into JSON, XML or other content types.
Let's create the mysite/serializers.py:
(env) [mythcat@desk mysite]$ cd mysite/
(env) [mythcat@desk mysite]$ vim serializers.py 
The code for this python script is this:
from django.contrib.auth.models import User, Group
from rest_framework import serializers

class UserSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = User
        fields = ['url', 'username', 'email', 'groups']

class GroupSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = Group
        fields = ['url', 'name']
The next changes will be on urls.py and views.py.
My urls.py file from the Django-chart project is this:
from django.contrib import admin
from django.urls import path
from test001.views import home_page
from test001.views import Test001ChartView
#
from django.urls import include, path
from rest_framework import routers
from test001 import views

router = routers.DefaultRouter()
router.register(r'users', views.UserViewSet)
router.register(r'groups', views.GroupViewSet)

app_name = 'test001'
urlpatterns = [
    path('admin/', admin.site.urls),
    #path('', home_page, name ='home'),
    path('', Test001ChartView.as_view(), name = 'home'), 
    # Use automatic URL routing
    # Can also include login URLs for the browsable API
    path('', include(router.urls)),
    path('api-auth/', include('rest_framework.urls', namespace='rest_framework'))
]
My views.py file is this:
from django.http import HttpResponse
from django.shortcuts import render
# snippet 
from django.shortcuts import get_object_or_404
# for chart 
from django.views.generic import TemplateView
from .models import Test001, Snippet
#
#def home_page(request):
#    return HttpResponse('Home page!')

# django framework
from django.contrib.auth.models import User, Group
from rest_framework import viewsets
from mysite.serializers import UserSerializer, GroupSerializer

def home_page(request):
    return render(request, 'test001/home.html',{
        'name':'CGF',
        'html_items': ['a','b','c','d','e']
    })


# define view for chart 
class Test001ChartView(TemplateView):
    template_name = 'test001/chart.html'

    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        context["qs"] = Test001.objects.all()
        return context


def snippet_detail(request, id):
    snippet = get_object_or_404(Snippet, id=id)
    return render(request, 'test001/snippets_detail.html', {'snippet': snippet})

class UserViewSet(viewsets.ModelViewSet):
    """
    API endpoint  allows users to be viewed or edited.
    """
    queryset = User.objects.all().order_by('-date_joined')
    serializer_class = UserSerializer

class GroupViewSet(viewsets.ModelViewSet):
    """
    API endpoint  allows groups to be viewed or edited.
    """
    queryset = Group.objects.all()
    serializer_class = GroupSerializer
The settings module for this my project is stored in mysite/settings.py and I add this:
INSTALLED_APPS = [
    ...
    'rest_framework',
]
I run the django project and works well:
python3 manage.py runserver
The http://127.0.0.1:8000/users/ page come with this output:


Sunday, December 22, 2019

Python 3.7.5 : The new PyQt5 released.

The PyQt v5.14.0 has been released with support for Qt v5.14.0.
[mythcat@desk ~]$ pip3 install --upgrade PyQt5 --user
Collecting PyQt5
...
Installing collected packages: PyQt5-sip, PyQt5
Successfully installed PyQt5-5.14.0 PyQt5-sip-12.7.0
Let's see how can see the version:
[mythcat@desk ~]$ python3 
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5.Qt import PYQT_VERSION_STR
>>> print("PyQt version:", PYQT_VERSION_STR)
PyQt version: 5.14.0


Saturday, December 21, 2019

Python 3.7.5 : Simple web search with google python package.

This is a simple search on the web with python google package.
[mythcat@desk ~]$ pip3 install google --user
Collecting google
...
Installing collected packages: google
Successfully installed google-2.0.3
This is a simple example for search on web with this words: protv news 2019.
From the python package, I need to import just the search and used it.
The python package need a variable string named query.
The search wants to know the words from query and arguments, see the help:
Help on function search in module googlesearch:

search(query, tld='com', lang='en', tbs='0', safe='off', num=10, start=0, stop=None, 
domains=None, pause=2.0, tpe='', country='', extra_params=None, user_agent=None)
    Search the given query string using Google.
    
    :param str query: Query string. Must NOT be url-encoded.
    :param str tld: Top level domain.
    :param str lang: Language.
    :param str tbs: Time limits (i.e "qdr:h" => last hour,
        "qdr:d" => last 24 hours, "qdr:m" => last month).
    :param str safe: Safe search.
    :param int num: Number of results per page.
    :param int start: First result to retrieve.
    :param int stop: Last result to retrieve.
        Use None to keep searching forever.
    :param list domains: A list of web domains to constrain
        the search.
    :param float pause: Lapse to wait between HTTP requests.
        A lapse too long will make the search slow, but a lapse too short may
        cause Google to block your IP. Your mileage may vary!
    :param str tpe: Search type (images, videos, news, shopping, books, apps)
        Use the following values {videos: 'vid', images: 'isch',
        news: 'nws', shopping: 'shop', books: 'bks', applications: 'app'}
    :param str country: Country or region to focus the search on. Similar to
        changing the TLD, but does not yield exactly the same results.
        Only Google knows why...
    :param dict extra_params: A dictionary of extra HTTP GET
        parameters, which must be URL encoded. For example if you don't want
        Google to filter similar results you can set the extra_params to
        {'filter': '0'} which will append '&filter=0' to every query.
    :param str user_agent: User agent for the HTTP requests.
        Use None for the default.
    
    :rtype: generator of str
    :return: Generator (iterator) that yields found URLs.
        If the stop parameter is None the iterator will loop forever.
This is the script:
[mythcat@desk ~]$ python3
Python 3.7.5 (default, Dec 15 2019, 17:54:26) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from googlesearch import search
>>> query = "protv news 2019"
>>> my_results_list = []
>>> for url in search(query,        
...     tld = 'com',  
...     lang = 'en',       
...     num = 10,     
...     start = 0,    
...     stop = None,  
...     pause = 2.0,):
...     my_results_list.append(url)
...     print(url)
... 
https://stirileprotv.ro/protvnews/
https://stirileprotv.ro/
https://stirileprotv.ro/superbun/protv-news.html
https://www.facebook.com/ProTvNews/
https://www.youtube.com/playlist?list=PLCJaU-QvLGR_FSZw6yeqBJHDe9LgFDRdy
https://www.youtube.com/watch?v=HaiQtDlaNic
https://www.youtube.com/watch?v=hxMEgAANSl4
https://www.youtube.com/channel/UCbbDChpDluLkdnH8QMwN6qA
https://www.youtube.com/watch?v=5zuN9uWFcTE
https://protvplus.ro/tv-live/1-pro-tv
https://en.wikipedia.org/wiki/Pro_TV
https://pro-tv.com/news/page/2/
https://m.youtube.cat/channel/UCbbDChpDluLkdnH8QMwN6qA
...

Sunday, December 15, 2019

Python 3.7.5 : Simple intro in CSRF.

CSRF or Cross-Site Request Forgery is a technique used by cyber-criminals to force users into executing unwanted actions on a web application.
To protect against web form CSRF attacks, it's isn't sufficient for web applications to trust authenticated users, must be equipped with a unique identifier called a CSRF token similar to a session identifier.
Django 3.0 can be used with CSRF, see the documentation page.
The CSRF process is a simple one in Django framework.
However, it's highly recommended to use the CsrfViewMiddleware instead.
To activate the django.middleware.csrf.CsrfViewMiddleware in the settings.py file.
Basically, you can use the decorator method in the view.py file.
For example, see documentation:
@csrf_protect
However csrf_protect will check only POST requests.
This annotation CSRF tells Django I'm not handling CSRF properly and don't fail this.
If I don't use this annotation then the CSRF is on if the post request is on by default and I will receive a 403 error.
As the developer using templates you don't have to know anything about that and you can use this in your template HTML5 file.
{% csrf_token %}
I can use the token with my code if I want to send it to the website.
from django.middleware.csrf import get_token
Then this can be used in view.py with a hidden form field with the name csrfmiddlewaretoken present in all outgoing POST forms.
You can used in any forms tag area like: input ...

def my_csrf_form(request):
    response = """ ... type = "hidden" name = "csrfmiddlewaretoken" value = "__token__" ... """
This can be used with:
token = get_token(request)
response = response.replace('__token__', html.escape(token))
response += dumpdata('POST', request.POST)
return HttpResponse(response)
This will get a token for the current request and fill into value = "__token__".
Using my old example from GitHub.
First, I start the project:

[mythcat@desk projects]$ source django/env/bin/activate
(env) [mythcat@desk projects]$ ls
cache  django  kaggle  logs  OSMnx  pygal_ex  SantaClaus.py
(env) [mythcat@desk projects]$ cd django/
(env) [mythcat@desk django]$ ls
env  mysite  venv
(env) [mythcat@desk django]$ cd mysite/
I change the chart.html file with this before javascript script:
... {% csrf_token %} ... 
You can use it in javascript - jQuery like this, if you want:
var csrftoken = jQuery("[name=csrfmiddlewaretoken]").val();
The CSRF protection on Django on my form can bu used in view.py:
...
# use csrf_protect
from django.views.decorators.csrf import csrf_protect
from django.utils.decorators import method_decorator
...
# define view for chart 
class Test001ChartView(TemplateView):
    csrf_protected_method = method_decorator(csrf_protect)
I test it and works well.
The documentation tells us:
The CSRF protection cannot protect against man-in-the-middle attacks, so use HTTPS with HTTP Strict Transport Security. It also assumes validation of the HOST header and that there aren’t any cross-site scripting vulnerabilities on your site (because XSS vulnerabilities already let an attacker do anything a CSRF vulnerability allows and much worse).

Saturday, December 14, 2019

Python 3.7.5 : Django admin shell by Grzegorz Tężycki.

Today I tested another python package for Django named django-admin-shell.
This package created by Grzegorz Tężycki can be found on GitHub and come with the intro:
Django application can execute python code in your project’s environment on django admin site. You can use similar as python manage shell without reloading the environment.
[mythcat@desk ~]$ cd projects/
[mythcat@desk projects]$ cd django/
...
[mythcat@desk projects]$ source django/env/bin/activate
(env) [mythcat@desk projects]$ 

(env) [mythcat@desk projects]$ pip3 install django-admin-shell --user
Collecting django-admin-shell
...
Installing collected packages: django-admin-shell
Successfully installed django-admin-shell-0.1 
First, I test my project to see if this working.
(env) [mythcat@desk projects]$ cd django/
(env) [mythcat@desk django]$ ls
env  mysite  venv
(env) [mythcat@desk django]$ cd mysite/
(env) [mythcat@desk mysite]$ ls
db.sqlite3  manage.py  mysite  test001
(env) [mythcat@desk mysite]$ python3 manage.py runserver
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
December 14, 2019 - 18:23:47
Django version 3.0, using settings 'mysite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C. 
You need to add this package into the settings.py file:
INSTALLED_APPS = [
    ...
    'django_admin_shell',
    ...
]
I add this package into my url.py file:
...
from django.conf.urls import include, url
...
urlpatterns = [
...
    # admin shell 
    url(r'^admin/shell/', include('django_admin_shell.urls')),
]
After that, you can start the project and see if this working: http://127.0.0.1:8000/admin/shell/
This screenshot shows me is working well: