Python 3.7.3 : Tonny I.D.E. for python programmers.

Today I tested the Thonny I.D.E. from thonny.org official webpage.
Yesterday I tried several editors for python programming language and did not work.
One of these is the spyder editor that does not work with python 3.7.3 - we have not discovered why.
The Mu is a simple Python editor for beginner programmers and has a strange working I.D.E. for good and fast development.
The PyCharm, it is developed by the Czech company JetBrains is not fully free ...

---

The Thonny I.D.E. is fast and comes with two good features:
Code completion and Highlights syntax errors.
The install steps are easy with a wizard application.
After that, you can test with any python source code.
The source code from the image is a simple example from this webpage.
Let's see a screenshot:

This is a video tutorial with this I.D.E. from Aivar Annamaa user:

Python 3.7.3 : Using the flask - part 002.

Let's see some tips for starting any project with flask python module.
Use these python modules to work with databases: flask-sqlalchemy and flask_marshmallow.
The Flask-SQLAlchemy is an extension for Flask that adds support for SQLAlchemy to your application.
The marshmallow is an ORM/ODM/framework-agnostic library for converting complex datatypes, such as objects, to and from native Python datatypes.
Let's install it:

C:\Python373>cd Scripts

C:\Python373\Scripts>pip install flask-sqlalchemy
Collecting flask_sqlalchemy
...
Successfully installed flask-sqlalchemy-2.4.0
C:\Python373\Scripts>pip install flask_marshmallow
Collecting flask_marshmallow
...
Successfully installed flask-marshmallow-0.10.1 marshmallow-2.19.5
C:\Python373\Scripts>pip install marshmallow-sqlalchemy
Collecting marshmallow-sqlalchemy
...
Successfully installed marshmallow-sqlalchemy-0.17.0

Use these python modules for login and authorization issues: flask-login
The Flask-Login provides user session management for Flask. It handles the common tasks of logging in, logging out, and remembering your users’ sessions over extended periods of time.
Flask-login doesn't actually have a user backend, it just handles the session machinery to help you to log in and logout users.
Flask-Login can work with user models that are based on any database system with four required items: is_authenticated, is_active, is_anonymous and get_id.
Let's install it:

C:\Python373\Scripts>pip install flask-login
Collecting flask-login
...
Successfully installed flask-login-0.4.1

This python module named python-dotenv let us to reads the key-value pair from .env file and adds them to an environment variable.

C:\Python373\Scripts>pip install python-dotenv
Collecting python-dotenv
...
Successfully installed python-dotenv-0.10.3

Flask-Login provides a UserMixin which makes it easy to create a user class, see example class User:

from flask_login import UserMixin
...
class User(UserMixin, db.Model):

Another tip is to use the get_or_404 and first_or_404 to avoid show errors on the webpage.
Reduce the passing of multiple variables to render_template, let's see the example:

from flask import render_template
...
return render_template('index.html', fn=name.first, ln=name.last, day=student.day, 
.... ')

This can de easy fix with this code to send value I want to pass:

context = {
'fn': name.first,
'ln': name.last,
'day': student.day,
...
}
return render_template('index.html', **context)

Use query to solve complex tasks.
Let's see a simple example of delete from a Student database class:

...
class Student(db.Model):
id=db.Column(db.Integer, primary_key=True)
...
country=db.Column(db.String(2))
...

Let's delete this with a query and return the number of deleted by delete_count:

del_ids= db.session.query(Student.id).filter(Student.country=='Ro')
delete_count=db.session.query(Student).filter(Student.id.in_(del_ids.subquery())).delete(synchronize_session=False)

I used two files in the main project named .env and .flaskenv:
The file .env comes with:

SQLALCHEMY_DATABASE_URI=sqlite:///db.sqlite3
SECRET_KEY = abcdef

The file .flaskenv has these lines of source code:

FLASK_ENV = development
FLASK_APP = my_flask_app

This can be used into a new file named settings.py:

import os 
SQLALCHEMY_DATABASE_URI = os.environ.get('SQLALCHEMY_DATABASE_URI')
SECRET_KEY = os.environ.get('SECRET_KEY')
SQLALCHEMY_TRACK_MODIFICATIONS = False

Using this the main flask application will start like this:

from flask import Flask
def create_my_app(config_file='settings.py'):
    app = Flask(__name__)
    app.config.from_pyfile(config_file)
    return app

There are plenty of tips and tricks that can be found based on the project in the bottle.
These are the most used.

Python 3.7.3 : Testing the timeit and Bokeh python module.

The tutorial today has several goals:

• testing the timeit function to measure the execution time of a line of print code;
• using python lists to record data;
• generate lists using the range function;
• multiplying a number with an entire list of numbers;
• how to use the bokeh module with CustomJSHover and HoverTool

Let's test with the timeit python module, see the official webpage.
This module provides a simple way to time small bits of Python code. It has both a Command-Line Interface as well as a callable one. It avoids a number of common traps for measuring execution times.

C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Inte
l)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import timeit
>>> import random
>>> timeit.timeit('print("a")',number=1)
a
0.00013114200010022614
>>> timeit.timeit('print("a")',number=100)
...
a
0.007177434000027461
>>> timeit.timeit('print("a")',number=1000)
...
a
0.07585798000002342

Let make more visible the results with the bokeh python module:

import timeit
import random

# if you want to use numpy 
#import numpy as np

# import bokeh python module 
from bokeh.plotting import figure, show, output_file
# for show values
from bokeh.models.tools import CustomJSHover
from bokeh.models import HoverTool
print ('''
You can tests a number of times named l
with a number of print named i
and see the result value with bokeh python module
timeit.timeit('print("a")',number = i)
''')
i = int(input('The increment number i:'))
l = int(input('The number of test l:'))

# create the list with the output values
timit_list = []
# create the test function for l and i and return the list
def get_timeit(i,l):
 while i < l:
  i += 1
  out=timeit.timeit('print("a")',number = i)
  timit_list.append(out)
 return timit_list

# run the test function with l and i
# this will be coordinate y
yt=get_timeit(i,l)

# show result of the test 
print(yt)

# create the coordinate x
xt = [i for i in range(0, len(yt))]

#xt = np.linspace(0, l, l)
# print the coordinate x
#print(xt)

# create the output HTML file to see the result
output_file("test.html")

# create a figure with a timeit type y-axis
fig = figure(title='timit_list values for print("a")',
             plot_height=400, plot_width=700,
             x_axis_label='x has each incrementation of i', y_axis_label='value of timeit of print the char a',
             x_minor_ticks=3, 
             toolbar_location=None)

# create a circle for each value

# see new multiplication with a list 

# y=[i * 100 for i in yt]
fig.circle(x=xt, y=[i * 100 for i in yt], 
         color='blue', size=5,
         legend='Values')

x_custom = CustomJSHover(code="""
    return '' + special_vars.data_x""")

y_custom = CustomJSHover(code="""
    return '' + special_vars.data_y""")

fig.add_tools(
    HoverTool(
        show_arrow=True, 
        tooltips=[
 ('xt', '$data_x'),
 ('yt', '$data_y')
        ],
        formatters=dict(
            xt=x_custom,
            yt=y_custom
        )
    )
)

# Put the legend in the upper left corner
fig.legend.location = 'top_left'

# Let's check it out
show(fig)

The result of this python source code can be found on my YouTube channel:

Python 3.7.3 : Using the flask - part 001.

A short intro into this python module can be found at the PyPI website:
Flask is a lightweight WSGI web application framework. It is designed to make getting started quick and easy, with the ability to scale up to complex applications. It began as a simple wrapper around Werkzeug and Jinja and has become one of the most popular Python web application frameworks.

Flask offers suggestions but doesn’t enforce any dependencies or project layout. It is up to the developer to choose the tools and libraries they want to use. There are many extensions provided by the community that makes adding new functionality easy.
The reason I used a series of tutorials with this python module is the complexity of the features of this python module.
Let's briefly outline some of the essential aspects of flask programming.

• Flask is a simple, lightweight, and minimalist web framework;
• Flask is developed based on the Jinja2 template engine;
• Flask depends on the Jinja template engine and the Werkzeug WSGI toolkit;
• Flask does not provide a built-in ORM system (Object Relation Mapping);
• in Flask web applications to perform CRUD operations on a database can be tedious (see: ORM techniques of Flask-SQLAlchemy);
• The Flask has a simple and customizable architecture;
• the Flask to accelerate the development of simple websites that use static content;
• has the option to extend and customize Flask according to precise project requirements;

A list of companies using the Flask framework - who is using Flask?
The install process is very simple using the pip tool:

C:\Python373>cd Scripts

C:\Python373\Scripts>pip install flask
Collecting flask
...
Installing collected packages: itsdangerous, Werkzeug, flask
Successfully installed Werkzeug-0.15.4 flask-1.1.1 itsdangerous-1.1.0

Let's start initializing the first flask application into the server.py file:

from flask import Flask 
app = Flask (__name__)
# create a wrarp of localhost
@app.route('/')
def home():
 return 'Hello world'
# the default name main 
if __name__ == '__main__':
 app.run()

About Routing and Variable Rules.
This can change it into the server.py script like this:

@app.route('/about')
def about():
 return 'The about page'
@app.route('/blog')
def blog():
 return 'This is the blog'

The next step is the variable rule issue:

# creaza a regula variabila ( variable rules) 
# the type of the variable can be set, see int 
# @app.route('/blog/')
@app.route('/blog/')
def blogpost(blog_id):
 return 'This is the post '+str(blog_id)

To run it, just use:

C:\Python373\my_flask>python server.py
 * Serving Flask app "server" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

The full source code can be found at my GitHub account.

Python 3.7.3 : The sip python module.

The official webpage pypi.org comes with this intro:
One of the features of Python that makes it so powerful is the ability to take existing libraries, written in C or C++, and make them available as Python extension modules. Such extension modules are often called bindings for the library.

SIP is a tool that makes it very easy to create Python bindings for C and C++ libraries. It was originally developed to create PyQt, the Python bindings for the Qt toolkit, but can be used to create bindings for any C or C++ library.

SIP comprises a code generator and a Python module. The code generator processes a set of specification files and generates C or C++ code which is then compiled to create the bindings extension module. The sip Python module provides support functions to the automatically generated code.
The SIP is copyright (c) Riverbank Computing Limited and its homepage is this webpage.
Support may be obtained from the PyQt mailing list at here.
The SIP is a tool for quickly writing Python modules that interface with C++ and C libraries.
The SIP comprises a code generator and a Python module.
About the code generator:
First, the install:

C:\Python373\Scripts>pip3 install sip
Collecting sip
...
Installing collected packages: sip
Successfully installed sip-4.19.8

If you using the PyQt5 then this version includes a private copy of the module.

C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Inte
l)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5 import sip

If you want to see backward compatibility the module then needs to imported and will only work if another PyQt5 module is imported first.

C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Inte
l)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5 import QtCore
>>> import sip

A good example of how can you build with SIP can be seen here.
About the SIP Python module:
This provides support functions to the automatically generated code.
The import python module sip let you use all functions build by the code generator:

>>> dir(sip)
['SIP_VERSION', 'SIP_VERSION_STR', '_C_API', '__doc__', '__file__', '__loader__'
, '__name__', '__package__', '__spec__', '_unpickle_enum', '_unpickle_type', 'as
sign', 'cast', 'delete', 'dump', 'enableautoconversion', 'enableoverflowchecking
', 'getapi', 'isdeleted', 'ispycreated', 'ispyowned', 'setapi', 'setdeleted', 's
etdestroyonexit', 'settracemask', 'simplewrapper', 'transferback', 'transferto',
 'unwrapinstance', 'voidptr', 'wrapinstance', 'wrapper', 'wrappertype']

If you want to improve or create python modules with C or C ++, then this tool can help.

Python 3.7.3 : The IMDbPY python module version 6.8.

The GitHub official webpage comes with this intro:
IMDbPY is a Python package for retrieving and managing the data of the IMDb movie database about movies, people and companies.
The last release version 6.8 was at 2019 Jul 20.
The official webpage tells us:
In the release 6.8 (codename "Apollo 11") of IMDbPY, multiple parsers were added and fixes; the new search_movie_advanced method allows advanced movie searches...
The changes of the version 6.8 can be found at GitHub webpage and come with these new features:
#224: introduce the search_movie_advanced(title, adult=None, results=None, sort=None, sort_dir=None) method
#145: names are stored in normal format (Name Surname)
#225: remove the obsolete cookie
#182: box office information
#168: parse series and episode number searching for movies
#217: grab poster from search
#218: extract MPAA rating
#220: extract actor headshot from full credits
The install on Python 3.7.3 is easy with pip3 tool:

C:\Python373\Scripts>pip3 install imdbpy
Collecting imdbpy
...
Installing collected packages: imdbpy
Successfully installed imdbpy-6.8

Let's test the new features:

C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Inte
l)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import imdb
>>> ia = imdb.IMDb()
>>> movies = ia.search_movie_advanced('debby does dallas', adult=True)
>>> print(movies)
...
>>> people = ia.search_person('Clark Gregg')
>>> print(people)

Let's test it:

import imdb
from imdb import IMDb

ia = imdb.IMDb()

# create a file to put the output 
file1 = open("_imdb_data.txt","w", encoding='utf-8') 

# get movies by movie 
# example: Alien 

def get_by_movie():
 my_movie = str(input('Type the movie name: '))
 movies = ia.search_movie_advanced(my_movie, adult=True)
 print(type(movies))
 return movies

# get filmography by id 
filmography_list = []
def get_filmography_by_id(id):
 actor_results = ia.get_person_filmography(id)
 for item in actor_results['data']['filmography']:
  filmography_list.append(str(item))
 return filmography_list

# the main function 
def main():
 a = get_by_movie()
 for i in a:
  print("________________________")
  print("i: ",i)
  # you can uncomment this to test Movie class functions
  #print("Type:",type(i))
  #print("Summary:",i.summary())
  #print("ID: ",i.getID())
  #print("Smart cannonical title: ",i.smartCanonicalTitle())
  #print("caracters ref: ",i.get_charactersRefs())
  #print("current info: ",i.get_current_info())
  #print("cinematographic process: ",i.get('cinematographic process'))
  #print(i["title"])
  #print informations items from Movie class
  print("~~~~~~~~~~~~~~~~~~~~~~~~")
  for k, v in i.items():
   print(k, v)
   # write to the file the value of a
   txt = str(k)+":"+str(v)+"\n" 
   file1.write(txt)
  print("------------------------")
  # add a new line on each movie
  file1.write('-----^-----\n')
 #get filmography by id 
 id_filmography=get_filmography_by_id('0078748')
 # print the filmography
 for item in id_filmography:
  print(item)

 #after write, close the file 
 file1.close()

if __name__ == '__main__':
    main()

This is the first part of the output file named _imdb_data:

title:Alien
certificates:['R']
runtimes:['117']
genres:['Horror', 'Sci-Fi']
rating:8.5
votes:719508
metascore:89
gross:78900000
plot:After a space merchant vessel perceives an unknown transmission as a distress call, its landing on the source moon finds one of the crew attacked by a mysterious lifeform, and they soon realize that its life cycle has merely begun.
directors:[]
cast:[, , , ]
cover url:https://m.media-amazon.com/images/M/MV5BMmQ2MmU3NzktZjAxOC00ZDZhLTk4YzEtMDMyMzcxY2IwMDAyXkEyXkFqcGdeQXVyNzkwMjQ5NzM@._V1_UX67_CR0,0,67,98_AL_.jpg
year:1979
kind:movie
canonical title:Alien
long imdb title:Alien (1979)
long imdb canonical title:Alien (1979)
smart canonical title:Alien
smart long imdb canonical title:Alien (1979)
full-size cover url:https://m.media-amazon.com/images/M/MV5BMmQ2MmU3NzktZjAxOC00ZDZhLTk4YzEtMDMyMzcxY2IwMDAyXkEyXkFqcGdeQXVyNzkwMjQ5NzM@.jpg
-----^-----
title:Aliens
certificates:['R']
runtimes:['137']
...

Python 3.7.3 : Use BeautifulSoup to parse Instagram account.

This example is a bit more complex because it parses the source code in a more particular way depending on it.
The basic idea of this script is to take the content of an Instagram account in the same way as a web browser.
For my account I found a parsing error, I guess the reason is using the points, see festila.george.catalin.

    scripts_content = json.loads(scripts[0].text.strip())
IndexError: list index out of range

In this case comment this line of code and will work:
For the other accounts I've tried, it works very well with the default script.
This is the script I used:

import requests
from bs4 import BeautifulSoup
import json
import re

from pprint import pprint

instagram_url = 'https://instagram.com'
#example user instagram profile_url = sherwoodseries
profile_url=str(input("name of the instagram user: "))


#UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f48b' in posit ion 5022: character maps to 
#fix write text file with  encoding='utf-8'
file1 = open("_shared_data.txt","w", encoding='utf-8') 

#profile_url = 'festila.george.catalin'
headers = {'User-Agent': 'Mozilla/5.0'}
response = requests.get(f"{instagram_url}/{profile_url}", headers = headers)

if response.ok:
    html = response.text
    bs_html = BeautifulSoup(html, "html.parser")
    print(bs_html)
    # get info from ... type="application/ld+json">{"@context":"http:\/\/schema.org","@type":"Person","name":
    scripts = bs_html.select('script[type="application/ld+json"]')
    #scripts_content = json.loads(scripts[0].text.strip())
    #pprint(scripts_content)

    #print scripts_content like json 
    #print(json.dumps(scripts_content,indent = 4,sort_keys = True))

    #print just part of source code get by 'script' (0 .. n), see n = 6 
    #print(bs_html.find_all('script')[6])
    script_tag = bs_html.find('script', text=re.compile('window\._sharedData'))
    shared_data = script_tag.string.partition('=')[-1].strip(' ;')

    #get item from shared data, see "language_code":"en"
    rex_item  = re.compile('(?<=\"language_code\":\")[a-zA-Z_\- ]+(?=\")')
    rex_get_item = rex_item.findall(shared_data)  
    print(rex_get_item)
    #get url image from shared data
    rex_url  = re.compile('(?<=\"display_url\":\")[^\s\"]+(?=\")')
    rex_get_url = rex_url.findall(shared_data)  
    print(rex_get_url)
 
    # load like a json 
    result_json = json.loads(shared_data)
    pprint(result_json)
    
    data = bs_html.find_all('meta', attrs={'property': 'og:description'})
    bb = data[0].get('content').split()
    user = '%s %s %s' % (bb[-3], bb[-2], bb[-1])
    # get from bb parts 
    posts = bb[4]
    print('all string: ',bb)
    print('number of posts: ',posts)
    print('name and the user: ',user)

    # write any output show by print into _a.txt file, see example
    #file1.write(str(bs_html.find_all('script')[4]))
    #example: write to _shared_data.txt file the shared_data
    #file1.write(str(shared_data))
#after write, close the file 
#file1.close() 

This is a part of the output for sherwoodseries account:

...
all string:  ['95', 'Followers,', '24', 'Following,', '56', 'Posts', '-', 'See',
 'Instagram', 'photos', 'and', 'videos', 'from', 'Sherwood', 'Series', '(@sherwo
odseries)']
number of posts:  56
name and the user:  Sherwood Series (@sherwoodseries)

Python 3.7.3 : The pandas python module.

Since I started learning python programming language I have not found a more complex and complete module for viewing complex data.
The official documentation of this python module tells us:
pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real-world data analysis in Python. Additionally, it has the broader goal of becoming the most powerful and flexible open-source data analysis/manipulation tool available in any language. It is already well on its way toward this goal.
The official webpage can be found here.
This python module is one of the most popular Python libraries for Data Science and Analytics.
You can install this python module with pip tool:

C:\Python373\Scripts>pip install pandas
Requirement already satisfied: pandas in c:\python373\lib\site-packages (0.24.2)

You can find many tutorials on web with this python module.
Today I will show you a short tutorial about this python module.
Most users use this both python modules:

import numpy as np
import pandas as pd

Most area of the pandas python module has a target into this list:
Window Functions, Aggregations, Missing Data, GroupBy, Merging/Joining, Concatenation, Date Functionality, Timedelta, Categorical Data,
Visualization, IO Tools, Sparse Data, Caveats & Gotchas, Comparison with SQL
There are two types of data structures in pandas: Series and DataFrames.
The pandas Series is a one-dimensional data structure.
The pandas DataFrame is a two (or more) dimensional data structure, like a table
Pandas provide few variants rolling, expanding and exponentially moving weights for window statistics.
Also have the sum, mean, median, variance, covariance, correlation, etc.
The several methods are available to perform aggregations on data.
Pandas provide functions for missing data like the isnull() and notnull().
Let's test the DataFrames with pandas and the Wikipedia example from my tutorial

C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Inte
l)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> # get table from wikipedia
... import requests
>>> from bs4 import BeautifulSoup
>>> website_url = requests.get('https://en.wikipedia.org/w/index.php?title=Table
_of_food_nutrients').text
>>> soup = BeautifulSoup(website_url,'lxml')
>>>
>>> my_table = soup.find('table',{'class':'wikitable collapsible collapsed'})
>>> links = my_table.findAll('a')
>>> Food = []
>>> for link in links:
...     Food.append(link.get('title'))
...
>>> print(Food)
["Cows' milk (page does not exist)", 'Buttermilk', 'Fortified milk (page does no
t exist)', 'Powdered milk', "Goats' milk", 'Malted milk', 'Hot chocolate', 'Yogu
rt', 'Milk pudding (page does not exist)', 'Custard', 'Ice cream', 'Ice milk', '
Cream', 'Cheese', 'Cheddar cheese', 'American cheese', 'Processed cheese', 'Egg
(food)', 'Scrambled', 'Omelet', 'Yolk']
>>> import pandas
>>> import pandas as pd
>>> df = pd.DataFrame()
>>> df['Foods'] = Food
>>> print(df)
                                   Foods
0       Cows' milk (page does not exist)
1                             Buttermilk
2   Fortified milk (page does not exist)
3                          Powdered milk
4                            Goats' milk
5                            Malted milk
6                          Hot chocolate
7                                 Yogurt
8     Milk pudding (page does not exist)
9                                Custard
10                             Ice cream
11                              Ice milk
12                                 Cream
13                                Cheese
14                        Cheddar cheese
15                       American cheese
16                      Processed cheese
17                            Egg (food)
18                             Scrambled
19                                Omelet
20                                  Yolk
>>> df.describe()
              Foods
count            21
unique           21
top     Malted milk
freq              1
>>> df.apply(pd.Series.value_counts)
                                      Foods
Malted milk                               1
Ice milk                                  1
Omelet                                    1
Goats' milk                               1
Custard                                   1
Cheddar cheese                            1
American cheese                           1
Ice cream                                 1
Yolk                                      1
Cream                                     1
Cows' milk (page does not exist)          1
Yogurt                                    1
Fortified milk (page does not exist)      1
Egg (food)                                1
Powdered milk                             1
Milk pudding (page does not exist)        1
Cheese                                    1
Hot chocolate                             1
Buttermilk                                1
Processed cheese                          1
Scrambled                                 1

The last example is to show data with

>>> import numpy as np
>>> import pandas as pd
>>> import matplotlib.pyplot as plt
>>> ts = pd.Series(np.random.randn(76), index=pd.date_range('1/1/76', periods=76
))
>>> ts.plot()

>>> plt.show()

Python 3.7.3 : Using the pipenv tool.

The goal of this tutorial is how to use pipenv and manage dependencies and development environments on a collaborative Python project.
The documentation area can be found here.

C:\Python373>cd Scripts

C:\Python373\Scripts>pip install --user pipenv
Collecting pipenv
...
Successfully installed pipenv-2018.11.26 virtualenv-clone-0.5.3

If you see errors like this:
WARNING: The scripts pipenv-resolver.exe and pipenv.exe are installed in 'C:\Users\....'

which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, 
use --no-warn-script-location.

You need to add this path show into the WARNING to the windows path, see command:

setx path "%path%;C:\Users\..\Python37\Scripts"

Let use this tool to install a requests package:

C:\Python373\Scripts>cd ..

C:\Python373>cd my_flask

C:\Python373\my_flask>pipenv install requests
Creating a virtualenv for this project.
Pipfile: C:\Python373\my_flask\Pipfile
Using c:\python373\python.exe (3.7.3) to create virtualenv.
[  ==] Creating virtual environment...Already using interpreter c:\python373\pyt
hon.exe
Using base prefix 'c:\\python373'
New python executable in C:\Users\catafest\.virtualenvs\my_flask-j9hGDZgP\Script
s\python.exe
Installing setuptools, pip, wheel...
done.
Successfully created virtual environment!
Virtualenv location: C:\Users\catafest\.virtualenvs\my_flask-j9hGDZgP
Creating a Pipfile for this project.
Installing requests.
Adding requests to Pipfile's [packages].
Installation Succeeded
Pipfile.lock not found, creating.
Locking [dev-packages] dependencies.
Locking [packages] dependencies.
Success!
Updated Pipfile.lock (444a6d)!
Installing dependencies from Pipfile.lock (444a6d).
  ================================ 5/5 - 00:00:04
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

The syntax for the Pipfile is TOML, and the file is separated into sections. [dev-packages] for development-only packages, [packages] for minimally required packages, and [requires] for other requirements like a specific version of Python
If you got this error you need restarts the command prompt:

'pipenv' is not recognized as an internal or external command,
operable program or batch file.

The next command let us see output diagnostic information for use in GitHub issues:

C:\Python373\my_flask>pipenv --support | clip
C:\Python373\my_flask>pipenv --support

Check for security vulnerabilities (and PEP 508 requirements) in your environment with:

C:\Python373\my_flask>pipenv check
Checking PEP 508 requirements.
Passed!
Checking installed package safety.
All good!

Let's use the example from the documentation webpage:

import requests

response = requests.get('https://httpbin.org/ip')

print('Your IP is {0}'.format(response.json()['origin']))

The result works great.

C:\Python373\my_flask>pipenv run python my_ip.py
Your IP is ...

You can print out a tree-like structure showing your dependencies.

C:\Python373\my_flask>pipenv graph
requests==2.22.0
  - certifi [required: >=2017.4.17, installed: 2019.6.16]
...

Using this command will see a reversed tree may be more useful when you are trying to figure out conflicting sub-dependencies.

C:\Python373\my_flask>pipenv graph --reverse
certifi==2019.6.16
  - requests==2.22.0 [requires: certifi>=2017.4.17]
chardet==3.0.4
...

This command installs all the dependencies needed for development:

C:\Python373\my_flask>pipenv install --dev
Installing dependencies from Pipfile.lock (444a6d).
  ================================ 5/5 - 00:00:10
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

You can uninstall a python module:

pipenv uninstall numpy

You can replace --all with --all-dev to just remove dev packages.
Let’s start a shell in a virtual environment to isolate the development of this app:

C:\Python373\my_flask>pipenv shell
Launching subshell in virtual environment.
Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

(my_flask-j9hGDZgP) C:\Python373\my_flask>

The package name, together with its version and a list of its own dependencies, can be frozen by updating the Pipfile.lock with the lock keyword.

(my_flask-j9hGDZgP) C:\Python373\my_flask>pipenv lock
Locking [dev-packages] dependencies.
Locking [packages] dependencies.
Success!
Updated Pipfile.lock (444a6d)!

Clears caches (pipenv, pip, and pip-tools) with:

pipenv --clear

Remove the virtualenv created:

(my_flask-j9hGDZgP) C:\Python373\my_flask>pipenv --rm

Close the pipenv shell:

(my_flask-j9hGDZgP) C:\Python373>exit

C:\Python373\my_flask>

This tool requires special attention in developing and programming with Python, so we have only traveled some of its real possibilities.

Python 3.7.3 : Using the werkzeug.

From the official webpage, you can see all the features of this python module:
Werkzeug is a comprehensive WSGI web application library. It began as a simple collection of various utilities for WSGI applications and has become one of the most advanced WSGI utility libraries.
It includes:
An interactive debugger that allows inspecting stack traces and source code in the browser with an interactive interpreter for any frame in the stack.
A full-featured request object with objects to interact with headers, query args, form data, files, and cookies.
A response object that can wrap other WSGI applications and handle streaming data.
A routing system for matching URLs to endpoints and generating URLs for endpoints, with an extensible system for capturing variables from URLs.
HTTP utilities to handle entity tags, cache control, dates, user agents, cookies, files, and more.
A threaded WSGI server for use while developing applications locally.
A test client for simulating HTTP requests during testing without requiring running a server.
The documentation of this python module can be found here.
You can install this python module with the pip tool:

pip install -U Werkzeug

This module can install it if you install the flask python module with python 3.7.3.
Let's test it:

C:\Python373>python
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from werkzeug.wrappers import Request, Response
>>>
>>> @Request.application
... def application(request):
...     return Response('Hello, World!')
...
>>> if __name__ == '__main__':
...     from werkzeug.serving import run_simple
...     run_simple('localhost', 4000, application)
...
 * Running on http://localhost:4000/ (Press CTRL+C to quit)
127.0.0.1 - - [16/Jul/2019 13:29:44] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [16/Jul/2019 13:29:46] "GET /favicon.ico HTTP/1.1" 200 -

If you run this source code and open your browser with this http://localhost:4000/ address then you will see a text with Hello, World!.
The common uses of this python package are:

>>> from werkzeug.wrappers import Request, Response
>>> from werkzeug.routing import Map, Rule
>>> from werkzeug.exceptions import HTTPException, NotFound
>>> from werkzeug.wsgi import SharedDataMiddleware
>>> from werkzeug.formparser import parse_form_data
>>> from werkzeug.utils import redirect
>>> from werkzeug.utils import escape
>>> from werkzeug.serving import run_simple

Let's test another simple application with werkzeug python module:

from werkzeug.serving import run_simple
from werkzeug.wrappers import Request
from werkzeug.wrappers import Response

class Application(object):
    def __init__(self):
        print('init Application class')

    def dispatch_request(self, request):
        return Response("Logged in as %s" % request.authorization)

    def __call__(self, environ, start_response):
        request = Request(environ)
        auth = request.authorization
        response = self.dispatch_request(request)
        return response(environ, start_response)

if __name__ == "__main__":
    application = Application()
    run_simple("localhost", 5000, application)

The result will be this:

C:\Python373>python.exe werkzeug_001.py
init Application class
 * Running on http://localhost:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [16/Jul/2019 14:05:31] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [16/Jul/2019 14:05:32] "GET /favicon.ico HTTP/1.1" 200 -

You can add more features to this source code and check with werkzeug features of Web Server Gateway Interface (WSGI).
A full list with examples can be found here.

Python 3.7.3 : Programming Krita.

Today I wrote a python tutorial about Krita software and programming python.
The Krita software use python version 3.6.2.

==== Warning: Script not saved! ====
3.6.2 (v3.6.2:5fd33b5, Jul  8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)]

The full tutorial can be found at my Blogspot (a Blogspot about the graphics area).

Python 3.7.3 : Simple tests with zipfile python module.

You can read about this python module here.
The ZIP file format is a common archive and compression standard. This module provides tools to create, read, write, append, and list a ZIP file. Any advanced use of this module will require an understanding of the format, as defined in PKZIP Application Note.

This module does not currently handle multi-disk ZIP files. It can handle ZIP files that use the ZIP64 extensions (that is ZIP files that are more than 4 GiB in size). It supports decryption of encrypted files in ZIP archives, but it currently cannot create an encrypted file. Decryption is extremely slow as it is implemented in native Python rather than C.

C:\Python373> python -m zipfile -c test.zip test.html  textalongpath.pdf
C:\Python373> python -m zipfile -c test_folder.zip test.html  temp
C:\Python373>python -m zipfile -l  test.zip
File Name                                             Modified             Size
test.html                                      2019-07-11 10:46:58         6115
textalongpath.pdf                              2019-06-08 22:55:50           84

C:\Python373>python -m zipfile -l  test_folder.zip
File Name                                             Modified             Size
test.html                                      2019-07-11 10:46:58         6115
temp/                                          2019-07-07 21:36:42            0
temp/wlop/                                     2019-07-07 21:36:42            0

This lines of code will create two archives named test.zip and test_folder.zip with the files shown on each command.
For extraction, is need to use the -e argument:

C:\Python373>python -m zipfile -e test.zip zipfiles/

C:\Python373>python -m zipfile -e test_folder.zip zipfiles/

C:\Python373>cd zipfiles

C:\Python373\zipfiles>dir
...

Let's using this python module inside python:

C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Inte
l)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import zipfile
>>> import datetime
>>> my_zip = zipfile.ZipFile('test.zip','r')
>>> print (my_zip.namelist())
['test.html', 'textalongpath.pdf']
>>> def print_info(archive_name):
...     my_zip = zipfile.ZipFile(archive_name)
...     for info in my_zip.infolist():
...             print (info.filename)
...             print ('Comment: ', info.comment)
...             print ('Modified: ', datetime.datetime(*info.date_time))
...             print ('System: ', info.create_system, '(0 = Windows, 3 = Unix)'
)
...             print ('ZIP version: ', info.create_version)
...             print ('Compressed: ', info.compress_size, 'bytes')
...             print ('Uncompressed: ', info.file_size, 'bytes')
...
>>> print_info('test_folder.zip')
test.html
Comment:  b''
Modified:  2019-07-11 10:46:58
System:  0 (0 = Windows, 3 = Unix)
ZIP version:  20
Compressed:  1679 bytes
Uncompressed:  6115 bytes
temp/
Comment:  b''
Modified:  2019-07-07 21:36:42
...

Extract all files from an archive:

>>> from zipfile import ZipFile
>>> with ZipFile('test.zip','r') as zipObj:
...     zipObj.extractall()

Extract files by extension:

>>> with ZipFile('test.zip', 'r') as zipObj:
...    listOfFileNames = zipObj.namelist()
...    for fileName in listOfFileNames:
...        if fileName.endswith('.html'):
...            zipObj.extract(fileName, 'new.html')
...
'new.html\\test.html'

Create a new arhive named The_new.zip and add the new.html file on it.

>>> zipObj = ZipFile('The_new.zip','w')
>>> zipObj.write('new.html')
>>> zipObj.close()
>>> print_info('The_new.zip')
new.html/
Comment:  b''
Modified:  2019-07-14 22:15:58
System:  0 (0 = Windows, 3 = Unix)
ZIP version:  20
Compressed:  0 bytes
Uncompressed:  0 bytes

Python 3.7.3 : Three examples with BeautifulSoup.

Beautiful Soup is a library that makes it easy to scrape information from web pages. It sits atop an HTML or XML parser, providing Pythonic idioms for iterating, searching, and modifying the parse tree., see the pypi webpage.
This python module was created by Leonard Richardson.
A large definition can be this:
Web Scraping (also termed Screen Scraping, Web Data Extraction, Web Harvesting, etc.) is a technique employed to extract large amounts of data from websites whereby the data is extracted and saved to a local file in your computer or to a database in table (spreadsheet) format.
This python module can do that but the input format and output format is different.
The input can be a webpage like an URL or webpage with all pieces of information and the output depends by the this and the user choices.
Les's see some examples:
First example show you how to take content of the first row table from a wikipedia webpage.

# get table from wikipedia 
import requests
from bs4 import BeautifulSoup
website_url = requests.get('https://en.wikipedia.org/w/index.php?title=Table_of_food_nutrients').text
soup = BeautifulSoup(website_url,'lxml')

my_table = soup.find('table',{'class':'wikitable collapsible collapsed'})
links = my_table.findAll('a')
Food = []
for link in links:
    Food.append(link.get('title'))

print(Food)

The next example takes all files from a page

# get links using the url
import urllib
from bs4 import BeautifulSoup
page = urllib.request.urlopen('http://____share.net/filmes/').read()
soup = BeautifulSoup(page)
soup.prettify()
for anchor in soup.findAll('a', href=True):
    print (anchor['href'])

The last example takes all images from the search query of imgur website:

# get images from imgur search query
import urllib
from bs4 import BeautifulSoup
url = 'https://imgur.com/search/score?q=cyborg'
with urllib.request.urlopen(url) as f:
    soup = BeautifulSoup(f.read(),'lxml')

a_tags = soup.findAll("a",{"class":"image-list-link"})
img_tags = [a.find("img") for a in a_tags]
print(img_tags)
srcs = []
for s in img_tags:
    src_tags=('http:'+s['src'])
    srcs.append(src_tags)

print(srcs)

As a conclusion, this module will pose problems for those who do not understand how to scroll through the source code, the content of web pages, how to read 'lxml', 'page', etc.
It will greatly help your Chrome F12 key to access parts of web content.