Python 3.7.6 : The new concepts of execution in python 3 - part 001.

The main goal of these tutorials series is learning to deal with python source code using the new concepts of execution in python 3.
When two or more events are concurrent it means that they are happening at the same time.
Concurrent programming is not equivalent to parallel execution.
In computing, concurrency is the execution of pieces of work or tasks by a computer at the same time.
Concurrency is a property which more than one operation can be run simultaneously.
When multiple computations or operations are carried out at the same time or in parallel with the goal of speeding up the computation process then this process is named parallelism.
Parallelism is a property which operations are actually being run simultaneously using the multiprocessing.
Multiprocessing, on the other hand, involves utilizing two or more processor units on a computer to achieve parallelism.
Multithreading is a property that refers to the ability of a CPU to execute multiple threads concurrently.
Python’s concurrency methods including threading, multiprocessing, and asyncio.
The difference between the threading and multiprocessing is this: the threading module uses threads, the multiprocessing module uses processes.
The threading is the package that provides API to create and manage threads.
With multiprocessing, Python creates new processes using an API similar to the threading module.
The asyncio is a library to write concurrent code using the async/await syntax.
The keyword async indicates that our function is a coroutine meaning they choose when to pause and let others execute and run coroutines multitask cooperatively.
The three fundamental advantages of async and await over threads are:

• cooperative multitasking - you can reasonably have millions of concurrent tasks;
• using await makes visible where the schedule points;
• if a task doesn’t yield then it can accidentally block all other tasks from running;
• tasks can support cancellation.

The next source of code show us how can deal with the execution code in Python 3 using the threading and multiprocessing python packages.
The timeit python package is used to benchmark the code write in the code_to_test variable:
Let's test the multi-threading execution with python:

[mythcat@desk ~]$ python3 
Python 3.7.6 (default, Jan 30 2020, 09:44:41) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import timeit
>>> code_to_test = """
... import threading
... 
... text = "Hello World"
... 
... def print_text(text):
...     for char in text:
...             print (char)
... 
... # multi-threading execution
... def multi_threads():
...     thread_1 = threading.Thread(target=print_text(text))
...     thread_2 = threading.Thread(target=print_text(text))
...     thread_1.start()
...     thread_2.start()
...     thread_1.join()
...     thread_2.join()
... multi_threads()
... """
>>> 
>>> elapsed_time = timeit.timeit(code_to_test, number=1)
H
e
l
l
o
 
W
o
r
l
d
H
e
l
l
o
 
W
o
r
l
d
>>> print(elapsed_time)
0.010613240000566293

Let's test the serially execution with python:

[mythcat@desk ~]$ python3 
Python 3.7.6 (default, Jan 30 2020, 09:44:41) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import timeit
>>> code_to_test = """
... import threading
... 
... text = "Hello World"
... 
... def print_text(text):
...     for char in text:
...             print (char)
... # serially execution
... def serially():
...     print_text(text)
...     print_text(text)
... serially()
... """
>>> elapsed_time = timeit.timeit(code_to_test, number=1)
H
e
l
l
o
 
W
o
r
l
d
H
e
l
l
o
 
W
o
r
l
d
>>> print(elapsed_time)
0.011771811000471644

Let's test the multiprocessing execution with python:

[mythcat@desk ~]$ python3 
Python 3.7.6 (default, Jan 30 2020, 09:44:41) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import timeit
>>> code_to_test = """
... import multiprocessing
... 
... text = "Hello World"
... 
... def print_text(text):
...     for char in text:
...             print (char)
... 
... # multiprocessing execution
... def multiprocessing_test():
...      process_1 = multiprocessing.Process(target=print_text(text))
...      process_2 = multiprocessing.Process(target=print_text(text))
...      process_1.start()
...      process_2.start()
...      process_1.join()
...      process_2.join()
... multiprocessing_test()
... """
>>> elapsed_time = timeit.timeit(code_to_test, number=1)
H
e
l
l
o
 
W
o
r
l
d
H
e
l
l
o
 
W
o
r
l
d
>>> print(elapsed_time)
0.3649730779998208

Since asyncio is a little complex, I will write about this in the next tutorial.

Python 3.7.3 : Testing the timeit and Bokeh python module.

The tutorial today has several goals:

• testing the timeit function to measure the execution time of a line of print code;
• using python lists to record data;
• generate lists using the range function;
• multiplying a number with an entire list of numbers;
• how to use the bokeh module with CustomJSHover and HoverTool

Let's test with the timeit python module, see the official webpage.
This module provides a simple way to time small bits of Python code. It has both a Command-Line Interface as well as a callable one. It avoids a number of common traps for measuring execution times.

C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Inte
l)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import timeit
>>> import random
>>> timeit.timeit('print("a")',number=1)
a
0.00013114200010022614
>>> timeit.timeit('print("a")',number=100)
...
a
0.007177434000027461
>>> timeit.timeit('print("a")',number=1000)
...
a
0.07585798000002342

Let make more visible the results with the bokeh python module:

import timeit
import random

# if you want to use numpy 
#import numpy as np

# import bokeh python module 
from bokeh.plotting import figure, show, output_file
# for show values
from bokeh.models.tools import CustomJSHover
from bokeh.models import HoverTool
print ('''
You can tests a number of times named l
with a number of print named i
and see the result value with bokeh python module
timeit.timeit('print("a")',number = i)
''')
i = int(input('The increment number i:'))
l = int(input('The number of test l:'))

# create the list with the output values
timit_list = []
# create the test function for l and i and return the list
def get_timeit(i,l):
 while i < l:
  i += 1
  out=timeit.timeit('print("a")',number = i)
  timit_list.append(out)
 return timit_list

# run the test function with l and i
# this will be coordinate y
yt=get_timeit(i,l)

# show result of the test 
print(yt)

# create the coordinate x
xt = [i for i in range(0, len(yt))]

#xt = np.linspace(0, l, l)
# print the coordinate x
#print(xt)

# create the output HTML file to see the result
output_file("test.html")

# create a figure with a timeit type y-axis
fig = figure(title='timit_list values for print("a")',
             plot_height=400, plot_width=700,
             x_axis_label='x has each incrementation of i', y_axis_label='value of timeit of print the char a',
             x_minor_ticks=3, 
             toolbar_location=None)

# create a circle for each value

# see new multiplication with a list 

# y=[i * 100 for i in yt]
fig.circle(x=xt, y=[i * 100 for i in yt], 
         color='blue', size=5,
         legend='Values')

x_custom = CustomJSHover(code="""
    return '' + special_vars.data_x""")

y_custom = CustomJSHover(code="""
    return '' + special_vars.data_y""")

fig.add_tools(
    HoverTool(
        show_arrow=True, 
        tooltips=[
 ('xt', '$data_x'),
 ('yt', '$data_y')
        ],
        formatters=dict(
            xt=x_custom,
            yt=y_custom
        )
    )
)

# Put the legend in the upper left corner
fig.legend.location = 'top_left'

# Let's check it out
show(fig)

The result of this python source code can be found on my YouTube channel:

Python 3.7.3 : How fast are Data Classes.

This simple tutorial follows the PEP 0557 subject of Data Classes.
About the python classes then you can define into three ways:

• standard_class;
• slot_class ( standard_class with __slots__);
• the new dataclass

Today I will show you how this works and how fast are these classes.
First, let's import some python modules: dataclasses and timeit.
The dataclasses python module lets us use the new dataclass.
This dataclass is a class decorator as defined in PEP 526.
For testing, I define a function to show us the output:

def  _str(n): return f'({n.x})'

This will get value and return it.
I define three classes for each type of these with one value named x and set to 1.0.
Each of these classes can have values and be used:

sc = standard_class(1.0)
sl = slot_class(1.0)
dc = new_dataclass(1.0)

This will set for each class the value 1.0 to x parameter.
The next step will start with the measure execution time of the time object creation for each of these classes.
Let's see the source code:

from dataclasses import dataclass
from timeit import timeit

def  _str(n): return f'({n.x})'

class standard_class:
 def __init__(self, x=0.0):
  self.x = x
 def __str__(self): return _str(self)

class slot_class:
 __slots__ = 'x'
 def __init__(self, x=0):
  self.x =x
 def __str__(self): return _str(self)

@dataclass
class new_dataclass:
 x: float = 0.0
 def __str__(self): return _str(self)

sc = standard_class(1.0)
sl = slot_class(1.0)
dc = new_dataclass(1.0)
print(sc,sl,dc)

time_sc=timeit('standard_class()', setup = 'from __main__ import standard_class')
print(f'standard class: {time_sc:.5f}')
time_sc=timeit('slot_class()', setup = 'from __main__ import slot_class')
print(f'standard class: {time_sc:.5f}')
time_sc=timeit('new_dataclass()', setup = 'from __main__ import new_dataclass')
print(f'standard class: {time_sc:.5f}')

The output is:

C:\Python373>python.exe dataclasses_001.py
(1.0) (1.0) (1.0)
standard class: 0.48912
standard class: 0.41349
standard class: 0.48514

As you can see, the new types of the class definition are not very fast but allow for other advantages and disadvantages.
You can read more about this at PEP 0557.