News : All Books Bundle from Michael Driscoll

For $60 you can buy a lot of all the card decks from Michael Driscoll.

• Get a copy of all my self-published Python eBooks :
• Python 101 - 2nd Edition
• Python 201: Intermediate Python
• ReportLab: PDF Processing with Python
• Jupyter Notebook 101
• Creating GUI Applications with Python
• Pillow: Image Processing with Python
• Automating Excel with Python
• The Python Quiz Book

News : Website for Python users.

Here is a site called clcoding.com for those who use the python language with many simple examples and even a test fairy: python coding challenge day.

Python tool oletools.

The recommended Python version to run oletools is the latest Python 3.x (3.9 for now). Python 2.7 is still supported for the moment, even if it reached end of life in 2020 (for projects still using Python 2/PyPy 2 such as ViperMonkey). It is highly recommended to switch to Python 3 if possible.

You can find it on this GitHub project.

See the all tools : mraptor, msodde, olebrowse, oledir, oleid, olemap, olemeta, oleobj, oletimes, olevba, pyxswf, rtfobj.

Python 3.10.12 : My colab tutorials and news from colab - part 036.

Today I recapitulated a bit the artificial intelligence part and a simple example for google drive.

I created two notebooks in collaboration and added them to my github repo.

The most interesting is the one with textgenrnn.

textgenrnn is an modern neural network architecture which utilizes new techniques as attention-weighting and skip-embedding to accelerate training

The last notebook colab is catafest_045.

Python 3.10.12 : My colab tutorials and news from colab - part 035.

In this notebook I will show you how to use python to run a program written in the programming language for CUDA.

This allows you to use NVIDIA CUDA Compiler Driver NVCC, see this official webpage.

NVCC Plugin for Jupyter Notebook by https://github.com/andreinechaev/nvcc4jupyter.

The example I tested is simple:

# This is formatted as CUDA code
__global__ void cuda_hello(){
    printf("Hello World from GPU!\n");
}

int main() {
    cuda_hello<<<1,1>>>();
    return 0;
}

See my GitHub project for colab.

Python 3.8.10 : My colab tutorials and news from colab - part 034.

I add a new colab notebook with a simple source code to list all running VM processes from the colab notebook

You can see more examples on my GitHub colab google repo.

This is the source code:

%%sh
echo "List all running VM processes."
ps -ef
echo "Done"

Python Qt6 : Download for youtube with PyQt6.

Simple example with PyQt6 to create an interface to download a video using a URL from youtube.

This simple example has some limitations, the filtering of the results is done according to the possibilities of the pytube mode and only after video, it does not use multithread and it does not have multiple selection possibilities and options. In conclusion, it offers a simple download functionality.

You can see more on my GitHub account.

import sys
from PyQt6.QtWidgets import QApplication, QMainWindow, QWidget, QVBoxLayout, QHBoxLayout, QLineEdit, QPushButton, QProgressBar, QDialog, QComboBox, QLabel, QMessageBox
from PyQt6.QtGui import QIcon, QPixmap
from pytube import YouTube
from PyQt6.QtCore import Qt
from PyQt6.QtWidgets import QDialogButtonBox


class FormatChecker:
    def __init__(self, url):
        self.url = url

    def check_formats(self):
        try:
            yt = YouTube(self.url)
            formats = []
            streams = yt.streams.filter(only_video=True)
            for stream in streams:
                if stream.url:
                    format_info = {
                        'resolution': stream.resolution,
                        'file_extension': stream.mime_type.split("/")[-1]
                    }
                    formats.append(format_info)
                    print(" format_info ",format_info)
            return formats
        except Exception as e:
            print("Error:", str(e))
            return []


class FormatInfo:
    def __init__(self, resolution, file_formats):
        self.resolution = resolution
        self.file_formats = file_formats


class ResolutionDialog(QDialog):
    def __init__(self, formats, parent=None):
        super().__init__(parent)
        self.setWindowTitle("Select Resolution and File Format")
        self.formats = formats

        layout = QVBoxLayout(self)

        self.resolution_combo = QComboBox(self)
        for format_info in formats:
            resolution = format_info.resolution
            self.resolution_combo.addItem(resolution)
        layout.addWidget(self.resolution_combo)

        self.file_format_combo = QComboBox(self)
        self.update_file_formats(self.resolution_combo.currentText())
        layout.addWidget(self.file_format_combo)

        button_box = QDialogButtonBox(QDialogButtonBox.StandardButton.Ok | QDialogButtonBox.StandardButton.Cancel)
        button_box.accepted.connect(self.accept)
        button_box.rejected.connect(self.reject)
        layout.addWidget(button_box)

        self.resolution_combo.currentIndexChanged.connect(self.on_resolution_changed)

    def update_file_formats(self, resolution):
        self.file_format_combo.clear()
        for format_info in self.formats:
            if format_info.resolution == resolution:
                file_formats = format_info.file_formats
                self.file_format_combo.addItems(file_formats)

    def selected_resolution(self):
        return self.resolution_combo.currentText()

    def selected_file_format(self):
        return self.file_format_combo.currentText()

    def on_resolution_changed(self, index):
        resolution = self.resolution_combo.currentText()
        self.update_file_formats(resolution)


class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()
        self.setWindowTitle("YouTube Downloader - selected - only_video =True")
        self.setFixedWidth(640)

        central_widget = QWidget(self)
        self.setCentralWidget(central_widget)

        layout = QVBoxLayout(central_widget)

        self.url_edit = QLineEdit()
        layout.addWidget(self.url_edit)

        download_button = QPushButton("Download")
        download_button.clicked.connect(self.show_resolution_dialog)
        layout.addWidget(download_button)

        progress_layout = QHBoxLayout()
        layout.addLayout(progress_layout)

        self.progress_bar = QProgressBar()
        self.progress_bar.setTextVisible(True)
        progress_layout.addWidget(self.progress_bar)

        self.progress_icon_label = QLabel(self)
        pixmap = QPixmap("youtube.png")  # Înlocuiți "path_to_icon.png" cu calea către iconul dorit
        self.progress_icon_label.setPixmap(pixmap)
        progress_layout.addWidget(self.progress_icon_label)

    def show_resolution_dialog(self):
        url = self.url_edit.text()
        if url:
            format_checker = FormatChecker(url)
            formats = format_checker.check_formats()
            format_infos = []
            for format in formats:
                resolution = format['resolution']
                file_extension = format['file_extension']
                format_info = next((info for info in format_infos if info.resolution == resolution), None)
                if format_info:
                    format_info.file_formats.append(file_extension)
                else:
                    format_info = FormatInfo(resolution, [file_extension])
                    format_infos.append(format_info)

            dialog = ResolutionDialog(format_infos, self)
            if dialog.exec() == QDialog.DialogCode.Accepted:
                resolution = dialog.selected_resolution()
                file_format = dialog.selected_file_format()
                self.download_video(url, resolution, file_format)
        else:
            print("Please enter a valid YouTube URL.")

    def download_video(self, url, resolution, file_format):
        try:
            yt = YouTube(url)
            stream = yt.streams.filter(only_video=True, resolution=resolution, mime_type="video/" + file_format).first()
            if stream:
                stream.download()
                print("Download completed!")
                QMessageBox.question(self, "Download Completed", "The video has been downloaded successfully.", QMessageBox.StandardButton.Ok)
            else:
                print("Error: The selected video format is not available for download.")
                QMessageBox.question(self, "Download Error", "The selected video format is not available for download.", QMessageBox.StandardButton.Ok)
        except Exception as e:
            print("Error:", str(e))
            QMessageBox.question(self, "Download Error", "An error occurred during the download.", QMessageBox.StandardButton.Ok)


def main():
    app = QApplication(sys.argv)
    window = MainWindow()
    window.show()
    sys.exit(app.exec())


if __name__ == "__main__":
    main()

Python 3.11.0 : The yattag python package.

Yattag is a Python library for generating HTML or XML in a pythonic way.

You can find more information on the official webpage.

Installation can be done easily with the pip utility:

pip install yattag --user

I test a simple example code from the official documentation and works well:

from yattag import Doc

doc, tag, text = Doc(
    defaults = {
        'title': 'Untitled',
        'contact_message': 'You just won the lottery!'
    },
    errors = {
        'contact_message': 'Your message looks like spam.'
    }
).tagtext()

with tag('h1'):
    text('Contact form')
with tag('form', action = ""):
    doc.input(name = 'title', type = 'text')
    with doc.textarea(name = 'contact_message'):
        pass
    doc.stag('input', type = 'submit', value = 'Send my message')

print(doc.getvalue())

Python Qt5 : OpenStreetMap with PyQtWebEngine.

You need to install the PyQtWebEngine python package and to have the PyQt5 python package installed.

PyQt6 works in the same way as PyQt5 but with small differences ...

pip install PyQtWebEngine --user
Collecting PyQtWebEngine
  Downloading PyQtWebEngine-5.15.6-cp37-abi3-win_amd64.whl (182 kB)
     ---------------------------------------- 182.7/182.7 kB 580.6 kB/s eta 0:00:00
...
     ---------------------------------------- 60.0/60.0 MB 655.7 kB/s eta 0:00:00
...
Installing collected packages: PyQtWebEngine-Qt5, PyQtWebEngine
Successfully installed PyQtWebEngine-5.15.6 PyQtWebEngine-Qt5-5.15.2

import sys
from PyQt5.QtWebEngineWidgets import QWebEngineView
from PyQt5.QtWidgets import QApplication, QWidget, QVBoxLayout

class MapWidget(QWidget):
    def __init__(self, parent=None):
        super().__init__(parent)

        # Create a QWebEngineView object to display the map
        self.map = QWebEngineView()
        self.map.setHtml("""
            <!DOCTYPE html>
            <html>
                <head>
                    <meta name="viewport" content="width=device-width, initial-scale=1.0">
                    <style>
                        #map {
                            height: 100%;
                        }
                    </style>
                </head>
                <body>
                    <div id="map"></div>
                    <script src="https://openlayers.org/en/v4.6.5/build/ol.js"></script>
                    <script>
                        var map = new ol.Map({
                            target: 'map',
                            layers: [
                                new ol.layer.Tile({
                                    source: new ol.source.OSM()
                                })
                            ],
                            view: new ol.View({
                                center: ol.proj.fromLonLat([0, 0]),
                                zoom: 2
                            })
                        });
                    </script>
                </body>
            </html>
        """)

        # Create a QVBoxLayout to hold the map widget
        layout = QVBoxLayout()
        layout.addWidget(self.map)
        self.setLayout(layout)

if __name__ == '__main__':
    app = QApplication(sys.argv)
    widget = MapWidget()
    widget.show()
    sys.exit(app.exec_())

The run of this python script will show a window with the OpenStreetMap map.

Python : Issues with python install - part 001.

I use python as an additional programming language for useful little things, testing for tutorials, and because I use the Blender 3D program.

This involves using it in Visual Code and command line with various versions of python that require separate installations.

Although the development team of the python program makes major changes to maintain and improve these separate installations on various versions, especially on the Windows operating system, they put their management in trouble.

There are plenty of examples on the web with questions and answers.

Here's a simple one, if you run the python command line and the Windows store opens then some settings are not set correctly

Use the Start button and on the Windows search bar to find Manage application run aliases.

There should be two python aliases, deselect them, this will allow the usual python aliases python and python3.

This deselect will delete two files on this folder:

cd %localappdata%\Microsoft\WindowsApps

C:\Users\YOUR_USER\AppData\Local\Microsoft\WindowsApps>dir python*

If you select these the file will be in that folder.

To solve this issue you need to deselect all python or remove the python files from the folder: \YOUR_USER\AppData\Local\Microsoft\WindowsApps.

After that, you need to go on Add or remove programs windows settings area from your operating system and repair select your python install press Modify button and select: Add python to environment variables.

This will fix the path for your python installation.

Now the python 3.10.2 installer come with one update feature, see the next screenshot:

If I run the python command python --version to see my python install on folder C:\Python310, the result is this: Python 3.10.2.

When you enter the python command, it searches the directories listed in your path environment variables page from top to bottom.

Will be great if these python installations will be managed, updated, and fixed properly by the installer, I see something similar in the Unity 3D Hub.

Let's see another example that shows us why it needs well-managed software for python versions.

If you try to install gravityai python package then you can do it easy on colab google with python version 3.7.12.

The bad part comes when you can try to use this version with a simple python windows installer because not all Windows executable installers are available for this python version.

This research needs time and finally, I found a good python version 3.7.9 for this package.

First I got this error:

C:\Python379>python.exe -m pip install --upgrade pip
Requirement already satisfied: pip in c:\users\
...
ERROR: Exception:
...
ValueError: Unable to find resource t64.exe in package pip._vendor.distlib
WARNING: You are using pip version 21.1.3; however, version 22.0.4 is available.
You should consider upgrading via the 'C:\Python379\python.exe -m pip install --upgrade pip' command.

I uninstall the setuptools with this command because python is set to the old install python 3.10.2 and give that error:

python -m pip uninstall pip setuptools
Found existing installation: pip 21.1.3
Uninstalling pip-21.1.3:
...
Uninstalling setuptools-57.2.0:

I upgrade with the pip tool my new python version 3.7.9 version.

python.exe -m pip install --upgrade pip
Requirement already satisfied: pip in c:\python379\lib\site-packages (22.0.4)

The last step is to use python.exe not the python command from my Python379 folder to install the gravityai python package.

C:\Python379>python.exe -m pip install gravityai
Collecting gravityai
  Using cached gravityai-0.1.3.post1.tar.gz (6.5 kB)
  Preparing metadata (setup.py) ... done
Collecting pathlib~=1.0.1
  Using cached pathlib-1.0.1.tar.gz (49 kB)
  Preparing metadata (setup.py) ... done
Collecting websockets~=9.1
  Downloading websockets-9.1-cp37-cp37m-win_amd64.whl (90 kB)
     ---------------------------------------- 90.2/90.2 KB 850.2 kB/s eta 0:00:00
Collecting asyncio~=3.4.3
  Downloading asyncio-3.4.3-py3-none-any.whl (101 kB)
     ---------------------------------------- 101.8/101.8 KB 1.2 MB/s eta 0:00:00
...
Successfully built gravityai pathlib
Installing collected packages: pathlib, asyncio, websockets, gravityai
Successfully installed asyncio-3.4.3 gravityai-0.1.3.post1 pathlib-1.0.1 websockets-9.1

Now I can use this package with python and python.exe commands in the Python379 folder.

Another useful command to have a good image of the install for pip tool is this:

python -m pip install --no-cache-dir  --force-reinstall -Iv gravityai
...
full command: 'C:\Python310\python.exe' -c '
...
exec(compile(setup_py_code, filename, "exec"))

This full output will give you information about the steps for installing the package, in this case, the uninstall and install the setuptools.

This option for the install process is --ignore-installed and let you install a new version of the package and keep the old one, see example command for gravityai package.

pip install gravityai --ignore-installed

Python Qt5 : Parse files and show image with QPixmap .

This tutorial is about how to create a simple script in python with a few features:

You can see I used these python packages: argparse, PIL, glob, io, PyQt5 to fulfill the issue.

The PIL python package is used to use PNG images.

The argparse python package is used to get inputs from arguments.

The glob python package is used to parse folders and files.

The io python package is used to open images:

The PyQt5 python package is used to show images on canvas:

This is the source code in python I used to parse a folder with images and show one image:

import os
import io
import sys
import glob

from PIL import Image
from PIL import UnidentifiedImageError

from PyQt5.QtGui import QPixmap
from PyQt5.QtWidgets import QMainWindow, QApplication, QLabel

import argparse
from pathlib import Path

parser = argparse.ArgumentParser()

parser.add_argument("files_path", type=Path, help='PATH of folder with files ')
parser.add_argument('-p','--print_files', action='store_true', 
    help='Print files from folder ')
parser.add_argument('-s','--show', action='store_true', 
    help='Show image in PyQt5 canvas!')
args = vars(parser.parse_args())

p = parser.parse_args()
print(p.files_path, type(p.files_path), p.files_path.exists())
print(p.show)
files = os.listdir(p.files_path)
file_list = []
image_list = []
bad_files_list =[]

if args['print_files']: 
    print("These are files from folder "+str(p.files_path))
    for f in file_list:
	    print(f)

images_ext = str(Path(p.files_path))+'/*.png'
print("images_ext: "+images_ext)
for filename in glob.glob(images_ext): #assuming png
    try:
        f = open(filename, 'rb')
        file = io.BytesIO(f.read())
        im = Image.open(file)
        image_list.append(filename)
        print(str(len(image_list))+" good file is"+filename)
    except Image.UnidentifiedImageError:
        bad_files_list.append(str(p.files_path)+"/"+str(filename))

for f in bad_files_list:
	print("bad file is : " + f)


if args['show']:
    value = input("Please enter the index of PNG image :\n")
    try:
        int(value)
        print("Image number select default : " + value)      
        value = int(value)
        if value &lt= len(image_list):
            value = int(value)
        else:
            print("The number image selected is greater then len of list images!")
            value = len(image_list)
    except:
        print("This is not a number, I set first image number.")
        value = 1

    class MainWindow(QMainWindow):

        def __init__(self):
            super(MainWindow, self).__init__()
            self.title = "Image Viewer"
            self.setWindowTitle(self.title)

            label = QLabel(self)
            pixmap = QPixmap(image_list[int(value)])
            label.setPixmap(pixmap)
            self.setCentralWidget(label)
            self.resize(pixmap.width(), pixmap.height())

    app = QApplication(sys.argv)
    w = MainWindow()
    w.show()
    sys.exit(app.exec_())

These are the output of the script:

[mythcat@desk PIL001]$ python  tool001.py 
usage: tool001.py [-h] [-p] [-s] files_path
tool001.py: error: the following arguments are required: files_path
[mythcat@desk PIL001]$ python  tool001.py ~/Pictures/
/home/mythcat/Pictures  True
False
images_ext: /home/mythcat/Pictures/*.png
1 good file is/home/mythcat/Pictures/keyboard.png
2 good file is/home/mythcat/Pictures/Fedora_The_Pirate_CaribbeanHunt.png
3 good file is/home/mythcat/Pictures/website.png
4 good file is/home/mythcat/Pictures/Screenshot from 2021-02-19 19-24-32.png
...
97 good file is/home/mythcat/Pictures/Screenshot from 2021-07-03 17-31-23.png
98 good file is/home/mythcat/Pictures/Screenshot from 2021-07-03 17-46-05.png
bad file is : /home/mythcat/Pictures//home/mythcat/Pictures/Screenshot from 2021-02-07 14-58-56.png
bad file is : /home/mythcat/Pictures//home/mythcat/Pictures/evolution_logo.png
[mythcat@desk PIL001]$ python  tool001.py ~/Pictures/ -p
/home/mythcat/Pictures  True
False
These are files from folder /home/mythcat/Pictures
images_ext: /home/mythcat/Pictures/*.png
1 good file is/home/mythcat/Pictures/keyboard.png
2 good file is/home/mythcat/Pictures/Fedora_The_Pirate_CaribbeanHunt.png
3 good file is/home/mythcat/Pictures/website.png
...
97 good file is/home/mythcat/Pictures/Screenshot from 2021-07-03 17-31-23.png
98 good file is/home/mythcat/Pictures/Screenshot from 2021-07-03 17-46-05.png
bad file is : /home/mythcat/Pictures//home/mythcat/Pictures/Screenshot from 2021-02-07 14-58-56.png
bad file is : /home/mythcat/Pictures//home/mythcat/Pictures/evolution_logo.png
[mythcat@desk PIL001]$ python  tool001.py ~/Pictures/ -s
/home/mythcat/Pictures  True
True
images_ext: /home/mythcat/Pictures/*.png
...
97 good file is/home/mythcat/Pictures/Screenshot from 2021-07-03 17-31-23.png
98 good file is/home/mythcat/Pictures/Screenshot from 2021-07-03 17-46-05.png
bad file is : /home/mythcat/Pictures//home/mythcat/Pictures/Screenshot from 2021-02-07 14-58-56.png
bad file is : /home/mythcat/Pictures//home/mythcat/Pictures/evolution_logo.png
Please enter the index of PNG image :
6
Image number select default : 6

Python 3.9.0 : Union and in-place union operators

Python introduces two new operators for dictionaries named union used in code with pipe operator | and in-place union used in python code with this |=.

I this tutorial I will show you how can be used:

[mythcat@desk ~]$ python3.9 
Python 3.9.0 (default, Oct  6 2020, 00:00:00) 
[GCC 10.2.1 20200723 (Red Hat 10.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> step_one = {"worker_one":"task_one", "worker_two":"task_two"}
>>> step_two = {"worker_three":"task_one", "worker_fouw":"task_two"}
>>> merged_step_one_step_two = {**step_one, **step_two}
>>> empty = {}
>>> empty |= step_one
>>> empty
{'worker_one': 'task_one', 'worker_two': 'task_two'}
>>> step_one 
{'worker_one': 'task_one', 'worker_two': 'task_two'}
>>> step_two
{'worker_three': 'task_one', 'worker_fouw': 'task_two'}
>>> all = step_one | step_two 
>>> all 
{'worker_one': 'task_one', 'worker_two': 'task_two', 'worker_three': 'task_one', 'worker_fouw': 'task_two'} 
>>> copy_step_one = empty | step_one
>>> copy_step_one 
{'worker_one': 'task_one', 'worker_two': 'task_two'} 
>>> copy_step_one |= [("manager", "steps")]
>>> copy_step_one 
{'worker_one': 'task_one', 'worker_two': 'task_two', 'manager': 'steps'}

You can easily see how to use these operators with the dictionaries created and how to add lists to dictionaries. Operations with these operators always result in a dictionary. The order of these operations is important, see example:

>>> numbers = {0: "zero", 1: "one"}
>>> numbers_one = {0: "zero", 1: "one"}
>>> numbers_two = {0: "zero", 2: "two"}
>>> numbers_one | numbers_two
{0: 'zero', 1: 'one', 2: 'two'}
>>> numbers_two | numbers_one
{0: 'zero', 2: 'two', 1: 'one'}
>>> numbers_three = {0: 'zero', 11: 'error 1', 22: 'error 2', 33:'error 3'}
>>> numbers_three | numbers_one
{0: 'zero', 11: 'error 1', 22: 'error 2', 33: 'error 3', 1: 'one'}
>>> numbers_one | numbers_three
{0: 'zero', 1: 'one', 11: 'error 1', 22: 'error 2', 33: 'error 3'}

You can see how the dictionaries are overwritten by the union operation. These operations are implemented through dunder operations for most the dictionary objects except abstract classes. Dunder or magic methods in Python are the methods having two prefix and suffix underscores in the method name. You can find on this page.

Python 3.9.0 : Introduction to release 3.9.0.

This is a short introduction to release 3.9.0. 

Five days ago, a new release of version 3.9 appeared with a series of improvements and new python packages, see the official website. 

You can install easily on Fedora 32 with dnf tool.

[root@desk mythcat]# dnf install  python39.x86_64
...
Installing:
 python39                x86_64                3.9.0-1.fc32
...
Installed:
  python39-3.9.0-1.fc32.x86_64                                                                             

Complete!

A new article written about this release shows some of the advantages of this programming language, you can read it here. 

You can run easily with this command;

[root@desk mythcat]# python3.9
Python 3.9.0 (default, Oct  6 2020, 00:00:00) 
[GCC 10.2.1 20200723 (Red Hat 10.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information. 

You can configure python with this command, see an example:

[root@desk mythcat]# python3.9-x86_64-config --libs 
 -lcrypt -lpthread -ldl  -lutil -lm -lm 

This release of Python 3.9 uses a new parser, based on parsing expression grammar (PEG) instead of LL(1) know as LL parser (Left-to-right, Leftmost derivation). PEG parsers are more powerful than LL(1) parsers and avoid the need for special hacks. This the same abstract syntax tree (AST) as the old LL(1) parser. The PEG parser is the default, but you can run your program using the old parser, see the next example:

[mythcat@desk ~]$ python3.9 -X oldparser my_script_name.py
...

One of the most important improvements for me is PEP 614 -- Relaxing Grammar Restrictions On Decorators. In Python 3.9, these restrictions are lifted and you can now use any expression. 

For example including one accessing items in a dictionary, like this simple example:

buttons = {
  "hello": QPushButton("hello word!"),
  "bye": QPushButton("bye word!"),
  "buy": QPushButton("buy!!"),
}
...
@buttons["hello"].clicked.connect
def speak_hello():
... 

Yes, comes with another Python Enhancement Proposals - PEP 615 with support for the IANA Time Zone Database in the Standard Library. This acronym is similar to I.A.N.A known as the Internet Assigned Numbers Authority, but it is not the same. 

In this case, the zoneinfo module provides a concrete time zone implementation to support the IANA time zone database. 

This support contains code and data that represent the history of local time for many representative locations around the globe. Many features for math, strings, union operator for the dictionary, HTTP codes, and more. 

Completion of variable and module names is automatically enabled at interpreter startup so that the Tab key invokes the completion function. 

You can see a simple example with zoneinfo python module and completion feature.

[mythcat@desk ~]$ python3.9
Python 3.9.0 (default, Oct  6 2020, 00:00:00) 
[GCC 10.2.1 20200723 (Red Hat 10.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from zoneinfo import ZoneInfo
>>> import zoneinfo
>>> zoneinfo.available_timezones()
{'Hongkong', 'America/Iqaluit', 'America/Indianapolis', 'America/Louisville', 'America/New_York', 
'America/Mazatlan', 'Australia/Yancowinna', 'Africa/Ndjamena', 'Portugal', 'Africa/Bujumbura', 
'America/Rosario', 'America/Antigua','America/Indiana/Tell_City', 'America/Managua', 
'Europe/Paris', 'Europe/Oslo', 
...
>>> tz = ZoneInfo("Europe/Bucharest")
>>> tz.
tz.clear_cache(  tz.from_file(    tz.key           tz.tzname(       
tz.dst(          tz.fromutc(      tz.no_cache(     tz.utcoffset(    
>>> tz. 

On my desktop I got this result:

[mythcat@desk ~]$ python var_access_benchmark.py 
Variable and attribute read access:
   6.0 ns	read_local
   6.5 ns	read_nonlocal
  10.7 ns	read_global
  10.7 ns	read_builtin
  25.2 ns	read_classvar_from_class
  23.4 ns	read_classvar_from_instance
  34.3 ns	read_instancevar
  29.4 ns	read_instancevar_slots
  26.8 ns	read_namedtuple
  42.4 ns	read_boundmethod

Variable and attribute write access:
   6.6 ns	write_local
   7.0 ns	write_nonlocal
  23.3 ns	write_global
  54.1 ns	write_classvar
  46.4 ns	write_instancevar
  39.4 ns	write_instancevar_slots

Data structure read access:
  26.1 ns	read_list
  28.1 ns	read_deque
  27.5 ns	read_dict
  25.8 ns	read_strdict

Data structure write access:
  29.6 ns	write_list
  31.7 ns	write_deque
  34.4 ns	write_dict
  31.7 ns	write_strdict

Stack (or queue) operations:
  55.5 ns	list_append_pop
  49.7 ns	deque_append_pop
  51.0 ns	deque_append_popleft

Timing loop overhead:
   0.4 ns	loop_overhead

Python Qt5 : Use QStandardItem with Images.

This tutorial show you how to use QStandardItem with Images.

The source code is simple to understand.

import sys
from PyQt5.QtWidgets import QApplication, QMainWindow, QTreeView
from PyQt5.QtCore import  Qt
from PyQt5.QtGui import QFont, QColor, QImage, QStandardItemModel, QStandardItem

class ItemImage(QStandardItem):
    def __init__(self, txt='', image_path='', set_bold=False, color=QColor(0, 0, 0)):
        super().__init__()

        self.setEditable(False)
        self.setForeground(color)
        self.setText(txt)

        if image_path:
            image = QImage(image_path)
            self.setData(image, Qt.DecorationRole)

class MyApp(QMainWindow):
    def __init__(self):
        super().__init__()
        self.resize(1200, 1200)

        treeView = QTreeView()
        treeView.setHeaderHidden(True)
        treeView.header().setStretchLastSection(True)

        treeModel = QStandardItemModel()
        rootNode = treeModel.invisibleRootItem()

        robots = ItemImage('Robots', '', set_bold=True)

        aaa = ItemImage('aaa.jpg', 'aaa.jpg', 14)
        robots.appendRow(aaa)

        bbb = ItemImage('bbb.jpg', 'bbb.jpg', 16)
        robots.appendRow(bbb)

        robots2 = ItemImage('Robots 2', '', set_bold=False)

        aaa = ItemImage('ccc.png', 'ccc.png', 14)
        robots2.appendRow(aaa)

        bbb = ItemImage('ddd.jpg', 'ddd.jpg', 16)
        robots2.appendRow(bbb)

        rootNode.appendRow(robots)
        rootNode.appendRow(robots2)
        treeView.setModel(treeModel)
        treeView.expandAll()

        self.setCentralWidget(treeView)

app = QApplication(sys.argv)        
demo = MyApp()
demo.show()
sys.exit(app.exec_())

The result is this:

Python 3.8.5 : A sphere in Cartesian coordinates - part 001.

I like the equation of a sphere of radius R centered at the origin is given in Cartesian coordinates:

x*x + y*y + z*z = r*r

It is one of the first elements that helped me better understand mathematics and later the dynamics and theory of electromagnetic fields.

I did not find a graphical representation using python as accurately as possible without eliminating the discretion of the range from -1 and 1 and radius * radius = 1.

The main reason is the plot_surface from matplotlib python package.

This is output of my script:

[mythcat@desk ~]$ python sphere_xyz.py 
[-1.         -0.91666667 -0.83333333 -0.75       -0.66666667 -0.58333333
 -0.5        -0.41666667 -0.33333333 -0.25       -0.16666667 -0.08333333
  0.          0.08333333  0.16666667  0.25        0.33333333  0.41666667
  0.5         0.58333333  0.66666667  0.75        0.83333333  0.91666667
  1.        ]
sphere_xyz.py:7: RuntimeWarning: invalid value encountered in sqrt
  return np.sqrt(1-x**2 - y**2)
sphere_xyz.py:18: UserWarning: Z contains NaN values. This may result in rendering artifacts.
  surface1 = ax.plot_surface(X2, Y2, -Z2,rstride=1, cstride=1, linewidth=0,antialiased=True)
sphere_xyz.py:19: UserWarning: Z contains NaN values. This may result in rendering artifacts.
  surface2 = ax.plot_surface(X2, Y2, Z2,rstride=1, cstride=1, linewidth=0,antialiased=True)

The image result is this:

The source code is this:

import matplotlib.pyplot as plt
import numpy as np
from matplotlib.colors import LightSource

#@np.vectorize
def solve_Z2(x,y):
    return np.sqrt(1-x**2 - y**2)

fig = plt.figure()
ax = fig.gca(projection='3d')

xrange2 = np.linspace(-1.0, 1.0, 25)
yrange2 = np.linspace(-1.0, 1.0, 25)
print(xrange2)
X2, Y2 = np.meshgrid(xrange2, yrange2)
Z2 = solve_Z2(X2, Y2)

surface1 = ax.plot_surface(X2, Y2, -Z2,rstride=1, cstride=1, linewidth=0,antialiased=True)
surface2 = ax.plot_surface(X2, Y2, Z2,rstride=1, cstride=1, linewidth=0,antialiased=True)

plt.show()

Python 3.8.5 : Linked List - part 001.

In computer science, a linked list is a linear collection of data elements whose order is not given by their physical placement in memory. see wikipedia.org.

In this tutorial I will show you how these linked list works and how can build with python programming language.
Let's start with some basic elements:

• Linked List is a linear data structure;
• Linked List are not array;
• Linked List are not stored at a contiguous location because use pointes;
• Each element use a linked pointe;
• Linked List has a dynamic size;
• Linked List use operations like insertion and deletion for each element;
• The access to elements is sequentially;
• Using reference to pointer foe each element needs extra memory space and is not cache friendly;

The Linked List consists of at least two parts: data and pointer to the next node.
The first element of the list is called the head.

The last node has a reference to null.
The each element named node of the list has a similar structure like Linked List: data node and pointer to the next node.
The data from Linked list is represent like a sequence.

A number of operations are required to use the Linked List.

The basic operations supported by a list can be:

• insertion - adds an element at the beginning of the list;
• deletion - deletes an element at the beginning of the list;
• display - displays the complete list;
• search - searches an element using the given key;
• delete - deletes an element using the given key.

The Advantages of Linked List elements can be easily inserted or removed without reallocation or reorganization of the entire structure because the data items need not be stored contiguously in memory or on disk using operations.

The Linked List structure allows several ways to link nodes resulting in a number of types of Linked List:

• Simple linked list;
• Doubly linked list;
• Multiply linked list;
• Circular linked list;
• Sentinel nodes;
• Empty lists;
• Hash linking;
• List handles;
• Combining alternatives;

Let's start with first source code for the first type of Linked List named Simple linked list.

Python 3.8.5 (default, Aug 12 2020, 00:00:00) 
[GCC 10.2.1 20200723 (Red Hat 10.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> # create a node class 
>>> class Node:
...     # first initialise of the node object
...     def __init__(self, data):
...             # assign the node data 
...             self.data = data
...             # initialize next element as null
...             self.next = None
... 
>>> # create a simple linked list
>>> class Simple_LinkedList:
...     # first initialize of element named head
...     def __init__(self):
...             self.head = None
>>> # use the list 
>>> if __name__=='__main__': 
...     simple_list = Simple_LinkedList()
...     simple_list.head = Node(1)
...     second = Node(2)
...     third = Node(3)
...     #link first node with the second
...     simple_list.head.next = second
...     #link second  node with the third
...     second.next = third
...     # now the list is linked as a simple Linked List

This source code don't include any basic operations.

If you put this source code into python script and run it you will see this:

[mythcat@desk ~]$ python simple_LinkedList.py 
<__main__.Node object at 0x7f35163e1ac0>
<__main__.Node object at 0x7f351638b0a0>
<__main__.Node object at 0x7f351638b130>

I create a basic operation for this simple Linked List to display the content of this list:

# create a simple linked list with a basic diplay operation
class Simple_LinkedList:
    # first initialize of element named head
    def __init__(self):
            self.head = None
    # use the basic operation for display
    def display_Simple_LinkedList(self): 
        points_to_node = self.head
	#will traverse the list to the last node. 
        while (points_to_node): 
            # print data linked to points_to_node
            print (points_to_node.data) 
            # print next node linked to points_to_node
            points_to_node = points_to_node.next

I run with this new python source code and result is this:

[mythcat@desk ~]$ python simple_LinkedList_basic_display.py 
1
2
3 

Python Qt5 : Get item data from QTreeWidgets.

In this example, I create a tree view with QTreeView with all folders tree.
I add a context_menu with two options.
One option is to get the data from item and is the name of the folder.
The second option is to close the application.
Let's see the source code:

import sys
from PyQt5.QtWidgets import QApplication, QFileSystemModel, QDesktopWidget
from PyQt5.QtWidgets import QTreeView, QWidget, QVBoxLayout, QMenu
from PyQt5.QtGui import QIcon
from PyQt5 import QtCore
from PyQt5.QtCore import Qt, QObject

class my_app_tree(QWidget):

    def __init__(self):
        super().__init__()
        self.title = "show files and folders on tree view"
        #self.left = 0
        #self.top = 0
        #self.width = 640
        #self.height = 480
        self.center()
        self.resize(640,480)
        self.initUI()

    def center(self):
        frame_geometry = self.frameGeometry()
        center_position = QDesktopWidget().availableGeometry().center()
        frame_geometry.moveCenter(center_position)
        self.move(frame_geometry.topLeft())

    def context_menu(self, position):
        menu = QMenu()
        copy_action = menu.addAction("Get folder")
        quit_action = menu.addAction("Quit")
        action = menu.exec_(self.tree.mapToGlobal(position))
        # quit application
        if action == quit_action:
            my_application.quit()
        # copy folder name from item
        elif action == copy_action:
            item = self.tree.selectedIndexes()[0].data()
            print("name folder is: "+str(item))

    def initUI(self):
        self.setWindowTitle(self.title)
        #the next source code line is used with left, top, width, height from __init__
        #self.setGeometry(self.left, self.top, self.width, self.height)
        
        self.model = QFileSystemModel()
        self.model.setRootPath('')
        self.tree = QTreeView()
        self.tree.setModel(self.model)
        
        self.tree.setAnimated(False)
        self.tree.setIndentation(20)
        self.tree.setSortingEnabled(True)
        
        self.tree.setWindowTitle("Dir View")
        self.tree.resize(640, 480)
        
        windowLayout = QVBoxLayout()
        windowLayout.addWidget(self.tree)
        self.setLayout(windowLayout)

        self.tree.setContextMenuPolicy(Qt.CustomContextMenu)
        self.tree.customContextMenuRequested.connect(self.context_menu)
        
        self.show()

if __name__ == '__main__':
    my_application = QApplication(sys.argv)
    example = my_app_tree()
    sys.exit(my_application.exec_())

The result of this source code can be see in the next image:

Python Qt5 : Add and remove items between two QTreeWidgets.

Today's tutorial will show you how to add and remove items between two QTreeWidgets.
The source code is very simple to understand: the user interface is created with two QTreeWidgets.
One is completed with elements and when the buttons are pressed, the elements are interchanged.

import sys
from PyQt5.QtWidgets import QApplication, QWidget, QDesktopWidget, QPushButton
from PyQt5.QtWidgets import QBoxLayout,QTreeWidget,QTreeWidgetItem

class my_app_class(QWidget):
    def __init__(self):
        super().__init__()

        self.add_button=QPushButton('Add item here')
        self.remove_button=QPushButton('Remove item')
        self.wishlist=QTreeWidget(self)
        self.tree_list=QTreeWidget(self)
        # init the UI
        self.initUI()

    def initUI(self):
        # set title of window
        self.setWindowTitle('add and remove items from QTreeWidget!')

        self.init_tree()

        self.resize(800, 480)
        self.center()
        self.show()

    def center(self):
        geometry_frame = self.frameGeometry()
        center_pos = QDesktopWidget().availableGeometry().center()
        geometry_frame.moveCenter(center_pos)
        self.move(geometry_frame.topLeft())


    def init_tree(self):
        headers = ['A','B','C','D']

        self.tree_list.setColumnCount(len(headers))
        self.tree_list.setHeaderLabels(headers)

        self.wishlist.setColumnCount(len(headers))
        self.wishlist.setHeaderLabels(headers)


        list_layout = QBoxLayout(QBoxLayout.LeftToRight)
        list_layout.addWidget(self.tree_list)
        list_layout.addWidget(self.wishlist)

        tree_root = QTreeWidget.invisibleRootItem(self.tree_list)
        # add data to QTreeWidget with QTreeWidgetItem
        my_data = ['1','2','3','4']
        item = QTreeWidgetItem()
        for idx, data in enumerate(my_data):
            item.setText(idx, data)

        tree_root.addChild(item)

        my_data = ['11','10','01','D']
        item = QTreeWidgetItem()
        for idx, data in enumerate(my_data):
            item.setText(idx, data)

        tree_root.addChild(item)

        my_data = ['s', 'c', 'c', 'c']
        item = QTreeWidgetItem()
        for idx, data in enumerate(my_data):
            item.setText(idx, data)

        tree_root.addChild(item)

        btn_layout = QBoxLayout(QBoxLayout.RightToLeft)
        btn_layout.addWidget(self.add_button)
        btn_layout.addWidget(self.remove_button)

        main_layout = QBoxLayout(QBoxLayout.TopToBottom)
        main_layout.addLayout(list_layout)
        main_layout.addLayout(btn_layout)

        self.add_button.clicked.connect(self.move_item)
        self.remove_button.clicked.connect(self.move_item)

        self.setLayout(main_layout)
        return main_layout


    def move_item(self):
        sender = self.sender()

        if self.add_button == sender:
            source = self.tree_list
            target = self.wishlist
        else:
            source = self.wishlist
            target = self.tree_list

        item = QTreeWidget.invisibleRootItem(source).takeChild(source.currentIndex().row())
        QTreeWidget.invisibleRootItem(target).addChild(item)

if __name__=='__main__':
    # start the QApplication
    my_application = QApplication(sys.argv)
    # create aplication with the class
    example = my_app_class()
    # use exit for QApplication
    sys.exit(my_application.exec_())

Python 3.8.5 : The hashlib python package - part 001.

The tutorial for today is about hashlib python module.
The official webpage comes for this python package has this intro:
This module implements a common interface to many different secure hash and message digest algorithms. Included are the FIPS secure hash algorithms SHA1, SHA224, SHA256, SHA384, and SHA512 (defined in FIPS 180-2) as well as RSA’s MD5 algorithm (defined in Internet RFC 1321).
The example source code to test a simple hash is this:

import hashlib
import os

def file_sha1(filename):
    BUF_SIZE = 65536  # read stuff in 64kb chunks!
    get_sha1 = hashlib.sha1()
    with open(filename, 'rb') as f:
        while True:
            data = f.read(BUF_SIZE)
            if not data:
                break
            get_sha1.update(data)
    return get_sha1.hexdigest()

# I add this comment after first to see the hash difference.
files = [f for f in os.listdir('.') if os.path.isfile(f)]
for f in files:
    h = file_sha1(f)
    print(h) 

Let's test the source code with the default directory and two files.
I run it first with default source code and then I add a comment to test_hash_file.py file.
You can see the hash is changed from b222523567a8a806382b86578717ddbd00e0f4b4 to 2134660551cc67812413a3a75fd12efb05d591ef.

[mythcat@desk Projects_Python]$ ls
test_hash_file.py  test_numpy_001.py
[mythcat@desk Projects_Python]$ python test_hash_file.py 
98b2833527ad3d9fe263542c6aa06c04182d3dfb
b222523567a8a806382b86578717ddbd00e0f4b4
[mythcat@desk Projects_Python]$ python test_hash_file.py 
98b2833527ad3d9fe263542c6aa06c04182d3dfb
2134660551cc67812413a3a75fd12efb05d591ef

Python 3.8.5 : Pearson Product Moment Correlation with corrcoef from numpy.

The python package named numpy come with corrcoef function to return Pearson product-moment correlation coefficients.
This method has a limitation in that it can compute the correlation matrix between two variables only.
The full name is the Pearson Product Moment Correlation (PPMC).
The PPMC is not able to tell the difference between dependent variables and independent variables.
The documentation about this function can be found here.
More examples of Pearson Correlation can be found on this website.
My example presented in this tutorial, use the random packet to randomly generate integers and then calculate the correlation coefficients.
All of these are calculated five times in a for a cycle and each time the seed parameters are changed randomly.
Each time the correlation matrices are printed and then the random number graphs are displayed.
Let's see the source code:

import random

import numpy as np

nr_integers = 100
size_integers = 100

import matplotlib
import matplotlib.pyplot as plt

# set from 0 to 4 seed for random and show result 
for e in range(5):
    # change random seed
    np.random.seed(e)
    # nr_integers random integers between 0 and size_integers
    x = np.random.randint(0, size_integers, nr_integers)
    # Positive Correlation with some noise created with
    # nr_integers random integers between 0 and size_integers
    positive_y = x + np.random.normal(0, size_integers, nr_integers)
    correlation_positive = np.corrcoef(x, positive_y)
    # show matrix for correlation_positive
    print(correlation_positive)
    # Negative Correlation with same noise created with 
    # nr_integers random integers between 0 and size_integers
    negative_y = 100 - x + np.random.normal(0, size_integers, nr_integers)
    correlation_negative = np.corrcoef(x, negative_y)
    # show matrix for output with plt
    print(correlation_negative)
    # set graphic for plt with two graphics for each output with subplot
    plt.subplot(1, 2, 1)
    plt.scatter(x,positive_y)
    plt.subplot(1, 2, 2)
    plt.scatter(x,negative_y)
    # show the graph 
    plt.show()