Sunday, July 15, 2018

Python 3.6.4 : Microsoft Azure - bad answer .

The Azure from Microsoft come with the free account and $200 credit to explore services for 30 days.
I only have 7 days left from this free account.
During this time I tried to use it from scratch.
In this short tutorial I will talk about this period,
First I used the python version 3.6.4 and the azure development tools.
These are the links I follow:
I install the azure for python with :
C:\Python364\Scripts>pip install azure
You can use a preview version of this package, which you can access using the --pre flag:
C:\Python364\Scripts>pip install --pre azure
I used the example from azure:
git clone
Into azure account I create a storage account named pythonazure.

You can configure the Access control (IAM) to use with your account.
This storage account need to use keys, see Access keys area.
They tell us about this feature:
Use access keys to authenticate your applications when making requests to this Azure storage account. Store your access keys securely - for example, using Azure Key Vault - and don't share them. We recommend regenerating your access keys regularly. You are provided two access keys so that you can maintain connections using one key while regenerating the other.
I change the accountname with pythonazure and account_key with my account key.
The result is this:

In view of my previous experiences with Google, Firebase and Azure on the ease of assimilation and understanding of the content and the time allocated to making a simple task according to the azure documentation , then azure get the last place.
In azure services we can see the errors that occur and which are non-existent on google or firebase.

I was interested in ease and fast access to python cloud services. However, you can read this point of view written by google here.

Tuesday, June 26, 2018

Python 3.6.4 : Trying to use the python azurure on the windows.

In this tutorial I used the python version 3.6.4 and Windows 8.1 OS.
You can start with pip install tool for some azure modules:
Install the latest Batch runtime library
C:\Python364\Scripts>pip install azure-batch 
This will install the latest Storage management library
C:\Python364\Scripts>pip install azure-mgmt-scheduler 
Will install only the latest Compute installed using the --pre flag:
C:\Python364\Scripts>pip install --pre azure-mgmt-compute 
Finally the storage I used into this tutorial:
C:\Python364\Scripts>pip install azure-storage --upgrade
You can install all of the azure python module:
C:\Python364\Scripts>pip install --pre azure
Installing collected packages: azure-storage-nspkg, azure-storage-common, azure-
storage-queue, azure-servicebus, azure-servicefabric, azure-storage-file, azure-
servicemanagement-legacy, azure-mgmt-consumption, azure-mgmt-media, azure-mgmt-d
ns, azure-mgmt-search, azure-mgmt-cdn, azure-mgmt-compute, azure-mgmt-eventhub,
azure-mgmt-containerinstance, azure-mgmt-datalake-nspkg, azure-mgmt-datalake-ana
lytics, azure-mgmt-recoveryservices, azure-mgmt-authorization, azure-mgmt-adviso
r, azure-mgmt-recoveryservicesbackup, azure-mgmt-billing, azure-mgmt-devtestlabs
, azure-mgmt-network, azure-mgmt-web, azure-mgmt-applicationinsights, azure-mgmt
-cognitiveservices, azure-mgmt-rdbms, azure-mgmt-monitor, azure-mgmt-reservation
s, azure-mgmt-notificationhubs, azure-mgmt-loganalytics, azure-mgmt-logic, azure
-mgmt-iothubprovisioningservices, azure-mgmt-marketplaceordering, azure-mgmt-res
ource, azure-mgmt-scheduler, azure-mgmt-powerbiembedded, azure-mgmt-servicefabri
c, azure-mgmt-commerce, azure-mgmt-sql, azure-mgmt-cosmosdb, azure-mgmt-relay, a
zure-mgmt-storage, azure-mgmt-redis, azure-mgmt-managementpartner, azure-mgmt-tr
afficmanager, azure-mgmt-machinelearningcompute, azure-mgmt-datafactory, azure-m
gmt-hanaonazure, azure-mgmt-iothub, azure-mgmt-servermanager, azure-mgmt-batch,
azure-mgmt-keyvault, azure-mgmt-subscription, azure-mgmt-eventgrid, azure-mgmt-s
ervicebus, azure-mgmt-batchai, azure-mgmt-containerservice, azure-mgmt-container
registry, azure-mgmt-msi, azure-mgmt-datalake-store, azure-mgmt, azure-datalake-
store, azure-eventgrid, azure-keyvault, azure-cosmosdb-nspkg, futures, azure-cos
mosdb-table, azure-graphrbac, azure-storage-blob, azure
I tested with all azure python modules, but you can used just you need.
The next step is to open azure account - I have a trial free account.
I create a Resource Group named python-azure and a Storage account named pythonazure.
Into pythonazure I add the Files service and I upload a HTML file named default.html .
See next images with the steps I make:
Let's make one simple test:
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)]
 on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from azure.common.credentials import UserPassCredentials
>>> from azure.mgmt.resource import ResourceManagementClient
>>> from import StorageManagementClient
>>> from import CloudStorageAccount
>>> from import ContentSettings, PublicAccess
I have not been able to authenticate myself with python script in azure.
I found some inconsistencies and issues on github so I still have to document.

Any help in this regard is welcome.

Sunday, June 24, 2018

Python 3.6.4 : Using python client with blogger API .

I used a new project into console cloud google.
I used the google-api-python-client from here with OAuth service.
You need to create a project and add the blogger API to use it.
For credentials I used OAuth with the json file from google .
This is the source code:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function

__author__ = 'Catalin George Festila ( catafest , mythcat )'

import sys

from oauth2client import client
from googleapiclient import sample_tools

def main(argv):
  # Authenticate and construct service.
  service, flags = sample_tools.init(
      argv, 'blogger', 'v3', __doc__, __file__,


      users = service.users()

      # Retrieve this user's profile information
      thisuser = users.get(userId='self').execute()
      print('This user\'s display name is: %s' % thisuser['displayName'])

      blogs = service.blogs()

      # Retrieve the list of Blogs this user has write privileges on
      thisusersblogs = blogs.listByUser(userId='self').execute()
      for blog in thisusersblogs['items']:
        print('The blog named \'%s\' is at: %s' % (blog['name'], blog['url']))

      posts = service.posts()

      # List the posts for each blog this user has
      for blog in thisusersblogs['items']:
        print('The posts for %s:' % blog['name'])
        request = posts.list(blogId=blog['id'])
        while request != None:
          posts_doc = request.execute()
          if 'items' in posts_doc and not (posts_doc['items'] is None):
            for post in posts_doc['items']:
              print('  %s (%s)' % (post['title'], post['url']))
          request = posts.list_next(request, posts_doc)

  except client.AccessTokenRefreshError:
    print ('The credentials have been revoked or expired, please re-run'
      'the application to re-authorize')

if __name__ == '__main__':
The content of the folder project named google_api_python.
06/24/2018  08:27 PM             1,096 blogger.dat
06/24/2018  07:49 PM             2,599
06/24/2018  08:24 PM               309 client_secrets.json
06/24/2018  08:26 PM                76 debug.log
               4 File(s)          4,080 bytes
               2 Dir(s)  201,382,006,784 bytes free
You need to add the client_id and client_secret from google project into file client_secrets.json.
Run the python script:
The google will ask you about the authentification and will run the script:

Google show us all the infos about this project into dashboard:

Wednesday, June 6, 2018

Python 3.6.4 : The qrcode python module .

This python module named qrcode is a suite of tools for Generate QR codes.
Let's start the tutorial with the install steep :
c:\Python364\Scripts>pip install qrcode[pil]
Collecting qrcode[pil]
Successfully installed qrcode-6.0 
Let's see the output for dir:
>>> import qrcode
>>> from qrcode import *
>>> dir(qrcode)
T', 'QRCode', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__',
 '__name__', '__package__', '__path__', '__spec__', 'base', 'constants', 'except
ions', 'image', 'main', 'make', 'run_example', 'util']
>>> dir(qrcode.util)
['ALPHA_NUM', 'BCH_digit', 'BCH_type_info', 'BCH_type_number', 'BIT_LIMIT_TABLE'
, 'BitBuffer', 'G15', 'G15_MASK', 'G18', 'LUT', 'MODE_8BIT_BYTE', 'MODE_ALPHA_NU
'RE_ALPHA_NUM', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__
', '__name__', '__package__', '__spec__', '_data_count', '_lost_point_level1', '
_lost_point_level2', '_lost_point_level3', '_lost_point_level4', '_optimal_split
', 'base', 'create_bytes', 'create_data', 'exceptions', 'length_in_bits', 'lost_
point', 'mask_func', 'math', 'mode_sizes_for_version', 'optimal_data_chunks', 'o
ptimal_mode', 'pattern_position', 're', 'six', 'to_bytestring', 'xrange']
>>> dir(qrcode.image)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__',
'__package__', '__path__', '__spec__', 'base']
>>> dir(qrcode.run_example)
['__annotations__', '__call__', '__class__', '__closure__', '__code__', '__defau
lts__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__',
 '__ge__', '__get__', '__getattribute__', '__globals__', '__gt__', '__hash__', '
__init__', '__init_subclass__', '__kwdefaults__', '__le__', '__lt__', '__module_
_', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex_
_', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']
This is the test python script I used to test this python module:
import qrcode
from  qrcode import *
qr = qrcode.QRCode(

img = qr.make_image(fill_color="black", back_color="white")
with open('my_qr_test.png', 'wb') as qrfile:
This is the output of QR file:

Saturday, May 26, 2018

Blender 3D and Roblox with Python .

I spend my free time with my son playing Roblox and in the meantime I try to introduce him to the world of computers.
However, you can download the player as a 3D object and use it as an avatar.
Here's a issue: The 3D object is hard to set with origins for animation but python and Blender 3D can easily solve this.
You can use BMesh.
As you know:
BMesh is the new Blender mesh system in 2.63, with full support for N-sided polygons instead of only triangles and quads.
The result of this download 3D object has a bad origin:

Let's see the source code:
import bpy
import bmesh
import mathutils 
from mathutils import Vector

context = bpy.context

def origin_to_bottom(obj):
    matrix_world = obj.matrix_world
    local_verts = [Vector(v[:]) for v in obj.bound_box]
    blender_mesh =
    x, y, z = 0, 0, 0
    l = len(local_verts)
    z = min([v.z for v in local_verts])
    local_origin = Vector((0, 0, 0))
    global_origin = matrix_world * local_origin
    for v in blender_mesh.verts:
        v.coord = v.coord - local_origin
    matrix_world.translation = global_origin

mesh_objs = [mesh_object for mesh_object in context.selected_objects if mesh_object.type == 'MESH']

for my_objects in mesh_objs:
The result is this:

Saturday, April 28, 2018

Python 3.6.4 : Testing OpenCV default Hough Line Transform.

This tutorial is about Hough Line Transform and OpenCV python module.
This can be a good example for Hough Line Transform.
See the source code:
import cv2
import numpy as np
img = cv2.imread('test_lines.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# filter black and gray pixels
thresh = cv2.threshold(gray, 100, 255, cv2.THRESH_BINARY_INV)[1]

# find lines
lines = cv2.HoughLinesP(thresh, 1, np.pi/180,360,18)

# output lines onto image
for line in lines:
    x1,y1,x2,y2 = line[0]

# show image
cv2.imshow('threshold houghlines', img)
This is the result for : test_lines.jpg .

You can test by make changes into this line of code:
lines = cv2.HoughLinesP(thresh, 1, np.pi/180,360,18)
According with documentation the changes are influenced by the range parameters.

Friday, April 27, 2018

Python 3.6.4 : Testing the wit python module .

Today I tested the wit python module.
This python module is a Python library for
You can use for this issues:
  • Bots
  • Mobile apps
  • Home automation
  • Wearable devices
  • Robots
This support languages like:
Afrikaans, Albanian, Arabic, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Catalan, Central Khmer, Chinese, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, Georgian, German, Greek, Hausa, Hebrew, Hindi, Hungarian, Icelandic, Igbo, Indonesian, Inuktitut, Italian, Japanese, Kannada, Kinyarwanda, Korean, Latin, Latvian, Lithuanian, Macedonian, Malay, Maori, Mongolian, Norwegian, Persian, Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovenian, Somali, Southern Ndebele, Southern Sotho, Spanish, Swahili, Swati, Swedish, Tagalog, Tamil, Thai, Tsonga, Tswana, Turkish, Ukrainian, Urdu, Uzbek, Venda, Vietnamese, Xhosa, Yoruba and Zulu.
About Wit is free, including for commercial use. So both private and public Wit apps are free and are governed our terms.
For this tutorial I use python 3.6.4 , see :
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)]on win32
Type "help", "copyright", "credits" or "license" for more information.
The install of wit python module is simple:
C:\Python364>cd Scripts

C:\Python364\Scripts>pip install wit
Collecting wit
Successfully built wit
Installing collected packages: wit
Successfully installed wit-5.1.0

Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)]
 on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from wit import Wit
>>> w=Wit('3ODKKNB---------')
>>> w.message('Python este un limbaj de programare')
{'_text': 'Python este un limbaj de programare', 'entities': {}, 'msg_id': '0pNT
>>> dir(w)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__form
at__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_s
ubclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__',
 '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclas
shook__', '__weakref__', '_sessions', 'access_token', 'interactive', 'logger', '
message', 'speech']
>>> file = open('C:\Python364\hello_world.wav', 'rb')
>>> w.speech(file)
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python364\lib\site-packages\wit\", line 88, in speech
    data=audio_file, headers=headers)
  File "C:\Python364\lib\site-packages\wit\", line 41, in req
    ' (' + rsp.reason + ')')
wit.wit.WitError: Wit responded with status: 400 (Bad Request)
The error has a open issue.
It does not seem to work properly.
There are some open issues for this python module.
The examples on the internet are not very concise with how to use this python module.

Thursday, April 26, 2018

Python Qt5 - menu example.

This simple tutorial is about PyQt5 and menu window example.
I have a similar examples with Qt4 on this blog.
The main reason for this tutorial come from the idea of simplicity and reuse the source code from PyQt4 and PyQt5.
I do not know if there are significant changes to the Qt5 base IU. However, it is good to check on the official pages. Let's look at the example with comments specific to source code lines:
# -*- coding: utf-8 -*-
Created on Thu Apr 26 17:20:02 2018

@author: catafest
import sys
from PyQt5.QtWidgets import QMainWindow, QAction, qApp, QApplication, QDesktopWidget
from PyQt5.QtGui import QIcon

class Example(QMainWindow):
    #init the example class to draw the window application    
    def __init__(self):
    #create the def center to select the center of the screen         
    def center(self):
        # geometry of the main window
        qr = self.frameGeometry()
        # center point of screen
        cp = QDesktopWidget().availableGeometry().center()
        # move rectangle's center point to screen's center point
        # top left of rectangle becomes top left of window centering it
    #create the init UI to draw the application
    def initUI(self):               
        #create the action for the exit application with shortcut and icon
        #you can add new action for File menu and any actions you need
        exitAct = QAction(QIcon('exit.png'), '&Exit', self)        
        exitAct.setStatusTip('Exit application')
        #create the status bar for menu 
        #create the menu with the text File , add the exit action 
        #you can add many items on menu with actions for each item
        menubar = self.menuBar()
        fileMenu = menubar.addMenu('&File')
        #resize the window application 
        self.resize(640, 480)
        #draw on center of the screen
        #add title on windows application 
        self.setWindowTitle('Simple menu')
        #show the application
        #close the UI class
if __name__ == '__main__':
    #create the application 
    app = QApplication(sys.argv)
    #use the UI with new  class
    ex = Example()
    #run the UI 
The result of this code is this:

Monday, April 2, 2018

The jdoodle online tool for python 3.

This online tool from jdoodle website let you to programming online with python 3 version.
To see all python modules used by this editor just add this python script and use Execute button.
import sys
import os 

Tuesday, March 20, 2018

Python 3.6.4 : Testing PyQt5 with Spyder I.D.E.

Today I tested the PyQt5 python module with python version 3.6.4 .
The script was created and tested with Spyder I.D.E. version 3.2.8 .
The PyQt5 version is:
from PyQt5.Qt import PYQT_VERSION_STR
print("PyQt version:", PYQT_VERSION_STR)
PyQt version: 5.9.2
This is the python script:
import sys
from PyQt5.QtWidgets import (QWidget, QCalendarWidget,
    QLabel, QApplication)
from PyQt5.QtCore import QDate
class Calendar(QWidget):
    def __init__(self):
    def initUI(self):
        calendar = QCalendarWidget(self)
        calendar.move(0, 15)
        calendar.currentPageChanged[int, int].connect(self.currentPageChanged)
        self.lbl = QLabel(self)
        date = calendar.selectedDate()
        self.lbl.move(0, 0)
        self.setGeometry(300, 300, 300, 300)
    def showDate(self, date):
    def currentPageChanged(self, year, month):
        print(year, month)
if __name__ == '__main__':
    app = QApplication(sys.argv)
    calendar_test = Calendar()
The result of this script is:

Saturday, March 17, 2018

The Google Cloud SDK - part 003 .

The webapp2 is a lightweight Python web framework compatible with Google App Engine’s.
The webapp2 project, by Rodrigo Moraes, started as a fork of the App Engine webapp framework.
The webapp2 includes a number of features such as improved support for URI routing, session management and localization.
You can see google documentation about this python module this link.
They say:
"webapp2 is compatible with the WSGI standard for Python web applications. You don't have to use webapp2 to write Python applications for App Engine. Other web application frameworks, such as Django, work with App Engine, and App Engine supports any Python code that uses the CGI standard. "
This is default start python example from Google Cloud SDK tested in the last tutorial.
import webapp2

class MainPage(webapp2.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.write('Hello, World!')

app = webapp2.WSGIApplication([
    ('/', MainPage),
], debug=True)
Remember how to deploy your project to google:
C:\Python27>cd python-docs-samples\appengine\standard\hello_world

C:\Python27\python-docs-samples\appengine\standard\hello_world>gcloud app deploy app.yaml
Services to deploy:

descriptor:      [C:\Python27\python-docs-samples\appengine\standard\hello_world\app.yaml]
source:          [C:\Python27\python-docs-samples\appengine\standard\hello_world]
target project:  [xxxxxx]
target service:  [default]
target version:  [xxxxxxxxxxxxx]
target url:      []

Do you want to continue (Y/n)?  y

Beginning deployment of service [default]...
Now I make some changes into file to show you how easy is to use it.
This file: is set into setting file app.yaml like: script:
Let's make some changes.

The default project is set with --promote true the result is : after a number of upload you got this error:
ERROR: ( Error Response: [400] Your app may not have more than 15 versions.
Please delete one of the existing versions before trying to create a new version.
To fix go to App Engine - Versions with selected versions and press Delete button.
Then you can make the upload with command :
gcloud app deploy app.yaml
Also you can use this:
gcloud app deploy app.yaml --stop-previous-version
Some info about your project can be see with this command:
gcloud config list
You can see the gcloud versions with:
gcloud version
I tested also with python version 3.6.4 :
C:\Python364\Scripts>pip install webapp2
Requirement already satisfied: webapp2 in c:\python364\lib\site-packages
      2 python-dateutil-2.7.0 setuptools-39.0.0 

Testing the new Django web framework with Fedora 27 distro.

Today I make a new tutorial about Django web framework with Fedora 27 distro.
The Fedora 27 distro come with the Django web framework version 2.0.3 and python 3.6.4 .
I follow first steps to install and create settings to run the website.
You can see the full tutorial here.

Wednesday, March 14, 2018

The regex online tool for python and any programming languages.

Today I tested this online tool.
Is a tool for regular expression (regex or regexp for short) for many progamming languages.
This programming languages are : php, javascript , golang and python.
The tool is easy to use it.
First you need to select the programming language that is used for regular expression.
The next step is to put the regular expression into edit box and add your text to be parsed by this regular expression.
For example if you use this inputs for regular expression:
([a-zA-Z]+) \d+
and this text example:
March 7 1976, June 1, August 9, Dec 25
the result output will be this:
March , June , August , Dec

Sunday, March 11, 2018

Python 3.6.4 : Testing OpenCV default GrabCut algorithm.

The main goal for me was to test the new install of python 3.6.4 and python modules with Windows operating system version 8.1.
For this tutorial I chose these python modules: cv2, numpy and matplotlib .
I have tested the GrabCut algorithm article from here.
The article comes with a python script that includes the modules I tested in this programming language.
They tell us:
User inputs the rectangle. Everything outside this rectangle will be taken as sure background (That is the reason it is mentioned before that your rectangle should include all the objects). Everything inside rectangle is unknown. Similarly any user input specifying foreground and background are considered as hard-labelling which means they won't change in the process.
From my point of view, it is not a very successful algorithm to crop off the background, but is working well.
import numpy as np
import cv2
from matplotlib import pyplot as plt

img = cv2.imread('test_python_opencv.jpg')
mask = np.zeros(img.shape[:2],np.uint8)

bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)

rect = (57,58,476,741)

mask2 = np.where((mask==2)|(mask==0),0,1).astype('uint8')
img = img*mask2[:,:,np.newaxis]

The intersection areas are eliminated exactly as in the documentation.
See my first test on an image taken from the internet.

Saturday, March 3, 2018

News: The Spyder IDE - new release .

Many python users use the Spyder IDE.
This IDE comes with many features and is easy to use, see wikipedia page:
Spyder (formerly Pydee[3]) is an open source cross-platform integrated development environment (IDE) for scientific programming in the Python language. Spyder integrates NumPy, SciPy, Matplotlib and IPython, as well as other open source software.[4][5] It is released under the MIT license.[6]
Six days ago, a release of this IDE with version 3.2.7 was announced.
This IDE can be download from github page.

Friday, February 23, 2018

Use IMDB website with IMDbPY python module .

This python package is written in pure Python 3 to access the IMDb's database and used it.

You can read about this python module from github docs weboage
The development team come with this DISCLAIMER:

IMDbPY and the authors are not affiliated with Internet Movie Database Inc.

IMDb is a trademark of Internet Movie Database Inc. and all contents
and data included on the IMDb's site is the property of IMDb or its
content suppliers and protected by United States and international
copyright laws.

Please, read the IMDb's conditions of use in their website:
- any other notice in the site.

First I start the install process with pip tool:
C:\Python364\Scripts>pip install IMDbPY
Requirement already satisfied: IMDbPY in c:\python364\lib\site-packages
Requirement already satisfied: lxml in c:\python364\lib\site-packages (from IMDbPY)
Requirement already satisfied: sqlalchemy-migrate in c:\python364\lib\site-packages (from IMDbPY)
Requirement already satisfied: SQLAlchemy in c:\python364\lib\site-packages (from IMDbPY)
Requirement already satisfied: pbr>=1.8 in c:\python364\lib\site-packages (from sqlalchemy-migrate->IMDbPY)
Requirement already satisfied: decorator in c:\python364\lib\site-packages (from sqlalchemy-migrate->IMDbPY)
Requirement already satisfied: six>=1.7.0 in c:\python364\lib\site-packages (from sqlalchemy-migrate->IMDbPY)
Requirement already satisfied: sqlparse in c:\python364\lib\site-packages (from sqlalchemy-migrate->IMDbPY)
Requirement already satisfied: Tempita>=0.4 in c:\python364\lib\site-packages (from sqlalchemy-migrate->IMDbPY)
This is my source code to test it and working well.
# start with IMDb python class
from imdb import IMDb
imd = IMDb('http')
# search movies by title
# and show the long imdb canonical title and movieID of the results.
title = imd.search_movie("Under the Dome")
for item in title:
   print(item['long imdb canonical title'], item.movieID)
# search for a person
for person in imd.search_person("Ana de Armas"):
    print(person.personID, person['name'])
# get 5 movies tagged with a keyword
movies_keyword = imd.get_keyword('novel', results=5)
for item in movies_keyword:
   print(item['long imdb canonical title'], item.movieID)
# get top 250  from top movies
top250 = imd.get_top250_movies()
for item in top250:
   print(item['long imdb canonical title'], item.movieID)
print("top 250 -=> ")
# get bottom 100 from top movies
bottom100 = imd.get_bottom100_movies()
print("bottom 100 -=> ")
for item in top250:
   print(item['long imdb canonical title'], item.movieID)

Thursday, February 8, 2018

Python 2.7 : Testing the pefile python module.

The pefile is a python module to read and work with PE (Portable Executable) files.
The install of this python module is very easy with the pip tool.
I tested the default example create with FASM to see if this is working well:
This is the source code:
; Example of 64-bit PE program
format PE64 GUI
entry start

section '.text' code readable executable

        sub     rsp,8*5         ; reserve stack for API use and make stack dqword aligned

        mov     r9d,0
        lea     r8,[_caption]
        lea     rdx,[_message]
        mov     rcx,0
        call    [MessageBoxA]

        mov     ecx,eax
        call    [ExitProcess]

section '.data' data readable writeable

  _caption db 'Win64 assembly program',0
  _message db 'Hello World!',0

section '.idata' import data readable writeable

  dd 0,0,0,RVA kernel_name,RVA kernel_table
  dd 0,0,0,RVA user_name,RVA user_table
  dd 0,0,0,0,0

    ExitProcess dq RVA _ExitProcess
    dq 0
    MessageBoxA dq RVA _MessageBoxA
    dq 0

  kernel_name db 'KERNEL32.DLL',0
  user_name db 'USER32.DLL',0

  _ExitProcess dw 0
    db 'ExitProcess',0
  _MessageBoxA dw 0
    db 'MessageBoxA',0  
The python script I used to test this python module is this:
import sys
from sys import argv
import mmap
import pefile

fp = open(argv[1],"r")
map = mmap.mmap(fp.fileno(),0,access=mmap.ACCESS_READ)
pe = pefile.PE(data=map[:])
print pe
The output is this:
C:\Python27>python.exe PE64DEMO.EXE
----------Parsing Warnings----------

Byte 0x00 makes up 87.5488% of the file's contents. This may indicate truncation / malformation.


0x0 0x0 e_magic: 0x5A4D
0x2 0x2 e_cblp: 0x80
0x4 0x4 e_cp: 0x1
0x6 0x6 e_crlc: 0x0
0x8 0x8 e_cparhdr: 0x4
0xA 0xA e_minalloc: 0x10
0xC 0xC e_maxalloc: 0xFFFF
0xE 0xE e_ss: 0x0
0x10 0x10 e_sp: 0x140
0x12 0x12 e_csum: 0x0
0x14 0x14 e_ip: 0x0
0x16 0x16 e_cs: 0x0
0x18 0x18 e_lfarlc: 0x40
0x1A 0x1A e_ovno: 0x0
0x1C 0x1C e_res:
0x24 0x24 e_oemid: 0x0
0x26 0x26 e_oeminfo: 0x0
0x28 0x28 e_res2:
0x3C 0x3C e_lfanew: 0x80


0x80 0x0 Signature: 0x4550


0x84 0x0 Machine: 0x8664
0x86 0x2 NumberOfSections: 0x3
0x88 0x4 TimeDateStamp: 0x5A1954AF [Sat Nov 25 11:31:59 2017 UTC]
0x8C 0x8 PointerToSymbolTable: 0x0
0x90 0xC NumberOfSymbols: 0x0
0x94 0x10 SizeOfOptionalHeader: 0xF0
0x96 0x12 Characteristics: 0x2F


0x98 0x0 Magic: 0x20B
0x9A 0x2 MajorLinkerVersion: 0x1
0x9B 0x3 MinorLinkerVersion: 0x49
0x9C 0x4 SizeOfCode: 0x200
0xA0 0x8 SizeOfInitializedData: 0x400
0xA4 0xC SizeOfUninitializedData: 0x0
0xA8 0x10 AddressOfEntryPoint: 0x1000
0xAC 0x14 BaseOfCode: 0x1000
0xB0 0x18 ImageBase: 0x400000
0xB8 0x20 SectionAlignment: 0x1000
0xBC 0x24 FileAlignment: 0x200
0xC0 0x28 MajorOperatingSystemVersion: 0x1
0xC2 0x2A MinorOperatingSystemVersion: 0x0
0xC4 0x2C MajorImageVersion: 0x0
0xC6 0x2E MinorImageVersion: 0x0
0xC8 0x30 MajorSubsystemVersion: 0x5
0xCA 0x32 MinorSubsystemVersion: 0x0
0xCC 0x34 Reserved1: 0x0
0xD0 0x38 SizeOfImage: 0x4000
0xD4 0x3C SizeOfHeaders: 0x200
0xD8 0x40 CheckSum: 0xECAF
0xDC 0x44 Subsystem: 0x2
0xDE 0x46 DllCharacteristics: 0x0
0xE0 0x48 SizeOfStackReserve: 0x1000
0xE8 0x50 SizeOfStackCommit: 0x1000
0xF0 0x58 SizeOfHeapReserve: 0x10000
0xF8 0x60 SizeOfHeapCommit: 0x0
0x100 0x68 LoaderFlags: 0x0
0x104 0x6C NumberOfRvaAndSizes: 0x10

----------PE Sections----------

0x188 0x0 Name: .text
0x190 0x8 Misc: 0x2D
0x190 0x8 Misc_PhysicalAddress: 0x2D
0x190 0x8 Misc_VirtualSize: 0x2D
0x194 0xC VirtualAddress: 0x1000
0x198 0x10 SizeOfRawData: 0x200
0x19C 0x14 PointerToRawData: 0x200
0x1A0 0x18 PointerToRelocations: 0x0
0x1A4 0x1C PointerToLinenumbers: 0x0
0x1A8 0x20 NumberOfRelocations: 0x0
0x1AA 0x22 NumberOfLinenumbers: 0x0
0x1AC 0x24 Characteristics: 0x60000020
Entropy: 0.540255 (Min=0.0, Max=8.0)
MD5 hash: 54edeb1437149ccc09183b623e3be7b8
SHA-1 hash: c473f3db5ca81084db3489ab3519832ded9cc28c
SHA-256 hash: 74e9ff7d6902292d9a8ad93174aef46596f8f1fe9eb5dd72b9ebc99f8bd2ecfb
SHA-512 hash: 070610baa66d6efcbb2cc7e935c2afd2686068818c00b772b3e62de103389cecbc6c309976e10860a974532a1018fba9da50effb64c60f533433dbb808ba088c

0x1B0 0x0 Name: .data
0x1B8 0x8 Misc: 0x24
0x1B8 0x8 Misc_PhysicalAddress: 0x24
0x1B8 0x8 Misc_VirtualSize: 0x24
0x1BC 0xC VirtualAddress: 0x2000
0x1C0 0x10 SizeOfRawData: 0x200
0x1C4 0x14 PointerToRawData: 0x400
0x1C8 0x18 PointerToRelocations: 0x0
0x1CC 0x1C PointerToLinenumbers: 0x0
0x1D0 0x20 NumberOfRelocations: 0x0
0x1D2 0x22 NumberOfLinenumbers: 0x0
0x1D4 0x24 Characteristics: 0xC0000040
Entropy: 0.627189 (Min=0.0, Max=8.0)
MD5 hash: 6684d4efed7dc864e5bbb0280faa841b
SHA-1 hash: 0214a59237a9020d3fa41419107a59f276a95e5f
SHA-256 hash: 23ae47e7bfb842935b35775428fe9c5df5c3f46fa46c2da2e93a27ba031ae091
SHA-512 hash: 60eeefcb47e1e63584342049a66d4539ab4b580190faf9d2629e0db1336933835c207e419060cce08cfec430e2f1e13a90cac7abfb05679ed5d84dac8997f12f

0x1D8 0x0 Name: .idata
0x1E0 0x8 Misc: 0x90
0x1E0 0x8 Misc_PhysicalAddress: 0x90
0x1E0 0x8 Misc_VirtualSize: 0x90
0x1E4 0xC VirtualAddress: 0x3000
0x1E8 0x10 SizeOfRawData: 0x200
0x1EC 0x14 PointerToRawData: 0x600
0x1F0 0x18 PointerToRelocations: 0x0
0x1F4 0x1C PointerToLinenumbers: 0x0
0x1F8 0x20 NumberOfRelocations: 0x0
0x1FA 0x22 NumberOfLinenumbers: 0x0
0x1FC 0x24 Characteristics: 0xC0000040
Entropy: 0.996929 (Min=0.0, Max=8.0)
MD5 hash: 073b9b0656f7ca77d968f183a1ceb909
SHA-1 hash: acefe438c7bfef7362b87519349c5a7b251aa43d
SHA-256 hash: 016761b2d3b31ed8eeddccc9f56e6338978171a0082c066cbf2b28cecd77566a
SHA-512 hash: a5fb7ace9108f63c96c9da239fc5077106cf3ffe8e31a1ab0a11b589a8e6af9e66d23c38060c157a3e34125bc5af495c770e48bc00172a5c8ec78b34794628b3


0x108 0x0 VirtualAddress: 0x0
0x10C 0x4 Size: 0x0
0x110 0x0 VirtualAddress: 0x3000
0x114 0x4 Size: 0x90
0x118 0x0 VirtualAddress: 0x0
0x11C 0x4 Size: 0x0
0x120 0x0 VirtualAddress: 0x0
0x124 0x4 Size: 0x0
0x128 0x0 VirtualAddress: 0x0
0x12C 0x4 Size: 0x0
0x130 0x0 VirtualAddress: 0x0
0x134 0x4 Size: 0x0
0x138 0x0 VirtualAddress: 0x0
0x13C 0x4 Size: 0x0
0x140 0x0 VirtualAddress: 0x0
0x144 0x4 Size: 0x0
0x148 0x0 VirtualAddress: 0x0
0x14C 0x4 Size: 0x0
0x150 0x0 VirtualAddress: 0x0
0x154 0x4 Size: 0x0
0x158 0x0 VirtualAddress: 0x0
0x15C 0x4 Size: 0x0
0x160 0x0 VirtualAddress: 0x0
0x164 0x4 Size: 0x0
0x168 0x0 VirtualAddress: 0x0
0x16C 0x4 Size: 0x0
0x170 0x0 VirtualAddress: 0x0
0x174 0x4 Size: 0x0
0x178 0x0 VirtualAddress: 0x0
0x17C 0x4 Size: 0x0
0x180 0x0 VirtualAddress: 0x0
0x184 0x4 Size: 0x0

----------Imported symbols----------

0x600 0x0 OriginalFirstThunk: 0x0
0x600 0x0 Characteristics: 0x0
0x604 0x4 TimeDateStamp: 0x0 [Thu Jan 01 00:00:00 1970 UTC]
0x608 0x8 ForwarderChain: 0x0
0x60C 0xC Name: 0x305C
0x610 0x10 FirstThunk: 0x303C

KERNEL32.DLL.ExitProcess Hint[0]

0x614 0x0 OriginalFirstThunk: 0x0
0x614 0x0 Characteristics: 0x0
0x618 0x4 TimeDateStamp: 0x0 [Thu Jan 01 00:00:00 1970 UTC]
0x61C 0x8 ForwarderChain: 0x0
0x620 0xC Name: 0x3069
0x624 0x10 FirstThunk: 0x304C

USER32.DLL.MessageBoxA Hint[0]

Sunday, February 4, 2018

The collections python module .

This module named collections implements some nice data structures which will help you to solve various real life problems.
Let's start to see the content of this python module:

C:\Users\catafest>cd C:\Python27\

Python 2.7 (r27:82525, Jul  4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import collections
>>> from collections import *
>>> dir(collections)
['Callable', 'Container', 'Counter', 'Hashable', 'ItemsView', 'Iterable', 'Iterator', 'KeysView',
 'Mapping', 'MappingView', 'MutableMapping', 'MutableSequence', 'MutableSet', 'OrderedDict', 'Sequence',
 'Set', 'Sized', 'ValuesView', '__all__', '__builtins__', '__doc__', '__file__', '__name__', '__package__'
, '_abcoll', '_chain', '_eq', '_heapq', '_ifilter', '_imap', '_iskeyword', '_itemgetter', '_repeat', 
'_starmap', '_sys', 'defaultdict', 'deque', 'namedtuple']
Now I will tell you about some
First is Counter and is a dict subclass which helps to count hashable objects.
The elements are stored as dictionary keys and counts are stored as values which can be zero or negative.
Next is defaultdict and is a dictionary object which provides all methods provided by dictionary.
This takes first argument (default_factory) as default data type for the dictionary.
The namedtuple helps to have meaning of each position in a tuple.
This allow us to code with better readability and self-documenting code.
Let's try some examples:
>>> from collections import Counter
>>> from collections import defaultdict
>>> from collections import namedtuple
>>> import re
>>> path = 'C:/yara_reg_rundll32.txt'
>>> output = re.findall('\w+', open(path).read().lower())
>>> Counter(output).most_common(5)
[('a', 2), ('nocase', 2), ('javascript', 2), ('b', 2), ('rundll32', 2)]
>>> d = defaultdict(list)
>>> colors = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]
>>> for k, v in colors:
...     d[k].append(v)
>>> d.items()
[('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])]
>>> Vertex = namedtuple('vertex', ['x', 'y'])
>>> v = Vertex(5,y = 9)
>>> v
vertex(x=5, y=9)
>>> v.x*v.y
>>> v[0]
>>> v[0]+v[1]
>>> x,y = v
>>> v
vertex(x=5, y=9)
>>> x
>>> y
The content of the yara_reg_rundll32.txt file is:
rule poweliks_rundll32_exe_javascript
description = "detect Poweliks' autorun rundll32.exe javascript:..."
$a = "rundll32.exe" nocase
$b = "javascript" nocase
$a and $b

I used vertex variables into my example because can be used with Blender 3D.
You can see many examples at official documentation website.

Sunday, January 14, 2018

The trinket website for learning.

This website come with this feature:
Trinket lets you run and write code in any browser, on any device.
Trinkets work instantly, with no need to log in, download plugins, or install software.
Easily share or embed the code with your changes when you're done.

  • Just create Your Free Account then use the web interface to play with turtle python module:
  • Trinket lets you run and write code in any browser, on any device.
  • Trinkets work instantly, with no need to log in, download plugins, or install software.
  • Easily share or embed the code with your changes when you're done.

Friday, January 12, 2018

Python 2.7 : Python and BigQuery service object.

Here's another tutorial about python and google. I thought it would be useful for the beginning of 2018.
The goole team tell us:

What is BigQuery?

Storing and querying massive datasets can be time consuming and expensive without the right hardware and infrastructure. Google BigQuery is an enterprise data warehouse that solves this problem by enabling super-fast SQL queries using the processing power of Google's infrastructure. Simply move your data into BigQuery and let us handle the hard work. You can control access to both the project and your data based on your business needs, such as giving others the ability to view or query your data.

This tutorial it follows more precisely the steps from here.
First of all, you must create an authentication file by using the Create service account from your google project.
Go to Google Console, navigate to the Create service account key page.
From the Service account dropdown, select New service account.
Input a name into the form field.
From the Role dropdown, select Project and Owner.
The result is a JSON file type (this is for authentificate with google) download it rename and put into your project folder .
Like into the next image:

Now, select from left area the Library do add the BigQuery API, try this link.
Search for BigQuery API and the use the button ENABLE to use it.
The next step is to install this python modules: pyopenssl and google-cloud-bigquery.
C:\Python27\Scripts>pip install -U pyopenssl
C:\Python27\Scripts>pip install --upgrade google-cloud-bigquery
Add this JSON file to windows path from my test folder:
set GOOGLE_APPLICATION_CREDENTIALS=C:\test\python_doc.json
Because my json file is named python_doc.json then this is the name I will use with my python script.
Let's see the script:
import google
from import bigquery

def query_shakespeare():
    client = bigquery.Client()
    client = client.from_service_account_json('python_doc.json')
    query_job = client.query("""
        SELECT corpus AS title, COUNT(*) AS unique_words
        FROM `bigquery-public-data.samples.shakespeare`
        GROUP BY title
        ORDER BY unique_words DESC
        LIMIT 10""")

    results = query_job.result()  # Waits for job to complete.

    for row in results:
        print("{}: {}".format(row.title, row.unique_words))

if __name__ == '__main__':
The result is:
hamlet: 5318
kinghenryv: 5104
cymbeline: 4875
troilusandcressida: 4795
kinglear: 4784
kingrichardiii: 4713
2kinghenryvi: 4683
coriolanus: 4653
2kinghenryiv: 4605
antonyandcleopatra: 4582
NOTE: Take care of the json file because it gives access to your google account and tries to use the restrictions according to the application's requirements.

Thursday, January 4, 2018

Python 2.7 : InsecurePlatformWarning error.

This is not a common error and can be solve it easy like any python issue.
The result of this error can be show like into the next example:
SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension 
to TLS is not available on this platform. This may cause the server to present an incorrect TLS 
certificate, which can cause validation failures. You can upgrade to a newer version of Python to
 solve this. For more information, see
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from
 configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade 
to a newer version of Python to solve this. For more information, see https://urllib3.readthe
The simple way to test this python error is to install this python modules:
pip install urllib3 
pip install requests
This last python module named requests come with:
Successfully installed certifi-2017.11.5 chardet-3.0.4 idna-2.6 requests-2.18.4
What is this python module named requests?
Is a security the requests python module inject pyopenssl into urllib3
Python 2.7 (r27:82525, Jul  4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> help()

Welcome to Python 2.7!  This is the online help utility.

If this is your first time using Python, you should definitely check out
the tutorial on the Internet at

Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules.  To quit this help utility and
return to the interpreter, just type "quit".

To get a list of available modules, keywords, or topics, type "modules",
"keywords", or "topics".  Each module also comes with a one-line summary
of what it does; to list the modules whose summaries contain a given word
such as "spam", type "modules spam".

help> modules requests

Here is a list of matching modules.  Enter any module name to get more help.

pip._vendor.cachecontrol.controller - The httplib2 algorithms ported for use with requests.
pip._vendor.requests - Requests HTTP library
pip._vendor.requests.adapters - requests.adapters
pip._vendor.requests.api - requests.api
pip._vendor.requests.auth - requests.auth
pip._vendor.requests.certs - requests.certs
pip._vendor.requests.compat - requests.compat
pip._vendor.requests.cookies - requests.cookies
pip._vendor.requests.exceptions - requests.exceptions
pip._vendor.requests.hooks - requests.hooks
pip._vendor.requests.models - requests.models
pip._vendor.requests.packages.chardet.chardetect - Script which takes one or more file paths 
and reports on their detected
pip._vendor.requests.packages.urllib3 - urllib3 - Thread-safe connection pooling and re-using.
pip._vendor.requests.packages.urllib3.contrib.ntlmpool - NTLM authenticating pool, 
contributed by erikcederstran
pip._vendor.requests.packages.urllib3.contrib.socks - SOCKS support for urllib3
pip._vendor.requests.packages.urllib3.packages.six - Utilities for writing code that runs on 
Python 2 and 3
pip._vendor.requests.packages.urllib3.packages.ssl_match_hostname._implementation - The match_hostname() 
function from Python 3.3.3, essential when using SSL.
pip._vendor.requests.sessions - requests.session
pip._vendor.requests.structures - requests.structures
pip._vendor.requests.utils - requests.utils
requests - Requests HTTP Library
requests._internal_utils - requests._internal_utils
requests.adapters - requests.adapters
requests.api - requests.api
requests.auth - requests.auth
requests.certs - requests.certs
requests.compat - requests.compat
requests.cookies - requests.cookies
requests.exceptions - requests.exceptions - Module containing bug report helper(s).
requests.hooks - requests.hooks
requests.models - requests.models
requests.sessions - requests.session
requests.structures - requests.structures
requests.utils - requests.utils
You are now leaving help and returning to the Python interpreter.
If you want to ask for help on a particular object directly from the
interpreter, you can type "help(object)".  Executing "help('string')"
has the same effect as typing a particular string at the help> prompt.

Wednesday, January 3, 2018

The ebooklib python module .

Happy new year 2018 !
The official webpage of this python module come with this intro:
EbookLib is a Python library for managing EPUB2/EPUB3 and Kindle files. It's capable of reading and writing EPUB files programmatically (Kindle support is under development).
First the installation of this python module named ebooklib.
C:\>cd Python27

C:\Python27>cd Script
The system cannot find the path specified.

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install ebooklib
Collecting ebooklib
  Downloading EbookLib-0.16.tar.gz
Requirement already satisfied: lxml in c:\python27\lib\site-packages (from ebooklib)
Requirement already satisfied: six in c:\python27\lib\site-packages (from ebooklib)
Installing collected packages: ebooklib
  Running install for ebooklib ... done
Successfully installed ebooklib-0.16
If you don't see the Scripts folder into your Python27 folder you need do install pip tool.
Just download the script into your Python27 folder and run it with python .
Let's test some default example:
The next step is to test a simple example:
from ebooklib import epub

book = epub.EpubBook()

# set metadata
book.set_title('Sample book')

book.add_author('Author Python')
book.add_author('catafest', file_as='', role='writer', uid='author')

# create chapter
c1 = epub.EpubHtml(title='Intro', file_name='chap_01.xhtml', lang='hr')
c1.content=u'Intro heading.Python is a interpreted high-level programming language ...'

# add chapter

# define Table Of Contents
book.toc = (epub.Link('chap_01.xhtml', 'Introduction', 'intro'),
(epub.Section('Simple book'),
(c1, ))

# add default NCX and Nav file

# define CSS style
style = 'BODY {color: white;}'
nav_css = epub.EpubItem(uid="style_nav", file_name="style/nav.css", media_type="text/css", content=style)

# add CSS file

# basic spine
book.spine = ['nav', c1]

# write to the file
epub.write_epub('test.epub', book, {})
You can update and make more good your epub book with HTML5 tags .
I used this example with headings and paragraph to change the text, see the result: