The Google Cloud Pub/Sub python module.

This is a test of google feature from cloud.google.com/pubsub web page.
The Google development team tell us about this service:
The Google Cloud Pub/Sub service allows applications to exchange messages reliably, quickly, and asynchronously. To accomplish this, a producer of data publishes a message to a Cloud Pub/Subtopic. A subscriber client then creates a subscription to that topic and consumes messages from the subscription. Cloud Pub/Sub persists messages that could not be delivered reliably for up to seven days. This page shows you how to get started publishing messages with Cloud Pub/Sub using client libraries.
The simple idea about this is:
Publisher applications can send messages to a topic, and other applications can subscribe to that topic to receive the messages.
I start with the installation of the python module using python version 2.7 and pip tool.

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install --upgrade google-cloud-pubsub
Collecting google-cloud-pubsub
  Downloading google_cloud_pubsub-0.28.4-py2.py3-none-any.whl (79kB)
    100% |################################| 81kB 300kB/s
...
Successfully installed google-cloud-pubsub-0.28.4 grpc-google-iam-v1-0.11.4 ply-3.8 
psutil-5.4.0 pyasn1-modules-0.1.5 setuptools-36.6.0

The next steps come with some settings on google console, see this google page.
The default settings can be started and set with this command: gcloud init .
You need to edit this settings and app.yaml at ~/src/.../appengine/flexible/pubsub$ nano app.yaml.
After that, you set all of this using the command gcloud app deploy you can see the output at https://[YOUR_PROJECT_ID].appspot.com.
The main goal of this tutorial was to start and run the Google Cloud Pub/Sub service with python and this has been achieved.

The online editor for python and google .

This is a good online editor for python and google.
Like any online editor, some python modules are not available for online security reasons.
I do not know what python modules are implemented in this online editor.
I tested just sys and math python modules.
The Google Apps come with this tool integration like application for Google drive:
Edit your python file directly in your browser:
- Save it to Google Drive integrated with Google Drive
- Test it in your browser with Skulpt
- Use autocompletion code (CTRL+SPACE)
- No registration required and totally free
- Export your file
- Work offline
New python libraries partially supported: numpy, matplotlib.

The capstone python module - disassembly framework.

The official python module comes with this info about this python module:
Capstone is a disassembly framework with the target of becoming the ultimate
the disasm engine for binary analysis and reversing in the security community.

Created by Nguyen Anh Quynh, then developed and maintained by a small community,
Capstone offers some unparalleled features:

- Support multiple hardware architectures: ARM, ARM64 (ARMv8), Mips, PPC & X86.

- Having clean/simple/lightweight/intuitive architecture-neutral API.

- Provide details on disassembled instruction (called “decomposer” by others).

- Provide semantics of the disassembled instruction, such as list of implicit
registers read & written.

- Implemented in pure C language, with lightweight wrappers for C++, Python,
Ruby, OCaml, C#, Java and Go available.

- Native support for Windows & *nix platforms (with OSX, Linux, *BSD & Solaris
have been confirmed).

- Thread-safe by design.

- Distributed under the open source BSD license.
Today I tested this python module with python version 2.7.
First I need to use a build of this python module from the official website.
I used binaries 32 bits like my python 2.7 and I tested with pip 2.7:

C:\Python27\Scripts>pip install capstone
Requirement already satisfied: capstone in c:\python27\lib\site-packages

Let's make a simple test with this python module:

C:\Python27>python.exe
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from capstone import (
...     Cs,
...     CS_ARCH_X86,
...     CS_MODE_32,
...     CS_OPT_SYNTAX_ATT,
... )
>>> mode=Cs(CS_ARCH_X86, CS_MODE_32)
>>> mode.syntax = CS_OPT_SYNTAX_ATT
>>> def D_ASM(code):
...     for address, size, mnemonic, op_str in mode.disasm_lite(code, offset=0x08048060):
...         print("0x{0:x}\t{1:d}\t{2:s}\t{3:s}".format(address, size,mnemonic, op_str))
...
>>> D_ASM(b"\xe1\x0b\x40\xb9\x20\x04\x81\xda\x20\x08\x02\x8b")
0x8048060       2       loope   0x804806d
0x8048062       1       incl    %eax
0x8048063       5       movl    $0xda810420, %ecx
0x8048068       2       andb    %cl, (%eax)

It seems to work very well.

The python-vlc python module.

The python module for vlc is named python-vlc.
This python module let you test libvlc API like the VLC video player.
You can install it easily with pip python tool.

C:\Python27\Scripts>pip2.7.exe install python-vlc
Collecting python-vlc
  Downloading python-vlc-1.1.2.tar.gz (201kB)
    100% |################################| 204kB 628kB/s
Installing collected packages: python-vlc
  Running setup.py install for python-vlc ... done
Successfully installed python-vlc-1.1.2

Let's see a simple example with this python module:

import os
import sys
import vlc
import pygame
 
def call_vlc(self, player):
 
    player.get_fps()
    player.get_time()
 
if len( sys.argv )< 2 or len( sys.argv )> 3:
        print 'Help: python vlc_001.py your_video.mp4'
else:
    pygame.init()
    screen = pygame.display.set_mode((800,600),pygame.RESIZABLE)
    pygame.display.get_wm_info()
    pygame.display.get_driver()

 
    # get path to movie specified as command line argument
    movie = os.path.expanduser(sys.argv[1])
    # see if movie is accessible
    if not os.access(movie, os.R_OK):
        print('Error: %s wrong read file: ' % movie)
        sys.exit(1)
 
    # make instane of VLC and create reference to movie.
    vlcInstance = vlc.Instance()
    media = vlcInstance.media_new(movie)
 
    # make new instance of vlc player
    player = vlcInstance.media_player_new()
 
    # start with a callback
    em = player.event_manager()
    em.event_attach(vlc.EventType.MediaPlayerTimeChanged, \
        call_vlc, player)
 
    # set pygame window id to vlc player
    win_id = pygame.display.get_wm_info()['window']
    if sys.platform == "win32": # for Windows
        player.set_hwnd(win_id)
 
    # load movie into vlc player instance
    player.set_media(media)
 
    # quit pygame mixer to allow vlc full access to audio device
    pygame.mixer.quit()
 
    # start movie play
    player.play()
 
    while player.get_state() != vlc.State.Ended:
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                sys.exit(2)

The base of this python script is to make an instance of vlc and put into the pygame display.
Another simple example:

C:\Python27>python.exe
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vlc
>>> inst = vlc.Instance()
Warning: option --plugin-path no longer exists.
Warning: option --plugin-path no longer exists.
>>> med = inst.media_new('rain.mp4')
>>> p = med.player_new_from_media()
>>> p.play()
0
>>>

The numba python module - part 002 .

Today I tested how fast is jit from numba python and fibonacci math function.
You will see strange output I got for some values.
First example:

import numba
from numba import jit
from timeit import default_timer as timer

def fibonacci(n):
    a, b = 1, 1
    for i in range(n):
        a, b = a+b, a
    return a
fibonacci_jit = jit(fibonacci)

start = timer()
fibonacci(100)
duration = timer() - start

startnext = timer()
fibonacci_jit(100)
durationnext = timer() - startnext

print(duration, durationnext)

The result of this run is:

C:\Python27>python numba_test_003.py
(0.00018731270733896962, 0.167499256682878)

C:\Python27>python numba_test_003.py
(1.6357787798437412e-05, 0.1683614083221368)

C:\Python27>python numba_test_003.py
(2.245186560569841e-05, 0.1758382003097716)

C:\Python27>python numba_test_003.py
(2.3093347480146938e-05, 0.16714964906130353)

C:\Python27>python numba_test_003.py
(1.5395564986764625e-05, 0.17471143739730277)

C:\Python27>python numba_test_003.py
(1.5074824049540363e-05, 0.1847134227837042)

As you can see the fibonacci function is not very fast.
The jit - just-in-time compile is very fast.
Let's see if the python source code may slow down.
Let's see the new source code with jit will not work well:

import numba
from numba import jit
from timeit import default_timer as timer

def fibonacci(n):
    a, b = 1, 1
    for i in range(n):
        a, b = a+b, a
    return a
fibonacci_jit = jit(fibonacci)

start = timer()
print fibonacci(100)
duration = timer() - start

startnext = timer()
print fibonacci_jit(100)
durationnext = timer() - startnext

print(duration, durationnext)

The result is this:

C:\Python27>python numba_test_003.py
927372692193078999176
1445263496
(0.0002334994022992635, 0.17628787910376)

C:\Python27>python numba_test_003.py
927372692193078999176
1445263496
(0.0006886307922204926, 0.17579169287387408)

C:\Python27>python numba_test_003.py
927372692193078999176
1445263496
(0.0008105123483657127, 0.18209553525407973)

C:\Python27>python numba_test_003.py
927372692193078999176
1445263496
(0.00025466830415606486, 0.17186550306131188)

C:\Python27>python numba_test_003.py
927372692193078999176
1445263496
(0.0007348174871807866, 0.17523103771560608)

The result for value 100 is not the same: 927372692193078999176 and 1445263496.
The first problem is:
The problem is that numba can't intuit the type of lookup. If you put a print nb.typeof(lookup) in your method, you'll see that numba is treating it as an object, which is slow.
The second problem is the output but can be from the same reason.
I test with value 5 and the result is :

C:\Python27>python numba_test_003.py
13
13
13
13
(0.0007258367409385072, 0.17057997338491704)

C:\Python27>python numba_test_003.py
13
13
(0.00033709872502270044, 0.17213235952108247)

C:\Python27>python numba_test_003.py
13
13
(0.0004836773333341886, 0.17184433415945508)

C:\Python27>python numba_test_003.py
13
13
(0.0006854233828482501, 0.17381272129120037)

The numba python module - part 001 .

Today I tested the numba python module.
This python module allows us to speed up applications with high-performance functions written directly in Python.
The numba python module works by generating optimized machine code using the LLVM compiler infrastructure at import time, runtime, or statically.
The code can be just-in-time compiled to native machine instructions, similar in performance to C, C++ and Fortran.
For the installation I used the pip tool:

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install numba
Collecting numba
  Downloading numba-0.35.0-cp27-cp27m-win32.whl (1.4MB)
    100% |################################| 1.4MB 497kB/s
...
Installing collected packages: singledispatch, funcsigs, llvmlite, numba
Successfully installed funcsigs-1.0.2 llvmlite-0.20.0 numba-0.35.0 singledispatch-3.4.0.3

C:\Python27\Scripts>pip install numpy
Requirement already satisfied: numpy in c:\python27\lib\site-packages

The example test from official website working well:
The example source code is:

from numba import jit
from numpy import arange

# jit decorator tells Numba to compile this function.
# The argument types will be inferred by Numba when function is called.
@jit
def sum2d(arr):
    M, N = arr.shape
    result = 0.0
    for i in range(M):
        for j in range(N):
            result += arr[i,j]
    return result

a = arange(9).reshape(3,3)
print(sum2d(a))

The result of this run python script is:

C:\Python27>python.exe numba_test_001.py
36.0

Another example using just-in-time compile is used with Numba’s jit function:

import numba
from numba import jit

def fibonacci(n):
    a, b = 1, 1
    for i in range(n):
        a, b = a+b, a
    return a

print fibonacci(10)

fibonacci_jit = jit(fibonacci)
print fibonacci_jit(14)

Also, you can use jit is as a decorator:

@jit
def fibonacci_jit(n):
    a, b = 1, 1
    for i in range(n):
        a, b = a+b, a

    return a

Numba is a complex python module because use compiling.
First, compiling takes time, but will work especially for small functions.
The Numba python module tries to do its best by caching compilation as much as possible though.
Another note: not all code is compiled equally.