Linux: OpenCV and using Lucas-Kanade Optical Flow function.

Fist I install OpenCV python module and I try using with Fedora 25.
I used python 2.7 version.

[root@localhost mythcat]# dnf install opencv-python.x86_64 
Last metadata expiration check: 0:21:12 ago on Sat Feb 25 23:26:59 2017.
Dependencies resolved.
================================================================================
 Package              Arch          Version                Repository      Size
================================================================================
Installing:
 opencv               x86_64        3.1.0-8.fc25           fedora         1.8 M
 opencv-python        x86_64        3.1.0-8.fc25           fedora         376 k
 python2-nose         noarch        1.3.7-11.fc25          updates        266 k
 python2-numpy        x86_64        1:1.11.2-1.fc25        fedora         3.2 M

Transaction Summary
================================================================================
Install  4 Packages

Total download size: 5.6 M
Installed size: 29 M
Is this ok [y/N]: y
Downloading Packages:
(1/4): opencv-python-3.1.0-8.fc25.x86_64.rpm    855 kB/s | 376 kB     00:00    
(2/4): opencv-3.1.0-8.fc25.x86_64.rpm           1.9 MB/s | 1.8 MB     00:00    
(3/4): python2-nose-1.3.7-11.fc25.noarch.rpm    543 kB/s | 266 kB     00:00    
(4/4): python2-numpy-1.11.2-1.fc25.x86_64.rpm   2.8 MB/s | 3.2 MB     00:01    
--------------------------------------------------------------------------------
Total                                           1.8 MB/s | 5.6 MB     00:03     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Installing  : python2-nose-1.3.7-11.fc25.noarch                           1/4 
  Installing  : python2-numpy-1:1.11.2-1.fc25.x86_64                        2/4 
  Installing  : opencv-3.1.0-8.fc25.x86_64                                  3/4 
  Installing  : opencv-python-3.1.0-8.fc25.x86_64                           4/4 
  Verifying   : opencv-python-3.1.0-8.fc25.x86_64                           1/4 
  Verifying   : opencv-3.1.0-8.fc25.x86_64                                  2/4 
  Verifying   : python2-numpy-1:1.11.2-1.fc25.x86_64                        3/4 
  Verifying   : python2-nose-1.3.7-11.fc25.noarch                           4/4 

Installed:
  opencv.x86_64 3.1.0-8.fc25            opencv-python.x86_64 3.1.0-8.fc25       
  python2-nose.noarch 1.3.7-11.fc25     python2-numpy.x86_64 1:1.11.2-1.fc25    

Complete!
[root@localhost mythcat]# 

This is my test script with opencv to detect flow using Lucas-Kanade Optical Flow function.
This tracks some points in a black and white video.
First you need:
- one black and white video;
- not mp4 file type file;
- the color args need to be under 4 ( see is 3);
- I used this video:
I used cv2.goodFeaturesToTrack().
We take the first frame, detect some Shi-Tomasi corner points in it, then we iteratively track those points using Lucas-Kanade optical flow.
The function cv2.calcOpticalFlowPyrLK() we pass the previous frame, previous points and next frame.
The returns next points along with some status numbers which has a value of 1 if next point is found, else zero.
That iteratively pass these next points as previous points in next step.
See the code below:

import numpy as np
import cv2

cap = cv2.VideoCapture('candle')

# params for ShiTomasi corner detection
feature_params = dict( maxCorners = 77,
                       qualityLevel = 0.3,
                       minDistance = 7,
                       blockSize = 7 )

# Parameters for lucas kanade optical flow
lk_params = dict( winSize  = (17,17),
                  maxLevel = 1,
                  criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))

# Create some random colors
color = np.random.randint(0,255,(100,3))

# Take first frame and find corners in it
ret, old_frame = cap.read()
old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)
p0 = cv2.goodFeaturesToTrack(old_gray, mask = None, **feature_params)

# Create a mask image for drawing purposes
mask = np.zeros_like(old_frame)

while(1):
    ret,frame = cap.read()
    frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # calculate optical flow
    p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)

    # Select good points
    good_new = p1[st==1]
    good_old = p0[st==1]

    # draw the tracks
    for i,(new,old) in enumerate(zip(good_new,good_old)):
        a,b = new.ravel()
        c,d = old.ravel()
        mask = cv2.line(mask, (a,b),(c,d), color[i].tolist(), 2)
        frame = cv2.circle(frame,(a,b),5,color[i].tolist(),-1)
    img = cv2.add(frame,mask)

    cv2.imshow('frame',img)
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break

    # Now update the previous frame and previous points
    old_gray = frame_gray.copy()
    p0 = good_new.reshape(-1,1,2)

cv2.destroyAllWindows()
cap.release()

The output of this file is:

The bad and good urllib.

This is a simple python script:

import urllib
opener = urllib.FancyURLopener({})
f = opener.open("http://www.ra___aer.ro/")
d=f.read()
fo = open('workfile.txt', 'w')
fo.write(d)
fo.close()

The really bad news comes from here:
http://blog.blindspotsecurity.com/2017/02/advisory-javapython-ftp-injections.html

The twill python module with Fedora 25.

Today I tested the twill python module with python 2.7 and Fedora 25.
This is: a scripting system for automating Web browsing. Useful for testing Web pages or grabbing data from password-protected sites automatically.
To install this python module I used pip command:
[root@localhost mythcat]# pip install twill
Collecting twill
Downloading twill-1.8.0.tar.gz (176kB)
100% |████████████████████████████████| 184kB 2.5MB/s
Installing collected packages: twill
Running setup.py install for twill ... done
Successfully installed twill-1.8.0

Let's try some tests:

[mythcat@localhost ~]$ python
Python 2.7.13 (default, Jan 12 2017, 17:59:37) 
[GCC 6.3.1 20161221 (Red Hat 6.3.1-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from twill import get_browser
>>> b = get_browser()
>>> 
>>> from twill.commands import *
>>> go("http://www.python.org/")
==> at https://www.python.org/
u'https://www.python.org/'
>>> b.showforms()

Form #1
## ## __Name__________________ __Type___ __ID________ __Value__________________
1     q                        search    id-searc ...   

To talk to the Web browser directly, call the get_browser function.
You can see most of the twill commands by using:

>>> import twill.shell
>>> twill.shell.main()

 -= Welcome to twill! =-

current page: https://www.python.org/widgets
>> ?

Undocumented commands:
======================
add_auth             fa           info             save_html           title
add_extra_header     find         load_cookies     setglobal           url  
agent                follow       notfind          setlocal          
back                 formaction   redirect_error   show              
clear_cookies        formclear    redirect_output  show_cookies      
clear_extra_headers  formfile     reload           show_extra_headers
code                 formvalue    reset_browser    showforms         
config               fv           reset_error      showhistory       
debug                get_browser  reset_output     showlinks         
echo                 getinput     run              sleep             
exit                 getpassword  runfile          submit            
extend_with          go           save_cookies     tidy_ok           

current page: https://www.python.org/widgets
>> 

Basic is used by setlocal to fill website forms and the go function.
Ban can be very good for some tasks.
The twill python module also provides a simple wrapper for mechanizing functionality with the API is still unstable.

Compare two images: the histogram method.

This is a very simple example about how to compare the histograms of both images and print the inconsistencies are bound to arise.
The example come with alternative solution: Histogram method.
The script was run under Fedora 25.
If the images are the same the result will be 0.0.
For testing I change the image2.png by make a line into this with a coverage of 10%.
The result of the script was:
1116.63243729
The images come with this dimensions: 738 x 502 px.

import math
import operator
from math import *
import PIL

from PIL import Image
h1 = Image.open("image1.png").histogram()
h2 = Image.open("image2.png").histogram()

rms = math.sqrt(reduce(operator.add,
        map(lambda a,b: (a-b)**2, h1, h2))/len(h1))
print rms

About the operator module exports a set of efficient functions corresponding to the intrinsic operators of Python.
Example:
operator.lt(a, b)
operator.le(a, b)
operator.eq(a, b)
operator.ne(a, b)
operator.ge(a, b)
operator.gt(a, b)
operator.__lt__(a, b)
operator.__le__(a, b)
operator.__eq__(a, b)
operator.__ne__(a, b)
operator.__ge__(a, b)
operator.__gt__(a, b)
This is like math operators:
lt(a, b) is equivalent to a < b
le(a, b) is equivalent to a <= b
Another example:
>>> # Elementwise multiplication
>>> map(mul, [0, 1, 2, 3], [10, 20, 30, 40])
[0, 20, 60, 120]

>>> # Dot product
>>> sum(map(mul, [0, 1, 2, 3], [10, 20, 30, 40]))
200


The kivy pyhon module for android.

First you need to install kivy python module.
The kivy python module
C:\>cd Python27

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install kivy
Collecting kivy
Downloading Kivy-1.9.1-cp27-none-win32.whl (7.4MB)
100% |################################| 7.4MB 50kB/s
Collecting Kivy-Garden>=0.1.4 (from kivy)
Downloading kivy-garden-0.1.4.tar.gz
Requirement already satisfied: requests in c:\python27\lib\site-packages (from Kivy-Garden>=0.1.4->kivy)
Installing collected packages: Kivy-Garden, kivy
Running setup.py install for Kivy-Garden ... done
Successfully installed Kivy-Garden-0.1.4 kivy-1.9.1

Use this to add new pthon modules:
python -m pip install --upgrade docutils pygments pypiwin32 kivy.deps.sdl2 kivy.deps.glew kivy.deps.gstreamer --extra-index-url https://kivy.org/downloads/packages/simple/
If the python kivy.deps.gstreamer don't working then is not problem will try without this python module.
To see outdated python modules , use this commnds:
python -m pip install --upgrade pip wheel setuptools
pip list outdated
Let's upgrade all of python modules with this:
pip freeze > requirements.txt && pip install --upgrade -r requirements.txt && del requirements.txt
Another way to install this python module can be found here.
The demo can be star with this comman line:
C:\Python27>python share\kivy-examples\demo\showcase\main.py
You can also test one simple python script:
from kivy.app import App
from kivy.uix.button import Button

class TestApp(App):
def build(self):
return Button(text='Hello World')

TestApp().run()
To use with android operating system then test Buildozer.
This tool supports packaging for Android via the python-for-android project, and for iOS via the kivy-ios project. Support for other operating systems is intended in the future.

The python arch module for financial econometrics.

This python module arch: is a work-in-progress for ARCH and other tools for financial econometrics, written in Python (and Cython).
The arch python module come with tools for:

•    Univariate volatility models

•    Bootstrapping

•    Multiple comparison procedures

•    Unit root tests

You can read and see many examples here.
Let's start with instalation into my python 2.7.12 version.
First you need to install this python module with pip tool:
C:\Python27\Scripts>pip install Arch
Collecting Arch
Downloading arch-4.0.tar.gz (107kB)
100% |################################| 112kB 390kB/s
Requirement already satisfied: matplotlib>=1.4 in c:\python27\lib\site-packages (from Arch)
Requirement already satisfied: scipy>=0.15 in c:\python27\lib\site-packages (from Arch)
Collecting patsy>=0.2 (from Arch)
Downloading patsy-0.4.1-py2.py3-none-any.whl (233kB)
100% |################################| 235kB 906kB/s
Collecting statsmodels>=0.6 (from Arch)
Downloading statsmodels-0.6.1.tar.gz (7.0MB)
100% |################################| 7.0MB 85kB/s
Collecting pandas>=0.16 (from Arch)
Downloading pandas-0.19.2-cp27-cp27m-win32.whl (6.8MB)
100% |################################| 6.8MB 81kB/s
Requirement already satisfied: numpy>=1.6 in c:\python27\lib\site-packages (from matplotlib>=1.4->Arch)
Requirement already satisfied: python-dateutil in c:\python27\lib\site-packages (from matplotlib>=1.4->Arch)
Requirement already satisfied: cycler in c:\python27\lib\site-packages (from matplotlib>=1.4->Arch)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,>=1.5.6 in c:\python27\lib\site-packages (from matplotlib>=1.4->Arch)
Requirement already satisfied: pytz in c:\python27\lib\site-packages (from matplotlib>=1.4->Arch)
Requirement already satisfied: six in c:\python27\lib\site-packages (from patsy>=0.2->Arch)
Installing collected packages: patsy, pandas, statsmodels, Arch
Running setup.py install for statsmodels ... done
Running setup.py install for Arch ... done
Successfully installed Arch-4.0 pandas-0.19.2 patsy-0.4.1 statsmodels-0.6.1
This python module is very mathematical and technical so I'll refer you to a few examples.
Can be used in statistical research and econometrics, or the application of mathematics, statistics, and computer science to economic data. 

NVIDIA python module Theano.

I try to use python module Theano.
First I install this python module.
C:\WINDOWS\system32>cd C:\Python27

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install Theano
Collecting Theano
Using cached Theano-0.8.2.tar.gz
Requirement already satisfied: numpy>=1.7.1 in c:\python27\lib\site-packages (from Theano)
Requirement already satisfied: scipy>=0.11 in c:\python27\lib\site-packages (from Theano)
Requirement already satisfied: six>=1.9.0 in c:\python27\lib\site-packages (from Theano)
Installing collected packages: Theano
Running setup.py install for Theano ... done
Successfully installed Theano-0.8.2

When I try to used I got this error:
import theano
WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.
I try to fix that error, but I don't find any solution.
This python module work. I tested with examples from NVIDIA, see:

Use the twitter python module - part 002.

Using the twitter python module named python-twitter you can search twitter query into the local area.
The default tutorial is here.
The source code to change is that line:
results = api.GetSearch(raw_query="q=from%3Asomething"
with:
results = api.GetSearch(raw_query="q=&geocode=lat,long,10km")
Also, you need to put your lat and long and the area sized.
The good point of this you will be able to spell time with twitter posts.