analitics

Pages

Showing posts with label 2016. Show all posts
Showing posts with label 2016. Show all posts

Friday, December 30, 2016

The python arch module for financial econometrics.

This python module arch: is a work-in-progress for ARCH and other tools for financial econometrics, written in Python (and Cython).
The arch python module come with tools for:
  •    Univariate volatility models
  •    Bootstrapping
  •    Multiple comparison procedures
  •    Unit root tests
You can read and see many examples here.
Let's start with instalation into my python 2.7.12 version.
First you need to install this python module with pip tool:
C:\Python27\Scripts>pip install Arch
Collecting Arch
Downloading arch-4.0.tar.gz (107kB)
100% |################################| 112kB 390kB/s
Requirement already satisfied: matplotlib>=1.4 in c:\python27\lib\site-packages (from Arch)
Requirement already satisfied: scipy>=0.15 in c:\python27\lib\site-packages (from Arch)
Collecting patsy>=0.2 (from Arch)
Downloading patsy-0.4.1-py2.py3-none-any.whl (233kB)
100% |################################| 235kB 906kB/s
Collecting statsmodels>=0.6 (from Arch)
Downloading statsmodels-0.6.1.tar.gz (7.0MB)
100% |################################| 7.0MB 85kB/s
Collecting pandas>=0.16 (from Arch)
Downloading pandas-0.19.2-cp27-cp27m-win32.whl (6.8MB)
100% |################################| 6.8MB 81kB/s
Requirement already satisfied: numpy>=1.6 in c:\python27\lib\site-packages (from matplotlib>=1.4->Arch)
Requirement already satisfied: python-dateutil in c:\python27\lib\site-packages (from matplotlib>=1.4->Arch)
Requirement already satisfied: cycler in c:\python27\lib\site-packages (from matplotlib>=1.4->Arch)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,>=1.5.6 in c:\python27\lib\site-packages (from matplotlib>=1.4->Arch)
Requirement already satisfied: pytz in c:\python27\lib\site-packages (from matplotlib>=1.4->Arch)
Requirement already satisfied: six in c:\python27\lib\site-packages (from patsy>=0.2->Arch)
Installing collected packages: patsy, pandas, statsmodels, Arch
Running setup.py install for statsmodels ... done
Running setup.py install for Arch ... done
Successfully installed Arch-4.0 pandas-0.19.2 patsy-0.4.1 statsmodels-0.6.1

This python module is very mathematical and technical so I'll refer you to a few examples.
Can be used in statistical research and econometrics, or the application of mathematics, statistics, and computer science to economic data. 

Sunday, December 18, 2016

NVIDIA python module Theano.

I try to use python module Theano.
First I install this python module.
C:\WINDOWS\system32>cd C:\Python27

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install Theano
Collecting Theano
Using cached Theano-0.8.2.tar.gz
Requirement already satisfied: numpy>=1.7.1 in c:\python27\lib\site-packages (from Theano)
Requirement already satisfied: scipy>=0.11 in c:\python27\lib\site-packages (from Theano)
Requirement already satisfied: six>=1.9.0 in c:\python27\lib\site-packages (from Theano)
Installing collected packages: Theano
Running setup.py install for Theano ... done
Successfully installed Theano-0.8.2

When I try to used I got this error:
import theano
WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.

I try to fix that error, but I don't find any solution.
This python module work. I tested with examples from NVIDIA, see:

Thursday, December 15, 2016

Use the twitter python module - part 002.

Using the twitter python module named python-twitter you can search twitter query into the local area.
The default tutorial is here.
The source code to change is that line:
results = api.GetSearch(raw_query="q=from%3Asomething"
with:
results = api.GetSearch(raw_query="q=&geocode=lat,long,10km")
Also, you need to put your lat and long and the area sized.
The good point of this you will be able to spell time with twitter posts.

Wednesday, December 14, 2016

Use the twitter python module - part 001.

About this python module python-twitter you can read here.

C:\>cd Python27
C:\Python27>cd Scripts
C:\Python27\Scripts>pip install python-twitter
Collecting python-twitter
Downloading python_twitter-3.2-py2-none-any.whl (71kB)
100% |################################| 81kB 292kB/s
Requirement already satisfied: requests in c:\python27\lib\site-packages (from python-twitter)
Requirement already satisfied: requests-oauthlib in c:\python27\lib\site-packages (from python-twitter)
Collecting future (from python-twitter)
Downloading future-0.16.0.tar.gz (824kB)
100% |################################| 829kB 485kB/s
Requirement already satisfied: oauthlib>=0.6.2 in c:\python27\lib\site-packages
(from requests-oauthlib->python-twitter)
Installing collected packages: future, python-twitter
Running setup.py install for future ... done
Successfully installed future-0.16.0 python-twitter-3.2


Let's see one simple example with one authentication key and token and one query:

import os
import json
import twitter
from twitter import *
CONSUMER_KEY=""
CONSUMER_SECRET=""

ACCESS_TOKEN=""
ACCESS_TOKEN_SECRET=""

api = Api(CONSUMER_KEY,
          CONSUMER_SECRET,
          ACCESS_TOKEN,
          ACCESS_TOKEN_SECRET)
def main():
    with open('output.txt', 'a') as f:
        for line in api.GetStreamFilter(track='something', languages=LANGUAGES):
            print line
    results = api.GetSearch(raw_query="q=from%3Asomething")
    print results
if __name__ == '__main__':
    main()

Monday, December 12, 2016

Use the tweepy to deal with twitter api - part 001.

I will show you how to install the python module named tweepy  and how to make authentication into twitter webpage.
This will install the tweepy python module.
C:\>cd Python27
C:\Python27>cd Scripts
C:\Python27\Scripts>pip install tweepy
Collecting tweepy
Downloading tweepy-3.5.0-py2.py3-none-any.whl
Collecting requests>=2.4.3 (from tweepy)
Downloading requests-2.12.3-py2.py3-none-any.whl (575kB)
100% |################################| 583kB 556kB/s
Collecting requests-oauthlib>=0.4.1 (from tweepy)
Downloading requests_oauthlib-0.7.0-py2.py3-none-any.whl
Requirement already satisfied: six>=1.7.3 in c:\python27\lib\site-packages (from
tweepy)
Collecting oauthlib>=0.6.2 (from requests-oauthlib>=0.4.1->tweepy)
Downloading oauthlib-2.0.1.tar.gz (122kB)
100% |################################| 133kB 506kB/s
Installing collected packages: requests, oauthlib, requests-oauthlib, tweepy
Running setup.py install for oauthlib ... done
Successfully installed oauthlib-2.0.1 requests-2.12.3 requests-oauthlib-0.7.0 tweepy-3.5.0

To deal with twitter api then you need to create a new application into this webpage.
This webpage will give for authentication your date to connect to twitter:
Application Settings
consumer_key=""
consumer_secret=""

Your Access Token
access_token=""
access_token_secret=""

Let's start with a simple example, by using your application settings and access token:
import tweepy
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream

consumer_key=""
consumer_secret=""

access_token=""
access_token_secret=""

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

api = tweepy.API(auth)

print(api.me().name)

class StdOutListener(StreamListener):
""" A listener handles tweets that are received from the stream.
This is a basic listener that just prints received tweets to stdout.
"""
def on_data(self, data):
print(data)
return True

def on_error(self, status):
print(status)

if __name__ == '__main__':
lista = StdOutListener()
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

stream = Stream(auth, lista)
stream.filter(track=['internet'])

Using python shell to run this script will show all about 'internet'.
The output will come into raw format.
Let's try another example to update your status with this message:
I using OAuth authentication via Tweepy!
Just add this into my code before class definition:
api.update_status(status='I using OAuth authentication via Tweepy!')
You can rad more about streaming by reading this docs.
For example, I used track from here.



Sunday, December 11, 2016

The morse python module with pip installation versus precompiled archive wheel.

Today I try to deal with morse python module and it took me a while to install python modules because of their dependencies.
I started with the pip install of morse module but I got dependency errors in python modules
Because I try to fix this issue the result of this tutorial is more about how to install some python module: matplotlib, scipy, numpy, mkl and morse.
The all installation process will help you to understand how can be fix some pip installation versus precompiled archive wheel.
C:\Python27\Scripts>pip install matplotlib
Collecting matplotlib
  Downloading matplotlib-1.5.3-cp27-cp27m-win32.whl (6.0MB)
    100% |################################| 6.0MB 98kB/s
Requirement already satisfied: numpy>=1.6 in c:\python27\lib\site-packages (from
 matplotlib)
Collecting python-dateutil (from matplotlib)
  Downloading python_dateutil-2.6.0-py2.py3-none-any.whl (194kB)
    100% |################################| 194kB 1.4MB/s
Collecting cycler (from matplotlib)
  Downloading cycler-0.10.0-py2.py3-none-any.whl
Collecting pyparsing!=2.0.4,!=2.1.2,>=1.5.6 (from matplotlib)
  Downloading pyparsing-2.1.10-py2.py3-none-any.whl (56kB)
    100% |################################| 61kB 2.0MB/s
Collecting pytz (from matplotlib)
  Downloading pytz-2016.10-py2.py3-none-any.whl (483kB)
    100% |################################| 491kB 656kB/s
Collecting six>=1.5 (from python-dateutil->matplotlib)
  Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, cycler, pyparsing, pytz, ma
tplotlib
Successfully installed cycler-0.10.0 matplotlib-1.5.3 pyparsing-2.1.10 python-da
teutil-2.6.0 pytz-2016.10 six-1.10.0

Download SciPy wheel file from here.
Install this file with:
C:\Python27\Scripts>pip install scipy-0.18.1-cp27-cp27m-win32.whl
Processing c:\python27\scripts\scipy-0.18.1-cp27-cp27m-win32.whl
Installing collected packages: scipy
Successfully installed scipy-0.18.1

Now you can install morse pyhon module.
pip install morse
If you installed the numpy by pip, but the scipy was installed by precompiled archive then expects numpy+mkl.
So to fix that issue, I download the numpy-1.12.0b1+mkl-cp27-cp27m-win32.whl file and install this with pip:
C:\Python27\Scripts>pip install "numpy-1.12.0b1+mkl-cp27-cp27m-win32.whl"
Processing c:\python27\scripts\numpy-1.12.0b1+mkl-cp27-cp27m-win32.whl
Installing collected packages: numpy
Found existing installation: numpy 1.11.2
Uninstalling numpy-1.11.2:
Successfully uninstalled numpy-1.11.2
Successfully installed numpy-1.12.0b1+mkl

Let's test the morse python module.
C:\Python27>python
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (
Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import morse
>>> dir(morse)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'lookup', 'st
ring_to_morse']
>>> dir(morse.lookup)
['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__', '__doc__'
, '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__',
'__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '_
_new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__'
, '__sizeof__', '__str__', '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get
', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'po
pitem', 'setdefault', 'update', 'values', 'viewitems', 'viewkeys', 'viewvalues']
>>> print morse.lookup.keys()
['"', '$', '&', '(', ',', '.', '0', '2', '4', '6', '8', ':', '@', 'B', 'D', 'F',
'H', 'J', 'L', 'N', 'P', 'R', 'T', 'V', 'X', 'Z', '!', "'", ')', '+', '-', '/',
'1', '3', '5', '7', '9', ';', '=', '?', 'A', 'C', 'E', 'G', 'I', 'K', 'M', 'O',
'Q', 'S', 'U', 'W', 'Y', '_']
>>> print morse.lookup.items()
[('"', '.-..-.'), ('$', '...-..-'), ('&', '.-...'), ('(', '-.--.'), (',', '--..-
-'), ('.', '.-.-.-'), ('0', '-----'), ('2', '..---'), ('4', '....-'), ('6', '-..
..'), ('8', '---..'), (':', '---...'), ('@', '.--.-.'), ('B', '-...'), ('D', '-.
.'), ('F', '..-.'), ('H', '....'), ('J', '.---'), ('L', '.-..'), ('N', '-.'), ('
P', '.--.'), ('R', '.-.'), ('T', '-'), ('V', '...-'), ('X', '-..-'), ('Z', '--..
'), ('!', '-.-.--'), ("'", '.----.'), (')', '-.--.-'), ('+', '.-.-.'), ('-', '-.
...-'), ('/', '-..-.'), ('1', '.----'), ('3', '...--'), ('5', '.....'), ('7', '-
-...'), ('9', '----.'), (';', '-.-.-.'), ('=', '-...-'), ('?', '..--..'), ('A',
'.-'), ('C', '-.-.'), ('E', '.'), ('G', '--.'), ('I', '..'), ('K', '-.-'), ('M',
'--'), ('O', '---'), ('Q', '--.-'), ('S', '...'), ('U', '..-'), ('W', '.--'), (
'Y', '-.--'), ('_', '..--.-')]

You can see the python morse module is working well.

LibreOffice and python scripts.

I try to deal with python scripts under LibreOffice applications.
The full tutorial can be found here: Python and LibreOffice – part 001.
I will come with new tutorials about LibreOffice and python.

Saturday, December 10, 2016

The python modules pygobject, pycairo and pygtk under Windows OS.

I used the python version 2.7 32 bits under .
C:\Python27>python
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (
Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

The issue is to instal PyGTK python module but you can see I got errors.
C:\>cd Python27

C:\Python27>cd script
The system cannot find the path specified.

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install PyGTK
Collecting PyGTK
Downloading pygtk-2.24.0.tar.bz2 (2.4MB)
100% |################################| 2.4MB 224kB/s
Complete output from command python setup.py egg_info:
ERROR: Could not import dsextras module: Make sure you have installed pygobj
ect.

----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in c:\users\my_account
\appdata\local\temp\pip-build-os60hf\PyGTK\

C:\Python27\Scripts>pip install pygobject
Collecting pygobject
Downloading pygobject-2.28.3.tar.bz2 (889kB)
100% |################################| 890kB 550kB/s
Complete output from command python setup.py egg_info:
ERROR: Could not find pkg-config: Please check your PATH environment variabl
e.

----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in c:\users\my_account
\appdata\local\temp\pip-build-lt0gbh\pygobject\

I got the pygobject-2.28.3.win32-py2.7.msi, pycairo-1.8.10.win32-py2.7.msi and pygtk-2.24.0.win32-py2.7.msi from gnome website.
I install this python modules using registry and this option: Python from another location and I set my python path: C:\Python27\
>>> import gobject
>>> dir(gobject)
['GBoxed', 'GEnum', 'GError', 'GFlags', 'GInterface', 'GObject', 'GObjectMeta',
...
>>> import cairo
>>> dir(cairo)
['ANTIALIAS_DEFAULT', 'ANTIALIAS_GRAY', 'ANTIALIAS_NONE', 'ANTIALIAS_SUBPIXEL',
>>> import pygtk
>>> dir(pygtk)
['__all__', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '_
get_available_versions', '_our_dir', '_pygtk_2_0_dir', '_pygtk_dir_pat', '_pygtk
_required_version', 'fnmatch', 'glob', 'os', 'require', 'require20', 'sys']


As you can see this python modules works well.


Thursday, December 8, 2016

Noise 2D, 3D, 4D with opensimplex python module.

OpenSimplex noise is an n-dimensional gradient noise function that was developed in order to overcome the patent-related issues surrounding Simplex noise, while continuing to also avoid the visually-significant directional artifacts characteristic of Perlin noise.
Let's start with instalation:
C:\Python27\Scripts>pip install OpenSimplex
Collecting OpenSimplex
Downloading opensimplex-0.2.tar.gz
Installing collected packages: OpenSimplex
Running setup.py install for OpenSimplex ... done
Successfully installed OpenSimplex-0.2

Test some examples from official page:
C:\Python27>python
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (
Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from opensimplex import OpenSimplex
>>> tmp = OpenSimplex()
>>> print (tmp.noise2d(x=10, y=10))
-0.297251513589
>>> tmp = OpenSimplex(seed=1)
>>> print (tmp.noise2d(x=10, y=10))
-0.734782324747
>>> dir(OpenSimplex)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribut
e__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_e
x__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_
_weakref__', '_extrapolate2d', '_extrapolate3d', '_extrapolate4d', 'noise2d', 'n
oise3d', 'noise4d']

Let's make a image example with noise 2D:
from opensimplex import OpenSimplex
from PIL import Image

height = int(input("Enter in the map height: "))
width = int(input("Enter in the map width: "))

def main():
    simplex = OpenSimplex()
    im = Image.new('L', (width, height))
    for y in range(0, height):
        for x in range(0, width):
            value = simplex.noise2d(x , y )
            color = int((value + 1) * 128)
            im.putpixel((x, y), color)
    im.save('noise2d.png')
    im.show()
if __name__ == '__main__':
    main()

Tuesday, December 6, 2016

The python-nmap python module fail.

You can read about this python module here.

First let's install this python module.
C:\Python27>cd Scripts

C:\Python27\Scripts>pip install python-nmap
Collecting python-nmap
Downloading python-nmap-0.6.1.tar.gz (41kB)
100% |################################| 51kB 240kB/s
Installing collected packages: python-nmap
Running setup.py install for python-nmap ... done
Successfully installed python-nmap-0.6.1

About this python-nmap version you can read here.
I try to run the example source code but not of this example working.
For example I got this:
>>> nm.scan('127.0.0.1', '22-443')
{'nmap': {'scanstats': {'uphosts': '1', 'timestr': 'Wed Dec 07 08:13:01 2016', '
downhosts': '-1', 'totalhosts': '0', 'elapsed': '10.74'}, 'scaninfo': {'tcp': {'
services': '22-443', 'method': 'syn'}, 'error': [u'dnet: Failed to open device l
o0\r\nQUITTING!\r\n', u'dnet: Failed to open device lo0\r\nQUITTING!\r\n']}, 'co
mmand_line': 'nmap -oX - -p 22-443 -sV 127.0.0.1'}, 'scan': {}}

Wednesday, November 30, 2016

OpenGL and OpenCV with python 2.7 - part 004.

Today I will continue the series of graphics processing in OpenGL and OpenCV.
The goal of this tutorial is the download and load into the python script of a youtube video.
To do that we need another two modules python.
First, I download and install the python 2.7.12 32 bit version from the internet.
C:\Python27>python.exe
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (
Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

First update the pip tool and install numpy python module:
C:\Python27\Scripts>python -m pip install --upgrade pip
Collecting pip
Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
100% |################################| 1.3MB 419kB/s
Installing collected packages: pip
Found existing installation: pip 8.1.1
Uninstalling pip-8.1.1:
Successfully uninstalled pip-8.1.1
Successfully installed pip-9.0.1
C:\Python27\Scripts>pip install numpy
Collecting numpy
Downloading numpy-1.11.2-cp27-none-win32.whl (6.5MB)
100% |################################| 6.5MB 79kB/s
Installing collected packages: numpy
Successfully installed numpy-1.11.2

The main reason to have the numpy python module: it is often used with OpenCV python module.
For OpenCV python module installation you need to see my tutorial.
After you install it, test the OpenCV python module:
C:\Python27>python.exe
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:19:22) [MSC v.1500 32 bit (
Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> print cv2.__version__
3.0.0

Install pafy python module.
This module help you to download video from youtube but you need also the youtube-dl python module.
So let's install the youtube-dl and pafy python modules.
C:\Python27>cd Scripts
C:\Python27\Scripts>pip install youtube-dl
Collecting youtube-dl
Downloading youtube_dl-2016.12.1-py2.py3-none-any.whl (1.5MB)
100% |################################| 1.5MB 377kB/s
Installing collected packages: youtube-dl
Successfully installed youtube-dl-2016.12.1
C:\Python27\Scripts>pip install pafy
Collecting pafy
Downloading pafy-0.5.2-py2.py3-none-any.whl
Installing collected packages: pafy
Successfully installed pafy-0.5.2

I make a simple python script named: get_yt.py.
The source code of this script is simple:
import os
import pafy
# Download the video
video = pafy.new('https://www.youtube.com/watch?v=O5VCjktWVD4')
print "video.title"
print video.title
print "video.rating"
print video.rating
print "video.viewcount, video.author, video.length"
print video.viewcount, video.author, video.length
print "video.duration, video.likes, video.dislikes"
print video.duration, video.likes, video.dislikes
print "video.description"
print video.description
resolution = video.getbestvideo(preftype="mp4")
print "resolution"
print resolution
input_movie = resolution.download(quiet=False)
print "input_movie"
print input_movie
print "delete movie"
os.remove(input_movie)

I used the URL of a video clip from youtube channel of Arden Cho to tested.
If you want to keep the video into your folder just remove the last line from the python script.
The result is this output:
C:\Python27>python.exe get_yt.py
video.title
Can't Help Falling in Love With You - Arden Cho
video.rating
4.99041318893
video.viewcount, video.author, video.length
10980 ardenBcho 168
video.duration, video.likes, video.dislikes
00:02:48 1665 4
video.description
Recorded this song a couple months ago when I was in Boston, this song always reminds
me of holidays and love so sharing that with you!

Guitar by Koo Chung https://youtube.com/koochung
Violin and Video editing/production by Daniel Jang https://www.youtube.com/metal
sides
Production + Keys by Tim Bongiovanni https://www.northgateproductions.net
Filmed by Rob Mark https://www.instagram.com/rmarq_

If you like my music comment and SHARE! You can also support me by buying & rati
ng my album on iTunes!! https://itunes.apple.com/us/album/my-true-happy/id592588
859

You can follow me at: SnapChat: ardencho
http://www.instagram.com/arden_cho
http://www.facebook.com/hiardencho
http://www.twitter.com/arden_cho
http://www.imdb.me/ardencho
resolution
video:mp4@1920x1080
input_movie5 Bytes [100.00%] received. Rate: [5371 KB/s]. ETA: [0 secs]
Can't Help Falling in Love With You - Arden Cho.mp4
delete movie


Sunday, November 6, 2016

Python and OAuth2 with Google API.


If you want to do this then is simple to do. The most complex part comes from google settings.
But that can be easy if you make it from a few times.
I used OAuth2 with JSON file to make authentification.
You can see my full tutorial here.

Sunday, October 23, 2016

Using text file with python 2.7 .

This simple tutorial is about text file. The text file come with txt extension.
It is a file containing text that can be used and further processing by functions and modules python.
I used Tkinter python module. The reason I chose this python mpodule is:
  • old python module;
  • rapidly developing graphical interfaces.
First step is to import the python module and then to put all of dir command into one text file.
The name of this file is: Tkinter_funct.txt
Open your python and try this source code:
C:\Python27>python.exe
Python 2.7.8 (default, Jun 30 2014, 16:08:48) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import Tkinter
>>> from Tkinter import *
>>> Tkinter_all=dir(Tkinter)
>>> Tkinter_file=open('Tkinter_funct.txt','w')
>>> Tkinter_file.write(str(Tkinter_all))
>>> print Tkinter_all

You will see the content of dir python module.
The text file is also have this text output.
To read the file is need to open the file with open function, to put position for read by using seek function and to read with read function.
This example will show you how is working:
>>> test=open('Tkinter_funct.txt','r+')
>>> test.seek(1)
>>> test.read()

Will open the file named Tkinter_funct.txt and r+ access to file.
The position it is set to 1.
The read function make all with full content output.
Now let's see the next steps, by change the read and seek values.

>>> test.read(1)
''
>>> test.read(2)
''
>>> test.read(10)
''
>>> test.seek(2)
>>> test.read(1)
'A'
>>> test.read(10)
"CTIVE', 'A"
>>> test.read(20)
"LL', 'ANCHOR', 'ARC'"

This outputs come with parts of all content and show you how it's working.

Monday, October 3, 2016

The python CacheControl module - part 002.

Today was a hard day and this is the reason I make this short tutorial.
Teory of HTTP:
HTTP specifies four response cache headers that you can set to enable caching:
  • Cache-Control
  • Expires
  • ETag
  • Last-Modified
These four headers are used to help cache your responses into two different models:
  • Expiration Caching - used to cache your entire response for a specific amount of time (e.g. 24 hours), simple, but cache invalidation is more difficult;
  • Validation Caching - this is more complex and used to cache your response, but allows you to dynamically invalidate it as soon as your content changes.
First you need to know about this code is a raw example about how we can access cache of the page.
Come with a simple class named DictCache. You can named with any name and is a BaseCache class.
The next step I make is to show you how can access it.
One simpe way is to see the page - first session.
The complex come when you need to access for example data and info like:
 'adapters', 'auth', 'cert', 'close', 'cookies', 'delete', 'get', 'get_adapter', 'head', 'headers', 'hooks', 'max_redirects', 'merge_environment_settings', 'mount', 'options', 'params', 'patch', 'post', 'prepare_request', 'proxies', 'put', 'rebuild_auth', 'rebuild_method', 'rebuild_proxies', 'redirect_cache', 'request', 'resolve_redirects', 'send', 'stream', 'trust_env', 'verify'
And this is come with teh second session from this source code:

import requests
from cachecontrol import CacheControl
from cachecontrol.cache import BaseCache

class DictCache(BaseCache):

    def __init__(self, init_dict=None):
        self.data = init_dict or {}

    def get(self, key):
        return self.data.get(key, None)

    def set(self, key, value):
        self.data.update({key: value})

    def delete(self, key):
        self.data.pop(key)

print "first session requests"
sess = requests.session()
cached_sess = CacheControl(sess)
response = cached_sess.get('http://google.com')
print '=================='
print 'see page by add this: print response.text'
print '=================='
print "second session BaseCache"
sess2 = requests.session()
base=DictCache(sess2)
print '=================='
print "dir(base)"
print dir(base)
print '=================='
print"dir(base.data)"
print dir(base.data)
print '=================='
print"base.data.max_redirects"
print base.data.max_redirects
print '=================='

Sunday, October 2, 2016

The python CacheControl module - part 001.

This tutorials series want to be a better approach to understand the several mechanisms that HTTP provides for web cache validation. Let's start with the first part.
You can install it with pip
C:\>cd Python27
C:\Python27>cd Scripts
C:\Python27\Scripts>pip install cachecontrol
Collecting cachecontrol
  Downloading CacheControl-0.11.7.tar.gz
Requirement already satisfied (use --upgrade to upgrade): 
requests in c:\python27\lib\site-packages (from cachecontrol)
Building wheels for collected packages: cachecontrol
  Running setup.py bdist_wheel for cachecontrol ... done
  Stored in directory: C:\Users\GeorgeCatalin\AppData\Local\pip\\
Cache\wheels\9b\94\d2\1793b004461b5bc238a89e260cd2b9f770437c42424fdd0943
Successfully built cachecontrol
Installing collected packages: cachecontrol
Successfully installed cachecontrol-0.11.7
First test come with the default example and show all with the text.
import requests
from cachecontrol import CacheControl
sess = requests.session()
cached_sess = CacheControl(sess)
response = cached_sess.get('http://google.com')
print response

print response.text
...
The requests python module is an Apache2 Licensed HTTP library to allow you to send HTTP/1.1 requests.
This help you to add headers, form data, multipart files, and parameters with simple
Python dictionaries, and access the response data in the same way.

The theory part.
You can use CacheControl with the basic wrapper way or via a requests Transport Adapter.
The Transport Adapters provide a mechanism to define interaction methods for an HTTP service.
The code will come with this template (docs example):
sess = requests.Session()
sess.mount('http://', CacheControlAdapter())
This mean the CacheControl assumes you are using a requests.Session for your requests.
So the Transport Adapter will cover the HTTPCore and WSGICore.
Now, both (the wrapper and adapter classes) allow providing a custom cache store object.
This is used for storing your cached data.
The next step will be
from cachecontrol.caches import FileCache
sess = CacheControl(requests.Session(),
                    cache=FileCache('.webcache'))
The result will create a directory called .webcache and store a file for each cached request.
Also the CacheControl python module comes with a few storage backends for storing your cache objects.
First is DictCache is the default cache, next is FileCache is similar to the caching mechanism provided by httplib2 and the last is RedisCache uses a Redis database to store values.
One note about requesting the filecache extra can use dependency with: pip install cachecontrol[filecache].
The CacheControl’s support of ETags by returns a response with the appropriate If-None-Match header.
Seem the ETag support only takes effect when the time has expired.
The ETag or entity tag, is part of HTTP, the protocol for the World Wide Web and provides for web cache validation. You can also take a look at Hypertext Transfer Protocol (HTTP/1.1): Caching.
The documentation of cachecontrol python module tells us:
Caching is hard! It is considered one of the great challenges of computer science.
Yes! you can agree with that, because some parts need to be understand well.
This issues: Timezones, Cached Responses and Query String Params are the most important parts.

Any info about this issue will be grea, just put your comments.



Another simple effect with pygame.

The pygame module come with many features for users.
I used the pygame version to make one simple tutorial about pallete functions :
>>> print pygame.version.ver
1.9.2b1

The result of my tutorial is this:


Thursday, September 22, 2016

Another learning python post with pygame.

This is a simple python script with pygame python module.
I make it for for educational purposes for the children.
I used words into romanian language for variables, functions and two python class.
See this tutorial here
pygame python

Thursday, September 8, 2016

OpenGL and OpenCV with python 2.7 - part 003.

If you have seen the last tutorial about OpenCV, then this tutorial comes to complete with one source code.
This source code will cut the background of webcam.
The webcam output is take by VideoCapture function.
This part of source code: np.zeros((1,65),np.float64) will return a new array of given shape and type, filled with zeros.
The result of this parts is used with function grabCut from cv2 python module.
This is the source code:

import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
    ret, img = cap.read()
    #img = cv2.imread('test002.jpg')
    mask = np.zeros(img.shape[:2],np.uint8)

    bgdModel = np.zeros((1,65),np.float64)
    fgdModel = np.zeros((1,65),np.float64)

    rect = (50,50,450,290)
    cv2.grabCut(img,mask,rect,bgdModel,fgdModel,5,cv2.GC_INIT_WITH_RECT)

    mask2 = np.where((mask==2)|(mask==0),0,1).astype('uint8')
    img = img*mask2[:,:,np.newaxis]
    cv2.imshow('frame',img)
    if 0xFF & cv2.waitKey(5) == 27:
        break
cap.release()
cv2.destroyAllWindows()
The end result will be something like:

Saturday, August 27, 2016

The python dizzy with clean instalation, is true !?

This will refer the python 2.7 working - but it can be extrapolated to other versions.
Many users have trouble installing python modules. The problem comes from old modules outdated or those who did not support.
I will present a few examples. I hope to come and support as necessary to remedy in time or exclusion through better solutions.
This is the main question for today: The python dizzy with clean instalation, is true !?
I don't think is a unwanted hacking of my python instalation using internet.
But I search and I saw many questions and erros over pip and Scripts folders.
I will deal just for this issue:
After I make one clean python 2.7 and all my python modules works well I used my windows to deal with some ssh software.
The next step I make it with python was to try to update with pip.
The strange think is with this files from Scripts folders:
12-Aug-16 06:41 PM 98,150 pyrsa-decrypt-bigfile.exe
12-Aug-16 06:41 PM 98,134 pyrsa-decrypt.exe
12-Aug-16 06:41 PM 98,150 pyrsa-encrypt-bigfile.exe
12-Aug-16 06:41 PM 98,134 pyrsa-encrypt.exe
12-Aug-16 06:41 PM 98,132 pyrsa-keygen.exe
12-Aug-16 06:41 PM 98,155 pyrsa-priv2pub.exe
12-Aug-16 06:41 PM 98,128 pyrsa-sign.exe
12-Aug-16 06:41 PM 98,132 pyrsa-verify.exe
and this file from same Script folder:
21-Jun-16 09:09 PM 0 python.exe
When I need to use pip I got errors.Then I try to fix with this:
pip install --upgrade ndg-httpsclient
and seem to be working now.
But I need to find from where come this file and why is this python file with:
C:\Python27\Scripts>python
Access is denied.
Maybe will be fix with a clean python instalation.
But the next step is and one of my concern is how to preserve this python instalation.
For example today the
pip update issue
come with many errors and this will be fixed.
Let's see how I fixed some of this.

First download Microsoft Visual C++ Compiler for Python 2.7.
This will fix this error:
error: Microsoft Visual C++ 9.0 is required. Get it from http://aka.ms/vcpython27
.

If you got this error: RuntimeError: Freetype library not found
C:\Python27>Scripts\pip install freetype-py
Collecting freetype-py
Downloading freetype-py-1.0.2.tar.gz (394kB)
100% |################################| 399kB 758kB/s
Building wheels for collected packages: freetype-py
Running setup.py bdist_wheel for freetype-py ... done
You can see also this freetype-py will not working:
C:\Python27>python.exe
Python 2.7.8 (default, Jun 30 2014, 16:08:48) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import freetype
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\freetype\__init__.py", line 21, in
from freetype.raw import *
File "C:\Python27\lib\site-packages\freetype\raw.py", line 37, in
raise RuntimeError('Freetype library not found')
RuntimeError: Freetype library not found
Also some python module has into cloud dizzy stuff.
For example the pycryptodome come with many features and working great.
Also some alternative is a bad solution.
It is hard to find a solution to the problem of leaving all these modules.
Any solutions?

The python module pycryptodome - part 001.

The tutorial for today come with this subject: python module pycryptodome.
According to the official website:
PyCryptodome is a self-contained Python package of low-level cryptographic primitives.
It supports Python 2.4 or newer, all Python 3 versions and PyPy.
official website.
Also this python module can be used with Windows and Linux (Ubuntu and Fedora distro linux).
I don't see anything about Mac OS - Apple OS Mac_OS - wikipedia.
First step of this tutorial:

C:\Python27\Scripts>pip install pycryptodome
Collecting pycryptodome
Downloading pycryptodome-3.4-cp27-cp27m-win_amd64.whl (7.5MB)
100% |################################| 7.5MB 88kB/s
Installing collected packages: pycryptodome
Successfully installed pycryptodome-3.4
You need to have command.com shell admin rights or you got errors:
C:\Python27\Scripts>python -m Crypto.SelfTest
Access is denied.
You can test it the instalation with:
C:\Python27>python -m Crypto.SelfTest
Skipping AESNI tests
...........................................................................................................................................................................................................................................................................................................................................................................

.......................
----------------------------------------------------------------------
Ran 22263 tests in 171.266s

OK
One simple test with this module:
C:\Python27>python.exe
Python 2.7.8 (default, Jun 30 2014, 16:08:48) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from Crypto.Cipher import AES
>>> from Crypto.Random import get_random_bytes
>>>
Most of resurces and features can be found here also can have some example at Matthew Green.
I will come with another tutorial about this python module.
Have a great day.