analitics

Pages

Friday, January 12, 2018

Python 2.7 : Python and BigQuery service object.

Here's another tutorial about python and google. I thought it would be useful for the beginning of 2018.
The Google team tell us:

What is BigQuery?

Storing and querying massive datasets can be time consuming and expensive without the right hardware and infrastructure. Google BigQuery is an enterprise data warehouse that solves this problem by enabling super-fast SQL queries using the processing power of Google's infrastructure. Simply move your data into BigQuery and let us handle the hard work. You can control access to both the project and your data based on your business needs, such as giving others the ability to view or query your data.


This tutorial it follows more precisely the steps from here.
First of all, you must create an authentication file by using the Create service account from your Google project.
Go to Google Console, navigate to the Create service account key page.
From the Service account drop-down, select the New service account.
Input a name into the form field.
From the Role drop-down, select Project and Owner.
The result is a JSON file type (this is for authenticating with google) download it renames and put into your project folder.
Like into the next image:

Now, select from the left area the Library does add the BigQuery API, try this link.
Search for BigQuery API and then use the button ENABLE to use it.
The next step is to install these python modules: pyopenssl and google-cloud-bigquery.
C:\Python27\Scripts>pip install -U pyopenssl
C:\Python27\Scripts>pip install --upgrade google-cloud-bigquery
Add this JSON file to windows path from my test folder:
set GOOGLE_APPLICATION_CREDENTIALS=C:\test\python_doc.json
Because my JSON file is named python_doc.json then this is the name I will use with my python script.
Let's see the script:
import google
from google.cloud import bigquery

def query_shakespeare():
    client = bigquery.Client()
    client = client.from_service_account_json('python_doc.json')
    query_job = client.query("""
        #standardSQL
        SELECT corpus AS title, COUNT(*) AS unique_words
        FROM `bigquery-public-data.samples.shakespeare`
        GROUP BY title
        ORDER BY unique_words DESC
        LIMIT 10""")

    results = query_job.result()  # Waits for job to complete.

    for row in results:
        print("{}: {}".format(row.title, row.unique_words))

if __name__ == '__main__':
    query_shakespeare()
The result is:
C:\Python27>python.exe goo_test_bquerry.py
hamlet: 5318
kinghenryv: 5104
cymbeline: 4875
troilusandcressida: 4795
kinglear: 4784
kingrichardiii: 4713
2kinghenryvi: 4683
coriolanus: 4653
2kinghenryiv: 4605
antonyandcleopatra: 4582
NOTE: Take care of the JSON file because it gives access to your google account and tries to use the restrictions according to the application's requirements.

Thursday, January 4, 2018

Python 2.7 : InsecurePlatformWarning error.

This is not a common error and can be solve it easy like any python issue.
The result of this error can be shown like into the next example:
c:\python27\lib\site-packages\pip\_vendor\requests\packages\urllib3\util\ssl_.py:318: 
SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension 
to TLS is not available on this platform. This may cause the server to present an incorrect TLS 
certificate, which can cause validation failures. You can upgrade to a newer version of Python to
 solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html
#snimissingwarning.
  SNIMissingWarning
c:\python27\lib\site-packages\pip\_vendor\requests\packages\urllib3\util\ssl_.py:122: 
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from
 configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade 
to a newer version of Python to solve this. For more information, see https://urllib3.readthe
docs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
The simple way to test this python error is to install these python modules:
pip install urllib3 
pip install requests
This last python module named requests to come with:
Successfully installed certifi-2017.11.5 chardet-3.0.4 idna-2.6 requests-2.18.4
What is this python module named requests?
Is a security the requests python module inject pyopenssl into urllib3
.
C:\Python27>python
Python 2.7 (r27:82525, Jul  4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> help()

Welcome to Python 2.7!  This is the online help utility.

If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://docs.python.org/tutorial/.

Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules.  To quit this help utility and
return to the interpreter, just type "quit".

To get a list of available modules, keywords, or topics, type "modules",
"keywords", or "topics".  Each module also comes with a one-line summary
of what it does; to list the modules whose summaries contain a given word
such as "spam", type "modules spam".

help> modules requests

Here is a list of matching modules.  Enter any module name to get more help.

pip._vendor.cachecontrol.controller - The httplib2 algorithms ported for use with requests.
pip._vendor.requests - Requests HTTP library
pip._vendor.requests.adapters - requests.adapters
pip._vendor.requests.api - requests.api
pip._vendor.requests.auth - requests.auth
pip._vendor.requests.certs - requests.certs
pip._vendor.requests.compat - requests.compat
pip._vendor.requests.cookies - requests.cookies
pip._vendor.requests.exceptions - requests.exceptions
pip._vendor.requests.hooks - requests.hooks
pip._vendor.requests.models - requests.models
pip._vendor.requests.packages
pip._vendor.requests.packages.chardet
pip._vendor.requests.packages.chardet.big5freq
pip._vendor.requests.packages.chardet.big5prober
pip._vendor.requests.packages.chardet.chardetect - Script which takes one or more file paths 
and reports on their detected
pip._vendor.requests.packages.chardet.chardistribution
pip._vendor.requests.packages.chardet.charsetgroupprober
pip._vendor.requests.packages.chardet.charsetprober
pip._vendor.requests.packages.chardet.codingstatemachine
pip._vendor.requests.packages.chardet.compat
pip._vendor.requests.packages.chardet.constants
pip._vendor.requests.packages.chardet.cp949prober
pip._vendor.requests.packages.chardet.escprober
pip._vendor.requests.packages.chardet.escsm
pip._vendor.requests.packages.chardet.eucjpprober
pip._vendor.requests.packages.chardet.euckrfreq
pip._vendor.requests.packages.chardet.euckrprober
pip._vendor.requests.packages.chardet.euctwfreq
pip._vendor.requests.packages.chardet.euctwprober
pip._vendor.requests.packages.chardet.gb2312freq
pip._vendor.requests.packages.chardet.gb2312prober
pip._vendor.requests.packages.chardet.hebrewprober
pip._vendor.requests.packages.chardet.jisfreq
pip._vendor.requests.packages.chardet.jpcntx
pip._vendor.requests.packages.chardet.langbulgarianmodel
pip._vendor.requests.packages.chardet.langcyrillicmodel
pip._vendor.requests.packages.chardet.langgreekmodel
pip._vendor.requests.packages.chardet.langhebrewmodel
pip._vendor.requests.packages.chardet.langhungarianmodel
pip._vendor.requests.packages.chardet.langthaimodel
pip._vendor.requests.packages.chardet.latin1prober
pip._vendor.requests.packages.chardet.mbcharsetprober
pip._vendor.requests.packages.chardet.mbcsgroupprober
pip._vendor.requests.packages.chardet.mbcssm
pip._vendor.requests.packages.chardet.sbcharsetprober
pip._vendor.requests.packages.chardet.sbcsgroupprober
pip._vendor.requests.packages.chardet.sjisprober
pip._vendor.requests.packages.chardet.universaldetector
pip._vendor.requests.packages.chardet.utf8prober
pip._vendor.requests.packages.urllib3 - urllib3 - Thread-safe connection pooling and re-using.
pip._vendor.requests.packages.urllib3._collections
pip._vendor.requests.packages.urllib3.connection
pip._vendor.requests.packages.urllib3.connectionpool
pip._vendor.requests.packages.urllib3.contrib
pip._vendor.requests.packages.urllib3.contrib.appengine
pip._vendor.requests.packages.urllib3.contrib.ntlmpool - NTLM authenticating pool, 
contributed by erikcederstran
pip._vendor.requests.packages.urllib3.contrib.pyopenssl
pip._vendor.requests.packages.urllib3.contrib.socks - SOCKS support for urllib3
pip._vendor.requests.packages.urllib3.exceptions
pip._vendor.requests.packages.urllib3.fields
pip._vendor.requests.packages.urllib3.filepost
pip._vendor.requests.packages.urllib3.packages
pip._vendor.requests.packages.urllib3.packages.ordered_dict
pip._vendor.requests.packages.urllib3.packages.six - Utilities for writing code that runs on 
Python 2 and 3
pip._vendor.requests.packages.urllib3.packages.ssl_match_hostname
pip._vendor.requests.packages.urllib3.packages.ssl_match_hostname._implementation - The match_hostname() 
function from Python 3.3.3, essential when using SSL.
pip._vendor.requests.packages.urllib3.poolmanager
pip._vendor.requests.packages.urllib3.request
pip._vendor.requests.packages.urllib3.response
pip._vendor.requests.packages.urllib3.util
pip._vendor.requests.packages.urllib3.util.connection
pip._vendor.requests.packages.urllib3.util.request
pip._vendor.requests.packages.urllib3.util.response
pip._vendor.requests.packages.urllib3.util.retry
pip._vendor.requests.packages.urllib3.util.ssl_
pip._vendor.requests.packages.urllib3.util.timeout
pip._vendor.requests.packages.urllib3.util.url
pip._vendor.requests.sessions - requests.session
pip._vendor.requests.status_codes
pip._vendor.requests.structures - requests.structures
pip._vendor.requests.utils - requests.utils
requests - Requests HTTP Library
requests.__version__
requests._internal_utils - requests._internal_utils
requests.adapters - requests.adapters
requests.api - requests.api
requests.auth - requests.auth
requests.certs - requests.certs
requests.compat - requests.compat
requests.cookies - requests.cookies
requests.exceptions - requests.exceptions
requests.help - Module containing bug report helper(s).
requests.hooks - requests.hooks
requests.models - requests.models
requests.packages
requests.sessions - requests.session
requests.status_codes
requests.structures - requests.structures
requests.utils - requests.utils
help>
You are now leaving help and returning to the Python interpreter.
If you want to ask for help on a particular object directly from the
interpreter, you can type "help(object)".  Executing "help('string')"
has the same effect as typing a particular string at the help> prompt.
>>>
...

Wednesday, January 3, 2018

The ebooklib python module .

Happy new year 2018!
The official webpage of this python module comes with this intro:
EbookLib is a Python library for managing EPUB2/EPUB3 and Kindle files. It's capable of reading and writing EPUB files programmatically (Kindle support is under development).
First the installation of this python module named ebooklib.
C:\>cd Python27

C:\Python27>cd Script
The system cannot find the path specified.

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install ebooklib
Collecting ebooklib
  Downloading EbookLib-0.16.tar.gz
Requirement already satisfied: lxml in c:\python27\lib\site-packages (from ebooklib)
Requirement already satisfied: six in c:\python27\lib\site-packages (from ebooklib)
Installing collected packages: ebooklib
  Running setup.py install for ebooklib ... done
Successfully installed ebooklib-0.16
If you don't see the Scripts folder into your Python27 folder you need to install pip tool.
Just download the get-pip.py script into your Python27 folder and run it with python.
Let's test some default example:
C:\Python27>python.exe get-pip.py
The next step is to test a simple example:
from ebooklib import epub

book = epub.EpubBook()

# set metadata
book.set_identifier('id123456')
book.set_title('Sample book')
book.set_language('en')

book.add_author('Author Python')
book.add_author('catafest', file_as='', role='writer', uid='author')

# create chapter
c1 = epub.EpubHtml(title='Intro', file_name='chap_01.xhtml', lang='hr')
c1.content=u'Intro heading.Python is a interpreted high-level programming language ...'

# add chapter
book.add_item(c1)

# define Table Of Contents
book.toc = (epub.Link('chap_01.xhtml', 'Introduction', 'intro'),
(epub.Section('Simple book'),
(c1, ))
)

# add default NCX and Nav file
book.add_item(epub.EpubNcx())
book.add_item(epub.EpubNav())

# define CSS style
style = 'BODY {color: white;}'
nav_css = epub.EpubItem(uid="style_nav", file_name="style/nav.css", media_type="text/css", content=style)

# add CSS file
book.add_item(nav_css)

# basic spine
book.spine = ['nav', c1]

# write to the file
epub.write_epub('test.epub', book, {})
You can update and make more good your epub book with HTML5 tags.
I used this example with headings and paragraph to change the text, see the result:

Tuesday, December 26, 2017

The development with python-instagram - python 3.6.3 .

Today I will show you how to deal with Instagram API using python-instagram python module.
The version of the python I used is this version: Python 3.6.3 |Anaconda custom (64-bit)| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)] on win32
This is the python install of PyCharm IDE - Miniconda3 on my Windows 10 account.
The first step is the install module with the pip tool:
pip install python-instagram
Collecting python-instagram
  Downloading python-instagram-1.3.2.tar.gz
Collecting simplejson (from python-instagram)
  Downloading simplejson-3.13.2-cp36-cp36m-win_amd64.whl (70kB)
    100% |████████████████████████████████| 71kB 816kB/s
Collecting httplib2 (from python-instagram)
  Downloading httplib2-0.10.3.tar.gz (204kB)
    100% |████████████████████████████████| 204kB 1.1MB/s
Requirement already satisfied: six in c:\users\catafest\miniconda3\lib\site-packages (from python-instagram)
Building wheels for collected packages: python-instagram, httplib2
...
Successfully built python-instagram httplib2
Installing collected packages: simplejson, httplib2, python-instagram
Successfully installed httplib2-0.10.3 python-instagram-1.3.2 simplejson-3.13.2
The next step is to take your Client Secret and Client ID from your Instagram account:
The next step is to set your Instagram API:

You can try some example from here.
I just got this error :...instagram.bind.InstagramAPIError: (400) OAuthAccessTokenException-The access_token provided is invalid.
I think the problem is Instagram because I search on the internet and many people come with this issue.
The team development of Instagram tell us to set some Permission:
All permissions require approval to be used out of Sandbox. Make sure to review our Platform Policies before submitting your app for review. To learn more about the review process, please read the Permissions Review documentation.
I try to use this but is not very clear for me.

Tuesday, December 5, 2017

Fix PyCharm error install python module from conda .

Today I fix an error about PyCharm and conda.
As you know :
Conda is an open source package management system and environmental management system that runs on Windows, macOS and Linux.
Also, Conda quickly installs, runs and updates packages dependency and environment management for any language—Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, FORTRAN.
This error is from PyCharm install python modules using error check from PyCharm (Alt+Enter keys):

The result of this install come with this error from conda :

Close your PyCharm and use this command into your shell-like administrator:

C:\WINDOWS\system32>conda config --show
C:\WINDOWS\system32>conda config --set force True
C:\WINDOWS\system32>conda update conda
C:\WINDOWS\system32>conda install conda anaconda
Fetching package metadata .............
Solving package specifications: .

# All requested packages already installed.
# packages in environment at C:\Users\catafest\Miniconda3:
#
anaconda                  5.0.1            py36h8316230_2
conda                     4.3.30           py36h7e176b0_0
C:\WINDOWS\system32>conda update --prefix C:\Users\catafest\Miniconda3 anaconda
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment C:\Users\catafest\Miniconda3:

The following packages will be UPDATED:

    conda-env: 2.6.0-0 --> 2.6.0-h36134e3_1
Proceed ([y]/n)? y

conda-env-2.6. 100% |###############################| Time: 0:00:00 163.59 kB/s
This command installs anaconda and updates it using my account catafest .
Start the I.D.E. PyCharm and after indexing all you can try to fix the python install module (Alt+Enter keys).
If the python modules are not into conda repo from PyCharm then you can use this command:
C:\WINDOWS\system32>conda install -c conda-forge opencv
Fetching package metadata ...............
Solving package specifications: .

# All requested packages already installed.
# packages in environment at C:\Users\catafest\Miniconda3:
#
opencv                    3.3.0                  py36_202    conda-forge
In this example I used OpenCV python module named into python script like cv2, see the next image:






Saturday, November 11, 2017

Using kivy python module with PyCharm IDE.

First, you need to download the last version of PyCharm IDE.
My PyCharm IDE put the python version 3 into a folder named Miniconda3.
I use the command shell to go to Scripts and I used pip to install the kivy python module, see:
Scripts>pip install kivy
Collecting kivy
  Downloading Kivy-1.10.0-cp36-cp36m-win_amd64.whl (3.5MB)
    100% |████████████████████████████████| 3.5MB 380kB/s
Collecting pygments (from kivy)
  Downloading Pygments-2.2.0-py2.py3-none-any.whl (841kB)
    100% |████████████████████████████████| 849kB 1.3MB/s
Collecting Kivy-Garden>=0.1.4 (from kivy)
  Downloading kivy-garden-0.1.4.tar.gz
Collecting docutils (from kivy)
  Downloading docutils-0.14-py3-none-any.whl (543kB)
    100% |████████████████████████████████| 552kB 1.8MB/s
Requirement already satisfied: requests 
...
Building wheels for collected packages: Kivy-Garden
...
Successfully built Kivy-Garden
Installing collected packages: pygments, Kivy-Garden, docutils, kivy
Successfully installed Kivy-Garden-0.1.4 docutils-0.14 kivy-1.10.0 pygments-2.2.0
I got one error about SDL and I put the SDL 2.0 into my windows system 32 folders.
The next step is to try to use this steps from the official website into the Scripts folder from Miniconda3.
python -m pip install --upgrade pip wheel setuptools
python -m pip install docutils pygments pypiwin32 kivy.deps.sdl2 kivy.deps.glew
python -m pip install kivy.deps.gstreamer
python -m pip install kivy.deps.angle
python -m pip install kivy
I make one project into my PyCharm editor and I use the default script:
from kivy.app import App
from kivy.uix.button import Button

class TestApp(App):
    def build(self):
        return Button(text='Hello World')

TestApp().run()
The result can be see into the next image:

Sunday, October 22, 2017

The Google Cloud Pub/Sub python module.

This is a test of google feature from cloud.google.com/pubsub web page.
The Google development team tell us about this service:
The Google Cloud Pub/Sub service allows applications to exchange messages reliably, quickly, and asynchronously. To accomplish this, a producer of data publishes a message to a Cloud Pub/Subtopic. A subscriber client then creates a subscription to that topic and consumes messages from the subscription. Cloud Pub/Sub persists messages that could not be delivered reliably for up to seven days. This page shows you how to get started publishing messages with Cloud Pub/Sub using client libraries.
The simple idea about this is:
Publisher applications can send messages to a topic, and other applications can subscribe to that topic to receive the messages.
I start with the installation of the python module using python version 2.7 and pip tool.
C:\Python27>cd Scripts

C:\Python27\Scripts>pip install --upgrade google-cloud-pubsub
Collecting google-cloud-pubsub
  Downloading google_cloud_pubsub-0.28.4-py2.py3-none-any.whl (79kB)
    100% |################################| 81kB 300kB/s
...
Successfully installed google-cloud-pubsub-0.28.4 grpc-google-iam-v1-0.11.4 ply-3.8 
psutil-5.4.0 pyasn1-modules-0.1.5 setuptools-36.6.0
The next steps come with some settings on google console, see this google page.
The default settings can be started and set with this command: gcloud init .
You need to edit this settings and app.yaml at ~/src/.../appengine/flexible/pubsub$ nano app.yaml.
After that, you set all of this using the command gcloud app deploy you can see the output at https://[YOUR_PROJECT_ID].appspot.com.
The main goal of this tutorial was to start and run the Google Cloud Pub/Sub service with python and this has been achieved.

Tuesday, October 10, 2017

The online editor for python and google .

This is a good online editor for python and google.
Like any online editor, some python modules are not available for online security reasons.
I do not know what python modules are implemented in this online editor.
I tested just sys and math python modules.
The Google Apps come with this tool integration like application for Google drive:
Edit your python file directly in your browser:
- Save it to Google Drive integrated with Google Drive
- Test it in your browser with Skulpt
- Use autocompletion code (CTRL+SPACE)
- No registration required and totally free
- Export your file
- Work offline
New python libraries partially supported: numpy, matplotlib.

Sunday, October 1, 2017

The capstone python module - disassembly framework.

The official python module comes with this info about this python module:
Capstone is a disassembly framework with the target of becoming the ultimate
the disasm engine for binary analysis and reversing in the security community.

Created by Nguyen Anh Quynh, then developed and maintained by a small community,
Capstone offers some unparalleled features:

- Support multiple hardware architectures: ARM, ARM64 (ARMv8), Mips, PPC & X86.

- Having clean/simple/lightweight/intuitive architecture-neutral API.

- Provide details on disassembled instruction (called “decomposer” by others).

- Provide semantics of the disassembled instruction, such as list of implicit
registers read & written.

- Implemented in pure C language, with lightweight wrappers for C++, Python,
Ruby, OCaml, C#, Java and Go available.

- Native support for Windows & *nix platforms (with OSX, Linux, *BSD & Solaris
have been confirmed).

- Thread-safe by design.

- Distributed under the open source BSD license.

Today I tested this python module with python version 2.7.
First I need to use a build of this python module from the official website.
I used binaries 32 bits like my python 2.7 and I tested with pip 2.7:
C:\Python27\Scripts>pip install capstone
Requirement already satisfied: capstone in c:\python27\lib\site-packages
Let's make a simple test with this python module:

C:\Python27>python.exe
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from capstone import (
...     Cs,
...     CS_ARCH_X86,
...     CS_MODE_32,
...     CS_OPT_SYNTAX_ATT,
... )
>>> mode=Cs(CS_ARCH_X86, CS_MODE_32)
>>> mode.syntax = CS_OPT_SYNTAX_ATT
>>> def D_ASM(code):
...     for address, size, mnemonic, op_str in mode.disasm_lite(code, offset=0x08048060):
...         print("0x{0:x}\t{1:d}\t{2:s}\t{3:s}".format(address, size,mnemonic, op_str))
...
>>> D_ASM(b"\xe1\x0b\x40\xb9\x20\x04\x81\xda\x20\x08\x02\x8b")
0x8048060       2       loope   0x804806d
0x8048062       1       incl    %eax
0x8048063       5       movl    $0xda810420, %ecx
0x8048068       2       andb    %cl, (%eax)
It seems to work very well.


Friday, September 22, 2017

The python-vlc python module.

The python module for vlc is named python-vlc.
This python module let you test libvlc API like the VLC video player.
You can install it easily with pip python tool.
C:\Python27\Scripts>pip2.7.exe install python-vlc
Collecting python-vlc
  Downloading python-vlc-1.1.2.tar.gz (201kB)
    100% |################################| 204kB 628kB/s
Installing collected packages: python-vlc
  Running setup.py install for python-vlc ... done
Successfully installed python-vlc-1.1.2
Let's see a simple example with this python module:
import os
import sys
import vlc
import pygame
 
def call_vlc(self, player):
 
    player.get_fps()
    player.get_time()
 
if len( sys.argv )< 2 or len( sys.argv )> 3:
        print 'Help: python vlc_001.py your_video.mp4'
else:
    pygame.init()
    screen = pygame.display.set_mode((800,600),pygame.RESIZABLE)
    pygame.display.get_wm_info()
    pygame.display.get_driver()

 
    # get path to movie specified as command line argument
    movie = os.path.expanduser(sys.argv[1])
    # see if movie is accessible
    if not os.access(movie, os.R_OK):
        print('Error: %s wrong read file: ' % movie)
        sys.exit(1)
 
    # make instane of VLC and create reference to movie.
    vlcInstance = vlc.Instance()
    media = vlcInstance.media_new(movie)
 
    # make new instance of vlc player
    player = vlcInstance.media_player_new()
 
    # start with a callback
    em = player.event_manager()
    em.event_attach(vlc.EventType.MediaPlayerTimeChanged, \
        call_vlc, player)
 
    # set pygame window id to vlc player
    win_id = pygame.display.get_wm_info()['window']
    if sys.platform == "win32": # for Windows
        player.set_hwnd(win_id)
 
    # load movie into vlc player instance
    player.set_media(media)
 
    # quit pygame mixer to allow vlc full access to audio device
    pygame.mixer.quit()
 
    # start movie play
    player.play()
 
    while player.get_state() != vlc.State.Ended:
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                sys.exit(2)
The base of this python script is to make an instance of vlc and put into the pygame display.
Another simple example:
C:\Python27>python.exe
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vlc
>>> inst = vlc.Instance()
Warning: option --plugin-path no longer exists.
Warning: option --plugin-path no longer exists.
>>> med = inst.media_new('rain.mp4')
>>> p = med.player_new_from_media()
>>> p.play()
0
>>>

Tuesday, September 19, 2017

The numba python module - part 002 .

Today I tested how fast is jit from numba python and fibonacci math function.
You will see strange output I got for some values.
First example:
import numba
from numba import jit
from timeit import default_timer as timer

def fibonacci(n):
    a, b = 1, 1
    for i in range(n):
        a, b = a+b, a
    return a
fibonacci_jit = jit(fibonacci)

start = timer()
fibonacci(100)
duration = timer() - start

startnext = timer()
fibonacci_jit(100)
durationnext = timer() - startnext

print(duration, durationnext)
The result of this run is:
C:\Python27>python numba_test_003.py
(0.00018731270733896962, 0.167499256682878)

C:\Python27>python numba_test_003.py
(1.6357787798437412e-05, 0.1683614083221368)

C:\Python27>python numba_test_003.py
(2.245186560569841e-05, 0.1758382003097716)

C:\Python27>python numba_test_003.py
(2.3093347480146938e-05, 0.16714964906130353)

C:\Python27>python numba_test_003.py
(1.5395564986764625e-05, 0.17471143739730277)

C:\Python27>python numba_test_003.py
(1.5074824049540363e-05, 0.1847134227837042)
As you can see the fibonacci function is not very fast.
The jit - just-in-time compile is very fast.
Let's see if the python source code may slow down.
Let's see the new source code with jit will not work well:
import numba
from numba import jit
from timeit import default_timer as timer

def fibonacci(n):
    a, b = 1, 1
    for i in range(n):
        a, b = a+b, a
    return a
fibonacci_jit = jit(fibonacci)

start = timer()
print fibonacci(100)
duration = timer() - start

startnext = timer()
print fibonacci_jit(100)
durationnext = timer() - startnext

print(duration, durationnext)
The result is this:
C:\Python27>python numba_test_003.py
927372692193078999176
1445263496
(0.0002334994022992635, 0.17628787910376)

C:\Python27>python numba_test_003.py
927372692193078999176
1445263496
(0.0006886307922204926, 0.17579169287387408)

C:\Python27>python numba_test_003.py
927372692193078999176
1445263496
(0.0008105123483657127, 0.18209553525407973)

C:\Python27>python numba_test_003.py
927372692193078999176
1445263496
(0.00025466830415606486, 0.17186550306131188)

C:\Python27>python numba_test_003.py
927372692193078999176
1445263496
(0.0007348174871807866, 0.17523103771560608)
The result for value 100 is not the same: 927372692193078999176 and 1445263496.
The first problem is:
The problem is that numba can't intuit the type of lookup. If you put a print nb.typeof(lookup) in your method, you'll see that numba is treating it as an object, which is slow.
The second problem is the output but can be from the same reason.
I test with value 5 and the result is :
C:\Python27>python numba_test_003.py
13
13
13
13
(0.0007258367409385072, 0.17057997338491704)

C:\Python27>python numba_test_003.py
13
13
(0.00033709872502270044, 0.17213235952108247)

C:\Python27>python numba_test_003.py
13
13
(0.0004836773333341886, 0.17184433415945508)

C:\Python27>python numba_test_003.py
13
13
(0.0006854233828482501, 0.17381272129120037)

Monday, September 18, 2017

The numba python module - part 001 .

Today I tested the numba python module.
This python module allows us to speed up applications with high-performance functions written directly in Python.
The numba python module works by generating optimized machine code using the LLVM compiler infrastructure at import time, runtime, or statically.
The code can be just-in-time compiled to native machine instructions, similar in performance to C, C++ and Fortran.
For the installation I used the pip tool:
C:\Python27>cd Scripts

C:\Python27\Scripts>pip install numba
Collecting numba
  Downloading numba-0.35.0-cp27-cp27m-win32.whl (1.4MB)
    100% |################################| 1.4MB 497kB/s
...
Installing collected packages: singledispatch, funcsigs, llvmlite, numba
Successfully installed funcsigs-1.0.2 llvmlite-0.20.0 numba-0.35.0 singledispatch-3.4.0.3

C:\Python27\Scripts>pip install numpy
Requirement already satisfied: numpy in c:\python27\lib\site-packages
The example test from official website working well:
The example source code is:
from numba import jit
from numpy import arange

# jit decorator tells Numba to compile this function.
# The argument types will be inferred by Numba when function is called.
@jit
def sum2d(arr):
    M, N = arr.shape
    result = 0.0
    for i in range(M):
        for j in range(N):
            result += arr[i,j]
    return result

a = arange(9).reshape(3,3)
print(sum2d(a))
The result of this run python script is:
C:\Python27>python.exe numba_test_001.py
36.0
Another example using just-in-time compile is used with Numba’s jit function:
import numba
from numba import jit

def fibonacci(n):
    a, b = 1, 1
    for i in range(n):
        a, b = a+b, a
    return a

print fibonacci(10)

fibonacci_jit = jit(fibonacci)
print fibonacci_jit(14)
Also, you can use jit is as a decorator:
@jit
def fibonacci_jit(n):
    a, b = 1, 1
    for i in range(n):
        a, b = a+b, a

    return a
Numba is a complex python module because use compiling.
First, compiling takes time, but will work especially for small functions.
The Numba python module tries to do its best by caching compilation as much as possible though.
Another note: not all code is compiled equally.

YARA another python module - part 002 .

This is another part of YARA python tutorial and the goal of this part is to install the Yara modules.
This is another python module about Yara named yara-python from VirusTotal.
The last tutorial uses the Yara python module.
The YARA modules provide extending features to allow us to define data structures and functions which can be used in your rules to express more complex conditions.
You can also write your own modules.
Some known modules used by YARA are:
  • PE
  • ELF
  • Cuckoo
  • Magic
  • Hash
  • Math
First, you need to install or reinstall YARA to the last version:
>>> yara.__version__
'3.6.3'
The Cuckoo module enables you to create YARA rules based on behavioral information generated by a Cuckoo sandbox.
C:\Python27\Scripts>pip install yara-python
Collecting yara-python
  Downloading yara_python-3.6.3-cp27-cp27m-win32.whl (606kB)
    100% |################################| 614kB 1.3MB/s
Installing collected packages: yara-python
Successfully installed yara-python-3.6.3
pip install cuckoo
Collecting cuckoo
  Downloading Cuckoo-2.0.4.4.tar.gz (3.1MB)
    100% |################################| 3.1MB 255kB/s
...
Successfully installed Mako-1.0.7 alembic-0.8.8 androguard-3.0.1 beautifulsoup4-4.5.3 
capstone-windows-3.0.4 chardet-2.3.0 click-6.6 colorama-0.3.7 cuckoo-2.0.4.4 django-1.8.4 
django-extensions-1.6.7 dpkt-1.8.7 ecdsa-0.13 egghatch-0.2.1 elasticsearch-5.3.0 
flask-sqlalchemy-2.1 httpreplay-0.2.1 jsbeautifier-1.6.2 jsonschema-2.6.0 olefile-0.43 
oletools-0.42 peepdf-0.3.6 pefile2-1.2.11 pillow-3.2.0 pyelftools-0.24 pymisp-2.4.54 
pymongo-3.0.3 python-dateutil-2.4.2 python-editor-1.0.3 python-magic-0.4.12 pythonaes-1.0 
requests-2.13.0 sflock-0.2.16 sqlalchemy-1.0.8 tlslite-ng-0.6.0 unicorn-1.0.1 wakeonlan-0.2.2
Let's test this python module:
>>> import cuckoo
>>> from cuckoo import *
>>> dir(cuckoo)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__',
 'auxiliary', 'common', 'compat', 'core', 'machinery', 'misc', 'plugins', 'processing', 
'reporting', 'signatures', 'web']
Let's test some yara modules:
>>> import yara
>>> rule = yara.compile(source='import \"pe\"')
>>> rule = yara.compile(source='import \"elf\"')
>>> rule = yara.compile(source='import \"cuckoo\"')
>>> rule = yara.compile(source='import \"math\"')
I could not use the YARA modules: hash and magic.
I will solve this problem in the future.
You can also write your own modules ( see this webpage ).

Friday, September 1, 2017

The beauty of Python: subprocess module - part 004 .

This series of python tutorials that we started at the beginning of this blog and called "The beauty of Python" is part of the series of tutorials aimed at the simplicity and beauty of the Python programming language.
The main goal for us is how to use this programming language in everyday life with different tasks.
Today I will come up with examples to cover this goal and show you how to use the subprocess python module.
  • using the PowerShell with python :
  • >>> import subprocess
    >>> process=subprocess.Popen(["powershell","Get-Childitem C:\\Windows\\*.log"],stdout=subprocess.PIPE);
    >>> result=process.communicate()[0]
    >>> print result
  • get and print the hostname :
  • >>> print subprocess.check_output("hostname")
  • print the output of ping command :
  • >>> print subprocess.check_output("ping localhost", shell=True)
  • print the output of dir command :
  • >>> cmd = 'dir *'
    >>> supcmd = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    >>> print supcmd.communicate()[0]
    
  • run the python script like python shell :
  • >>> import sys
    >>> import subprocess
    >>> pid = subprocess.Popen([sys.executable, "calc.py"])

Monday, August 21, 2017

Using pip into shell to install and use pymunk.

The tutorial for today will show how to use pip into the python shell to install a python package.
The first step is shown in the next image:

Friday, August 18, 2017

The Google Cloud SDK - part 002 .

The next part of my tutorials about the Google Cloud SDK comes with some info about the project.
As you know I used the default sample app engine hello word standard application.
The goal is to understand how it works by working with Google's documentation and examples.
Into this project folder we have this files:
08/17/2017  11:12 PM                98 app.yaml
08/17/2017  11:12 PM               854 main.py
08/17/2017  11:12 PM               817 main_test.py
Let's see what these files contain:
First is app.yaml and come with:
runtime: python27
api_version: 1
threadsafe: true

handlers:
- url: /.*
  script: main.app
The next is main.py file:
# Copyright 2016 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import webapp2


class MainPage(webapp2.RequestHandler):
    def get(self):
        self.response.headers['Content-Type'] = 'text/plain'
        self.response.write('Hello, World!')


app = webapp2.WSGIApplication([
    ('/', MainPage),
], debug=True)
The last from this folder is main_test.py :
# Copyright 2016 Google Inc. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import webtest

import main


def test_get():
    app = webtest.TestApp(main.app)

    response = app.get('/')

    assert response.status_int == 200
    assert response.body == 'Hello, World!'
The app.yaml file is used to configure your App Engine application's settings of the project.
You can have many application-level configuration files (dispatch.yaml, cron.yaml, index.yaml, and queue.yaml).
This all type of configuration files are included in the top level app directory ( in this case: hello_world).
Let's see some common gcloud commands:
  • gcloud app deploy  --project XXXXXX - deploy your project;
  • gcloud app browse - show your project running into your browser;
  • gcloud components list - show all available components;
  • gcloud components update - update all gcloud components;
  • gcloud projects list --limit=10 - show all projects with a limit number;
Let's test some changes:
First, change the text from main.py file with something else:
self.response.write('Hello, World!')
Now use this commands:
C:\Python27\python-docs-samples\appengine\standard\hello_world>gcloud app deploy
C:\Python27\python-docs-samples\appengine\standard\hello_world>gcloud app browse
The result is shown in your browser.
You can read about this files into google documentation page - here.
Also some gcloud commands and reference you can read here.

Thursday, August 17, 2017

The Google Cloud SDK - part 001 .

This tutorial will cover this steps into development with Google Cloud SDK and Python version 2.7:

  • install the Google Cloud SDK on the computer;
  • make settings online for your Google project to use Google Cloud SDK;
  • run the online project of Google Cloud SDK;
  • make setting into your computer to run the local project ;

First, you need to download the Google Cloud SDK and run it.


After GUI install a window command will ask you to set the default project for your work.
Welcome to the Google Cloud SDK! Run "gcloud -h" to get the list of available commands.
---
Welcome! This command will take you through the configuration of gcloud.

Your current configuration has been set to: [default]

You can skip diagnostics next time by using the following flag:
  gcloud init --skip-diagnostics

Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic (1/1 checks) passed.

You must log in to continue. Would you like to log in (Y/n)?  Y
...
The next step is to start online to deploying a Hello World app with: Deploy a Hello World app:

This will start an online tutorial into the right area of the screen with all commands and steps for your Google Cloud SDK online project.
Follow this steps and in the end will see how the online Google Cloud SDK project will show: Hello, World! into your browser.
The next step is to make a local project and run it.
You can use the python docs sample from GoogleCloudPlatform but is not the same with the online example.
To download the GoogleCloudPlatform sample use git command:
C:\Python27>git clone https://github.com/GoogleCloudPlatform/python-docs-samples
Cloning into 'python-docs-samples'...
remote: Counting objects: 12126, done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 12126 (delta 1), reused 10 (delta 1), pack-reused 12106
Receiving objects: 100% (12126/12126), 3.37 MiB | 359.00 KiB/s, done.
Resolving deltas: 100% (6408/6408), done.

C:\Python27>cd python-docs-samples/appengine/standard/hello_world
To start this sample into your google project you need to use this:
C:\Python27\python-docs-samples\appengine\standard\hello_world>gcloud app deploy app.yaml --project encoded-metrics-147522
Services to deploy:

descriptor:      [C:\Python27\python-docs-samples\appengine\standard\hello_world\app.yaml]
source:          [C:\Python27\python-docs-samples\appengine\standard\hello_world]
target project:  [encoded-metrics-147522]
target service:  [default]
target version:  [20170817t234925]
target url:      [https://encoded-metrics-147522.appspot.com]


Do you want to continue (Y/n)?  Y

Beginning deployment of service [default]...
#============================================================#
#= Uploading 5 files to Google Cloud Storage                =#
#============================================================#
File upload done.
Updating service [default]...done.
Waiting for operation [apps/encoded-metrics-147522/operations/XXXXXX] to complete...done.
Updating service [default]...done.
Deployed service [default] to [https://XXXXXX.appspot.com]

You can stream logs from the command line by running:
  $ gcloud app logs tail -s default

To view your application in the web browser run:
  $ gcloud app browse

C:\Python27\python-docs-samples\appengine\standard\hello_world>gcloud app browse
Opening [https://XXXXXX.appspot.com] in a new tab in your default browser.

C:\Python27\python-docs-samples\appengine\standard\hello_world>
This will start your application with the text - Hello, World! into your browser address bar with this web address: XXXXXX.appspot.com.



Wednesday, August 16, 2017

The DreamPie - interactive shell .

The DreamPie was designed to bring you a great interactive shell Python experience.
There are two ways to install the DreamPie:
  • cloning the git repository;
  • downloading a release.
You can read about installation and download here.
To run it just try the dreampie.exe with your python shell, I used with my python 2.7 version:
C:\DreamPie>dreampie.exe --hide-console-window c:\Python27\python.exe
Let's see one screenshot of this running command:

Also, I tested with Python 3.6.2 and works well.
The main window is divided into the history box and the code box.
The history box lets you view previous commands and their output.
The code box for writing your code.
Some keys I used:

  • Ctr+Enter - run the code;
  • Ctr+up / down arrow - adds the previous / next source code;
  • Ctr+Space - show code completions;
  • Ctr+T - open a new tab code;
  • Ctr+W - close the tab code;
  • Ctr+S - save your work history into HTML file.

You can set your font, colors and many features.
I make the installation into C:\DreamPie folder, and comes with all these folders and files:
C:\DreamPie>tree
Folder PATH listing for volume free-tutorials
Volume serial number is 000000FF 0EB1:091D
C:.
├───data
│   ├───language-specs
│   ├───subp-py2
│   │   └───dreampielib
│   │       ├───common
│   │       └───subprocess
│   └───subp-py3
│       └───dreampielib
│           ├───common
│           └───subprocess
├───gtk-2.0
│   ├───cairo
│   ├───gio
│   ├───glib
│   ├───gobject
│   ├───gtk
│   └───runtime
│       ├───bin
│       ├───etc
│       │   ├───bash_completion.d
│       │   ├───fonts
│       │   ├───gtk-2.0
│       │   └───pango
│       ├───lib
│       │   ├───gdk-pixbuf-2.0
│       │   │   └───2.10.0
│       │   │       └───loaders
│       │   ├───glib-2.0
│       │   │   └───include
│       │   └───gtk-2.0
│       │       ├───2.10.0
│       │       │   └───engines
│       │       ├───include
│       │       └───modules
│       └───share
│           ├───aclocal
│           ├───dtds
│           ├───glib-2.0
│           │   ├───gdb
│           │   ├───gettext
│           │   │   └───po
│           │   └───schemas
│           ├───gtk-2.0
│           ├───gtksourceview-2.0
│           │   ├───language-specs
│           │   └───styles
│           ├───icon-naming-utils
│           ├───themes
│           │   ├───Default
│           │   │   └───gtk-2.0-key
│           │   ├───Emacs
│           │   │   └───gtk-2.0-key
│           │   ├───MS-Windows
│           │   │   └───gtk-2.0
│           │   └───Raleigh
│           │       └───gtk-2.0
│           └───xml
│               └───libglade
└───share
    ├───applications
    ├───man
    │   └───man1
    └───pixmaps

Saturday, August 12, 2017

The flickrapi python module .

Flickr is one of the most popular photo hosting websites.
The development team let us test the API and with a plenty of opportunities to use Flickr.
Today I tested the flickrapi python module.
I used the python 2.7 version and pip to install it.
C:\>cd Python27

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install flickrapi
Collecting flickrapi
  Downloading flickrapi-2.3-py2-none-any.whl
...
  Running setup.py install for oauthlib ... done
Successfully installed flickrapi-2.3 oauthlib-2.0.2 requests-oauthlib-0.8.0 requests-toolbelt-0.8.0
You need to have/make an API key into Account Settings, see my API key:
Your API keys You have 1 API key registered to this account.
Then you need to know your user_id.
This can be done by asking api online with your username and the result will be something like this: xxxxxxxx@Nxx.
The example I used is from documentation and I change with my API key to test this python module.


>>>import flickrapi
>>>
>>>api_key = u'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
>>>api_secret = u'YYYYYYYYYYYYYYYYYYYYYYY'
>>>
>>>flickr = flickrapi.FlickrAPI(api_key, api_secret)
>>>photos = flickr.photos.search(user_id='xxxxxxxx@Nxx', per_page='10')
>>>sets = flickr.photosets.getList(user_id='xxxxxxxx@Nxx')
>>> dir(sets)
['__class__', '__contains__', '__copy__', '__deepcopy__', '__delattr__', '__delitem__', '__doc__',
 '__format__', '__getattribute__', '__getitem__', '__hash__', '__init__', '__iter__', '__len__', 
'__new__', '__nonzero__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__setattr__',
 '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '_init', 'addnext', 'addprevious', 
'append', 'attrib', 'base', 'clear', 'cssselect', 'extend', 'find', 'findall', 'findtext', 'get', 
'getchildren', 'getiterator', 'getnext', 'getparent', 'getprevious', 'getroottree', 'index', 'insert',
 'items', 'iter', 'iterancestors', 'iterchildren', 'iterdescendants', 'iterfind', 'itersiblings', 
'itertext', 'keys', 'makeelement', 'nsmap', 'prefix', 'remove', 'replace', 'set', 'sourceline', 
'tag', 'tail', 'text', 'values', 'xpath']
>>> print sets.attrib['stat']
ok
>>> sets.find('photosets').attrib
{'total': '4', 'perpage': '4', 'page': '1', 'pages': '1'}
>>> print "upload a photo via browser - set permisions"
upload a photo via browser - set permisions
>>> flickr.authenticate_via_browser(perms='write')
127.0.0.1 - - [12/Aug/2017 22:05:00] "GET /?oauth_token=xxxxxx-xxxxx&oauth_verifier=xxxxxxxxx HTTP/1.1" 200 -
>>> flickr.upload("0001.gif")

>>> print "try show title of my photos and id"
try show title of my photos and id
for child in photos:
  for i in child:
    photo_set = []
    try:
      title = i.attrib['title']
      print title
    except Exception, e:
      print e
    sets = flickr.photos_getAllContexts(photo_id = i.attrib["id"])
    for j in sets:
      if j.tag == "set":
        photo_set.append(j.attrib["id"])
        print photo_set
The result is my photos title and id:
painting_world
DSC_0112
['72157632077721715']
DSC_0111
['72157632077721715']
DSC_0113
['72157632077721715']
DSC_0110
['72157632077721715']
DSC_0094
['72157632077721715']
DSC_0091
['72157632077721715']
DSC_0108
I tested wit Python 362 version:
C:\Python27>cd ..

C:\>cd Python362

C:\Python362>cd Scripts

C:\Python362\Scripts>pip install flickrapi
Collecting flickrapi
  Downloading flickrapi-2.3-py3-none-any.whl
Collecting requests>=2.2.1 (from flickrapi)
  Downloading requests-2.18.3-py2.py3-none-any.whl (88kB)
    100% |████████████████████████████████| 92kB 1.4MB/s
Collecting six>=1.5.2 (from flickrapi)
  Using cached six-1.10.0-py2.py3-none-any.whl
...
Successfully installed certifi-2017.7.27.1 chardet-3.0.4 flickrapi-2.3 idna-2.5 oauthlib-2.0.2 requests-2.18.3
 requests-oauthlib-0.8.0 requests-toolbelt-0.8.0 six-1.10.0 urllib3-1.22

Thursday, August 10, 2017

The pysolar - python module.

Today I tested another python module named pysolar with python 2.7 and 3.6.2 versions.
You can read more about this python module here:
Pysolar is a collection of Python libraries for simulating the irradiation of any point on earth by the sun. It includes code for extremely precise ephemeris calculations and more.
Using pip under Windows OS to install this python module:
C:\Python27\Scripts>pip install pysolar
...
Successfully built pysolar
Installing collected packages: pysolar
Successfully installed pysolar-0.7
I try also with python 3.6.2 version:
C:\Python362\Scripts>pip install pysolar
Collecting pysolar
  Using cached pysolar-0.7.tar.gz
Installing collected packages: pysolar
  Running setup.py install for pysolar ... done
Successfully installed pysolar-0.7
Let's start with a simple example to calculate the angle between the sun and a plane tangent to the earth where
you are:
C:\Python362>python.exe
Python 3.6.2 (v3.6.2:5fd33b5, Jul  8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pysolar.solar import *
>>> import datetime
>>> my_data = datetime.datetime.now()
>>> print (my_data)
2017-08-10 18:12:01.997373
>>> get_altitude_fast(47.4616,26.3015,datetime.datetime.now())
-6.393190003993212
Let's try the scattering of light by the atmosphere, though it uses an
atmospheric model based on data taken in the United States:
>>> latitude_deg =47.2741
>>> longitude_deg = 26.185
>>> my_data = datetime.datetime.now()
>>> altitude_deg = get_altitude(latitude_deg, longitude_deg,my_data)
C:\Python362\lib\site-packages\pysolar\time.py:105: UserWarning: I don't know about leap seconds after 2015
  (leap_seconds_base_year + len(leap_seconds_adjustments) - 1)
>>> azimuth_deg = get_azimuth(latitude_deg, longitude_deg,  my_data)
>>> radiation.get_radiation_direct(my_data, altitude_deg)
634.4916522219954
About this issue:I don't know about leap seconds after 2015 read here.

Using Flask and Flask-WTF to build websites in Python 2.7 and 3.6.2 - part 002.

This is one update of my old tutorial about Flask and python 2.7 under Linux from here.
The default install of python 2.7 and pip is very simple - read this tutorial.
First, you need to install the Flask python module under Windows 10:
C:\Python27\Scripts>pip install Flask
Collecting Flask
Downloading Flask-0.11.1-py2.py3-none-any.whl (80kB)
100% |################################| 81kB 564kB/s
Collecting click>=2.0 (from Flask)
Downloading click-6.6-py2.py3-none-any.whl (71kB)
100% |################################| 71kB 1.3MB/s
Collecting Werkzeug>=0.7 (from Flask)
Downloading Werkzeug-0.11.11-py2.py3-none-any.whl (306kB)
100% |################################| 307kB 231kB/s
Collecting Jinja2>=2.4 (from Flask)
Downloading Jinja2-2.8-py2.py3-none-any.whl (263kB)
100% |################################| 266kB 890kB/s
Collecting itsdangerous>=0.21 (from Flask)
Downloading itsdangerous-0.24.tar.gz (46kB)
100% |################################| 51kB 1.3MB/s
Collecting MarkupSafe (from Jinja2>=2.4->Flask)
Downloading MarkupSafe-0.23.tar.gz
Installing collected packages: click, Werkzeug, MarkupSafe, Jinja2, itsdangerous
, Flask
Running setup.py install for MarkupSafe ... done
Running setup.py install for itsdangerous ... done
Successfully installed Flask-0.11.1 Jinja2-2.8 MarkupSafe-0.23 Werkzeug-0.11.11
click-6.6 itsdangerous-0.24
The simple example of the running Flask python module is one script named hello.py:
from flask import Flask
app = Flask(__name__)

@app.route("/")
def hello():
    return "Hello World!"

if __name__ == "__main__":
    app.run()
Using the command to run the script and show the result into your browser:
C:\Python27>python.exe hello.py
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
Now the installation of Flask-WTF python module:
C:\Python27\Scripts>pip install Flask-WTF
Collecting Flask-WTF
  Downloading Flask_WTF-0.14.2-py2.py3-none-any.whl
Collecting WTForms (from Flask-WTF)
  Downloading WTForms-2.1.zip (553kB)
    100% |################################| 563kB 1.7MB/s
Requirement already satisfied: Flask in c:\python27\lib\site-packages (from Flask-WTF)
Requirement already satisfied: click>=2.0 in c:\python27\lib\site-packages (from Flask->Flask-WTF)
Requirement already satisfied: Werkzeug>=0.7 in c:\python27\lib\site-packages (from Flask->Flask-WTF)
Requirement already satisfied: Jinja2>=2.4 in c:\python27\lib\site-packages (from Flask->Flask-WTF)
Requirement already satisfied: itsdangerous>=0.21 in c:\python27\lib\site-packages (from Flask->Flask-WTF)
Requirement already satisfied: MarkupSafe>=0.23 in c:\python27\lib\site-packages (from Jinja2>=2.4->Flask->Flask-WTF)
Installing collected packages: WTForms, Flask-WTF
  Running setup.py install for WTForms ... done
Successfully installed Flask-WTF-0.14.2 WTForms-2.1
Now I try with Python 3.6.2 with both python modules and works great.
C:\Python362\Scripts>pip3.6.exe install flask
...
Successfully installed Jinja2-2.9.6 MarkupSafe-1.0 Werkzeug-0.12.2 click-6.7 flask-0.12.2 itsdangerous-0.24

C:\Python362\Scripts>Flask-WTF
'Flask-WTF' is not recognized as an internal or external command,
operable program or batch file.

C:\Python362\Scripts>pip3.6.exe install Flask-WTF
...
Successfully installed Flask-WTF-0.14.2 WTForms-2.1

Wednesday, July 26, 2017

The gtts python module.

This python module named gtts will create an mp3 file from spoken text via the Google TTS (Text-to-Speech) API.
The installation of the gtts python module under Windows 10.
C:\Python27\Scripts>pip install gtts
Collecting gtts
  Downloading gTTS-1.2.0.tar.gz
Requirement already satisfied: six in c:\python27\lib\site-packages (from gtts)
Requirement already satisfied: requests in c:\python27\lib\site-packages (from gtts)
Collecting gtts_token (from gtts)
  Downloading gTTS-token-1.1.1.zip
Requirement already satisfied: chardet<3 .1.0="">=3.0.2 in c:\python27\lib\site-packages (from requests->gtts)
Requirement already satisfied: certifi>=2017.4.17 in c:\python27\lib\site-packages (from requests->gtts)
Requirement already satisfied: idna<2 .6="">=2.5 in c:\python27\lib\site-packages (from requests->gtts)
Collecting urllib3<1 .22="">=1.21.1 (from requests->gtts)
  Using cached urllib3-1.21.1-py2.py3-none-any.whl
Installing collected packages: gtts-token, gtts, urllib3
  Running setup.py install for gtts-token ... done
  Running setup.py install for gtts ... done
  Found existing installation: urllib3 1.22
    Uninstalling urllib3-1.22:
      Successfully uninstalled urllib3-1.22
Successfully installed gtts-1.2.0 gtts-token-1.1.1 urllib3-1.21.1
Let's see a basic example:
from gtts import gTTS
import os
import pygame.mixer
from time import sleep
 
user_text=input("Type your text: ")
 
translate=gTTS(text=user_text ,lang='en')
translate.save('output.wav')

pygame.mixer.init()
path_name=os.path.realpath('output.wav')
real_path=path_name.replace('\\','\\\\')
pygame.mixer.music.load(open(real_path,"rb"))
pygame.mixer.music.play()
while pygame.mixer.music.get_busy():
    sleep(1)
The text will be taken by input into a user_text variable.
You need to type the text into quotes also you will get an error.
The result will be one audio file named output.wav and play it by pygame python module.
This uses the default voices for all languages. I don't find a way to change this voices with python.

Monday, July 24, 2017

Fix Gimp with python script.

Today I will show you how python language can help GIMP users.
From my point of view, Gimp does not properly import frames from GIF files.
This program imports GIF files in this way:

Using the python module, you can get the correct frames from the GIF file.
Here's my script that uses the python PIL module.
import sys
from PIL import Image, ImageSequence
try:
        img = Image.open(sys.argv[1])
except IOError:
        print "Cant load", infile
        sys.exit(1)

pal = img.getpalette()
prev = img.convert('RGBA')
prev_dispose = True
for i, frame in enumerate(ImageSequence.Iterator(img)):
    dispose = frame.dispose

    if frame.tile:
        x0, y0, x1, y1 = frame.tile[0][1]
        if not frame.palette.dirty:
            frame.putpalette(pal)
        frame = frame.crop((x0, y0, x1, y1))
        bbox = (x0, y0, x1, y1)
    else:
        bbox = None

    if dispose is None:
        prev.paste(frame, bbox, frame.convert('RGBA'))
        prev.save('result_%03d.png' % i)
        prev_dispose = False
    else:
        if prev_dispose:
            prev = Image.new('RGBA', img.size, (0, 0, 0, 0))
        out = prev.copy()
        out.paste(frame, bbox, frame.convert('RGBA'))
        out.save('result_%03d.png' % i)
Name the python script with convert_gif.py and then you can use it on the GIF file as follows:
C:\Python27>python.exe convert_gif.py 0001.gif
The final result has a smaller number of images than in Gimp, but this was to be expected.

Saturday, July 22, 2017

About py-translate python module.

This python module is used for translating text in the terminal.
You can read and see examples with this API on this web page.
Features

  • Fast! Translate an entire book in less than 5 seconds.
  • Made for Python 3 but still works on Python 2
  • Fast and easy to install, easy to use
  • Supports translation from any language
  • Highly composable interface, the power of Unix pipes and filters.
  • Simple API and documentation

Installation 
C:\>cd Python27

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install py-translate
Collecting py-translate
  Downloading py_translate-1.0.3-py2.py3-none-any.whl (61kB)
    100% |################################| 61kB 376kB/s
Installing collected packages: py-translate
Successfully installed py-translate-1.0.3

C:\Python27\Scripts>
Let's test it with a simple example:
>>> import translate
>>> dir(translate)
['TestLanguages', 'TestTranslator', '__author__', '__build__', '__builtins__', '__copyright__', '__doc__', '__file__', '__license__', '__name__', '__package__', '__path__', '__title__', '__version__', 'accumulator', 'coroutine', 'coroutines', 'languages', 'print_table', 'push_url', 'set_task', 'source', 'spool', 'tests', 'translation_table', 'translator', 'write_stream']
>>> from translate import translator
>>> translator('ro', 'en', 'Consider ca dezvoltarea personala este un pas important')
[[[u'I think personal development is an important step', u'Consider ca dezvoltarea personala este un pas important', None, None, 0]], None, u'ro']
>>>

Wednesday, July 19, 2017

PyCharm an integrated development environment.

The development team comes with this integrated development environment (I.D.E.) named: PyCharm.
They tell us about this integrated development environment:
Python IDE with a complete set of tools for productive development with Python programming language. In addition, the IDE provides high-class capabilities for professional Web development with Django framework and Google App Engine. It has powerful coding assistance, navigation, a lot of refactoring features, tight integration with various Version Control Systems, Unit testing, powerful all-singing all-dancing Debugger, and entire customization. PyCharm is developer driven IDE. It was developed with the aim of providing you almost everything you need for your comfortable and productive development!
I usually use it and from my point of view, it is very quick to learn and use.
Also, some features like Servers come is supported in the Professional edition only.
The menu is easy to remember and the project settings come automatically.
You can start from the main menu: File - New Project...
A little more complicated element is putting new scripts into the project File - New.
The python file can be added and the type of python version can by editing the configuration menu: Run - Edit Configurations...
The editor comes with unindent and convert fro tabs and spaces.
This I.D.E. comes with common version control operations integrations, like:
  • CVS Reference
  • Git Reference
  • Mercurial Reference
  • Perforce Reference
  • Subversion Reference
Also, some official tutorials can also be found on the official website.
You can read more about this software here.

The JetBrainsTV is the official youtube channel of this I.D.E.

Monday, July 10, 2017

Python tutor - web tool for python programming.

The website comes with this intro about this web tool.
Python Tutor, created by Philip Guo, helps people overcome a fundamental barrier to learning programming: understanding what happens as the computer runs each line of source code.
Using this tool, you can write Python, Java, JavaScript, TypeScript, Ruby, C, and C++ code in your web browser and visualize what the computer is doing step-by-step as it runs your code.
Over 3.5 million people in over 180 countries have used Python Tutor to visualize over 30 million pieces of code, often as a supplement to textbooks, lectures, and online tutorials.

I tested and worked very well.
You can use python programming language 2.7 and 3.6 versions.
No need to import python modules, you will get an error.
Just programming on the fly to test and see the result.
The website comes with some example to see how to deal with this tool.
Let's see some examples:

example with factorial :

# dumb recursive factorial
def fact(n):
    if (n <= 1):
        return 1
    else:
        return n * fact(n - 1)

print(fact(6))

example with for - else:

# find primes using a for-else construct
for n in range(2, 10):
    x_range = range(2, n)
    for x in x_range:
        if n % x == 0:
            break
    else:
        # loop fell through without finding a factor
        print(n)

example with inputs:

prefix = "Hello "

n1 = raw_input("Enter your name")

n2 = raw_input("Enter another name")

res = prefix + n1 + " and " + n2
print(res)

Run your script just press: Visualize Execution or Live Programming Mode buttons and the will run step by step with:
First, Back, Forward and Last.
One good feature of this tool - with a single line of JavaScript code, you can embed a Python Tutor visualization within any webpage.
Another good feature is COLLABORATE to learn together - this allow us to give and get direction with real-time python programming.
Can be a good tool for python chat users.
Let's show you a screenshot to see how this tool working with python scripting.

Sunday, July 9, 2017

The speech python module.

About this python module can be read here.
It's a little undocumented and I have not found tutorials about this python module but I tested with a simple example.
I'm sure he can do more than I tried with my example.
First, the install of this python module:
C:\Python27\Scripts>pip install speech
Collecting speech
  Downloading speech-0.5.2.tar.gz
Installing collected packages: speech
  Running setup.py install for speech ... done
Successfully installed speech-0.5.2
Let's see more about this python module:
>>> dir(speech)
['Listener', '_ListenerBase', '_ListenerCallback', '__builtins__', '__doc__', '__file__', '__name__', '__package__'
, '_constants', '_ensure_event_thread', '_eventthread', '_handlerqueue', '_listeners', '_recognizer', 
'_startlistening', '_voice', 'gencache', 'input', 'islistening', 'listenfor', 'listenforanything', 'pythoncom',
 'say', 'stoplistening', 'thread', 'time', 'win32com']>>> help(speech)
Help on module speech:

NAME
    speech - speech recognition and voice synthesis module.

FILE
    c:\python27\lib\site-packages\speech.py

DESCRIPTION
    Please let me know if you like or use this module -- it would make my day!

    speech.py: Copyright 2008 Michael Gundlach  (gundlach at gmail)
    License: Apache 2.0 (http://www.apache.org/licenses/LICENSE-2.0)

    For this module to work, you'll need pywin32 (http://tinyurl.com/5ezco9
    for Python 2.5 or http://tinyurl.com/5uzpox for Python 2.4) and
    the Microsoft Speech kit (http://tinyurl.com/zflb).


    Classes:
        Listener: represents a command to execute when phrases are heard.

    Functions:
        say(phrase): Say the given phrase out loud.
        input(prompt, phraselist): Block until input heard, then return text.
        stoplistening(): Like calling stoplistening() on all Listeners.
        islistening(): True if any Listener is listening.
        listenforanything(callback): Run a callback when any text is heard.
        listenfor(phraselist, callback): Run a callback when certain text is heard.
Let's make a simple example with one script that tells us something:
>>> speech.say('Hello Catalin George')
The result of this line of code will be heard into your audio device like Hello Catalin George.

Saturday, July 8, 2017

Python Qt4 - part 006.

Today I will deal with QFileDialog widget.
You can read the more about this widget here.
This allows us to open a dialog to load a resource - a file.
The example comes with the base PyQt4 application window with a my_example dialog from fileDialogSample python class.
Into this python class, I have some variable for the file: file_name, data and my_file_open.
The my_text_edit for text area and my_button to open the QFileDialog.
Also, the vbox variable to put all on QVBoxLayout from the application.
Let's see the example:
import sys
from PyQt4 import QtGui
from PyQt4 import QtCore

class fileDialogSample(QtGui.QWidget):
    def __init__(self):
        QtGui.QWidget.__init__(self)

        #make text area and button
        self.my_text_edit = QtGui.QTextEdit()
        self.my_button = QtGui.QPushButton('Select File', self)

        #open the showDialog
        self.my_button.clicked.connect(self.showDialog)

        #put all into application area
        vbox = QtGui.QVBoxLayout()
        vbox.addWidget(self.my_text_edit)
        vbox.addWidget(self.my_button)
        self.setLayout(vbox)

        #set title and geometry of application
        self.setWindowTitle('File Dialog example')
        self.setGeometry(50, 50, 300, 300)

    #make a function with seeting for my QFileDialog
    def showDialog(self):
        file_name = QtGui.QFileDialog.getOpenFileName(self, 'Open file', 'C://')
        my_file_open = open(file_name)
        data = my_file_open.read()
        self.my_text_edit.setText(data)
        my_file_open.close()

#run the application 
app = QtGui.QApplication(sys.argv)
my_dialog = fileDialogSample()
my_dialog.show()
sys.exit(app.exec_())
Now, just press the Select File button, take a text file and open it.

Friday, July 7, 2017

Python Qt4 - part 005.

Another example with PyQt4 that allow seeing images over the internet.
Here's another simple example with PyQt4 that allows you to view images on the internet.
You can use any image on the internet to display with this python script.
This example is done in two steps:
  • take a single image from the internet - httplib python module;
  • displaying - PyQt4 python module
from PyQt4 import QtGui
import sys
import httplib

def getTempPNG():
   conn = httplib.HTTPConnection("www.meteoromania.ro")
   conn.request("GET", "/sateliti/img/id814/id814_2017070718.png")
   return conn.getresponse().read()

def main():
   app = QtGui.QApplication(sys.argv)
   png = getTempPNG()
   pixmap = QtGui.QPixmap()
   pixmap.loadFromData(png)
   label = QtGui.QLabel()
   label.setPixmap(pixmap)
   label.setWindowTitle('METEOSAT-10 Thermal Infrared Channel 10.8 micrometers Glowing temperature')
   label.show()
   app.exec_()

if __name__ == '__main__':
   main()
The result can be seen in this screenshot:

Python Qt4 - part 004.

Another tutorial about PyQt4 with QLCDNumber widget displays a number with LCD-like digits.
This tutorial will show you how to deal with this widget.
First, you need to know more about QLCDNumber, so take a look here.
The first example is very simple and will show just one digit, see:
import sys
from PyQt4.QtCore import *
from PyQt4.QtGui import *

class Digit(QWidget):

    def __init__(self, parent=None):
        QWidget.__init__(self, parent)
        self.setWindowTitle("One digit")
        lcd = QLCDNumber(self)

app = QApplication(sys.argv)
ls = Digit()
ls.show()
sys.exit(app.exec_())
Now, the next step is to send data to this digit.
One good example is with one slider.
The position of the slider will be send to the QLCDNumber.
How can do that? Will need a vbox to put the QLCDNumber and the slider and then using signal and slot.
Let's see the example:
import sys
from PyQt4.QtCore import *
from PyQt4.QtGui import *
class Digit(QWidget):

    def __init__(self, parent=None):
        QWidget.__init__(self, parent)
        #make widgets
        self.setWindowTitle("One digit with slider")
        lcd = QLCDNumber(self)
        slider = QSlider(Qt.Horizontal, self)
        #set layout variable vbox
        vbox = QVBoxLayout()
        #add widgests
        vbox.addWidget(lcd)
        vbox.addWidget(slider)
        #set the vbox to layout
        self.setLayout(vbox)
        #create signal to slot
        self.connect(slider, SIGNAL("valueChanged(int)"),lcd, SLOT("display(int)"))
        self.resize(200, 170)
if __name__ == '__main__':
    app = QApplication(sys.argv)
    ls = Digit()
    ls.show()
    sys.exit(app.exec_())
In the source code in the example, you can see the comments that mark the steps of creating and executing the python script.
Let's try another example with a digital clock:
import sys
from PyQt4 import QtCore, QtGui

class digital_clock(QtGui.QLCDNumber):
    def __init__(self, parent=None):
        super(digital_clock, self).__init__(parent)
        self.setSegmentStyle(QtGui.QLCDNumber.Filled)
        #the defaul is 5 , change to 8 for seconds
        self.setDigitCount(5)
        self.setWindowTitle("Digital Clock")
        self.resize(200, 70)
        timer = QtCore.QTimer(self)
        timer.timeout.connect(self.showTime)
        timer.start(1000)
        self.showTime()

    def showTime(self):
        time = QtCore.QTime.currentTime()
        text = time.toString('hh:mm')
        #if you setDigitsCount to 8
        #uncomment the next line of code
        #text = time.toString('hh:mm:ss')
        if (time.second() % 2) == 0:
            text = text[:2] + ' ' + text[3:]
        self.display(text)

if __name__ == '__main__':
    app = QtGui.QApplication(sys.argv)
    clock = digital_clock()
    clock.show()
    sys.exit(app.exec_())
If you want to see seconds, then you need to set the digit count of the LCD to 8 ( it's 5 by default) of setDigitCount.
Also you need to uncomment this line of code: text = time.toString('hh:mm:ss') and comment the old one.
You can solve multiple issues with this widget, like: stopwatch, timer, clock down timer ...

Thursday, July 6, 2017

Python Qt4 - part 003.

Today I've taken a simple example with PyQt4 compared to the other tutorials we have done so far.
The main reason was to understand and use PyQt4 to display an important message.
To make this example I have set the following steps for my python program.
  • importing python modules
  • creating the application in PyQt4 as a tray icon class
  • establishing an exit from the application
  • setting up a message to display
  • display the message over a period of time
  • closing the application
  • running the python application
Let's see my source code of my python application:
#! /usr/bin/env python
import sys
from PyQt4 import QtGui, QtCore

class SystemTrayIcon(QtGui.QSystemTrayIcon):
    def __init__(self, parent=None):
        QtGui.QSystemTrayIcon.__init__(self, parent)
        self.setIcon(QtGui.QIcon("mess.svg"))
        menu = QtGui.QMenu(parent)
        exitAction = menu.addAction("Exit")
        self.setContextMenu(menu)
        QtCore.QObject.connect(exitAction,QtCore.SIGNAL('triggered()'), self.exit)

    def click_trap(self, value):
        if value == self.Trigger: #left click!
            self.left_menu.exec_(QtGui.QCursor.pos())

    def welcome(self):
        self.showMessage("Hello user!", "This is a message from PyQT4")

    def show(self):
        QtGui.QSystemTrayIcon.show(self)
        QtCore.QTimer.singleShot(600, self.welcome)

    def exit(self):
        QtCore.QCoreApplication.exit()

def main():
    app = QtGui.QApplication([])
    tray = SystemTrayIcon()
    tray.show()
    app.exec_()

if __name__ == '__main__':
    main()
I used PyQt4 python module to make the application and sys python module for exit from application.
About running application: the main function will run the application.
The python class SystemTrayIcon will work only if we used QApplication to make like any application.
This is the reason I used the variable app.
The tray variable is used to run it like tray icon application.
Into the SystemTrayIcon class, I put some functions to help me with my issue.
Under __init__ I used all settings for my tray icon application: the icon, the exit menu, signal for the exit.
The next functions come with:
  • click_trap - take the click of the user ;
  • welcome - make a message to display;
  • show - display the welcome message;
  • exit - exit from the application
The result of my python script is this message:

About the showMessage then this helps you:

QSystemTrayIcon.showMessage (self, QString title, QString msg, MessageIcon icon = QSystemTrayIcon.Information, int msecs = 10000)

Shows a balloon message for the entry with the given title, message and icon for the time specified in millisecondsTimeoutHint. title and message must be plain text strings.
The message can be clicked by the user; the messageClicked() signal will be emitted when this occurs.
Note that the display of messages is dependent on the system configuration and user preferences and that messages may not appear at all. Hence, it should not be relied upon as the sole means for providing critical information.
On Windows, the millisecondsTimeoutHint is usually ignored by the system when the application has focus.
On Mac OS X, the Growl notification system must be installed for this function to display messages.
This function was introduced in Qt 4.3.

Tuesday, July 4, 2017

The pdb python interactive debugging - part 001.

This is a short intro tutorial on python debugger to summarize this topic related to Python.
According to the development team, this python module called pdb has the following objectives:
The module pdb defines an interactive source code debugger for Python programs. It supports setting (conditional) breakpoints and single stepping at the source line level, inspection of stack frames, source code listing, and evaluation of arbitrary Python code in the context of any stack frame. It also supports post-mortem debugging and can be called under program control.

Let's start it with some example:
C:\Python27>python
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pdb
>>> pdb.pm()
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python27\lib\pdb.py", line 1270, in pm
    post_mortem(sys.last_traceback)
AttributeError: 'module' object has no attribute 'last_traceback'
>>> import os
>>> pdb.pm()
> c:\python27\lib\pdb.py(1270)pm()
-> post_mortem(sys.last_traceback)
(Pdb) ?

Documented commands (type help ):
========================================
EOF    bt         cont      enable  jump  pp       run      unt
a      c          continue  exit    l     q        s        until
alias  cl         d         h       list  quit     step     up
args   clear      debug     help    n     r        tbreak   w
b      commands   disable   ignore  next  restart  u        whatis
break  condition  down      j       p     return   unalias  where

Miscellaneous help topics:
==========================
exec  pdb

Undocumented commands:
======================
retval  rv
As you can see, this module will work correctly with just another python module. In the example presented above is the os python module.
With the argument? we can see the commands that we can execute.
Let's see the list command ( l ):
(Pdb) l
1265        p = Pdb()
1266        p.reset()
1267        p.interaction(None, t)
1268
1269    def pm():
1270 ->     post_mortem(sys.last_traceback)
1271
1272
1273    # Main program for testing
1274
1275    TESTCMD = 'import x; x.main()'
You can see the pm function loaded by pdb python module.
Post-mortem debugging is a method that requires an environment that provides dynamic execution of code.
One of the greatest benefits of post-mortem debugging is that you can use it directly after something has gone wrong.
You can move between frames within the current call stack using up and down.
This moves towards older frames on the stack.
The debugger prints the current location with where, see example:
(Pdb) where
  (1)()
> c:\python27\lib\pdb.py(1270)pm()
-> post_mortem(sys.last_traceback)
To execute the current line and then stop at the next execution point use step.
The until command can be used to step past the end of a loop.
Use break command used for setting breakpoints, example: (Pdb) break 4.
Turning off a breakpoint with disabling tells the debugger not to stop when that line is reached, example: (Pdb) disable 1.
Also, we can have the other breakpoints like conditional breakpoints and temporary breakpoint.
Use clear to delete a breakpoint entirely.
Changing execution flow with the jump command lets you alter the flow of your program.
The jump can be ahead and back and moves the point of execution past the location without evaluating any of the statements in between.
We can also have illegal jumps in and out of certain flow control statements prevented by the debugger.
When the debugger reaches the end of your program, it automatically starts it over.
With run command, the program can be restarted.
We can avoid typing complex commands repeatedly by using alias and unalias to define the shortcuts.
The pdb python module lets you save configuration using text files read and interpreted on startup.

Friday, June 30, 2017

Blender 3D - bpy and scripting - part 001.

The tutorial for today is bpy python module used by Blender 3D software for scripting.
The last Application Programming Interface (A.P.I.) for Blender 3D can be found here.
You need to take a look at the link to see the base of this python module.
The example I will start today is a python script that helps me to deal with Blender 3D.
First I put the script named catafest.py into Blender 3D path: C:\Program Files\Blender Foundation\Blender\2.78\python\lib.
The reason is to be load by Blender 3D application.
Let's start with the script:
First, you need to import the Blender 3D python module named bpy.
This allows us to deal with Blender 3D using python and Application Programming Interface (A.P.I.).
The script is an example not a lib script for Blender 3D.
The goal of this script is to show how to deal with python scripting into Blender 3D.
If you want to make a lib for the Blender 3D then you need to read about Python’s standard library - here.
My script just creates materials. To do that I used functions to make materials - mat, set the material to the object - set_mat and the run function to see how is working.
Let's see the python script:
import bpy

def mat(name,df_col,df_sh,df_int,sp_col,sp_sh,sp_int,alp,amb):
    mat = bpy.data.materials.new(name)
    mat.diffuse_color = df_col
    mat.diffuse_shader = df_sh
    mat.diffuse_intensity = df_int
    mat.specular_color = sp_col
    mat.specular_shader = sp_sh
    mat.specular_intensity = sp_int
    mat.alpha = alp
    mat.ambient = amb
    return mat

def set_mat(ob, mat):
    me = ob.data
    me.materials.append(mat)

def run(origin):
    # create two materials
    red = mat('Red', (1,0,0),'LAMBERT',1.0,(1,1,1),'COOKTORR',0.5, 1,1)
    blue = mat('Green', (0,1,0),'LAMBERT',1.0,(0,0.5,0.5),'COOKTORR',0.5, 1, 0.5)

    # create red cube
    bpy.ops.mesh.primitive_cube_add(location=origin)
    set_mat(bpy.context.object, red)
    # create a green torus
    bpy.ops.mesh.primitive_torus_add(location=origin)
    bpy.ops.transform.translate(value=(1,0,0))
    set_mat(bpy.context.object, blue)

if __name__ == "__main__":
    run((0,0,0))
To run this script put the script into the lib path of Blender 3D software.
Open the Blender 3D and go to the scripting area:

Now about the source code of this script.
As you know the Blender 3D interface let you add and change objects and materials.
Each material comes with a name, colors with a diffuse and specular property, alpha color, ambient color, etc. see the documentation.
Some values come like strings and can get it from Blender 3D interface, see:
diffuse_shader 

and the
specular_shader
.
The set_mat function just load one object using: me = ob.data
Then put the material to object using append function.
The run function is used to show the result with how to make materials, create objects with this materials.
I run into Blender 3D - Console area:
YTHON INTERACTIVE CONSOLE 3.5.2 (default, Dec  1 2016, 20:58:16) [MSC v.1800 64 bit (AMD64)]

Command History:     Up/Down Arrow
Cursor:              Left/Right Home/End
Remove:              Backspace/Delete
Execute:             Enter
Autocomplete:        Ctrl-Space
Zoom:                Ctrl +/-, Ctrl-Wheel
Builtin Modules:     bpy, bpy.data, bpy.ops, bpy.props, bpy.types, bpy.context, bpy.utils, bgl, blf, mathutils
Convenience Imports: from mathutils import *; from math import *
Convenience Variables: C = bpy.context, D = bpy.data

>>> import catafest 
>>> dir(catafest)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__',
 'bpy', 'mat', 'run', 'set_mat']

>>> catafest.run((0,0,0))
This is the result of the script: