analitics

Pages

Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Saturday, October 26, 2019

Python 3.7.4 : About with the PyOpenCL python module.

PyOpenCL lets you access GPUs and other massively parallel compute devices from Python.
It is important to note that OpenCL is not restricted to GPUs.
In fact, no special hardware is required to use OpenCL for computation–your existing CPU is enough.
The documentation of this project can be found at this website.
Let's install the python module for python 3 version:
[mythcat@desk ~]$ pip3 install pyopencl --user
Collecting pyopencl
...
Successfully built pytools
Installing collected packages: pytools, pyopencl
Successfully installed pyopencl-2019.1.1 pytools-2019.1.1
The install of OpenCL driver can be done with these commands:
# get OpenCL driver automated installer (installs kernel 4.7)
curl https://software.intel.com/sites/default/files/managed/f6/77/install_OCL_driver.sh_.txt > install_OCL\
_driver.sh
chmod +x install_OCL_driver.sh
# install OpenCL driver
sudo ./install_OCL_driver.sh install
# check
ls /boot/vmlinuz-*intel*
This is a simple python script to test the opencl context:
import pyopencl as cl
import numpy as np
ctx = cl.create_some_context()
# cet platforms, both CPU and GPU
my_plat= cl.get_platforms()
CPU = my_plat[0].get_devices()
try:
    GPU = my_plat[1].get_devices()
except IndexError:
    GPU = "none"
# create context for GPU/CPU
if GPU != "none":
    ctx = cl.Context(GPU)
else:
    ctx = cl.Context(CPU)
# create queue for each kernel execution
queue = cl.CommandQueue(ctx)
mf = cl.mem_flags
This is another simple python script:
# -*- coding: utf-8 -*-
import pyopencl as cl 
import numpy
a = numpy.random.rand(50000).astype(numpy.float32)
ctx = cl.create_some_context()
queue = cl.CommandQueue(ctx)
a_buf = cl.Buffer(ctx ,cl.mem_flags.READ_WRITE,size=a.nbytes)
cl.enqueue_write_buffer(queue, a_buf , a)

prg= cl.Program(ctx,
"""
__kernel void twice(__global float ∗a)
{
 int gid=get_global_id(0);
 a[gid] ∗= 2;
}"""
). build()

prg.twice(queue, a.shape, None,a_buf ).wait()

Sunday, October 20, 2019

Python 3.7.4 : Usinge pytesseract for text recognition.

About this python module named tesseract, you can read here.
I tested with the tesseract tool install on my Fedora 30 distro and python module pytesseract version 0.3.0.
[root@desk mythcat]# dnf install tesseract
Last metadata expiration check: 0:24:18 ago on Sun 20 Oct 2019 10:56:23 AM EEST.
Package tesseract-4.1.0-1.fc30.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
[root@desk mythcat]# whereis tesseract
tesseract: /usr/bin/tesseract /usr/share/tesseract
[mythcat@desk ~]$ pip3 install pytesseract --user
Collecting pytesseract
...
Installing collected packages: pytesseract
Successfully installed pytesseract-0.3.0
I test with many images and texts and works very well.
Text images with a printed font are very well recognized.
This test with this image does not have very good accuracy.

The result of the handwriting image.
[mythcat@desk ~]$ python3 ocr_image.py 001.png 
rake Yous mnislakes,
take you chances,
look silby,

bul hep. mv going
dont freeze up

Wednesday, October 16, 2019

Python 3.7.4 : Test the DHCP handshakes.

First, the DHCP is based on the earlier BOOTP protocol which uses well-known port numbers for both server and client instead of an ephemeral port. The server and the client communicate via broadcast and the server broadcasts the offered IP address to the client on UDP port 68.
This python example has a learning purpose and does not harm anyone.
import subprocess as sub
import re

def find_word(w):
    return re.compile(r'\b({0})\b'.format(w), flags=re.IGNORECASE).search

p = sub.Popen(('sudo', 'tcpdump', '-l', '-s 0', '-vvv', '-n', '((udp port 67) and (udp[8:1] = 0x1))'),
 stdout=sub.PIPE)
for row in iter(p.stdout.readline, b''):
    if find_word(row):
        print (row.split(' ')[-1])
    elif find_word(row):
        print (row.split(' ')[-1])
The result of my script ( I don't have inputs on this port).
[mythcat@desk scripts]$ python3 dhcpreq.py 
tcpdump: listening on ___, link-type EN10MB (Ethernet), capture size 262144 bytes
^CTraceback (most recent call last):
  File "dhcpreq.py", line 10, in 
    for row in iter(p.stdout.readline, b''):
KeyboardInterrupt
0 packets captured
0 packets received by filter
0 packets dropped by kernel
[mythcat@desk scripts]$ vim dhcpreq.py 

Tuesday, October 15, 2019

Python 3.8.0 : New release of python development.

Good news from the python development area with the new release of python development:
Python 3.7.5 Oct. 15, 2019 and Python 3.8.0 Oct. 14, 2019

Now you can use the new python version 3.8.0 from the official webpage.

Major new features of the 3.8 series, compared to 3.7 - release Date: Oct. 14, 2019:
  • PEP 572, Assignment expressions
  • PEP 570, Positional-only arguments
  • PEP 587, Python Initialization Configuration (improved embedding)
  • PEP 590, Vectorcall: a fast calling protocol for CPython
  • PEP 578, Runtime audit hooks
  • PEP 574, Pickle protocol 5 with out-of-band data
  • Typing-related: PEP 591 (Final qualifier), PEP 586 (Literal types), and PEP 589 (TypedDict)
  • Parallel filesystem cache for compiled bytecode
  • Debug builds share ABI as release builds
  • f-strings support a handy = specifier for debugging
  • continue is now legal in finally: blocks
  • on Windows, the default asyncio event loop is now ProactorEventLoop
  • on macOS, the spawn start method is now used by default in multiprocessing
  • multiprocessing can now use shared memory segments to avoid pickling costs between processes
  • typed_ast is merged back to CPython
  • LOAD_GLOBAL is now 40% faster
  • pickle now uses Protocol 4 by default, improving performance

Let's install on Fedora 30 Linux distro:
[mythcat@desk ~]$ cd Python-3.8.0/
[mythcat@desk Python-3.8.0]$ ls
aclocal.m4          Doc         m4               Parser         README.rst
CODE_OF_CONDUCT.md  Grammar     Mac              PC             setup.py
config.guess        Include     Makefile.pre.in  PCbuild        Tools
config.sub          install-sh  Misc             Programs
configure           Lib         Modules          pyconfig.h.in
configure.ac        LICENSE     Objects          Python
[mythcat@desk Python-3.8.0]$ ./configure 
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for python3.8... no
...
creating Makefile

If you want a release build with all stable optimizations active (PGO, etc),
please run ./configure --enable-optimizations
If you want then you can run the tool to prepare the build with optimizations:
[mythcat@desk Python-3.8.0]$ ./configure --enable-optimizations --with-ensurepip=install
...
creating Modules/Setup.local
creating Makefile
Use the make with the -j option to use building into parallel steps to speed up the compilation.
[mythcat@desk Python-3.8.0]$ make -j 2
...
make[1]: Leaving directory '/home/mythcat/Python-3.8.0'
Since you’re installing Python into /usr/bin, you’ll need to run as root:
[mythcat@desk Python-3.8.0]$ sudo make altinstall
...
Collecting setuptools
Collecting pip
Installing collected packages: setuptools, pip
Successfully installed pip-19.2.3 setuptools-41.2.0
Let's test it in this folder and with new python3.8 :
[mythcat@desk Python-3.8.0]$ ./python 
Python 3.8.0 (default, Oct 15 2019, 23:45:20) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
[mythcat@desk Python-3.8.0]$ python3.8
Python 3.8.0 (default, Oct 15 2019, 23:45:20) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

Python 3.7.4 : Testing python source code with streamlit tool.

The official webpage for this python package can be found at streamlit.io.
Let's install it with pip3 tool:
[mythcat@desk proiecte_github]$ mkdir streamlit_examples
[mythcat@desk proiecte_github]$ cd streamlit_examples/
[mythcat@desk streamlit_examples]$ pip3 install streamlit --user
Let's try some examples.
Create a file named 001.py
This simple example will show a map with randoms spots:
import pandas as pd
import numpy as np
import streamlit as st    
df = pd.DataFrame(
    np.random.randn(100, 2) / [50, 50] + [47.45, 26.3],
    columns=['lat', 'lon'])
st.map(df)
Let's run it with this command:
[mythcat@desk streamlit_examples]$ streamlit run 001.py 

  You can now view your Streamlit app in your browser.

  Local URL: http://localhost:8501
...
The next source code will show just the map because the df variable is empty:
import streamlit as st
df = []
st.deck_gl_chart(
    viewport={
        'latitude': 47.45,
        'longitude': 26.3,
        'zoom': 13,
        'pitch': 50,
    },
    layers=[{
        'type': 'HexagonLayer',
        'data': df,
        'radius': 200,
        'elevationScale': 4,
        'elevationRange': [0, 1000],
        'pickable': True,
        'extruded': True,
    }, {
        'type': 'ScatterplotLayer',
        'data': df,
    }])
The source code is added into another file named 002.py and can be run with this command:
[mythcat@desk streamlit_examples]$ streamlit run 002.py 

  You can now view your Streamlit app in your browser.

  Local URL: http://localhost:8501
...
You can see more about this tool at the official youtube channel:




Tuesday, October 1, 2019

Python 3.7.4 : Install the protobuf from sources on Fedora distro.

Today I will show you how to build protobuf from sources using the Fedora distro.
The google team comes with this intro:
Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler...
This google project comes with these tutorials.
The GitHub project can be found here.
To install the compiler, download the package.
[mythcat@desk ~]$ cd Downloads/
[mythcat@desk Downloads]$ cp protobuf-python-3.9.2.tar.gz ~/
[mythcat@desk Downloads]$ cd ..
[mythcat@desk ~]$ tar xvzf protobuf-python-3.9.2.tar.gz 
...
protobuf-3.9.2/aclocal.m4
protobuf-3.9.2/install-sh
protobuf-3.9.2/generate_descriptor_proto.sh
protobuf-3.9.2/CHANGES.txt
protobuf-3.9.2/configure.ac
protobuf-3.9.2/configure
Let's see the content:
[mythcat@desk ~]$ cd protobuf-3.9.2/
[mythcat@desk protobuf-3.9.2]$ ls
aclocal.m4                   config.sub                    ltmain.sh            README.md
ar-lib                       configure                     m4                   six.BUILD
autogen.sh                   configure.ac                  Makefile.am          src
benchmarks                   conformance                   Makefile.in          test-driver
BUILD                        CONTRIBUTORS.txt              missing              third_party
CHANGES.txt                  depcomp                       objectivec           update_file_lists.sh
cmake                        editors                       protobuf.bzl         util
compile                      examples                      protobuf_deps.bzl    WORKSPACE
compiler_config_setting.bzl  generate_descriptor_proto.sh  protobuf-lite.pc.in
config.guess                 install-sh                    protobuf.pc.in
config.h.in                  LICENSE                       python
[mythcat@desk protobuf-3.9.2]$ ./configure
...
checking how to run the C preprocessor... gcc -E
checking how to run the C++ preprocessor... /lib/cpp
configure: error: in `/home/mythcat/protobuf-3.9.2':
configure: error: C++ preprocessor "/lib/cpp" fails sanity check
See `config.log' for more details
...
[root@desk protobuf-3.9.2]# dnf install g++
...
Installed:
  gcc-c++-9.2.1-1.fc30.x86_64                                                                               

Complete![root@desk protobuf-3.9.2]# exit
exit
Let's build again:
[mythcat@desk protobuf-3.9.2]$ ./configure
checking whether to enable maintainer-specific portions of Makefiles... yes
checking build system type... x86_64-pc-linux-gnu
...
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating scripts/gmock-config
config.status: creating build-aux/config.h
config.status: executing depfiles commands
config.status: executing libtool commands
[mythcat@desk protobuf-3.9.2]$ nproc --all
2
[mythcat@desk protobuf-3.9.2]$ make -j2
...
  CXXLD    protoc
make[2]: Leaving directory '/home/mythcat/protobuf-3.9.2/src'
make[1]: Leaving directory '/home/mythcat/protobuf-3.9.2'
[mythcat@desk protobuf-3.9.2]$ make check -j2
...PASS: protobuf-lazy-descriptor-test
PASS: protobuf-lite-test
PASS: google/protobuf/compiler/zip_output_unittest.sh
PASS: google/protobuf/io/gzip_stream_unittest.sh
PASS: protobuf-lite-arena-test
PASS: no-warning-test
PASS: protobuf-test
============================================================================
Testsuite summary for Protocol Buffers 3.9.2
============================================================================
# TOTAL: 7
# PASS:  7
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[3]: Leaving directory '/home/mythcat/protobuf-3.9.2/src'
make[2]: Leaving directory '/home/mythcat/protobuf-3.9.2/src'
make[1]: Leaving directory '/home/mythcat/protobuf-3.9.2/src'
The last steps is for install:
[mythcat@desk protobuf-3.9.2]$ sudo make install
[sudo] password for mythcat: 
...
make[2]: Leaving directory '/home/mythcat/protobuf-3.9.2/src'
make[1]: Leaving directory '/home/mythcat/protobuf-3.9.2/src'
[mythcat@desk protobuf-3.9.2]$ sudo ldconfig 
Let's test it:
[mythcat@desk protobuf-3.9.2]$ protoc --version
libprotoc 3.9.2
Now the next step comes for python module:
[mythcat@desk protobuf-3.9.2]$ cd python/
[mythcat@desk python]$ ls
google  MANIFEST.in  mox.py  README.md  release  release.sh  setup.cfg  setup.py  stubout.py  tox.ini
[mythcat@desk python]$ python setup.py build
...
testSerialize (google.protobuf.internal.unknown_fields_test.UnknownFieldsTest) ... ok
testSerializeMessageSetWireFormatUnknownExtension 
(google.protobuf.internal.unknown_fields_test.UnknownFieldsTest) ... ok
testSerializeProto3 (google.protobuf.internal.unknown_fields_test.UnknownFieldsTest) ... ok
testByteSizeFunctions (google.protobuf.internal.wire_format_test.WireFormatTest) ... ok
testPackTag (google.protobuf.internal.wire_format_test.WireFormatTest) ... ok
testUnpackTag (google.protobuf.internal.wire_format_test.WireFormatTest) ... ok
testZigZagDecode (google.protobuf.internal.wire_format_test.WireFormatTest) ... ok
testZigZagEncode (google.protobuf.internal.wire_format_test.WireFormatTest) ... ok

----------------------------------------------------------------------
Ran 818 tests in 5.343s

OK (skipped=10)
Let's test the python module:
[mythcat@desk python]$ python3
Python 3.7.4 (default, Jul  9 2019, 16:32:37) 
[GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> from google.protobuf import descriptor as _descriptor
>>> from google.protobuf import message as _message
>>> from google.protobuf import reflection as _reflection
>>> from google.protobuf import symbol_database as _symbol_database
>>> from google.protobuf import descriptor_pb2
>>> from google.protobuf import text_format
>>> dir(_descriptor)
['Descriptor', 'DescriptorBase', 'DescriptorMetaclass', 'EnumDescriptor', 'EnumValueDescriptor', 
'Error', 'FieldDescriptor', 'FileDescriptor', 'MakeDescriptor', 'MethodDescriptor', 'OneofDescriptor', 
'ServiceDescriptor', 'TypeTransformationError', '_Lock', '_NestedDescriptorBase', '_OptionsOrNone', 
'_ParseOptions', '_ToCamelCase', '_ToJsonName', '_USE_C_DESCRIPTORS', '__author__', '__builtins__', 
'__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_lock', 
'api_implementation', 'six', 'threading']
Now I can to define the structure for the data structured as messages
These message is a small logical record of information containing a series of name-value pairs called fields.
The protobuffers compiler (protoc) to generate the source code in the language you need (from the .proto file).
To generate a Python file, you need to execute:
protoc -I=$SRC_DIR --python_out=$DST_DIR $SRC_DIR/my_example.proto
Let's test with am example (you can use any proto file) in default folder (use . for default folder):
[mythcat@desk python]$ protoc -I=. --python_out=. my_example.proto
[mythcat@desk python]$ ls my_example*
my_example_pb2.py  my_example.proto
[mythcat@desk python]$ python3
Python 3.7.4 (default, Jul  9 2019, 16:32:37) 
[GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import my_example_pb2
>>> dir(my_example_pb2)
['AddressBook', 'DESCRIPTOR', 'Person', '_ADDRESSBOOK', '_PERSON', '_PERSON_PHONENUMBER', '_PERSON_PHONETYPE',
 '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__',
 '_b', '_descriptor', '_message', '_reflection', '_sym_db', '_symbol_database', 'sys']

Monday, September 9, 2019

Python 3.7.3 : Using the flask - part 018.

In this tutorial, I will show you how to fix auto increment in Flask SQLAlchemy.
The old source code for the user model from the server.py was this:
class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String(80), unique=True)
    password = db.Column(db.String(120), unique=True)
    email = db.Column(db.String(120), unique=True)
    gender = db.Column(db.String(5), unique=True)
    work = db.Column(db.String(33), unique=True)
    city = db.Column(db.String(15), unique=True)
The server.sqlite will be this:
[mythcat@desk my_flask]$ sqlite3 server.sqlite 
SQLite version 3.26.0 2018-12-01 12:34:55
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE user (
 id INTEGER NOT NULL, 
 username VARCHAR(80), 
 password VARCHAR(120), 
 email VARCHAR(120), 
 gender VARCHAR(5), 
 work VARCHAR(33), 
 city VARCHAR(15), 
 PRIMARY KEY (id), 
 UNIQUE (username), 
 UNIQUE (password), 
 UNIQUE (email), 
 UNIQUE (gender), 
 UNIQUE (work), 
 UNIQUE (city)
);
CREATE TABLE alembic_version (
 version_num VARCHAR(32) NOT NULL, 
 CONSTRAINT alembic_version_pkc PRIMARY KEY (version_num)
);
CREATE TABLE sqlite_stat1(tbl,idx,stat);
sqlite> ^Z 
If you want to change the id into auto increment then you need to follow this steps:
class User(db.Model):
    __tablename__ = 'user'
    __table_args__ = {'sqlite_autoincrement': True}
    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String(80), unique=True)
    password = db.Column(db.String(120), unique=True)
    email = db.Column(db.String(120), unique=True)
    gender = db.Column(db.String(5), unique=True)
    work = db.Column(db.String(33), unique=True)
    city = db.Column(db.String(15), unique=True)
Delete the server.sqlite file or rename it.
Open python3 and create a new server.sqlite file:
[mythcat@desk my_flask]$ python3
Python 3.7.4 (default, Jul  9 2019, 16:32:37) 
[GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from server import db
>>> db.create_all()
>>> db.engine.table_names()
['sqlite_sequence', 'user']
>>> 
[5]+  Stopat                  python3 
Open the new file to see the changes:
[mythcat@desk my_flask]$ sqlite3 server.sqlite 
SQLite version 3.26.0 2018-12-01 12:34:55
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE user (
 id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, 
 username VARCHAR(80), 
 password VARCHAR(120), 
 email VARCHAR(120), 
 gender VARCHAR(5), 
 work VARCHAR(33), 
 city VARCHAR(15), 
 UNIQUE (username), 
 UNIQUE (password), 
 UNIQUE (email), 
 UNIQUE (gender), 
 UNIQUE (work), 
 UNIQUE (city)
);
CREATE TABLE sqlite_sequence(name,seq);
sqlite>  

Wednesday, September 4, 2019

Python 3.7.4 : Create an Stand Alone Executable on Fedora distro.

In this tutorial I will show you how to create an Stand Alone Executable with Python version 3.7.4 and Fedora 30 distro.
First you need to install using the dnf tool the python3 package.
You can test it easy with this command:
[mythcat@desk dist]$ python3
Python 3.7.4 (default, Jul  9 2019, 16:32:37) 
[GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information. 
I created a simple python script named test.py for testing:
import os
import sys
print("Hello!")
You need to install the pyinstaller python module.
Let's see this step with the output:
[mythcat@desk ~]$ pip3 install pyinstaller --user
Now you can use this python module to create Stand Alone Executable.
[mythcat@desk ~]$ pip3 install pyinstaller --user
Requirement already satisfied: pyinstaller in ./.local/lib/python3.7/site-packages (3.5)
Requirement already satisfied: setuptools in /usr/lib/python3.7/site-packages (from pyinstaller) (40.8.0)
Requirement already satisfied: altgraph in ./.local/lib/python3.7/site-packages (from pyinstaller) (0.16.1)
[mythcat@desk ~]$ pyinstaller --onefile test.py
561 INFO: PyInstaller: 3.5
561 INFO: Python: 3.7.4
571 INFO: Platform: Linux-5.2.9-200.fc30.x86_64-x86_64-with-fedora-30-Thirty
573 INFO: wrote /home/mythcat/test.spec
596 INFO: UPX is not available.
598 INFO: Extending PYTHONPATH with paths
['/home/mythcat', '/home/mythcat']
598 INFO: checking Analysis
624 INFO: Building because /home/mythcat/test.py changed
624 INFO: Initializing module dependency graph...
640 INFO: Initializing module graph hooks...
678 INFO: Analyzing base_library.zip ...
5247 INFO: running Analysis Analysis-00.toc
5352 INFO: Caching module hooks...
5376 INFO: Analyzing /home/mythcat/test.py
5396 INFO: Loading module hooks...
5397 INFO: Loading module hook "hook-xml.py"...
5775 INFO: Loading module hook "hook-pydoc.py"...
5800 INFO: Loading module hook "hook-encodings.py"...
5892 INFO: Looking for ctypes DLLs
5893 INFO: Analyzing run-time hooks ...
5901 INFO: Looking for dynamic libraries
6507 INFO: Looking for eggs
6507 INFO: Using Python library /lib64/libpython3.7m.so.1.0
6514 INFO: Warnings written to /home/mythcat/build/test/warn-test.txt
6547 INFO: Graph cross-reference written to /home/mythcat/build/test/xref-test.html
6604 INFO: checking PYZ
6607 INFO: Building because toc changed
6607 INFO: Building PYZ (ZlibArchive) /home/mythcat/build/test/PYZ-00.pyz
7030 INFO: Building PYZ (ZlibArchive) /home/mythcat/build/test/PYZ-00.pyz completed successfully.
7034 INFO: checking PKG
7035 INFO: Building because toc changed
7035 INFO: Building PKG (CArchive) PKG-00.pkg
10033 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
10036 INFO: Bootloader 
/home/mythcat/.local/lib/python3.7/site-packages/PyInstaller/bootloader/Linux-64bit/run
10036 INFO: checking EXE
10123 INFO: Building because toc changed
10124 INFO: Building EXE from EXE-00.toc
10163 INFO: Appending archive to ELF section in EXE /home/mythcat/dist/test
10337 INFO: Building EXE from EXE-00.toc completed successfully.
This command will create some folders: dist, build, ...
The dist folder will have the Stand Alone Executable named test:
[mythcat@desk ~]$ cd dist/
[mythcat@desk dist]$ ls
test
[mythcat@desk dist]$ ./test
Hello!

Thursday, August 22, 2019

Python 3.7.3 : Using the inotify.

About this tool you can read here:
inotify functionality is available from the Linux kernel and allows you to register one or more directories for watching, and to simply block and wait for notification events.
[mythcat@desk ~]$ pip3 install inotify --user
...
Successfully installed inotify-0.2.10 nose-1.3.7
Let's test it and see how this can be load it:
[mythcat@desk ~]$ python3
Python 3.7.4 (default, Jul  9 2019, 16:32:37) 
[GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import inotify
>>> from inotify import *
>>> print(dir(inotify))
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', 
'__path__', '__spec__', '__version__']
>>> import inotify.adapters
>>> print(dir(inotify))
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', 
'__path__', '__spec__', '__version__', 'adapters', 'calls', 'constants', 'library']
>>> print(dir(inotify.adapters))
['EINTR', 'EventTimeoutException', 'Inotify', 'InotifyTree', 'InotifyTrees', 'TerminalEventException', 
'_BaseTree', '_DEFAULT_EPOLL_BLOCK_DURATION_S', '_DEFAULT_TERMINAL_EVENTS', '_HEADER_STRUCT_FORMAT', 
'_INOTIFY_EVENT', '_IS_DEBUG', '_LOGGER', '_STRUCT_HEADER_LENGTH', '__builtins__', '__cached__', '__doc__',
 '__file__', '__loader__', '__name__', '__package__', '__spec__', 'collections', 'inotify', 'logging', 'os',
 'select', 'struct', 'time']
>>> import inotify.calls
>>> print(dir(inotify.calls))
['InotifyError', '_LIB', '_LOGGER', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', 
'__name__', '__package__', '__spec__', '_check_nonnegative', '_check_nonzero', '_check_zero', 'ctypes', 
'errno', 'inotify', 'inotify_add_watch', 'inotify_init', 'inotify_rm_watch', 'logging']
>>> import inotify.constants
>>> print(dir(inotify.constants))
['IN_ACCESS', 'IN_ALL_EVENTS', 'IN_ATTRIB', 'IN_CLOEXEC', 'IN_CLOSE', 'IN_CLOSE_NOWRITE', 'IN_CLOSE_WRITE', 
'IN_CREATE', 'IN_DELETE', 'IN_DELETE_SELF', 'IN_DONT_FOLLOW', 'IN_IGNORED', 'IN_ISDIR', 'IN_MASK_ADD', 
'IN_MODIFY', 'IN_MOVE', 'IN_MOVED_FROM', 'IN_MOVED_TO', 'IN_MOVE_SELF', 'IN_NONBLOCK', 'IN_ONESHOT', 
'IN_ONLYDIR', 'IN_OPEN', 'IN_Q_OVERFLOW', 'IN_UNMOUNT', 'MASK_LOOKUP', '__builtins__', '__cached__', 
'__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__']
>>> import inotify.library
>>> print(dir(inotify.library))
['_FILEPATH', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', 
'__spec__',
 'ctypes', 'instance']
Let's test it with a simple watch issue for a file named test_file.
First, let create this file:
[mythcat@desk ~]$ touch /tmp/test_file
[mythcat@desk ~]$ ll /tmp/test_file 
-rw-rw-r--. 1 mythcat mythcat 0 Aug 22 13:54 /tmp/test_file
Now, I can use the next script to watch on it:
import inotify.adapters
def notif_tmp():
    adapter_tmp = inotify.adapters.Inotify()

    adapter_tmp.add_watch('/tmp')

    with open('/tmp/test_file', 'w'):
        pass

    events = adapter_tmp.event_gen(yield_nones=False, timeout_s=1)
    events = list(events)

    print(events)
if __name__ == '__main__':
    notif_tmp()
The result is this:
$ python3 notif.py 
[(_INOTIFY_EVENT(wd=1, mask=2, cookie=0, len=16), ['IN_MODIFY'], '/tmp', 'test_file'), 
(_INOTIFY_EVENT(wd=1, mask=32, cookie=0, len=16), ['IN_OPEN'], '/tmp', 'test_file'), 
(_INOTIFY_EVENT(wd=1, mask=8, cookie=0, len=16), ['IN_CLOSE_WRITE'], '/tmp', 'test_file')]
If remove this file the result will be this:
$ python3 notif.py 
[(_INOTIFY_EVENT(wd=1, mask=256, cookie=0, len=16), ['IN_CREATE'], '/tmp', 'test_file'), 
(_INOTIFY_EVENT(wd=1, mask=32, cookie=0, len=16), ['IN_OPEN'], '/tmp', 'test_file'), 
(_INOTIFY_EVENT(wd=1, mask=8, cookie=0, len=16), ['IN_CLOSE_WRITE'], '/tmp', 'test_file')]
Let's test another example:
import inotify.adapters
import logging
_LOGGER = logging.getLogger(__name__)
def test():
    i = inotify.adapters.Inotify()
    i.add_watch('/tmp')

    try:
        for event in i.event_gen():
            if event is not None:
                (header, type_names, path, filename) = event
                _LOGGER.info("WD=(%d) MASK=(%d) COOKIE=(%d) LEN=(%d) MASK->NAMES=%s "
                             "FILENAME=[%s]",
                             header.wd, header.mask, header.cookie, header.len, type_names,
                             filename)
                print(header, type_names, path, filename)
    finally:
        i.remove_watch('/tmp')

if __name__ == '__main__':
    test()
If you run this and will try to edit with vim editor the /tmp/text.txt file the output will be this:
$ python3 notif_002.py
_INOTIFY_EVENT(wd=1, mask=32, cookie=0, len=16) ['IN_OPEN'] /tmp text.txt
_INOTIFY_EVENT(wd=1, mask=256, cookie=0, len=16) ['IN_CREATE'] /tmp .text.txt.swp
_INOTIFY_EVENT(wd=1, mask=32, cookie=0, len=16) ['IN_OPEN'] /tmp .text.txt.swp
...
If you want to use many paths then use this:
    paths = [
        '/tmp',
    ]
    i = Inotify(paths=paths)
This module is not well documented but can be successfully used for certain tasks.

Saturday, March 30, 2019

Testing the python IMDbPY module with simple commands.

Today we tested a more innovative but useful method with python aaa mode.
The main reason I used this method is the lack of documentation.
Using this method, we have reached elements related to the use of reported methods and errors.
The test was done on a Fedora 29 Linux system with a classic install with the pip utility:
[mythcat@desk ~]$ pip install imdbpy --user
Collecting imdbpy
...
Successfully installed SQLAlchemy-1.3.1 imdbpy-6.6 
I used an example of a person's search in the IMDB database to test this method.
>>> from imdb import IMDb, IMDbError
>>> try:
...     im=IMDb()
...     people = im.search_person('Mel Gibson')
... except IMDbError as exc:
...     print(exc) 
Using the dir and print function will show the resulting output configuration and will have the following form:
>>> print(people)
[, , , , , , , , , , , , , , , , , , , ] 
I have used the dir function for a relative view of the options we have:
>>> print(dir(people))
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__delslice__', '__doc__', 
'__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getslice__', 
'__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', 
'__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', 
'__rmul__', '__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__', '__subclasshook__', 
'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']
>>> print(dir(people[0]))
['_Container__role', '__bool__', '__class__', '__contains__', '__deepcopy__', '__delattr__', 
'__delitem__', '__dict__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', 
'__getitem__', '__gt__', '__hash__', '__init__', '__le__', '__len__', '__lt__', '__module__', 
'__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', 
'__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_additional_keys', '_clear', 
'_get_currentRole', '_get_roleID', '_getitem', '_image_key', '_init', '_reset', '_roleClass', 
'_roleIsPerson', '_set_currentRole', '_set_roleID', 'accessSystem', 'add_to_current_info', 
'append_item', 'asXML', 'billingPos', 'charactersRefs', 'clear', 'cmpFunct', 'copy', 'currentRole', 
'current_info', 'data', 'default_info', 'get', 'getAsXML', 'getID', 'get_charactersRefs', 
'get_current_info', 'get_fullsizeURL', 'get_namesRefs', 'get_titlesRefs', 'has_current_info', 
'has_key', 'infoset2keys', 'isSame', 'isSameName', 'isSamePerson', 'items', 'iteritems', 
'iterkeys', 'itervalues', 'key2infoset', 'keys', 'keys_alias', 'keys_tomodify', 'keys_tomodify_list', 
'modFunct', 'myID', 'myName', 'namesRefs', 'notes', 'personID', 'pop', 'popitem', 'reset', 'roleID', 
'set_current_info', 'set_data', 'set_item', 'set_mod_funct', 'set_name', 'setdefault', 'summary', 
'titlesRefs', 'update', 'update_charactersRefs', 'update_infoset_map', 'update_namesRefs', 
'update_titlesRefs', 'values']
Here are some simple examples of displaying using the print function to view content in output:

>>> print(people[0].values())
[u'Catalin', u'II', u'Catalin', u'Catalin (II)', u'Catalin (II)']
>>> print(people[0].data)
{u'name': u'Catalin', u'imdbIndex': u'II'}
>>> print(people[1].data.viewitems())
dict_items([(u'name', u'Moreno, Catalina Sandino')])
>>> print(people[1].data.values())
[u'Moreno, Catalina Sandino']
>>> print(people[0].getID())
2165704
>>> print(people[0].itervalues())
The built-in function iter takes an iterable object and returns an iterator.
>>> print(people[0].itervalues().next())
Catalin
>>> print(people[0].asXML()) 
The last line of code will return XML content.
This simple example simply illustrates how to access structured information through simple Python commands.

Sunday, June 4, 2017

The development with python-instagram .

The python-instagram python module is a Python 2/3 client for the Instagram REST and Search APIs.
This python module requires httplib2, simplejson and six.
Instagram API uses the OAuth2 protocol for authentication, see docs.
C:\Python27\Scripts>pip install --upgrade  --trusted-host  pypi.python.org  
python-instagram
Collecting python-instagram
  Downloading python-instagram-1.3.2.tar.gz
Collecting simplejson (from python-instagram)
  Downloading simplejson-3.10.0-cp27-cp27m-win32.whl (66kB)
    100% |################################| 71kB 1.1MB/s
Requirement already up-to-date: httplib2 in c:\python27\lib\site-packages 
(from python-instagram)
Requirement already up-to-date: six in c:\python27\lib\site-packages 
(from python-instagram)
Building wheels for collected packages: python-instagram
  Running setup.py bdist_wheel for python-instagram ... done
 ...
Installing collected packages: simplejson, python-instagram
Successfully installed python-instagram-1.3.2 simplejson-3.10.0
Now about this python module:
C:\Python27>python.exe
Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from instagram.client import InstagramAPI
>>> dir(InstagramAPI)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__',
 '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', 
'__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_make_relationship_shortcut', 
'_make_subscription_action', 'access_token_field', 'access_token_url', 'api_name', 
'approve_user_request', 'authorize_url', 'base_path', 'block_user', 'change_user_relationship',
 'create_media_comment', 'create_subscription', 'delete_comment', 'delete_subscriptions',
 'exchange_code_for_access_token', 'exchange_user_id_for_access_token', 
'exchange_xauth_login_for_access_token', 'follow_user', 'geography_recent_media', 
'get_authorize_login_url', 'get_authorize_url', 'host', 'ignore_user_request', 'like_media',
 'list_subscriptions', 'location', 'location_recent_media', 'location_search', 'media', 
'media_comments', 'media_likes', 'media_popular', 'media_search', 'media_shortcode', 'protocol',
 'redirect_uri', 'tag', 'tag_recent_media', 'tag_search', 'unblock_user', 'unfollow_user', 
'unlike_media', 'user', 'user_followed_by', 'user_follows', 'user_incoming_requests', 
'user_liked_media', 'user_media_feed', 'user_recent_media', 'user_relationship', 'user_search',
 'x_ratelimit', 'x_ratelimit_remaining']
If you have an Instagram account then just log in into instagram developer website.
Then fill the issue about your website the phone number and what do you want to build for your application check your agreement with Instagram.
Now you need to use Register Your Application and finally on Register a New Client.
About Register Your Application you need to fill them with data for your application ( basic info: Description, Company Name, Website URL, Contact email).
Select the tab Security and disable the Disable implicit OAuth.

About the token authorizations:

Is given to you with this words:

basic – to read a user’s profile info and media

or needs additional permission:

public_content – to read any public profile info and media on a user’s behalf
follower_list – to read the list of followers and followed-by users
comments – to post and delete comments on a user’s behalf
relationships – to follow and unfollow accounts on a user’s behalf
likes – to like and unlike media on a user’s behalf
The next step is to get access token then you need to add http://localhost link into Security tag from Manage Client.
Use this URL to get the access token by pasting it into your web browser.
https://instagram.com/oauth/authorize/?client_id=[CLIENT_ID_HERE]&redirect_uri=http://localhost&response_type=token&scope=public_content
Into the browser, you will see one page with one button for Authorizing access.
Press this button and into your browser address bar you will get the access token like:
http://localhost/#access_token=################
A simple python script to test it.
from time import sleep
from instagram.client import InstagramAPI

client_id="zzzzz"
client_secret="sssssssssssss"
redirect_uri= "http://xxxxx"
access_token="eeeee"

api = InstagramAPI(client_id=client_id, client_secret=client_secret)
print dir(api)
print api.api_name
To deal with python and Instagram is not very easy for me.
The main reason comes from errors and the Instagram API development way.
Some simple tasks are very hard to do.

Friday, May 5, 2017

The google-cloud-vision python module - part 001.

Google comes with $300 credit for free to sign up into Google Cloud Platform over the next 12 months.
This allows you to deal with access to all Cloud Platform Products.
Today I will show you how to install this platform into your Linux and Windows 10 OS.
For Linux, I used Fedora 26 distro.
Using the Windows 10 operating system and python 2.7 then you can use this command:
pip install --upgrade google-cloud-vision
If you got errors the fix with this command:
C:\Python27\Scripts>pip install --upgrade  --trusted-host  pypi.python.org google-cloud-vision
Collecting google-cloud-vision
  Downloading google_cloud_vision-0.24.0-py2.py3-none-any.whl (68kB)
    100% |################################| 71kB 270kB/s
Collecting google-cloud-core<0 .25dev="">=0.24.0 (from google-cloud-vision)
  Downloading google_cloud_core-0.24.1-py2.py3-none-any.whl (52kB)
    100% |################################| 61kB 1.6MB/s
...
Installing collected packages: appdirs, setuptools, protobuf, httplib2, rsa, pyasn1-modules,
 cachetools, google-auth, google-auth-httplib2, googleapis-common-protos, google-cloud-core,
 pyreadline, dill, futures, grpcio, oauth2client, ply, google-gax, proto-google-cloud-vision-v1,
 gapic-google-cloud-vision-v1, google-cloud-vision, pyparsing
  Found existing installation: appdirs 1.4.0
    Uninstalling appdirs-1.4.0:
      Successfully uninstalled appdirs-1.4.0
  Rolling back uninstall of appdirs
Exception:
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
...
    with open(path, 'rb') as stream:
IOError: [Errno 2] No such file or directory: 'c:\\python27\\lib\\site-packages\\
appdirs-1.4.0.dist-info\\METADATA'
I run again the command and I don't have errors:
C:\Python27\Scripts>pip install --upgrade  --trusted-host  pypi.python.org google-cloud-vision
Collecting google-cloud-vision
  Downloading google_cloud_vision-0.24.0-py2.py3-none-any.whl (68kB)
    100% |################################| 71kB 597kB/s
Collecting google-cloud-core<0 .25dev="">=0.24.0 (from google-cloud-vision)
...
  Downloading futures-3.1.1-py2-none-any.whl
Collecting pyparsing (from packaging>=16.8->setuptools->protobuf>=3.0.0->google-cloud-core<0 .25dev="">=0.24.0->google-cloud-vision)
  Downloading pyparsing-2.2.0-py2.py3-none-any.whl (56kB)
    100% |################################| 61kB 4.7MB/s
Installing collected packages: appdirs, setuptools, protobuf, httplib2, rsa, pyasn1-modules,
 cachetools, google-auth, google-auth-httplib2, googleapis-common-protos, google-cloud-core,
 oauth2client, ply, pyreadline, dill, futures, grpcio, google-gax, proto-google-cloud-vision-v1,
 gapic-google-cloud-vision-v1, google-cloud-vision, pyparsing
  Found existing installation: appdirs 1.4.0
    Uninstalling appdirs-1.4.0:
      Successfully uninstalled appdirs-1.4.0
  Found existing installation: setuptools 34.0.2
    Uninstalling setuptools-34.0.2:
      Successfully uninstalled setuptools-34.0.2
  Found existing installation: httplib2 0.9.2
    Uninstalling httplib2-0.9.2:
      Successfully uninstalled httplib2-0.9.2
  Found existing installation: pyparsing 2.1.10
    Uninstalling pyparsing-2.1.10:
      Successfully uninstalled pyparsing-2.1.10
Successfully installed appdirs-1.4.3 cachetools-2.0.0 dill-0.2.6 futures-3.1.1 
gapic-google-cloud-vision-v1-0.90.3 google-auth-1.0.0 google-auth-httplib2-0.0.2 
google-cloud-core-0.24.1 google-cloud-vision-0.24.0 google-gax-0.15.8 googleapis-common-protos-1.5.2
 grpcio-1.3.0 httplib2-0.10.3 oauth2client-3.0.0 ply-3.8 proto-google-cloud-vision-v1-0.90.3
 protobuf-3.2.0 pyasn1-modules-0.0.8 pyparsing-2.2.0 pyreadline-2.1 rsa-3.4.2 setuptools-35.0.2
For Fedora 26 distro I used this command to install the python module:
[root@localhost mythcat]# pip install --upgrade google-cloud-vision --ignore-installed
WARNING: Running pip install with root privileges is generally not a good idea. 
Try `pip install --user` instead.                                         
Collecting google-cloud-vision
  Using cached google_cloud_vision-0.24.0-py2.py3-none-any.whl
...
 google-auth-httplib2, google-cloud-core, google-cloud-vision
  Running setup.py install for dill ... done
  Running setup.py install for future ... done
  Running setup.py install for googleapis-common-protos ... done
  Running setup.py install for ply ... done
  Running setup.py install for google-gax ... done
  Running setup.py install for httplib2 ... done
  Running setup.py install for oauth2client ... done
  Running setup.py install for proto-google-cloud-vision-v1 ... done
  Running setup.py install for gapic-google-cloud-vision-v1 ... done
Successfully installed appdirs-1.4.3 cachetools-2.0.0 dill-0.2.6 enum34-1.1.6 
future-0.16.0 futures-3.1.1 gapic-google-cloud-vision-v1-0.90.3 google-auth-1.0.0
 google-auth-httplib2-0.0.2 google-cloud-core-0.24.1 google-cloud-vision-0.24.0
 google-gax-0.15.9 googleapis-common-protos-1.5.2 grpcio-1.3.0 httplib2-0.10.3
 oauth2client-3.0.0 packaging-16.8 ply-3.8 proto-google-cloud-vision-v1-0.90.3
 protobuf-3.2.0 pyasn1-0.2.3 pyasn1-modules-0.0.8 pyparsing-2.2.0 rsa-3.4.2 
setuptools-35.0.2 six-1.10.0


Wednesday, April 5, 2017

The scapy python module - part 001.

Today I will start with scapy python module.
This is a good python module to deal and interact with network packets.
[root@localhost mythcat]# pip install scapy
Collecting scapy
  Downloading scapy-2.3.3.tgz (1.4MB)
    100% |████████████████████████████████| 1.4MB 904kB/s 
Building wheels for collected packages: scapy
  Running setup.py bdist_wheel for scapy ... done
  Stored in directory: /root/.cache/pip/wheels/bd/cf/...
Installing collected packages: scapy
Successfully installed scapy-2.3.3
The first test is to test is the echo of Layer 3 ICMP.
Use the superuser shell to run this python script:
from scapy.all import *
dstip=raw_input("enter the ip address \n")
icmp=ICMP()
icmp.type=8
icmp.code=0
ip=IP()
ip.dst=dstip
p=sr1(ip/icmp,timeout=5, verbose=0)
if(p):
        print "Layer 3 is up"
else:
        print "Layer 3 status is down"
The next python script will about arp request:
from scapy.all import *
def arp_display(pkt):
    if pkt[ARP].op == 1: 
        return "Request: " + pkt[ARP].psrc + " is asking about " + pkt[ARP].pdst
    if pkt[ARP].op == 2: 
        return "*Response: " + pkt[ARP].hwsrc + " has address " + pkt[ARP].psrc
print sniff(prn=arp_display, filter="arp", store=0, count=10)
This will read the packages from source and destination and show me what ARP traffic my computer is seeing.

Saturday, March 4, 2017

Working with datetime python module.

This module is very good and I worked with this issue by using MySQL and python.
The version of python I used is 2.7 under Fedora distro.
You can take a look at the official page.
I use the pip and not the DNF fedora Linux tool.
 
[root@localhost lucru]# pip install datetime
Collecting datetime
Downloading DateTime-4.1.1.zip (66kB)
100% |████████████████████████████████| 71kB 703kB/s 
Collecting zope.interface (from datetime)
Downloading zope.interface-4.3.3.tar.gz (150kB)
100% |████████████████████████████████| 153kB 2.2MB/s 
Collecting pytz (from datetime)
Downloading pytz-2016.10-py2.py3-none-any.whl (483kB)
100% |████████████████████████████████| 491kB 2.4MB/s 
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from zope.interface->datetime)
Installing collected packages: zope.interface, pytz, datetime
Running setup.py install for zope.interface ... done
Running setup.py install for datetime ... done
Successfully installed datetime-4.1.1 pytz-2016.10 zope.interface-4.3.3

I solve this problem:
  • conversion using the lambda function
    parser.add_argument('date', type=lambda s: datetime.datetime.strptime(s, '%Y-%m-%d'))
  • solve last day
    datetime.datetime.strptime(new_value, '%Y-%m-%d %H:%M:%S')-timedelta(days=1)
  • print the today date
    print date.today()
  • show date using an explicit format string
    today=date.today()
    today.strftime("%A %d. %B %Y")
    'Sunday 05. March 2017'
    
  • using epoch issue [1]
    from datetime import datetime
    now_epoch = (datetime.utcnow() - datetime(1970, 1, 1)).total_seconds()
    datetime.utcfromtimestamp(now_epoch)
    datetime.datetime(2017, 3, 4, 22, 35, 13, 463409)
    datetime.fromtimestamp(now_epoch)
    datetime.datetime(2017, 3, 5, 0, 35, 13, 463409)
    import pytz
    datetime.fromtimestamp(now_epoch, pytz.utc)
    datetime.datetime(2017, 3, 4, 22, 35, 13, 463409, tzinfo=)
    
[1] The Unix epoch is the time 00:00:00 UTC on 1 January 1970. There is a problem with this definition, in that UTC did not exist in its current form until 1972;

Using pygeoip and maxmin database.


I try to locate one IP using the databases from maxmind website and is not good for me.
The database records show me the output from country area.
I read the docs from here.
This is the python script I used:
#wget -N -q http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
import pygeoip 
gip = pygeoip.GeoIP('GeoLiteCity.dat')
rec = gip.record_by_addr('___________________')
for key,val in rec.items():
    print "%s: %s" %(key,val)

Saturday, February 25, 2017

Linux: OpenCV and using Lucas-Kanade Optical Flow function.

Fist I install OpenCV python module and I try using with Fedora 25.
I used python 2.7 version.
[root@localhost mythcat]# dnf install opencv-python.x86_64 
Last metadata expiration check: 0:21:12 ago on Sat Feb 25 23:26:59 2017.
Dependencies resolved.
================================================================================
 Package              Arch          Version                Repository      Size
================================================================================
Installing:
 opencv               x86_64        3.1.0-8.fc25           fedora         1.8 M
 opencv-python        x86_64        3.1.0-8.fc25           fedora         376 k
 python2-nose         noarch        1.3.7-11.fc25          updates        266 k
 python2-numpy        x86_64        1:1.11.2-1.fc25        fedora         3.2 M

Transaction Summary
================================================================================
Install  4 Packages

Total download size: 5.6 M
Installed size: 29 M
Is this ok [y/N]: y
Downloading Packages:
(1/4): opencv-python-3.1.0-8.fc25.x86_64.rpm    855 kB/s | 376 kB     00:00    
(2/4): opencv-3.1.0-8.fc25.x86_64.rpm           1.9 MB/s | 1.8 MB     00:00    
(3/4): python2-nose-1.3.7-11.fc25.noarch.rpm    543 kB/s | 266 kB     00:00    
(4/4): python2-numpy-1.11.2-1.fc25.x86_64.rpm   2.8 MB/s | 3.2 MB     00:01    
--------------------------------------------------------------------------------
Total                                           1.8 MB/s | 5.6 MB     00:03     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Installing  : python2-nose-1.3.7-11.fc25.noarch                           1/4 
  Installing  : python2-numpy-1:1.11.2-1.fc25.x86_64                        2/4 
  Installing  : opencv-3.1.0-8.fc25.x86_64                                  3/4 
  Installing  : opencv-python-3.1.0-8.fc25.x86_64                           4/4 
  Verifying   : opencv-python-3.1.0-8.fc25.x86_64                           1/4 
  Verifying   : opencv-3.1.0-8.fc25.x86_64                                  2/4 
  Verifying   : python2-numpy-1:1.11.2-1.fc25.x86_64                        3/4 
  Verifying   : python2-nose-1.3.7-11.fc25.noarch                           4/4 

Installed:
  opencv.x86_64 3.1.0-8.fc25            opencv-python.x86_64 3.1.0-8.fc25       
  python2-nose.noarch 1.3.7-11.fc25     python2-numpy.x86_64 1:1.11.2-1.fc25    

Complete!
[root@localhost mythcat]# 
This is my test script with opencv to detect flow using Lucas-Kanade Optical Flow function.
This tracks some points in a black and white video.
First you need:
- one black and white video;
- not mp4 file type file;
- the color args need to be under 4 ( see is 3);
- I used this video:
I used cv2.goodFeaturesToTrack().
We take the first frame, detect some Shi-Tomasi corner points in it, then we iteratively track those points using Lucas-Kanade optical flow.
The function cv2.calcOpticalFlowPyrLK() we pass the previous frame, previous points and next frame.
The returns next points along with some status numbers which has a value of 1 if next point is found, else zero.
That iteratively pass these next points as previous points in next step.
See the code below:
import numpy as np
import cv2

cap = cv2.VideoCapture('candle')

# params for ShiTomasi corner detection
feature_params = dict( maxCorners = 77,
                       qualityLevel = 0.3,
                       minDistance = 7,
                       blockSize = 7 )

# Parameters for lucas kanade optical flow
lk_params = dict( winSize  = (17,17),
                  maxLevel = 1,
                  criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))

# Create some random colors
color = np.random.randint(0,255,(100,3))

# Take first frame and find corners in it
ret, old_frame = cap.read()
old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)
p0 = cv2.goodFeaturesToTrack(old_gray, mask = None, **feature_params)

# Create a mask image for drawing purposes
mask = np.zeros_like(old_frame)

while(1):
    ret,frame = cap.read()
    frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # calculate optical flow
    p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)

    # Select good points
    good_new = p1[st==1]
    good_old = p0[st==1]

    # draw the tracks
    for i,(new,old) in enumerate(zip(good_new,good_old)):
        a,b = new.ravel()
        c,d = old.ravel()
        mask = cv2.line(mask, (a,b),(c,d), color[i].tolist(), 2)
        frame = cv2.circle(frame,(a,b),5,color[i].tolist(),-1)
    img = cv2.add(frame,mask)

    cv2.imshow('frame',img)
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break

    # Now update the previous frame and previous points
    old_gray = frame_gray.copy()
    p0 = good_new.reshape(-1,1,2)

cv2.destroyAllWindows()
cap.release()
The output of this file is:

Sunday, September 22, 2013

Jython - funny and simple scripts - first steps .

I play today with jython and can be fun but seams to be to slow in a linux os.

Jython is invoked using the "jython" script and it's an implementation of Python for the JVM.

Install the package jython in your linux distro and you can start to deal with java and python.

When you use jython then script will start with :

#!/usr/bin/env jython

I make also some very simple scripts...

First script make one button and give a action to exit.


#!/usr/bin/env jython
from javax import *
import java
from java import *
import sys

frame = awt.Frame(size=(500,100))
frame.background = 255,255,0
def exit(event):
  java.lang.System.exit(0)

my_button = awt.Button("Exit!", actionPerformed=exit)
frame.add(my_button,"Center")
frame.pack()
frame.setVisible(1)

The output is:


The script is easy to make ... it's like gtk with add, pack and action ...

Let's see the next script : one list.

from javax import *
from java import awt
import sys
python_list=[]
python_list.append('text 1')
python_list.append('text 2')
python_list.append('text 3')
python_list.append('text 4')
python_list.append('text 5')

frame=awt.Frame("test list")
panel=swing.JList(python_list)
frame.add(panel,"Center")
frame.pack()
frame.setVisible(1)

... and this is the gui with the list:


I make a simple list and add to the gui using pack() function.

The jython is not easy is much to learn and if you want then go to this website.

Friday, August 24, 2012

Python 3.2 : Start with Django 1.4.

Although most of us prefer the python version 2.6, today I tried to install the latest version of django and python 2.3.2 .
Make a new folder , named test-dj .
$mkdir test-dj
$cd test-dj/
On the official site, I got the two archives:
django-django-1.4-919-ge57338f.zip
Python-3.2.3.tar.bz2
I will start with the installation of python. We unzip the archive:
$tar xvjf Python-3.2.3.tar.bz2 
We execute the following commands to install python:
$cd Python-3.2.3
$./configure
$make all
$sudo make altinstall
# python3.2 setup.py install 
Let's see what we have.
$ whereis  python3
python3: /usr/lib/python3.0 /usr/local/bin/python3.2m-config
/usr/local/bin/python3.2 /usr/local/bin/python3.2m 
/usr/local/lib/python3.2
As you see it's ...
python3.2
python3.2m
python3.2m-config
In accordance with the PEP-3149 we can got this:
Python implementations MAY include additional flags in the file name tag as appropriate. For example, on POSIX systems these flags will also contribute to the file name:

        * --with-pydebug (flag: d)
        * --with-pymalloc (flag: m)
        * --with-wide-unicode (flag: u)

Now we need to install django.
$unzip django-django-1.4-919-ge57338f.zip
Go to the django folder:
$cd django-django-e57338f/
# python3.2 setup.py install 
Now , we can test it:
# python3.2
Python 3.2.3 (default, Aug 24 2012, 19:24:21) 
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django 
>>> print(django.get_version())
1.5
>>> 
I will make another tutorial about how to configure the django to have one website.

Monday, March 1, 2010

Python and OpenGL - first steps

First you need to install python .
Second you need to install pyopengl module.
This is easy if we use some linux distributions.
On Fedora we can use command yum.
On Debian and Ubuntu we can use apt-get.
At last we need some basicaly examples.
We can get some examples from:PyOpenGL-Demo
But this is not all we will need more.
To go on next step we will need to read OpenGL specifications.
Try to use search on internet to see more about OpenGL.