Today I tested the urllib python module with python 3.7.3 and json python module.
The issue was to get the location of International Space Station - Open Notify. The International Space Station is moving at close to 28,000 km/h so its location changes really fast! Where is it right now?
This is an open source project to provide a simple programming interface for some of NASA’s awesome data.
I do some of the work to take raw data and turn them into APIs related to space and spacecraft.
C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD6
4)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request
>>> with urllib.request.urlopen('http://api.open-notify.org/iss-now.json') as f:
print(f.read(300))
...
b'{"iss_position": {"longitude": "-86.9247", "latitude": "-38.3744"}, "message":
"success", "timestamp": 1556575039}'
>>> with urllib.request.urlopen('http://api.open-notify.org/iss-now.json') as f:
... source = f.read()
... data = json.loads(source)
...
>>> print(data)
{'iss_position': {'longitude': '151.1941', 'latitude': '49.4702'}, 'message': 's
uccess', 'timestamp': 1556578621}
>>> print(data['iss_position']['longitude'])
151.1941
>>> print(data['iss_position']['latitude'])
49.4702
>>> print(data['message'])
success
Today I will come up with a simpler and more effective tutorial in python programming.
First, I need to install the psutil python module for the example of this tutorial.
Python memory monitor is very important for debugging application performance and fix bugs.
You can solve this issue is the python module named memory_profiler, see more here.
Let's install this python module with the pip python tool:
C:\Python373\Scripts>pip install memory_profiler
Let's start with a simple python script example:
import psutil
def test_psutil():
# gives a single float value
print(psutil.cpu_percent())
# gives an object with many fields
print(psutil.virtual_memory())
# you can convert that object to a dictionary
print(dict(psutil.virtual_memory()._asdict()))
if __name__ == '__main__':
test_psutil()
Let's decorate python source code with @profile annotation to have a good output.
import psutil
@profile
def test_psutil():
# gives a single float value
print(psutil.cpu_percent())
# gives an object with many fields
print(psutil.virtual_memory())
# you can convert that object to a dictionary
print(dict(psutil.virtual_memory()._asdict()))
if __name__ == '__main__':
test_psutil()
Sure, this error tells us the decorate not working in the default way.
C:\Python373>python psutil_001.py
Traceback (most recent call last):
File "psutil_001.py", line 2, in
@profile
NameError: name 'profile' is not defined
In this case, the decorate profile works great with the python module and give us all the information we need:
C:\Python373>python -m memory_profiler psutil_001.py
100.0
svmem(total=4171108352, available=1022672896, percent=75.5, used=3148435456, fre
e=1022672896)
{'total': 4171108352, 'available': 1022783488, 'percent': 75.5, 'used': 31483248
64, 'free': 1022783488}
Filename: psutil_001.py
Line # Mem usage Increment Line Contents
================================================
2 15.219 MiB 15.219 MiB @profile
3 def test_psutil():
4 # gives a single float value
5 15.230 MiB 0.012 MiB print(psutil.cpu_percent())
6 # gives an object with many fields
7 15.230 MiB 0.000 MiB print(psutil.virtual_memory())
8 # you can convert that object to a dicti
onary
9 15.234 MiB 0.004 MiB print(dict(psutil.virtual_memory()._asdi
ct()))
Let's test with another python script named 001.py :
from memory_profiler import profile
@profile(precision=4)
def test():
a = 0
a = a + 1
if __name__ == "__main__":
test()
The result with precision=4 is this:
C:\Python373>python -m memory_profiler 001.py
Filename: 001.py
Line # Mem usage Increment Line Contents
================================================
3 15.3945 MiB 15.3945 MiB @profile(precision=4)
4 def test():
5 15.3945 MiB 0.0000 MiB a = 0
6 15.3945 MiB 0.0000 MiB a = a + 1
If we change the precision=1 then this is the result:
C:\Python373>python -m memory_profiler 002.py
Filename: 002.py
Line # Mem usage Increment Line Contents
================================================
3 15.4 MiB 15.4 MiB @profile(precision=1)
4 def test():
5 15.4 MiB 0.0 MiB a = 0
6 15.4 MiB 0.0 MiB a = a + 1
A good source of information for this python module can be found at GitHub.
Today I will introduce you a tutorial to fix some of the necessary elements presented in the old tutorial.
The manage tool shell can also give us some info:
C:\Python373\Scripts\example>python manage.py shell
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD6
4)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from test001.models import test
>>> from django.db.models import Count, Min, Max, Avg
>>> out = test.objects.all
>>> out
...
One note about error method, do not use like this:
>>> out = test.objects.all
>>> out[0]
Traceback (most recent call last):
File "", line 1, in
TypeError: 'method' object is not subscriptable
See the result of the annotate add the last_name__min value.
I created another class named city with just one field named city_name and I fill with one value:
All changes are posted at the end of this tutorial.
Let's test with the shell some of these new changes:
C:\Python373\Scripts\example>python manage.py shell
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from test001.models import test,city
>>> from django.db.models import Count, Min, Max, Avg
>>> out_test = test.objects.all()
>>> out_city = city.objects.all()
>>> out_test
]>
>>> out_city
]>
>>> vars(out_city[0])
{'_state': , 'id'
: 1, 'city_name': 'Fălticeni'}
Feel free to test with my old and the new example:
# models.py
from django.db import models
class test(models.Model):
first_name = models.CharField(max_length=30)
last_name = models.CharField(max_length=30)
class city(models.Model):
city_name = models.CharField(max_length=30)
#admin.py
from django.contrib import admin
from .models import test, city
admin.site.register(test)
admin.site.register(city)
#serializers.py
from rest_framework import serializers
from .models import test, city
class test_serializer(serializers.ModelSerializer):
class Meta:
model = test
fields = ('id', 'first_name', 'last_name')
class city_serializer(serializers.ModelSerializer):
class Meta:
model = city
fields = ('id', 'city_name')
#views.py
from django.shortcuts import render
from rest_framework import viewsets
from .models import test, city
from .serializers import test_serializer, city_serializer
class test_view(viewsets.ModelViewSet):
#query to get all information from database
queryset = test.objects.all()
serializer_class = test_serializer
class city_view(viewsets.ModelViewSet):
#query to get all information from database
queryset = city.objects.all()
serializer_class = city_serializer
#urls.py
from django.urls import path, include
from . import views
from rest_framework import routers
router = routers.DefaultRouter()
router.register('test001', views.test_view)
router.register('city', views.city_view)
#add to path
urlpatterns = [
path('', include(router.urls))
]
Today I tested something simpler for beginners: Django REST framework.
Once you understand how it works then it's simple to use.
This tutorial does not address the security issues generated by the REST, Django framework.
The official webpage comes with many information and technical specifications for this API: Django REST framework is a powerful and flexible toolkit for building Web APIs.
The example I've submitted is built into the Scripts folder because I did not use the virtual environment.
Let's start installing the python module.
C:\Python373\> cd Scripts
C:\Python373\Scripts\>pip install djangorestframework
C:\Python373\Scripts\>django-admin startproject example
C:\Python373\Scripts\>cd example
C:\Python373\Scripts\>python manage.py migrate
C:\Python373\Scripts\>python manage.py createsuperuser
C:\Python373\Scripts\>python manage.py startapp test001
In the example folder we will make changes.
First is the settings.py file:
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('', include('test001.urls'))
]
Also make changes in the test001 folder
You must create a python script and call it urls.py:
from django.urls import path, include
# this urlpatterns will fill later
urlpatterns = []
You make changes to the file models.py and create a named class test.
This class will have two fields that we will update.
from django.db import models
class test(models.Model):
first_name = models.CharField(max_length=30)
last_name = models.CharField(max_length=30)
44/5000
The following commands will synchronize the database:
C:\Python373\Scripts\example>python manage.py makemigrations
...
test001\migrations\0001_initial.py
- Create model test
C:\Python373\Scripts\example>python manage.py migrate
...
Running migrations:
...
Another step is to add the test into admin.py script:
from django.contrib import admin
from .models import test
admin.site.register(test)
Serializers allow complex data such as querysets and model instances to be converted to native Python datatypes that can then be easily rendered into JSON , XML or other content types.
Let's create an serializers.py python script into test001 folder and use this code:
from rest_framework import serializers
from .models import test
class test_serializer(serializers.ModelSerializer):
class Meta:
model = test
fields = ('id', 'first_name', 'last_name')
Also is need to update the views.py python script:
from django.shortcuts import render
from rest_framework import viewsets
from .models import test
from .serializers import test_serializer
class test_view(viewsets.ModelViewSet):
#query to get all information from database
queryset = test.objects.all()
serializer_class = test_serializer
Now because is all create the last step is to fix the urls.py from test001 folder with the routers.
The REST framework adds support for automatic URL routing to Django.
from django.urls import path, include
from . import views
from rest_framework import routers
router = routers.DefaultRouter()
router.register('test001', views.test_view)
#add to path
urlpatterns = [
path('', include(router.urls))
]
If you want to make changes into models.py then you will need to use the commands to synchronize the database after these use"commands makemigrations and migrate to fix errors.
If you encounter such run-time errors "GET /static/assets/js/docs.min.js HTTP/1.1" 404 1667..."
These errors can be the result of settings like DEBUG, STATICFILES_DIRS or STATIC_ROOT from file settings.py.
Then you need to execute python manage.py collectstatic and Django goes through all directories where static files can be found and places them in your static root.
The result of this tutorial can be see on my youtube channel:
Here's something I like and I hope it should be known in the Python community.
Some people from Google want to attract the python community into a learning process.
Although most of the API documentation examples do not exist in the Python programming language, they have not disappeared.
Let's hope this little step will increase the chances of programming Google with the Python programming language.
This material was created by Nick Parlante working in the engEDU group at Google. Welcome to Google's Python Class -- this is a free class for people with a little bit of programming experience who want to learn Python. The class includes written materials, lecture videos, and lots of code exercises to practice Python coding. These materials are used within Google to introduce Python to people who have just a little programming experience. The first exercises work on basic Python concepts like strings and lists, building up to the later exercises which are full programs dealing with text files, processes, and http connections. The class is geared for people who have a little bit of programming experience in some language, enough to know what a "variable" or "if statement" is. Beyond that, you do not need to be an expert programmer to use this material.
Read more about this course here.
The tutorial for today consists of using the Firebase service with python version 3.7.3 .
As you know Firebase offers multiple free and paid services.
In order to use the Python programming language, we need to use the pip utility to enter the required modules.
If your installation requires other python modules then you will need to install them in the same way.
The next step is to log in with a firebase account and create or use a project with a database.
You must use this link to see your project data and create your JSON configuration file for access.
Here's a screenshot of an old project in another programming language that I used for this tutorial.
At point 1, you will find Project Settings - Service accounts and you will need to use the Generate new private key button.
This will generate a JSON file that we will use in the python script.
Be careful to define the path to the python script file.
At point 2, you will find the database or you will need to create it to use it.
The script I'm going to present has commented on the lines for a better understanding of how to use it.
import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore
# the go-test.json from my project settings
cred = credentials.Certificate("go-test.json")
# start using firebase python modules
firebase_admin.initialize_app(cred)
# access the database
database = firestore.client()
# access the collection with limit
my_ref = database.collection("test").limit(2)
# show the values from database
try:
docs = my_ref.stream()
for doc in docs:
print(u'Doc Data:{}'.format(doc.to_dict()))
except:
print(u'Missing data')
# result running
#Doc Data:{'nrcrt': 1, 'string_value': 'this is a text'}
#Doc Data:{'name': 'test', 'added': 'just now'}
# add data to database
my_ref_add = database.collection("test")
my_ref_add.add({u'nickname': u'catafest', u'friend': u'Last Ocelot'})
# show the new database values
try:
docs = my_ref_add.stream()
for doc in docs:
print(u'Doc Data:{}'.format(doc.to_dict()))
except:
print(u'Missing data')
# result running
#Doc Data:{'nrcrt': 1, 'string_value': 'this is a text'}
#Doc Data:{'friend': 'Last Ocelot', 'nickname': 'catafest'}
#Doc Data:{'name': 'test', 'added': 'just now'}
# get just one
doc = database.collection('test').document('doc_id').get()
# show create time
print(doc.create_time)
# result running
#seconds: 1556014478
#nanos: 460439000
A good definition for this tool can be found at the Wikipedia website: 2002: PsychoPy was originally written by Peirce as a proof of concept - that a high-level scripting language could generate experimental stimuli in real time (existing solutions, such as Psychtoolbox, had to pre-generate movies or use CLUT animation techniques).
The install of this python module is very simple:
C:\Python373\Scripts>pip install psychopy
Using this command to start this tool:
C:\Python373>psychopy
The tool starts with two graphical interfaces:
one for Coder area;
one for the project with an untitled.psyexp;
Let's see one screenshot with this tool:
This open source tool, written in the Python programming language, help you to run a wide range of neuroscience, psychology and psychophysics experiments.
I searched the internet for the capabilities of this tool and the obvious conclusion is a utility for building experiments with a wide range of applicability.
This tool lets you use the pavlovia website, see more: Pavlovia is a place for the wide community of researchers in the behavioural sciences to run, share, and explore experiments online.
This tool used commonly-used components for linguistic experiments:
Text Component (display text on the screen);
Sound Component (play sounds);
Keyboard Component (receive input from the keyboard);
RatingScale Component (collect a numeric rating or a choice from a few alternatives, via the mouse, the keyboard or both);
Code Component (insert short pieces of python code into your experiments (e.g. time stamp for the production task);
The easy way is to use the Builder tool to generate a wide range of experiments easily from the Builder using its intuitive, graphical user interface (GUI).
Today I will use the Coder with programming a simple example.
To access demos, you need to create a python script in the Coder area (use File menu - New) then use the Demos from the application menu.
Everything in a PsychoPy experiment needs a unique name.
The name must contain only letters, numbers and underscores and not contain spaces, punctuation or mathematical symbols.
This tool saves several data files: a Microsoft Excel (spreadsheet) file, a psydat file, and a log file.
The example I used today for this tutorial is how to import from a CSV file named colors_001.csv and used with psychopy python module to create stimuli:
3.3;"red"
1.1;"green"
4.4;"red"
2.2;"green"
This python script named colors.py I used to import the CSV file and use it:
from psychopy import core, visual, event
import csv
## Setup section, read experiment variables from file
win = visual.Window([400,300], monitor="testMonitor", units="cm", fullscr=False)
stimuli = []
datafile = open("colors_001.csv", "r",encoding="utf8")
reader = csv.reader(datafile, delimiter=";")
for row in reader:
if len(row)==2: # ignore empty and incomplete lines
size = float(row[0]) # the first element in the row converted to a floating point number
color = row[1] # the second element in the row
stimulus = visual.Rect(win, width=size, height=size)
stimulus.fillColor = color
stimuli.append(stimulus)
datafile.close()
## Experiment Section, use experiment variables here
for stimulus in stimuli:
stimulus.draw()
win.flip()
core.wait(1.000)
## Closing Section
win.close()
core.quit()
The result of this script will show you four squares colored by the CSV file.
If you don't like to use the Builder or Coder areas, then you can simply use the python:
C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from psychopy import visual, core
pygame 1.9.5
Hello from the pygame community. https://www.pygame.org/contribute.html
>>> win = visual.Window()
>>> msg = visual.TextStim(win, text=u"Hello python users!")
>>> msg.draw()
>>> win.flip()
47.541564770566765
Today we tested an older tool with the new version of python 3.7.3.
This is a tool that will help you update your python modules.
Here's how to install:
C:\Python373\Scripts>pip-review -h
usage: pip-review [-h] [--verbose] [--raw] [--interactive] [--auto]
Keeps your Python packages fresh.
optional arguments:
-h, --help show this help message and exit
--verbose, -v Show more output
--raw, -r Print raw lines (suitable for passing to pip install)
--interactive, -i Ask interactively to install updates
--auto, -a Automatically install every update found
Unrecognised arguments will be forwarded to pip list --outdated, so you can
pass things such as --user, --pre and --timeout and they will do exactly what
you expect. See pip list -h for a full overview of the options.
To update all python modules you can use:
C:\Python373\Scripts>pip-review --auto
To run interactively, you can ask to upgrade for each package:
Today I will show you a simple script using the ORB (oriented BRIEF), see C++ documentation / OpenCV. The algorithm uses FAST in pyramids to detect stable keypoints, selects the strongest features using FAST or Harris response, finds their orientation using first-order moments and computes the descriptors using BRIEF (where the coordinates of random point pairs (or k-tuples) are rotated according to the measured orientation).
One good feature of ORB is the is rotation invariant and resistant to noise.
The ORB descriptor use the Center of the mass of the patch of the Moment (sum of x,y), Centroid (the result of the matrix of all moment) and Orientation ( the atan2 of moment one and two).
One good article about ORB can be found here.
Let's see the script code of this python example:
The script use two file images 1.png and 2.png.
The result is an image composed of the two on which the areas of similitude are traced as detected by the mathematical algorithm.
What is music21? Music21 is a set of tools for helping scholars and other active listeners answer questions about music quickly and simply. If you’ve ever asked yourself a question like, “I wonder how often Bach does that” or “I wish I knew which band was the first to use these chords in this order,” or “I’ll bet we’d know more about Renaissance counterpoint (or Indian ragas or post-tonal pitch structures or the form of minuets) if I could write a program to automatically write more of them,” then music21 can help you with your work.
This toolkit for Computer-Aided Musical Analysis was developed at MIT by cuthbertLab. Michael Scott Cuthbert, Principal Investigator.
The development of music21 is supported by the generosity of the Seaver Institute and the NEH.
The tutorial today is about the python music21 module.
Let's start with the default pip tool to install this python module:
The next step tells us to install some additional python modules:
C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from music21 import *
music21: Certain music21 functions might need these optional packages: matplotlib,
scipy;
if you run into errors, install them by following the instructions at
http://mit.edu/music21/doc/installing/installAdditional.html
from music21 import stream, instrument
from music21.note import Note
cd = Note("C#", type='quarter')
test = stream.Part()
test.insert(0, instrument.AcousticGuitar())
test_measure = stream.Measure()
test_measure.append(cd)
test.append(test_measure)
test.show('midi')
You can see all the instruments here.
This python module is very complex and can be used with additional software, see here.
More details can be found in the documentation.
The TinEye API is ideally suited for image and profile verification, UGC moderation, copyright compliance and fraud detection.
Read more about the TinEye API here.
You need to use authentication for this API, read here.
To use the TinEye API you must purchase a search bundle.
The documentation for Python can be found here.
Let's start with the installation.
You need to download the zip file from GitHub and install using the pip tool:
Because I don't have search the python code return an error:
>>> myapi.search_url(url='http://en.es-static.us/upl/2019/03/rogue-planet-CFBD-S
IR-J214947.2-040308.9-2012-800x450.jpg')
Traceback (most recent call last):
File "", line 1, in
File "C:\Python373\lib\site-packages\pytineye\api.py", line 304, in search_url
obj = self._request('search', params, **kwargs)
File "C:\Python373\lib\site-packages\pytineye\api.py", line 276, in _request
raise TinEyeAPIError(obj['code'], obj.get('messages'))
pytineye.exceptions.TinEyeAPIError: APIError:
code = 401
message = ['AUTHORIZATION_ERROR', 'You have no more searches
available. Please purchase another bundle if you wish to make more searches.']
If you want to buy TinEye API searches then the prices start from:
5,000 searches $200.00 USD($0.04 per search)
10,000 searches $300.00 USD($0.03 per search)
50,000 searches $1,000.00 USD($0.02 per search)
1,000,000 searches $10,000.00 USD($0.01 per search)
The Python 3.7.3 is the third maintenance release of Python 3.7 and is released at March 25, 2019.
More about this new released version can be found at official website.
C:\Python373>python.exe
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'4.0.0'
Let test a simple example that computes a dense optical flow using the Gunnar Farneback’s algorithm.
import cv2
import numpy as np
cap = cv2.VideoCapture(0)
ret, frame1 = cap.read()
prvs = cv2.cvtColor(frame1,cv2.COLOR_BGR2GRAY)
hsv = np.zeros_like(frame1)
hsv[...,1] = 255
while(1):
ret, frame2 = cap.read()
next = cv2.cvtColor(frame2,cv2.COLOR_BGR2GRAY)
flow = cv2.calcOpticalFlowFarneback(prvs,next, None, 0.1, 1, 2, 3, 5, 11.2, 0)
mag, ang = cv2.cartToPolar(flow[...,0], flow[...,1])
hsv[...,0] = ang*180/np.pi/2
hsv[...,2] = cv2.normalize(mag,None,0,255,cv2.NORM_MINMAX)
bgr = cv2.cvtColor(hsv,cv2.COLOR_HSV2BGR)
cv2.imshow('frame2',bgr)
k = cv2.waitKey(30) & 0xff
if k == 27:
break
prvs = next
cap.release()
cv2.destroyAllWindows()
This I.D.E. is a very good tool for python programming and development.
The version of this tool is 2.3.
I tested with my Django project on Windows OS and works great.
The development team comes with this info:
NINJA-IDE (from the recursive acronym: "Ninja-IDE Is Not Just Another IDE"), is a cross-platform integrated development environment (IDE). NINJA-IDE runs on Linux/X11, Mac OS X and Windows desktop operating systems, and allows developers to create applications for several purposes using all the tools and utilities of NINJA-IDE, making the task of writing software easier and more enjoyable.
