Friday, February 23, 2018

Use IMDB website with IMDbPY python module .

This python package is written in pure Python 3 to access the IMDb's database and used it.

You can read about this python module from github docs weboage
The development team come with this DISCLAIMER:
# DISCLAIMER

IMDbPY and the authors are not affiliated with Internet Movie Database Inc.

IMDb is a trademark of Internet Movie Database Inc. and all contents
and data included on the IMDb's site is the property of IMDb or its
content suppliers and protected by United States and international
copyright laws.

Please, read the IMDb's conditions of use in their website:
- http://www.imdb.com/conditions
- http://www.imdb.com/licensing
- any other notice in the http://www.imdb.com/ site.

First I start the install process with pip tool:
C:\Python364\Scripts>pip install IMDbPY
Requirement already satisfied: IMDbPY in c:\python364\lib\site-packages
Requirement already satisfied: lxml in c:\python364\lib\site-packages (from IMDbPY)
Requirement already satisfied: sqlalchemy-migrate in c:\python364\lib\site-packages (from IMDbPY)
Requirement already satisfied: SQLAlchemy in c:\python364\lib\site-packages (from IMDbPY)
Requirement already satisfied: pbr>=1.8 in c:\python364\lib\site-packages (from sqlalchemy-migrate->IMDbPY)
Requirement already satisfied: decorator in c:\python364\lib\site-packages (from sqlalchemy-migrate->IMDbPY)
Requirement already satisfied: six>=1.7.0 in c:\python364\lib\site-packages (from sqlalchemy-migrate->IMDbPY)
Requirement already satisfied: sqlparse in c:\python364\lib\site-packages (from sqlalchemy-migrate->IMDbPY)
Requirement already satisfied: Tempita>=0.4 in c:\python364\lib\site-packages (from sqlalchemy-migrate->IMDbPY)
This is my source code to test it and working well.
# start with IMDb python class
from imdb import IMDb
imd = IMDb('http')
print("-===-")
# search movies by title
# and show the long imdb canonical title and movieID of the results.
title = imd.search_movie("Under the Dome")
for item in title:
   print(item['long imdb canonical title'], item.movieID)
print("-===-")
# search for a person
for person in imd.search_person("Ana de Armas"):
    print(person.personID, person['name'])
print("-===-")
# get 5 movies tagged with a keyword
movies_keyword = imd.get_keyword('novel', results=5)
for item in movies_keyword:
   print(item['long imdb canonical title'], item.movieID)
print("-===-")
# get top 250  from top movies
top250 = imd.get_top250_movies()
for item in top250:
   print(item['long imdb canonical title'], item.movieID)
print("-===-")
print("top 250 -=> ")
# get bottom 100 from top movies
bottom100 = imd.get_bottom100_movies()
print("bottom 100 -=> ")
for item in top250:
   print(item['long imdb canonical title'], item.movieID)

Thursday, February 8, 2018

Python 2.7 : Testing the pefile python module.

The pefile is a python module to read and work with PE (Portable Executable) files.
The install of this python module is very easy with the pip tool.
I tested the default example create with FASM to see if this is working well:
This is the source code:
; Example of 64-bit PE program
format PE64 GUI
entry start

section '.text' code readable executable

  start:
        sub     rsp,8*5         ; reserve stack for API use and make stack dqword aligned

        mov     r9d,0
        lea     r8,[_caption]
        lea     rdx,[_message]
        mov     rcx,0
        call    [MessageBoxA]

        mov     ecx,eax
        call    [ExitProcess]

section '.data' data readable writeable

  _caption db 'Win64 assembly program',0
  _message db 'Hello World!',0

section '.idata' import data readable writeable

  dd 0,0,0,RVA kernel_name,RVA kernel_table
  dd 0,0,0,RVA user_name,RVA user_table
  dd 0,0,0,0,0

  kernel_table:
    ExitProcess dq RVA _ExitProcess
    dq 0
  user_table:
    MessageBoxA dq RVA _MessageBoxA
    dq 0

  kernel_name db 'KERNEL32.DLL',0
  user_name db 'USER32.DLL',0

  _ExitProcess dw 0
    db 'ExitProcess',0
  _MessageBoxA dw 0
    db 'MessageBoxA',0  
The python script I used to test this python module is this:
import sys
from sys import argv
import mmap
import pefile

fp = open(argv[1],"r")
map = mmap.mmap(fp.fileno(),0,access=mmap.ACCESS_READ)
pe = pefile.PE(data=map[:])
print pe
The output is this:
C:\Python27>python.exe pe.py PE64DEMO.EXE
----------Parsing Warnings----------

Byte 0x00 makes up 87.5488% of the file's contents. This may indicate truncation / malformation.

----------DOS_HEADER----------

[IMAGE_DOS_HEADER]
0x0 0x0 e_magic: 0x5A4D
0x2 0x2 e_cblp: 0x80
0x4 0x4 e_cp: 0x1
0x6 0x6 e_crlc: 0x0
0x8 0x8 e_cparhdr: 0x4
0xA 0xA e_minalloc: 0x10
0xC 0xC e_maxalloc: 0xFFFF
0xE 0xE e_ss: 0x0
0x10 0x10 e_sp: 0x140
0x12 0x12 e_csum: 0x0
0x14 0x14 e_ip: 0x0
0x16 0x16 e_cs: 0x0
0x18 0x18 e_lfarlc: 0x40
0x1A 0x1A e_ovno: 0x0
0x1C 0x1C e_res:
0x24 0x24 e_oemid: 0x0
0x26 0x26 e_oeminfo: 0x0
0x28 0x28 e_res2:
0x3C 0x3C e_lfanew: 0x80

----------NT_HEADERS----------

[IMAGE_NT_HEADERS]
0x80 0x0 Signature: 0x4550

----------FILE_HEADER----------

[IMAGE_FILE_HEADER]
0x84 0x0 Machine: 0x8664
0x86 0x2 NumberOfSections: 0x3
0x88 0x4 TimeDateStamp: 0x5A1954AF [Sat Nov 25 11:31:59 2017 UTC]
0x8C 0x8 PointerToSymbolTable: 0x0
0x90 0xC NumberOfSymbols: 0x0
0x94 0x10 SizeOfOptionalHeader: 0xF0
0x96 0x12 Characteristics: 0x2F
Flags: IMAGE_FILE_EXECUTABLE_IMAGE, IMAGE_FILE_LARGE_ADDRESS_AWARE, IMAGE_FILE_LINE_NUMS_STRIPPED, IMAGE_FILE_LOCAL_SYMS_STRIPPED, IMAGE_FILE_RELOCS_STRIPPED

----------OPTIONAL_HEADER----------

[IMAGE_OPTIONAL_HEADER64]
0x98 0x0 Magic: 0x20B
0x9A 0x2 MajorLinkerVersion: 0x1
0x9B 0x3 MinorLinkerVersion: 0x49
0x9C 0x4 SizeOfCode: 0x200
0xA0 0x8 SizeOfInitializedData: 0x400
0xA4 0xC SizeOfUninitializedData: 0x0
0xA8 0x10 AddressOfEntryPoint: 0x1000
0xAC 0x14 BaseOfCode: 0x1000
0xB0 0x18 ImageBase: 0x400000
0xB8 0x20 SectionAlignment: 0x1000
0xBC 0x24 FileAlignment: 0x200
0xC0 0x28 MajorOperatingSystemVersion: 0x1
0xC2 0x2A MinorOperatingSystemVersion: 0x0
0xC4 0x2C MajorImageVersion: 0x0
0xC6 0x2E MinorImageVersion: 0x0
0xC8 0x30 MajorSubsystemVersion: 0x5
0xCA 0x32 MinorSubsystemVersion: 0x0
0xCC 0x34 Reserved1: 0x0
0xD0 0x38 SizeOfImage: 0x4000
0xD4 0x3C SizeOfHeaders: 0x200
0xD8 0x40 CheckSum: 0xECAF
0xDC 0x44 Subsystem: 0x2
0xDE 0x46 DllCharacteristics: 0x0
0xE0 0x48 SizeOfStackReserve: 0x1000
0xE8 0x50 SizeOfStackCommit: 0x1000
0xF0 0x58 SizeOfHeapReserve: 0x10000
0xF8 0x60 SizeOfHeapCommit: 0x0
0x100 0x68 LoaderFlags: 0x0
0x104 0x6C NumberOfRvaAndSizes: 0x10
DllCharacteristics:

----------PE Sections----------

[IMAGE_SECTION_HEADER]
0x188 0x0 Name: .text
0x190 0x8 Misc: 0x2D
0x190 0x8 Misc_PhysicalAddress: 0x2D
0x190 0x8 Misc_VirtualSize: 0x2D
0x194 0xC VirtualAddress: 0x1000
0x198 0x10 SizeOfRawData: 0x200
0x19C 0x14 PointerToRawData: 0x200
0x1A0 0x18 PointerToRelocations: 0x0
0x1A4 0x1C PointerToLinenumbers: 0x0
0x1A8 0x20 NumberOfRelocations: 0x0
0x1AA 0x22 NumberOfLinenumbers: 0x0
0x1AC 0x24 Characteristics: 0x60000020
Flags: IMAGE_SCN_CNT_CODE, IMAGE_SCN_MEM_EXECUTE, IMAGE_SCN_MEM_READ
Entropy: 0.540255 (Min=0.0, Max=8.0)
MD5 hash: 54edeb1437149ccc09183b623e3be7b8
SHA-1 hash: c473f3db5ca81084db3489ab3519832ded9cc28c
SHA-256 hash: 74e9ff7d6902292d9a8ad93174aef46596f8f1fe9eb5dd72b9ebc99f8bd2ecfb
SHA-512 hash: 070610baa66d6efcbb2cc7e935c2afd2686068818c00b772b3e62de103389cecbc6c309976e10860a974532a1018fba9da50effb64c60f533433dbb808ba088c

[IMAGE_SECTION_HEADER]
0x1B0 0x0 Name: .data
0x1B8 0x8 Misc: 0x24
0x1B8 0x8 Misc_PhysicalAddress: 0x24
0x1B8 0x8 Misc_VirtualSize: 0x24
0x1BC 0xC VirtualAddress: 0x2000
0x1C0 0x10 SizeOfRawData: 0x200
0x1C4 0x14 PointerToRawData: 0x400
0x1C8 0x18 PointerToRelocations: 0x0
0x1CC 0x1C PointerToLinenumbers: 0x0
0x1D0 0x20 NumberOfRelocations: 0x0
0x1D2 0x22 NumberOfLinenumbers: 0x0
0x1D4 0x24 Characteristics: 0xC0000040
Flags: IMAGE_SCN_CNT_INITIALIZED_DATA, IMAGE_SCN_MEM_READ, IMAGE_SCN_MEM_WRITE
Entropy: 0.627189 (Min=0.0, Max=8.0)
MD5 hash: 6684d4efed7dc864e5bbb0280faa841b
SHA-1 hash: 0214a59237a9020d3fa41419107a59f276a95e5f
SHA-256 hash: 23ae47e7bfb842935b35775428fe9c5df5c3f46fa46c2da2e93a27ba031ae091
SHA-512 hash: 60eeefcb47e1e63584342049a66d4539ab4b580190faf9d2629e0db1336933835c207e419060cce08cfec430e2f1e13a90cac7abfb05679ed5d84dac8997f12f

[IMAGE_SECTION_HEADER]
0x1D8 0x0 Name: .idata
0x1E0 0x8 Misc: 0x90
0x1E0 0x8 Misc_PhysicalAddress: 0x90
0x1E0 0x8 Misc_VirtualSize: 0x90
0x1E4 0xC VirtualAddress: 0x3000
0x1E8 0x10 SizeOfRawData: 0x200
0x1EC 0x14 PointerToRawData: 0x600
0x1F0 0x18 PointerToRelocations: 0x0
0x1F4 0x1C PointerToLinenumbers: 0x0
0x1F8 0x20 NumberOfRelocations: 0x0
0x1FA 0x22 NumberOfLinenumbers: 0x0
0x1FC 0x24 Characteristics: 0xC0000040
Flags: IMAGE_SCN_CNT_INITIALIZED_DATA, IMAGE_SCN_MEM_READ, IMAGE_SCN_MEM_WRITE
Entropy: 0.996929 (Min=0.0, Max=8.0)
MD5 hash: 073b9b0656f7ca77d968f183a1ceb909
SHA-1 hash: acefe438c7bfef7362b87519349c5a7b251aa43d
SHA-256 hash: 016761b2d3b31ed8eeddccc9f56e6338978171a0082c066cbf2b28cecd77566a
SHA-512 hash: a5fb7ace9108f63c96c9da239fc5077106cf3ffe8e31a1ab0a11b589a8e6af9e66d23c38060c157a3e34125bc5af495c770e48bc00172a5c8ec78b34794628b3

----------Directories----------

[IMAGE_DIRECTORY_ENTRY_EXPORT]
0x108 0x0 VirtualAddress: 0x0
0x10C 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_IMPORT]
0x110 0x0 VirtualAddress: 0x3000
0x114 0x4 Size: 0x90
[IMAGE_DIRECTORY_ENTRY_RESOURCE]
0x118 0x0 VirtualAddress: 0x0
0x11C 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_EXCEPTION]
0x120 0x0 VirtualAddress: 0x0
0x124 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_SECURITY]
0x128 0x0 VirtualAddress: 0x0
0x12C 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_BASERELOC]
0x130 0x0 VirtualAddress: 0x0
0x134 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_DEBUG]
0x138 0x0 VirtualAddress: 0x0
0x13C 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_COPYRIGHT]
0x140 0x0 VirtualAddress: 0x0
0x144 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_GLOBALPTR]
0x148 0x0 VirtualAddress: 0x0
0x14C 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_TLS]
0x150 0x0 VirtualAddress: 0x0
0x154 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG]
0x158 0x0 VirtualAddress: 0x0
0x15C 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_BOUND_IMPORT]
0x160 0x0 VirtualAddress: 0x0
0x164 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_IAT]
0x168 0x0 VirtualAddress: 0x0
0x16C 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT]
0x170 0x0 VirtualAddress: 0x0
0x174 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR]
0x178 0x0 VirtualAddress: 0x0
0x17C 0x4 Size: 0x0
[IMAGE_DIRECTORY_ENTRY_RESERVED]
0x180 0x0 VirtualAddress: 0x0
0x184 0x4 Size: 0x0

----------Imported symbols----------

[IMAGE_IMPORT_DESCRIPTOR]
0x600 0x0 OriginalFirstThunk: 0x0
0x600 0x0 Characteristics: 0x0
0x604 0x4 TimeDateStamp: 0x0 [Thu Jan 01 00:00:00 1970 UTC]
0x608 0x8 ForwarderChain: 0x0
0x60C 0xC Name: 0x305C
0x610 0x10 FirstThunk: 0x303C

KERNEL32.DLL.ExitProcess Hint[0]

[IMAGE_IMPORT_DESCRIPTOR]
0x614 0x0 OriginalFirstThunk: 0x0
0x614 0x0 Characteristics: 0x0
0x618 0x4 TimeDateStamp: 0x0 [Thu Jan 01 00:00:00 1970 UTC]
0x61C 0x8 ForwarderChain: 0x0
0x620 0xC Name: 0x3069
0x624 0x10 FirstThunk: 0x304C

USER32.DLL.MessageBoxA Hint[0]

Sunday, February 4, 2018

The collections python module .

This module named collections implements some nice data structures which will help you to solve various real life problems.
Let's start to see the content of this python module:
C:\Users\catafest>python

C:\Users\catafest>cd C:\Python27\

C:\Python27>python
Python 2.7 (r27:82525, Jul  4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import collections
>>> from collections import *
>>> dir(collections)
['Callable', 'Container', 'Counter', 'Hashable', 'ItemsView', 'Iterable', 'Iterator', 'KeysView',
 'Mapping', 'MappingView', 'MutableMapping', 'MutableSequence', 'MutableSet', 'OrderedDict', 'Sequence',
 'Set', 'Sized', 'ValuesView', '__all__', '__builtins__', '__doc__', '__file__', '__name__', '__package__'
, '_abcoll', '_chain', '_eq', '_heapq', '_ifilter', '_imap', '_iskeyword', '_itemgetter', '_repeat', 
'_starmap', '_sys', 'defaultdict', 'deque', 'namedtuple']
Now I will tell you about some
First is Counter and is a dict subclass which helps to count hashable objects.
The elements are stored as dictionary keys and counts are stored as values which can be zero or negative.
Next is defaultdict and is a dictionary object which provides all methods provided by dictionary.
This takes first argument (default_factory) as default data type for the dictionary.
The namedtuple helps to have meaning of each position in a tuple.
This allow us to code with better readability and self-documenting code.
Let's try some examples:
>>> from collections import Counter
>>> from collections import defaultdict
>>> from collections import namedtuple
>>> import re
>>> path = 'C:/yara_reg_rundll32.txt'
>>> output = re.findall('\w+', open(path).read().lower())
>>> Counter(output).most_common(5)
[('a', 2), ('nocase', 2), ('javascript', 2), ('b', 2), ('rundll32', 2)]
>>> 
>>> d = defaultdict(list)
>>> colors = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]
>>> for k, v in colors:
...     d[k].append(v)
...
>>> d.items()
[('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])]
>>> 
>>> Vertex = namedtuple('vertex', ['x', 'y'])
>>> v = Vertex(5,y = 9)
>>> v
vertex(x=5, y=9)
>>> v.x*v.y
45
>>> v[0]
5
>>> v[0]+v[1]
14
>>> x,y = v
>>> v
vertex(x=5, y=9)
>>> x
5
>>> y
9
>>>
The content of the yara_reg_rundll32.txt file is:
rule poweliks_rundll32_exe_javascript
{
meta:
description = "detect Poweliks' autorun rundll32.exe javascript:..."
string:
$a = "rundll32.exe" nocase
$b = "javascript" nocase
condition:
$a and $b
}

I used vertex variables into my example because can be used with Blender 3D.
You can see many examples at official documentation website.