The numba python module - part 001 .

Today I tested the numba python module.
This python module allows us to speed up applications with high-performance functions written directly in Python.
The numba python module works by generating optimized machine code using the LLVM compiler infrastructure at import time, runtime, or statically.
The code can be just-in-time compiled to native machine instructions, similar in performance to C, C++ and Fortran.
For the installation I used the pip tool:

C:\Python27>cd Scripts

C:\Python27\Scripts>pip install numba
Collecting numba
  Downloading numba-0.35.0-cp27-cp27m-win32.whl (1.4MB)
    100% |################################| 1.4MB 497kB/s
...
Installing collected packages: singledispatch, funcsigs, llvmlite, numba
Successfully installed funcsigs-1.0.2 llvmlite-0.20.0 numba-0.35.0 singledispatch-3.4.0.3

C:\Python27\Scripts>pip install numpy
Requirement already satisfied: numpy in c:\python27\lib\site-packages

The example test from official website working well:
The example source code is:

from numba import jit
from numpy import arange

# jit decorator tells Numba to compile this function.
# The argument types will be inferred by Numba when function is called.
@jit
def sum2d(arr):
    M, N = arr.shape
    result = 0.0
    for i in range(M):
        for j in range(N):
            result += arr[i,j]
    return result

a = arange(9).reshape(3,3)
print(sum2d(a))

The result of this run python script is:

C:\Python27>python.exe numba_test_001.py
36.0

Another example using just-in-time compile is used with Numba’s jit function:

import numba
from numba import jit

def fibonacci(n):
    a, b = 1, 1
    for i in range(n):
        a, b = a+b, a
    return a

print fibonacci(10)

fibonacci_jit = jit(fibonacci)
print fibonacci_jit(14)

Also, you can use jit is as a decorator:

@jit
def fibonacci_jit(n):
    a, b = 1, 1
    for i in range(n):
        a, b = a+b, a

    return a

Numba is a complex python module because use compiling.
First, compiling takes time, but will work especially for small functions.
The Numba python module tries to do its best by caching compilation as much as possible though.
Another note: not all code is compiled equally.

YARA another python module - part 002 .

This is another part of YARA python tutorial and the goal of this part is to install the Yara modules.
This is another python module about Yara named yara-python from VirusTotal.
The last tutorial uses the Yara python module.
The YARA modules provide extending features to allow us to define data structures and functions which can be used in your rules to express more complex conditions.
You can also write your own modules.
Some known modules used by YARA are:

• PE
• ELF
• Cuckoo
• Magic
• Hash
• Math

First, you need to install or reinstall YARA to the last version:

>>> yara.__version__
'3.6.3'

The Cuckoo module enables you to create YARA rules based on behavioral information generated by a Cuckoo sandbox.

C:\Python27\Scripts>pip install yara-python
Collecting yara-python
  Downloading yara_python-3.6.3-cp27-cp27m-win32.whl (606kB)
    100% |################################| 614kB 1.3MB/s
Installing collected packages: yara-python
Successfully installed yara-python-3.6.3
pip install cuckoo
Collecting cuckoo
  Downloading Cuckoo-2.0.4.4.tar.gz (3.1MB)
    100% |################################| 3.1MB 255kB/s
...
Successfully installed Mako-1.0.7 alembic-0.8.8 androguard-3.0.1 beautifulsoup4-4.5.3 

capstone-windows-3.0.4 chardet-2.3.0 click-6.6 colorama-0.3.7 cuckoo-2.0.4.4 django-1.8.4 

django-extensions-1.6.7 dpkt-1.8.7 ecdsa-0.13 egghatch-0.2.1 elasticsearch-5.3.0 

flask-sqlalchemy-2.1 httpreplay-0.2.1 jsbeautifier-1.6.2 jsonschema-2.6.0 olefile-0.43 

oletools-0.42 peepdf-0.3.6 pefile2-1.2.11 pillow-3.2.0 pyelftools-0.24 pymisp-2.4.54 

pymongo-3.0.3 python-dateutil-2.4.2 python-editor-1.0.3 python-magic-0.4.12 pythonaes-1.0 

requests-2.13.0 sflock-0.2.16 sqlalchemy-1.0.8 tlslite-ng-0.6.0 unicorn-1.0.1 wakeonlan-0.2.2

Let's test this python module:

>>> import cuckoo
>>> from cuckoo import *
>>> dir(cuckoo)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__',

 'auxiliary', 'common', 'compat', 'core', 'machinery', 'misc', 'plugins', 'processing', 

'reporting', 'signatures', 'web']

Let's test some yara modules:

>>> import yara
>>> rule = yara.compile(source='import \"pe\"')
>>> rule = yara.compile(source='import \"elf\"')
>>> rule = yara.compile(source='import \"cuckoo\"')
>>> rule = yara.compile(source='import \"math\"')

I could not use the YARA modules: hash and magic.
I will solve this problem in the future.
You can also write your own modules ( see this webpage ).