Thursday, April 28, 2016

Maximum lines of code for Python script

I've created a script with 10 million non-empty and non-comment lines. Python process has consumed over 10 Gigs of RAM, but successfully executed :)

So what is the maximum lines of code permitted in Python?
No limitation.

Sunday, April 24, 2016

Python frozen modules __hello__ and __phello__

Under Python 2.7:
>>> import __hello__
Hello world...
>>> import __phello__
Hello world...
>>> import __phello__.spam
Hello world...

Under Python 3.x:
>>> import __hello__
Hello World!

If check a file of this modules the next result will be returned:
>>> __hello__.__file__
'<frozen>'

The byte-code of these modules (see Python' source file ./Python/frozen.c) is compiled into Python lib (python27.dll on Windows and libpython2.7.so on Linux).

To check whether the module is frozen it's possible to use imp.is_frozen:
>>> import imp
>>> imp.is_frozen('__hello__')
True
>>> imp.is_frozen('__phello__')
True
>>> imp.is_frozen('__phello__.spam')
True

It's also possible to get the code object of these modules and for example get the bytecode:
>>> imp.get_frozen_object('__phello__.spam').co_code
'd\x00\x00GHd\x01\x00S'

Or get the code object filename:
>>> imp.get_frozen_object('__hello__').co_filename
'hello.py'
>>> imp.get_frozen_object('__phello__').co_filename
'hello.py'
>>> imp.get_frozen_object('__phello__.spam').co_filename
'hello.py'

To load a frozen module Python C API function PyImport_ImportFrozenModule is used.

Thursday, April 14, 2016

Python anonymous class name with anonymous class variable name

In Python it is possible to create new type object without any name (or more precisely with an empty name).
Moreover, you can create an anonymous (empty) class variable.
For example to create Phantom class with anonymous (empty) class variable:
>>> Phantom = type('', (object,), {'': 'surprise'})
>>> p = Phantom()
>>> Phantom.__name__
''
>>> p.__class__
<class '__main__.'>
>>> getattr(p, '')
'surprise'

Sunday, April 10, 2016

Python hashlib algorithms

It's very interesting that some algorithms from Python hashlib module are not documented, for example SHA which is SHA-0.

>>> import hashlib
>>> h = hashlib.new('sha')
>>> h.update(b'')
>>> h.hexdigest()
'f96cea198ad1dd5617ac084a3d92c6107708c0ef'

The reason of such behavior is loading of these algorithms using _hashlib module, which is based on OpenSSL library available on your platform. Therefore, these additional algorithms may vary.

Here is how hashlib new() constructor is assigned based on _hashlib (OpenSSL) module availability (from hashlib.py):

try:
    import _hashlib
    new = __hash_new
    __get_hash = __get_openssl_constructor
    algorithms_available = algorithms_available.union(
            _hashlib.openssl_md_meth_names)
except ImportError:
    new = __py_new
    __get_hash = __get_builtin_constructor

By default only several hash algorithms are always available in hashlib module.
You can check all of them using hashlib.algorithms_available (was added in version 2.7.9):
>>> hashlib.algorithms
('md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512')

Additional algorithms available by hashlib.algorithms_available (was added in version 2.7.9):
>>> hashlib.algorithms_available
{'SHA384', 'MD5', 'sha512', 'MD4', 'RIPEMD160', 'dsaEncryption', 'SHA224', 'SHA', 'sha', 'ecdsa-with-SHA1', 'md5', 'whirlpool', 'dsaWithSHA', 'SHA1', 'sha1', 'sha384', '
DSA', 'sha224', 'md4', 'ripemd160', 'DSA-SHA', 'SHA512', 'SHA256', 'sha256'}