Question

I am running this code to see the performance impact of the keyczar encryption library from google:

from keyczar import keyczar, keys

def main(iters):
    key = keys.RsaPrivateKey.Generate()
    msg = "ciao"
    crypt = None
    for i in range(iters):
        print i, "\r",
        crypt = key.Encrypt(msg)
    for i in range(iters):
        print i, "\r",
        key.Decrypt(crypt)

if __name__ == '__main__':
    main(500)

Under Windows, 500 iterations takes about 16 minutes. Under an Ubuntu 9.04 partition on the same machine, 500 iterations takes about 6 seconds.

I’ve tried profiling this (cProfile + pstats) but I don’t have much experience in interpreting the results.

Can someone tell me why the same code runs 150+ times slower under Windows?

---

Edit 2010-01-16

Here is my generate_key.py script:

from keyczar import keyczar, keys

key = keys.RsaPrivateKey.Generate()

Here is my command line to create a stats file in generate_key:

C:\temp\python-keyczar-0.6b\tests\keyczar>python -m cProfile -o generate_key generate_key.py

Here is my python session to expand the results:

>>> import pstats
>>> p = pstats.Stats('generate_key')
>>> p.strip_dirs().sort_stats(-1).print_stats(25)
Sat Jan 16 12:18:43 2010    generate_key

         83493 function calls (82974 primitive calls) in 5.131 CPU seconds

   Ordered by: standard name
   List reduced from 564 to 25 due to restriction <25>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 <string>:1(<module>)
        1    0.000    0.000    0.210    0.210 AES.py:1(<module>)
        1    0.022    0.022    0.210    0.210 AES.py:1(__bootstrap__)
        1    0.000    0.000    0.000    0.000 DSA.py:115(DSAobj)
        1    0.001    0.001    0.005    0.005 DSA.py:14(<module>)
        1    0.000    0.000    0.000    0.000 DSA.py:174(DSAobj_c)
        1    0.000    0.000    0.000    0.000 DSA.py:26(error)
        1    0.000    0.000    0.001    0.001 RSA.py:125(size)
        1    0.000    0.000    0.000    0.000 RSA.py:13(<module>)
        1    0.000    0.000    0.000    0.000 RSA.py:140(publickey)
        1    0.000    0.000    0.000    0.000 RSA.py:146(RSAobj_c)
        1    0.000    0.000    0.000    0.000 RSA.py:23(error)
        1    0.000    0.000    4.816    4.816 RSA.py:26(generate)
        1    0.000    0.000    0.000    0.000 RSA.py:63(construct)
        1    0.000    0.000    0.000    0.000 RSA.py:85(RSAobj)
        1    0.003    0.003    0.004    0.004 SHA.py:4(<module>)
        1    0.000    0.000    0.000    0.000 __future__.py:48(<module>)
        1    0.000    0.000    0.000    0.000 __future__.py:74(_Feature)
        7    0.000    0.000    0.000    0.000 __future__.py:75(__init__)
        8    0.000    0.000    0.000    0.000 __init__.py:1(<module>)
        1    0.000    0.000    0.000    0.000 __init__.py:11(<module>)
        1    0.000    0.000    0.000    0.000 __init__.py:13(<module>)
        1    0.000    0.000    0.000    0.000 __init__.py:18(<module>)
        1    0.000    0.000    0.000    0.000 __init__.py:20(<module>)
        2    0.000    0.000    0.000    0.000 __init__.py:24(<module>)


<pstats.Stats instance at 0x023F5E40>
>>>

So the Windows code does execute in python. Most of the runtime is spent here:

def generate(bits, randfunc, progress_func=None):
    """generate(bits:int, randfunc:callable, progress_func:callable)

    Generate an RSA key of length 'bits', using 'randfunc' to get
    random data and 'progress_func', if present, to display
    the progress of the key generation.
    """
    obj=RSAobj()

    # Generate the prime factors of n
    if progress_func:
        progress_func('p,q\n')
    p = q = 1L
    while number.size(p*q) < bits:
        p = pubkey.getPrime(bits/2, randfunc)
        q = pubkey.getPrime(bits/2, randfunc)

    # p shall be smaller than q (for calc of u)
    if p > q:
        (p, q)=(q, p)
    obj.p = p
    obj.q = q

    if progress_func:
        progress_func('u\n')
    obj.u = pubkey.inverse(obj.p, obj.q)
    obj.n = obj.p*obj.q

    obj.e = 65537L
    if progress_func:
        progress_func('d\n')
    obj.d=pubkey.inverse(obj.e, (obj.p-1)*(obj.q-1))

    assert bits <= 1+obj.size(), "Generated key is too small"

    return obj

I am running PyCrypto downloade from here.

Answer 1

PyCrypto has a C module called _fastmath which uses GNU MP for the public key operations. If it is not available, it instead uses Python’s native long integers, which are much much slower.

The two files are src/_fastmath.c and lib/Crypto/PublicKey/_slowmath.py

It’s likely that Python on Windows does not include GNU MP, so on Windows it’s instead using _slowmath.py