Question

I’m trying to extract some string from a file using python re, then MD5ing this string using
something like:

    #MD5er.py
    salt = extract_salt(file_foo)
    print 'salt: %s' % salt
    from md5 import md5
    print 'hash: %s' % md5(salt).hexdigest()

$python MD5er

    salt: \0001\072\206\277\354\107\134\061\361\076\150\047\010\124\200\315\100
    hash: ce24166858853dfb12a86d7d602b0638

BUT, using iPython like that:

    In [40]: salt = '\0001\072\206\277\354\107\134\061\361\076\150\047\010\124\200\315\100'

    In [41]: salt
    Out[41]: "\x001:\x86\xbf\xecG\\1\xf1>h'\x08T\x80\xcd@"

    In [42]: print salt
    1:���G\1�>hT��@

    In [43]: from md5 import md5

    In [44]: md5(salt).hexdigest()
    Out[44]: 'ebae47a953591f7448ff7079837fb534'

Any clues why the MD5 is different in the 2 scenarios?
and why in ipython when I typed the variable name it appeared in a different format from the original string, and print() output was a third format!?

Hint:

    In [53]: import sys
    In [54]: sys.getdefaultencoding()
    Out[54]: 'ascii' 

Answer 1

The string in the first case is exactly what you saw printed:

>>> salt = '\\0001\\072\\206\\277\\354\\107\\134\\061\\361\\076\\150\\047\\010\\
124\\200\\315\\100'
>>> md5(salt).hexdigest()
'ce24166858853dfb12a86d7d602b0638'

Notice how I’ve escaped the backslashes to keep the digits from being interpreted as octal byte values.

The string in the first case is exactly what you saw printed:

>>> salt = '\\0001\\072\\206\\277\\354\\107\\134\\061\\361\\076\\150\\047\\010\\
124\\200\\315\\100'
>>> md5(salt).hexdigest()
'ce24166858853dfb12a86d7d602b0638'

Notice how I’ve escaped the backslashes to keep the digits from being interpreted as octal byte values.

Edit:

Assuming you want to create a byte string from the octal values in this list:

data = ['\\0001', '\\072', '\\206', '\\277', '\\354', '\\107', '\\134', 
        '\\061', '\\361', '\\076', '\\150', '\\047', '\\010', '\\124', 
        '\\200', '\\315', '\\100']

You can convert to an integer and then join the characters, but it’s different from what you got in IPython. The first value is 4 digits instead of 3. Should it be treated as ‘\0’ followed by an ASCII ‘1’, or should it be treated as ‘\1’? The following does the latter:

salt = ''.join(chr(int(d[1:], 8)) for d in data)
print repr(salt)
print md5(salt).hexdigest()

Output:

"\x01:\x86\xbf\xecG\\1\xf1>h'\x08T\x80\xcd@"
d2092426d1bd5bec1579c8b7ed9c73c2