Question

How can I get a MD5, SHA and other hashes from a file but only doing one pass? I have 100mb files, so I’d hate to process those 100MB files multiple times.

Answer 1

Here’s a modified @ʞɔıu‘s answer using @Jason S‘ suggestion.

from __future__ import with_statement from hashlib import md5, sha1  filename = 'hash_one-pass.py'  hashes = md5(), sha1() chunksize = max(4096, max(h.block_size for h in hashes)) with open(filename, 'rb') as f:     while True:         chunk = f.read(chunksize)         if not chunk:             break         for h in hashes:             h.update(chunk)  for h in hashes:     print h.name, h.hexdigest()