Question

A lot of modules I use import entire files into memory or trickle a file’s contents in while they process it. I’m wondering if there’s any way to track this sort of loading progress? Possibly a wrapper class that takes a callback?

Answer 1

I would do by this by determining the size of the file, and then simply dividing the total by the number of bytes read. Like this:

import os  def show_progress(file_name, chunk_size=1024):     fh = open(file_name, 'r')     total_size = os.path.getsize(file_name)     total_read = 0     while True:         chunk = fh.read(chunk_size)         if not chunk:              fh.close()             break         total_read += len(chunk)         print 'Progress: %s percent' % (total_read/total_size)         yield chunk  for chunk in show_progress('my_file.txt'):     # Process the chunk     pass  

Edit: I know it isn’t the best code, but I just wanted to show the concept.