Question

I am trying to read from a process that produces long and time-consuming output. However, I want to catch it’s output as and when it is produced. But using something like the following seems to be buffering the command’s output, so I end up getting the output lines all at once:

p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, bufsize=0)
    for line in p.stdout:
        print line

I am trying this on MacOS 10.5

Answer 1

The file iterator is doing some internal buffering on its own. Try this:

line = p.stdout.readline()
while line:
    print line
    line = p.stdout.readline()

You also need to make sure the process you are running is actually flushing its output buffers frequently.