Question

If this is my subprocess:

import time, sys for i in range(200):     sys.stdout.write( 'reading %i\n'%i )     time.sleep(.02) 

And this is the script controlling and modifying the output of the subprocess:

import subprocess, time, sys  print 'starting'      proc = subprocess.Popen(     'c:/test_apps/testcr.py',     shell=True,     stdin=subprocess.PIPE,     stdout=subprocess.PIPE  )  print 'process created'  while True:     #next_line = proc.communicate()[0]     next_line = proc.stdout.readline()     if next_line == '' and proc.poll() != None:         break     sys.stdout.write(next_line)     sys.stdout.flush()      print 'done' 

Why is readline and communicate waiting until the process is done running? Is there a simple way to pass (and modify) the subprocess’ stdout real-time?

I’m on Windows XP.

Answer 1

As Charles already mentioned, the problem is buffering. I ran in to a similar problem when writing some modules for SNMPd, and solved it by replacing stdout with an auto-flushing version.

I used the following code, inspired by some posts on ActiveState:

class FlushFile(object):     '''Write-only flushing wrapper for file-type objects.'''     def __init__(self, f):         self.f = f     def write(self, x):         self.f.write(x)         self.f.flush()  # Replace stdout with an automatically flushing version sys.stdout = FlushFile(sys.__stdout__)