Question

I was looking to implement a python script that called another script and captured its stdout. The called script will contain some input and output messages eg

print ("Line 1 of Text")
variable = raw_input("Input 1 :")
print "Line 2 of Text Input: ", vairable

The section of the code I’m running is

import subprocess

cmd='testfile.py'
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
so, se = p.communicate()
print(so)

The problem that is occurring is that the stdout is not printing until after the script has been executed. This leaves a blank prompt waiting for the user input. Is there a way to get stdout to print while the called script is still running?

Thanks,

Answer 1

There are two problems here.

1. Firstly, python is buffering output to stdout and you need to prevent this. You could insert a call to sys.stdout.flush() in testfile.py as Ilia Frenkel has suggested, or you could use python -u to execute testfile.py with unbuffered I/O. (See the other stack overflow question that Ilia linked to.)

2. You need a way of asynchronously reading data from the sub-process and then, when it is ready for input, printing the data you’ve read so that the prompt for the user appears. For this, it would be very helpful to have an asynchronous version of the subprocess module.

I downloaded the asynchronous subprocess and re-wrote your script to use it, along with using python -u to get unbuffered I/O:

import async_subprocess as subprocess

cmd = ['python', '-u', 'testfile.py']
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
so = p.asyncread()
print so,
(so, se) = p.communicate()
print so

When I run this script using python -u I get the following results:

$ python -u script.py
Line 1 of Text
Input 1:

and the script pauses, waiting for input. This is the desired result.

If I then type something (e.g. “Hullo”) I get the following:

$ python -u script.py
Line 1 of Text
Input 1:Hullo
 Line 2 of Text Input:  Hullo