Question

I have a program that I would like to launch via subprocess. It can output to fd other than stdout and stderr. Is there a way to capture data from an arbitrary fd? I’d like to do something like the following, if my process is sending output to fd 9:

import subprocess, StringIO
redirector = StringIO.StringIO()
errno = subprocess.call(cmd, fd9=redirector)
#process the error code and data in redirector

I do not want to redirect the called process’s output to fd 9 through stderr or stdout.

Answer 1

If the program opens file descriptor 9 (or whatever) after it starts running, then there is no way to do what you want.

If the program does not open file descriptor 9 itself, but gets it from the parent process, then you can do what you want with the preexec_fn argument to subprocess.Popen, but it is not as simple as setting something to a StringIO instance. You have to create a pipe by hand, and you have to arrange to read from it in a timely fashion, or you will cause a deadlock. Expect to write two or three hundred lines of code to get this 100% right. I’m not going to put any sample code here because I don’t want to make it look easier than it is.

In addition to subprocess.Popen, you will need os.pipe, os.dup2, and os.read, and you will need to understand the underlying system primitives:
pipe(2), dup2(2), read(2), fork(2), and execve(2).

Now might be a good time to invest in a copy of Advanced Programming in the Unix Environment.