Question

i have a construct like the following:

os.mkfifo('pipe.tmp')
enc = Popen(['encoder', '-i', 'pipe.tmp'])
cap = Popen(['capture', '-f', 'pipe.tmp'])

here cap is a process which normally writes to a file (specified by -f), but i can get it to spew data to the screen by supplying /dev/stdout as the output file. similarly, enc expects to read from a file-like object, and i am able to get it to read from pipe by supplying - as the input. so instead of using a named pipe in the os, i thought the special file may not be necessary, i can use an unnamed pipe like this..

cap = Popen(['capture', '-f', '/dev/stdout'], stdout=PIPE)
enc = Popen(['encoder', '-i', '-'], stdin=cap.stdout)
cap.stdout.close()

(note also the reversal of order of spawning). i like this better because a temporary file seems unnecessary, but i am a bit concerned about whether this construct will chain the processes in the way i expect.

1. is the /dev/stdout that cap is talking to distinct from the actual stdout in the OS? that is, with the input pipe - in enc will i get a clean channel of data between these two processes even if other processes are chatting away to /dev/stdout on the OS?
2. will there be any significant differences in behaviour with blocking/queuing ? i think in my first example the named pipe will be a buffered 4096 bytes, and will block at either end if cap/enc aren’t writing/reading fast enough, but correct me if i’m wrong.
3. is any special order of spawning or termination required, or any other gotchas i should be aware of?

Answer 1

1. /dev/stdout gives you the stdout for the current process, so you should be just fine to use that. (there isn’t really anything ‘global’ about /dev/stdout)
2. The size of the fifo in the first example depends on the configuration of your system (I’m not sure how to change that). But subprocess.Popen allows you to define the buffer size for its I/O operations, so you should be able to tune that. Update: a little more research in, I have found there is a 64kB limit to the pipes that is not affected by the bufsize argument. Not sure how to get around that (except to read more frequently and handle the buffering manually)
3. For your second example, it looks like you are required to start in the order you gave, since you need cap.stdout to be available before starting enc. Alternatively, you could leave the two processes disconnected and handle the communication between them manually.