Question

I have a Python program that uses Popen to call a test C++ program. The test C++ program simply writes 0-99999 to stdout. The Python program has two functions that should be run as seperate processes. One function, funcA, should launch the C++ program, read the integers from the stdout pipe, and insert those integers into a shared Queue. The other function, funcB, should read and print the integers in the Queue until the Queue is empty. I have some issues / questions that I will post below, along with my code below that.

1. What is the proper way for funcA to read from the C++ program’s stdout until it (the C++ program) terminates?
2. What is the proper way for funcB to read from the shared Queue until all ints have been processed?

My current method for question 1 works, I believe, but I know there may be some issues that I don’t check such as the Queue filling up. Also, all the numbers are not printed out (stops at about 98000) and I think this might have something to do with funcA terminating and disrupting the shared Queue? I am not exactly sure what to do for question 2 because the documentation says that one can’t rely on empty() in a concurrent processing atmosphere and I don’t want to use a while(1).

import multiprocessing
import subprocess
import Queue

def funcA(intQueue):
    # call C++ program
    handle = subprocess.Popen(['../C++/C++.exe'], stdout=subprocess.PIPE)

    while(handle.returncode == None):
        handle.stdout.readline()
        intQueue.put(handle.stdout.readline())
        handle.poll()

def funcB(intQueue):
    try:
        while(1):
            print intQueue.get(True, 2)
    except Queue.Empty:
        pass

if __name__ == "__main__":
    # shared Queue for all the processes
    intQueue = multiprocessing.Queue()

    # producer - receives ints from the C++ stdout and inserts into Queue
    multiprocessing.Process(target=funcA, args=(intQueue,)).start()

    # consumer - prints ints from the Queue
    multiprocessing.Process(target=funcB, args=(intQueue,)).start()

Answer 1

In case anyone comes across this same problem:

For question 1 I used a while(1) that breaks when the list returned from a splitting of handle.stdout.read() has a length of 1 (this means that nothing was returned from the pipe).

For question 2 I used the poison pill method described in this post: http://www.doughellmann.com/PyMOTW/multiprocessing/communication.html