Question

I have a server written in Python that basically accepts incoming connection from clients and feeds the data received from them into a subprocess (one instance per connection) which then processes the data and returns the result to the server so that it can send it back to the client.

The problem is that the data is streaming in and I need to be able to execute multiple read/write operations with no EOF in sight. So far, I’ve been unable to come up with a solution that would enable me to do just that without getting my server program blocked on reading. Any suggestions? Thanks.

Answer 1

You could use select.select (available on Unix and Windows).

while True:
    rlist, wlist, xlist = select.select([client, proc.stdout], [], [])

The call to select.select will block until either the client socket or proc.stdout is ready to be read.

rlist holds a subset of [client.stdin, proc.stdout] which is ready to be read.

An example of its usage (though for a different problem) can be found here.