Question

When I use multiprocessing.Queue.get I sometimes get an exception due to EINTR.

I know definitely that sometimes this happens for no good reason (I open another pane in a tmux buffr), and in such a case I would want to continue working and retry the operation.

I can imagine that in some other cases The error would be due to a good reason and I should stop running or fix some error.

How can I distinguish the two?

Thanks in advance

Answer 1

The EINTR error can be returned from many system calls when the application receives a signal while waiting for other input. Typically these signals can be quite benign and already handled by Python, but the underlying system call still ends up being interrupted. When doing C/C++ coding this is one reason why you can’t entirely rely on functions like sleep(). The Python libraries sometimes handle this error code internally, but obviously in this case they’re not.

You might be interested to read this thread which discusses this problem.

The general approach to EINTR is to simply handle the error and retry the operation again – this should be a safe thing to do with the get() method on the queue. Something like this could be used, passing the queue as a parameter and replacing the use of the get() method on the queue:

import errno

def my_queue_get(queue, block=True, timeout=None):
    while True:
        try:
            return queue.get(block, timeout)
        except IOError, e:
            if e.errno != errno.EINTR:
                raise

# Now replace instances of queue.get() with my_queue_get(queue), with other
# parameters passed as usual.

Typically you shouldn’t need to worry about EINTR in a Python program unless you know you’re waiting for a particular signal (for example SIGHUP) and you’ve installed a signal handler which sets a flag and relies on the main body of the code to pick up the flag. In this case, you might need to break out of your loop and check the signal flag if you receive EINTR.

However, if you’re not using any signal handling then you should be able to just ignore EINTR and repeat your operation – if Python itself needs to do something with the signal it should have already dealt with it in the signal handler.