Question

It seems that asynchronous signals in multithreaded programs are not correctly handled by Python. But, I thought I would check here to see if anyone can spot a place where I am violating some principle, or misunderstanding some concept.

There are similar threads I’ve found here on SO, but none that seem to be quite the same.

The scenario is: I have two threads, reader thread and writer thread (main thread). The writer thread writes to a pipe that the reader thread polls. The two threads are coordinated using a threading.Event() primitive (which I assume is implemented using pthread_cond_wait). The main thread waits on the Event while the reader thread eventually sets it.

But, if I want to interrupt my program while the main thread is waiting on the Event, the KeyboardInterrupt is not handled asynchronously.

Here is a small program to illustrate my point:

#!/usr/bin/python
import os
import sys
import select
import time
import threading

pfd_r = -1
pfd_w = -1
reader_ready = threading.Event()

class Reader(threading.Thread):
    """Read data from pipe and echo to stdout."""
    def run(self):
        global pfd_r
        while True:
            if select.select([pfd_r], [], [], 1)[0] == [pfd_r]:
                output = os.read(pfd_r, 1000)
                sys.stdout.write("R> '%s'\n" % output)
                sys.stdout.flush()
                # Suppose there is some long-running processing happening:
                time.sleep(10)
                reader_ready.set()


# Set up pipe.
(pfd_r, pfd_w) = os.pipe()
rt = Reader()
rt.daemon = True
rt.start()

while True:
    reader_ready.clear()
    user_input = raw_input("> ").strip()
    written = os.write(pfd_w, user_input)
    assert written == len(user_input)
    # Wait for reply -- Try to ^C here and it won't work immediately.
    reader_ready.wait()

Start the program with ‘./bug.py’ and enter some input at the prompt. Once you see the reader reply with the prefix ‘R>’, try to interrupt using ^C.

What I see (Ubuntu Linux 10.10, Python 2.6.6) is that the ^C is not handled until after the blocking reader_ready.wait() returns. What I expected to see is that the ^C is raised asynchronously, resulting in the program terminating (because I do not catch KeyboardInterrupt).

This may seem like a contrived example, but I’m running into this in a real-world program where the time.sleep(10) is replaced by actual computation.

Am I doing something obviously wrong, like misunderstanding what the expected result would be?

Edit: I’ve also just tested with Python 3.1.1 and the same problem exists.

Answer 1

The wait() method of a threading._Event object actually relies on a thread.lock‘s acquire() method. However, the thread documentation states that a lock’s acquire() method cannot be interrupted, and that any KeyboardInterrupt exception will be handled after the lock is released.

So basically, this is working as intended. Threading objects that implement this behavior rely on a lock at some point (including queues), so you might want to choose another path.