Question

I’ve heard there are problems when calling os.waitpid from within a thread. I have not experienced such problems yet (especially using os.WNOHANG option). However I have not paid much attention to the performance implications of such use.

Are there any performance penalties or any other issues one should be aware of?
Does this have to do with os.waitpid (potentially) using signals?

I don’t see how signals could be related, though, since otherwise (I suppose) I wouldn’t be able to get os.waitpid to return when calling it from a non-main thread.

Answer 1

By default, a child process dies, the parent is sent a SIGCHLD signal. Concern for calling os.waitpid() probably comes from this.

If you look in the Python “signal” module documentation the warning is pretty clear:

Some care must be taken if both signals and threads are used in the same program. The fundamental thing to remember in using signals and threads simultaneously is: always perform signal() operations in the main thread of execution. Any thread can perform an alarm(), getsignal(), pause(), setitimer() or getitimer(); only the main thread can set a new signal handler, and the main thread will be the only one to receive signals (this is enforced by the Python signal module, even if the underlying thread implementation supports sending signals to individual threads). This means that signals can’t be used as a means of inter-thread communication. Use locks instead.

http://docs.python.org/library/signal.html

BUT… if you leave the SIGCHLD signal alone, then you should be happily able to call os.waitpid() (or any other os.wait() variant) from a thread.

The main drawback then is that you’ll need to use os.waitpid() with WNOHANG and poll periodically, if you want any way to cancel the operation. If you don’t ever need to cancel the os.waitpid(), then you can just invoke it in blocking mode.