Question

I want to learn multiprocessing in python. I started reading http://www.doughellmann.com/PyMOTW/multiprocessing/basics.html and I am not able to understand the section on importing target functions.

In particular what does the following sentence mean..

“Wrapping the main part of the application in a check for __main__ ensures that it is not run recursively in each child as the module is imported.”

Can someone explain this in more detail with an example ?

Answer 1

On Windows, the multiprocessing module imports the __main__ module when spawning a new process. If the code that spawns the new process is not wrapped in a if __name__ == '__main__' block, then importing the main module will again spawn a new process. And so on, ad infinitum.

This issue is also mentioned in the multiprocessing docs in the section entitled “Safe importing of main module”. There, you’ll find the following simple example:

Running this on Windows:

from multiprocessing import Process

def foo():
    print 'hello'

p = Process(target=foo)
p.start()

results in a RuntimeError.

And the fix is to use:

if __name__ == '__main__':    
    p = Process(target=foo)
    p.start()