Question

Using Windows 7 + python 2.6, I am trying to run a simulation model in parallel. I can launch multiple instances of the executable by double-clicking on them in my file browser. However, asynchronous calls with Popen result in each successive instance interrupting the previous one. For what it’s worth, the executable returns text to the console, but I don’t need to collect results interactively.

Here’s where I am so far:

import multiprocessing, subprocess

def run(c):
    exe = os.path.join("<location>","folder",str(c),"program.exe")
    run = os.path.join("<location>","folder",str(c),"run.dat")
    subprocess.Popen([exe,run],creationflags = subprocess.CREATE_NEW_CONSOLE)

def main():
    pool = multiprocessing.Pool(3)
    for c in range(10):
        pool.apply_async(run,(str(c),))
 pool.close()
 pool.join()

if __name__ == '__main__':
    main()

After scouring SO for a solution, I’ve learned that using multiprocessing may be redundant, but I need some way to limit the number of cores working.

Answer 1

Enabled by @J.F. Sebastian’s comment regarding the cwd argument.

import multiprocessing, subprocess

def run(c):
    exe = os.path.join("<location>","folder",str(c),"program.exe")
    run = os.path.join("<location>","folder",str(c),"run.dat")
    subprocess.check_call([exe,run],cwd=os.path.join("<location>","folder"),creationflags = subprocess.CREATE_NEW_CONSOLE)

def main():
    pool = multiprocessing.Pool(3)
    for c in range(10):
        pool.apply_async(run,(str(c),))
    pool.close()
    pool.join()

if __name__ == '__main__':
    main()