Question

for example, the following code only run the task in 1 process:

from jug import TaskGenerator
evaluate = TaskGenerator(evaluate)

for p0 in [1,2,3]:
for p1 in xrange(10):
    for p2 in xrange(10,20):
         for p3 in [True, False]:
             for p4 in xrange(100):
                results.append(evaluate(p0,p1,p2,p3,p4))

where should I indicate the number of processes?
thanks!

Answer 1

Author of jug here.

To run this script, assuming it is a file called script.py, you will run the jug execute script.py command (and not python script.py).

You can simply run multiple of these in parallel. If you are using a Unix-like shell, the following should work just fine in order to use two processors:

jug execute script.py &
jug execute script.py &

The different processes will communicate with each other using the file system to divide up the work (this is the default, you can also use a redis backend if you want to use a cluster of machines that do not share a file system).

You can also use the following to check on the progress while they run:

jug status script.py