Question

I have a multiprocess program that fetches web pages that all have different response time. The results are stored in the process queue according to the FIFO rule. I would like to indentify the results from the queue by the process number. This is my test rig and what I have achieved so far using two queues. Any other way? I have tried to use a global list to store the results but the two processes doesn’t seem to share the same memory space.

#!/usr/bin/python3.2

import time
from multiprocessing import Process, Queue

def myWait(processNb, wait, resultQueues):
    startedAt = time.strftime("%H:%M:%S", time.localtime())
    time.sleep(wait)
    endedAt = time.strftime("%H:%M:%S", time.localtime())
    resultQueues[processNb].put('Process %s started at %s wait %s ended at %s' % (processNb, startedAt, wait, endedAt))

# queue initialisation
resultQueues = [Queue(), Queue()]

# process creation arg: (process number, sleep time, queue)
proc =  [
    Process(target=myWait, args=(0, 2, resultQueues,)),
    Process(target=myWait, args=(1, 1, resultQueues,))
    ]

# starting processes
for p in proc:
    p.start()

for p in proc:
    p.join()

# print results
print(resultQueues[0].get())
print(resultQueues[1].get())

Answer 1

• mp.Processs can be given a name parameter, which the target
  function myWait can then access with mp.current_process().name.
  So it is not necessary to pass processNb.
• Keep interprocess communication to a minimum. Instead of passing a
  formatted string through the queue, just pass the parts of the string
  that will change in a tuple: (name, wait, startedAt, endedAt).

---

So, you could do it with one queue like this:

import time
import multiprocessing as mp

def myWait(wait, resultQueue):
    startedAt = time.strftime("%H:%M:%S", time.localtime())
    time.sleep(wait)
    endedAt = time.strftime("%H:%M:%S", time.localtime())
    name = mp.current_process().name
    resultQueue.put(
        (name, wait, startedAt, endedAt))


# queue initialisation
resultQueue = mp.Queue()

# process creation arg: (process number, sleep time, queue)
proc =  [
    mp.Process(target=myWait, name = '0', args=(2, resultQueue,)),
    mp.Process(target=myWait, name = '1', args=(1, resultQueue,))
    ]

# starting processes
for p in proc:
    p.start()

for p in proc:
    p.join()

# print results
for p in proc:
    name, wait, startedAt, endedAt = resultQueue.get()
    print('Process %s started at %s wait %s ended at %s' %
          (name, startedAt, wait, endedAt))