Question

Given this bug (Python Issue 4892) that gives rise to the following error:

>>> import multiprocessing
>>> multiprocessing.allow_connection_pickling()
>>> q = multiprocessing.Queue()
>>> p = multiprocessing.Pipe()
>>> q.put(p)
>>> q.get()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File     "/.../python2.6/multiprocessing/queues.py", line 91, in get
    res = self._recv()
TypeError: Required argument 'handle' (pos 1) not found

Does anyone know of a workaround to pass a Connection object on a Queue?

Thank you.

Answer 1

(What I believe is) A better method, after some playing around (I was having the same problem. Wanted to pass a pipe through a pipe.) before discovering this post:

>>> from multiprocessing import Pipe, reduction
>>> i, o = Pipe()
>>> reduced = reduction.reduce_connection(i)
>>> newi = reduced[0](*reduced[1])
>>> newi.send("hi")
>>> o.recv()
'hi'

I’m not entirely sure why this is built this way (someone would need insight into what the heck the reduction part of multiprocessing is about for that) but it does definitely work, and requires no pickle import. Other than that, it’s pretty close to the above in what it does, but simpler. I also threw this into the python bug report so others know of the workaround.