Question

I am writing a sample program to test the usage of multiprocessing pool of workers in python 2.7.2+

This is the code i have written in the python ubuntu interpreter

>>> from multiprocessing import Pool
>>> def name_append(first_name,last_name):
...     return first_name+" "+last_name
...

>>> from functools import partial
>>> partial_name_append=partial(name_append,'kiran')
>>> partial_name_append('acb')
'kiran acb'
>>> abc='kiran'

>>> pool=Pool(processes=4)
>>> pool.map(partial_name_append,abc)
['kiran k', 'kiran i', 'kiran r', 'kiran a', 'kiran n']
>>> pool.close()
>>> pool.join()

>>> pool.map(partial_name_append,abc)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 226, in map
    assert self._state == RUN
AssertionError

After i got pickle errors, over my pool of workers code for large data sets, i am trying to do small examples and try to figure out what is the error.

I dont understand why the same statement ‘pool.map’ doesn’t work, when it has worked above. I think i have executed the ‘pool map’ correctly but i dont understand the reason.

Is this error related to “PicklingError: Can’t pickle : attribute lookup builtin.function failed”

Can someone help me out ?

Thanks

Answer 1

You typed:

>>> pool.close()

from the docs:

close()

Prevents any more tasks from being submitted to the pool. Once all the tasks
have been completed the worker processes will exit.

Of course you can’t use the pool anymore, you closed it.