Question

I’m trying to do a simple network job (ping) from django process. Django is deployed via apache’s mod_wsgi. But code works only during first run, returning the following error on subsequent runs.

Traceback:
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/var/www/propingui/ping/views.py" in ping
  32.             p = Pool(len(fls))
File "/usr/lib/python2.7/multiprocessing/pool.py" in __init__
  674.         Pool.__init__(self, processes, initializer, initargs)
File "/usr/lib/python2.7/multiprocessing/pool.py" in __init__
  134.         self._repopulate_pool()
File "/usr/lib/python2.7/multiprocessing/pool.py" in _repopulate_pool
  197.             w.start()
File "/usr/lib/python2.7/multiprocessing/dummy/__init__.py" in start
  73.         self._parent._children[self] = None

Exception Type: AttributeError at /
Exception Value: '_DummyThread' object has no attribute '_children'

The code is below:

from multiprocessing.pool import ThreadPool as Pool

...

def _ping((host, firing_location)):
    pinger = Pyro4.Proxy("PYRONAME:" + firing_location)
    return pinger.ping(host)

def ping(request):
    if request.method == 'POST':
        form = PingForm(request.POST)
        if form.is_valid():
            host = form.cleaned_data['host']
            fls = ['g1','a1']
            p = Pool(len(fls))
            noanswer = False
            try: 
                jobs = p.map(_ping, zip([host]*len(fls), fls) )
            except:
                jobs = []
            ...
            return ...

I’ve tried to google the error but found nothing and I do not understand the exact source of the problem. The interesting thing is that if I change ThreadPool to multiprocessing Pool everything works just fine. I think this is somehow caused by problems with spawning threads inside django.

Answer 1

Use threading, I had the same problem, eventually gave up and used threading module instead of ThreadPool and now everything works just fine, so I think the problem must be with ThreadPool not Python threads at all.

You can implement your own ThreadPool or reuse what others done (like this or this)