Home/ Questions/Q 8083237
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-05T17:23:52+00:00

I have Django set up under Windows (Win2008 Server R2, IIS 7.5, MS SQL).

  • 0

I have Django set up under Windows (Win2008 Server R2, IIS 7.5, MS SQL). I’m trying to implement celery 2.4.1 (with RabbitMQ) for task processing and running into a very frustrating problem.

I start celeryd with the recommended:

manage.py celeryd --settings=settings

and it runs OK. Then in my django project, I trigger a task to start (for testing I’m just using the simple “add” example:

@task
def add(x, y):
    return x + y

and that’s called in my views.py as:

tasks.add.delay(1,2)

When celeryd picks this up, a couple of things happen, sometimes:
1) I get a Windows dialog, “python.exe has stopped working”, and an option to close it.
2) After closing that, in the cmd where celeryd is running, I get:

[date-time: ERROR/MainProcess] Task [taskUUID] raised exception: WorkerLostError('Worker exited prematurely.',)
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\celery\concurrency\processes\pool.py", line 610, in _join_exited_workers
    raise WorkerLostError("Worker exited prematurely.")
WorkerLostError: Worker exited prematurely.

I’ve tried very tediously tinkering with just about every celery setting, one at a time, that could possibly affect workers (CELERYD_CONCURRENCY, CELERYD_PREFETCH_MULTIPLIER, CELERYD_MAX_TASKS_PER_CHILD, etc.) with no effect whatsoever on this.

What’s really strange is that sometimes, after that has happened (and I just leave celeryd running…it doesn’t kill that process…?), I can trigger the task again, with no python.exe crash, and the task will complete successfully.

My celery-relevant settings from settings.py:

BROKER_HOST = "localhost"
BROKER_PORT = 5672
BROKER_USER = "guest"
BROKER_PASSWORD = "guest"
BROKER_VHOST = "/"

CELERY_IMPORTS = ("myproject.tasks",)
CELERY_RESULT_BACKEND = "database"

CELERY_SEND_EVENTS=True #same as '-E' option from cmd

I strongly suspect a celery bug (possibly specific to Windows implementation?) but I don’t quite have the chops yet to identify it or propose a fix.

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Finally, after many months of tinkering on and off with this, I believe I have figured it out.

    The problem appears to be coming from the use of django-mssql as the Django database engine. django-mssql relies on pywin32, and one of the calls there uses ole32.dll and causes a crash there, which consequently brings down Python, hence the WorkerLostError.

    I’ve switched to django-pyodbc which does not use the problematic pywin32/ole32.dll call, and have not experienced an unexplained WorkerLostError such as these since switching.

    • 0