Question

I cannot figure out why celeryd is not pulling new tasks that are added to the queue. It will only retrieve tasks once it is started, then fails to monitor thereafter. I am running the Django development server with django-celery using the Django ORM for the message broker.

My Django settings file has this configuration for celery:

INSTALLED_APPS += ("djcelery", )
INSTALLED_APPS += ("djkombu", )

import djcelery
djcelery.setup_loader()
BROKER_TRANSPORT = "djkombu.transport.DatabaseTransport"

BROKER_HOST = "localhost"
BROKER_PORT = 5672
BROKER_USER = "guest"

BROKER_PASSWORD = "guest"
BROKER_VHOST = "/"

Furthermore if it matters, I am using a remote MySQL server as my database backend.

Edit: Resolved!

After spending several hours on this, I found an answer in the Celery FAQ:

MySQL is throwing deadlock errors, what can I do?

I wasn’t seeing the deadlock errors in the log, but modifying my /etc/my.cnf to include:

[mysqld]
transaction-isolation = READ-COMMITTED

resolved my issue.

Hope this helps someone else!

Answer 1

After spending several hours on this, I found an answer in the Celery FAQ:

MySQL is throwing deadlock errors, what can I do?

I wasn’t seeing the deadlock errors in the log, but modifying my /etc/my.cnf to include:

[mysqld]
transaction-isolation = READ-COMMITTED

resolved my issue.

Hope this helps someone else!