I might be misunderstanding how this works (which is why I’m asking), but I think when a celery worker consumes a task from RabbitMQ it puts a lock on it — so to speak — and then must acknowledge it completed that task onces it’s done. So say I have 4 workers which all have the prefetch setting at 1 and queue of 6 tasks which take a long time. Once I start those workers and I run:
rabbitmqctl -q list_queues name messages messages_ready messages_unacknowledged
I’d expect to see something like:
celery 6 2 4
indicating that 4 tasks are running (but not yet acknowledged) and 2 are ready to be consumed.
I think my understanding is wrong because what I actually see is:
celery 2 0 2
So it’s as if the acknowledging happens when a message is received by a worker, but before that worker finishes processing that task.
So to sum up, my question is, when does a celery worker acknowledge it has a task? It seems like it’s once it receives that task and starts working on it, not when it completes working on it. Can someone confirm?
This is mentioned in the FAQ, but I can’t blame you for not finding it:
http://docs.celeryproject.org/en/latest/faq.html#should-i-use-retry-or-acks-late
The default behavior of early ack is there because we don’t want to enforce users
to write idempotent tasks.