Question

Is there any standard/backend-independent method for querying pending tasks based on certain fields?

For example, I have a task which needs to run once after the “last user interaction”, and I’d like to implement it something like:

def user_changed_content():
    task = find_task(name="handle_content_change")
    if task is None:
        task = queue_task("handle_content_change")
    task.set_eta(datetime.now() + timedelta(minutes=5))
    task.save()

Or is it simpler to hook directly into the storage backend?

Answer 1

No, this is not possible.

Even if some transports may support accessing the “queue” out of order (e.g. Redis)
it is not a good idea.

The task may not be on the queue anymore, and instead reserved by a worker.

See this part in the documentation: http://docs.celeryproject.org/en/latest/userguide/tasks.html#state

Given that, a better approach would be for the task to check if it should reschedule itself
when it starts:

@task
def reschedules():
    new_eta = redis.get(".".join([reschedules.request.task_id, "new_eta"])
    if new_eta:
         return reschedules.retry(eta=new_eta)