Question

According to the documentation found in http://ask.github.com/celery/userguide/routing.html#manual-routing i can pass a queue parameter to apply_async in order to route a task to a particular queue. However when using TaskSet i get

TypeError at /some/path/
apply_async() got an unexpected keyword argument 'queue'

Which is undestandable given the following code in the TaskSet class https://github.com/ask/celery/blob/master/celery/task/sets.py#L122

def apply_async(self, connection=None, connect_timeout=None,
        publisher=None, taskset_id=None):
    """Apply taskset."""
    app = self.app

    if app.conf.CELERY_ALWAYS_EAGER:
        return self.apply(taskset_id=taskset_id)

    with app.default_connection(connection, connect_timeout) as conn:
        setid = taskset_id or uuid()
        pub = publisher or self.Publisher(connection=conn)
        try:
            results = self._async_results(setid, pub)
        finally:
            if not publisher:  # created by us.
                pub.close()

        return app.TaskSetResult(setid, results)

I have an undefined number of tasks to which i need to apply special routing on some situations, how should i handle this? not use a TaskSet?

Answer 1

You can use subtasks with options argument

>>> from celery.task.sets import TaskSet
>>> from tasks import add     
>>> 
>>> job = TaskSet(tasks=[add.subtask(args=(i, i),options={'queue':'celery'}) for i in range(10)])
>>> result = job.apply_async()
>>> result.ready()
True
>>> job
[tasks.add(0, 0), tasks.add(1, 1), tasks.add(2, 2), tasks.add(3, 3), tasks.add(4, 4), tasks.add(5, 5), tasks.add(6, 6), tasks.add(7, 7), tasks.add(8, 8), tasks.add(9, 9)]
>>>