Question

The celery documentation tells me that if multiple tasks are chained together, the result of the first task will be the first argument of the next. My problem is, I can’t get it to work when I have a task that returns multiple results.

Example:

@task()
def get_comments(url):
    #get the comments and the submission and return them as 2 objects
    return comments, submission

@task
def render_template(threadComments, submission):
    #render the objects into a html file
    #does not return anything

Now, if I call them in a chain like (get_comments(url) | render_template()).apply_asnc() python will throw an TypeError: render_template() takes exactly 2 arguments (0 given).

I can see that the results are not unwrapped and applied to the arguments. If I only call get_comments, I can do:

result = get_comments(url)
arg1, arg2 = result

and get both results.

Answer 1

There are two mistakes here.

First, you don’t have to call get_comments() and render_template(). Instead, you should use the .s() task method. Like:

( get_comments.s(url) | render_template.s()).apply_async()

In your case, you launch the function first, and then tries to join functions results to a chain.

Second, actually, you don’t return “two results” from your first task. Instead, you return a tuple, containing both results, and this tuple is passed to the second task as the single object.

Therefore, you should rewrite your second task as

@task
def render_template(comments_and_submission):
   comments, submission = comments_and_submission

If you fix these, it should work.