Question

I was working on a Flask project, getting some data from an API wrapper. The wrapper returned a generator object, so I print the values (for obj in gen_object: print obj) before passing it to Flask’s render_template().

When requesting the page while printing the objects, the page is empty. But removing the for loop the page renders the generator object’s content.

@app.route('/')
def front_page():
    top_stories = r.get_front_page(limit=10)
    # this for loop prevents the template from rendering the stories
    for s in top_stories:
        print s
    return render_template('template.html', stories=top_stories)

Answer 1

Yes, generators are ment to be consumed once. Each time we iterate a generator we ask it to give us another value, and if there’s no more values to give the StopIteration exception is thrown which would stop the iteration. There’s no way for the generator to know that we want to iterate it again without cloning it.

As long as the records can fit comfortably in memory, I would use a list instead:

top_stories = list(top_stories)

That way you can iterate top_stories multiple times.

There’s a function called itertools.tee to copy an iterator that also could help you but it’s sometimes slower than just using a list.
Reference:
http://docs.python.org/library/itertools.html?highlight=itertools.tee#itertools.tee