Question

I have a page that loads a lot of analytics & data on specific pages and so far its loading okay but I’m wondering if I can speed it up. The idea I had to try to speed it up was to render each part of the separately.

For example, look at your profile page on SO: it has Questions, Answers, Reputation, Tags, Accounts, Badges, Votes cast, which in my current model would be gathered in the view.py and then rendered by the template. But what if the “Reputation” or “tags” query is slow because of XYZ reason, it will slow down the entire page load even if its not super important for most users. I thought about solving this using JavaScript to make a request and render the json reply on the page asynchronously but I really like Pyramid and using Python, so if possible I would like to use tools I already know than learn javascript to do this. The benefit of this approach is I can load/display the top of the page first and render the rest of the page as the queries complete or order the rending by speed of queries.

Is this possible, or do you have any suggestions on how to do this?

Answer 1

This surely is possible but the implementation is almost completely has nothing to do with Pyramid or Jinja2. The way Pyramid (or any other web framework) works is that it gets a bit of text from the client (which is abstracted into the request object) and basically returns another blob of text in response, so the operation is more or less atomic – the response is not sent back to the client until the template engine finished generating the full text of the page.

Actually, there is a way to make Pyramid to send data to the client bit after bit, which will make the browser render the page incrementally (like back at the time of dial-up connections) – which is writing directly into the file-like object which represent response’s body:

import time
def some_view(request):
    for i in range(1, 31):
        request.response.body_file.write(".")
        time.sleep(1)

but this approach is a bit limited because you can only append data to what you’ve already sent, there’s no way to, say, display a “please wait, the chart is loading” message and replace it with the real chart, or even to load page header and footer first and then insert something in the middle. Also, this approach would require you to render the template manually or not to use templates at all. I only used this for some long-running batch operations

As you mentioned in the question, a better technique involves loading bits of the page from separate URLs with JavaScript. I believe the blocks loaded asynchronously are often called “partials”.

• you register a few separate views which represent your main page and the bits you want to insert, so, say, /questions/123454 would load the main part of the page, /questions/123454/badges would load the block with badges and /questions/123454/related would load the list of related questions

• you define placeholders in your page where the partials will be inserted:

  < div id=”badges-partial” >Loading…< /div >

• you add a small bit of javascript to your page to load the partials (well, it will be small if you use jquery 🙂 :

  $(function () {
      $("#badges-partial").load("/questions/123454/badges");
      $("#related-partial").load("/questions/123454/related");
  });
  

so, as you can see, the JavaScript solution is cleaner, easier and more flexible.