Question

How to show continuous real time updates in browser like facebook ticker, meetup.com home page does? In python, PHP, node.js and what would be the performance impact at the server side ?
Also how could we achieve the same update thing if the page is cached by an CDN like akamai?

Answer 1

You have two options (that others have detailed above). In case you are not familiar with some of the conceptual ideas behind each option, I figured I’d give a line or two about them. Note that I’m presenting these concepts at a very, very high-level.

Your three options are:

1. Short-Polling
2. Web Socket
3. Comet / Long-Polling

Short Polling

Short Polling overcomes the one-way communication between Client-Server by forcing the client to continuously send requests to the server of the form:

Client: Do you have a message for me?
Server: No.
Client: (wait x seconds)
Client: Do you have a message for me?
Server: No.
Client: (wait x seconds)
Client: Do you have a message for me?
Server: Yes. Here it is!
Client: Yay!
Client: (update message)

The constant nagging on behalf of the Client is called Polling. In order to implement this structure, you’ll need to set up your server to “listen” to these polling requests from the client. The server will also have to store those messages somewhere, so that when messages are ready, the server can deliver them. At a very high a simplistic level, your server needs to:

• Accept general web requests
• Accept polling requests
• Run background jobs that fetch messages
• Store these messages somewhere so that when polling requests come in, the server can check for them.

You will also need to tie these polling requests to some kind of session ID for the user, so that the right messages reach the right person. Overall, the paradigm is complicated and, in my opinion, inefficient.

Web Sockets

Web Sockets are new to HTML5. The basic idea behind them is that the Client can maintain a direct connection to the server and they can push information back and forth to each other. Therefore, instead of the usual: Clients sends GET request >> Server responds with content, Web Sockets allow you to maintain a continuous dialogue.

In order to set this up, however, you need:

• Browsers that are WebSocket compliant (not all are).
• A server that can handle web sockets (not sure how to articulate this, but not all servers are set up for this kind of arrangement).

The setup is somewhat complicated, albeit simpler than long polling:

• Client maintains connection to Web-Socket-enabled connection to Server
• Server pushes results to Client through web socket
• Client updates page according to results

You’ll see this pattern referred to as Push Notifications (certainly, if you own an iPhone you’ve experienced this) as the Server has been empowered to push “stuff” to the client (how impolite!). As there are lots of client and server nuances, I would recommend testing out something like Pusher, which is basically a web service to handle all the hard parts of Web Sockets. It’ll be an easy way for you to test and play with the pattern before embarking on setting it up yourself. It has both client and server-side libraries.

Hope that information gives you a baseline to solve your problem. The other answers have more direct information on how to solve each scenario.

Comet / Long-Polling

An alternative, seemingly cross-browser approach to Web Sockets is Long-Polling (see Comet). In this case, your client establishes a connection to the server and leaves it hanging, waiting for data to be pushed back. The setup for this is somewhat complicated, but it does represent a middle ground between Short Polling and Web Sockets.