Question

So I was listening to the latest Stackoverflow podcast (episode 19), and Jeff and Joel talked a bit about scaling server hardware as a website grows. From what Joel was saying, the first few steps are pretty standard:

1. One server running both the webserver and the database (the current Stackoverflow setup)
2. One webserver and one database server
3. Two load-balanced webservers and one database server

They didn’t talk much about what comes next though. Do you add more webservers? Another database server? Replicate this three-machine cluster in a different datacenter for redundancy? Where does a web startup go from here in the hardware department?

Answer 1

A reasonable setup supporting an ‘average’ web application might evolve as follows:

1. Single combined application/database server
2. Separate database on a different machine
3. Second application server with DNS round-robin (poor man’s load balancing) or, e.g. Perlbal
4. Second, replicated database server (for read loads, requires some application logic changes so eligible database reads go to a slave)

At this point, evaluating the current state of affairs would help to determine a better scaling path. For example, if read load is high and content doesn’t change too often, it might be better to emphasise caching and introduce dedicated front-end caches, e.g. Squid to avoid un-needed database reads, although you will need to consider how to maintain cache coherency, typically in the application.

On the other hand, if content changes reasonably often, then you will probably prefer a more spread-out solution; introduce a few more application servers and database slaves to help mitigate the effects, and use object caching, such as memcached to avoid hitting the database for the less volatile content.

For most sites, this is probably enough, although if you do become a global phenomenon, then you’ll probably want to start considering having hardware in regional data centres, and using tricks such as geographic load balancing to direct visitors to the closest ‘cluster’. By that point, you’ll probably be in a position to hire engineers who can really fine-tune things.

Probably the most valuable scaling advice I can think of would be to avoid worrying about it all far too soon; concentrate on developing a service people are going to want to use, and making the application reasonably robust. Some easy early optimisations are to make sure your database design is fairly solid, and that indexes are set up so you’re not doing anything painfully crazy; also, make sure the application emits cache-control headers that direct browsers on how to cache the data. Doing this sort of work early on in the design can yield benefits later, especially when you don’t have to rework the entire thing to deal with cache coherency issues.

The second most valuable piece of advice I want to put across is that you shouldn’t assume what works for some other web site will work for you; check your logs, run some analysis on your traffic and profile your application – see where your bottlenecks are and resolve them.