Question

It could be a stupid query, but i came across of this query multiple time. what if you have just one web service, doing a normal calculation (say A+B).

Now, this service is being used by billions of user. How can you make sure that your service is scalable? how can you say that it can handle billions of hits.

Initially it seems a stupid question to me even. But practically, if you have just one web service which is doing normal calculation even, it cannot handle billions of hits.

So, how can we achieve scalability where billions of growing users wants to use this service.

Note: It is requested to not to reply based on hardware part or load balancer. This query is just related to the software (other then loadbalancer)

Answer 1

At some point you are going to be constrained by limitations such as CPU, RAM, or OS/NIC connections. This question is remarkably vague, but things to look at in the general case:

• make appropriate use of caching (there isn’t anything to cache for A+B, but in most real cases there is)
• ensure the overheads are minimal – for example, there’s a difference in overheads between WCF with lots of protocol options enabled (for example, sessions + transactions + message encryption) and a raw socket server; ServiceStack is a nice compromise in the middle, offering lean throughput, without as much overhead as WCF
• if some parts of the service involve an external time-bound resource, you might need to consider an async-based implementation
• measure everything and optimize; we use mini-profiler to help show us where to spend time

but perhaps more important than anything:

• design the system such that you can scale out by adding another app-server

Sorry, but even if you don’t want to do it – it is the best tool you have here. There are lots of issues trying to do this on a single node (not least, uptime) – for example, it is going to want to run garbage collection at some point; if all your traffic is going to a single node, that could cause a backlog ripple big enough to hurt