Question

When ASP.NET receives a request, how does it determine whether to serve it or to queue it? I ask because I’m monitoring performance counters on a server and the CPU is not maxed out and there are a boatload of available worker threads, but I’m still seeing up to 200 requests get queued up.

Answer 1

I’ve been doing research and I believe I’ve come upon an acceptable answer. My primary source is this article: http://blogs.msdn.com/b/tmarq/archive/2007/07/21/asp-net-thread-usage-on-iis-7-0-and-6-0.aspx

As I understand there are two main ways the request processing gets throttled. The first is the MaxConcurrentRequestsPerCPU property. Before .NET 4 this was set to 12 by default. In .NET 4 it was changed to 5000. For async requests they wanted to allow a lot, and for synchronous requests they believe that the ASP.NET ThreadPool will throttle synchronous requests well enough. The second of course is the ThreadPool itself. After ASP.NET posts the request there it can decide when it will go.

If you’re doing async processing, your limiting factors are likely to be CPU, network and disk and not any ASP.NET request throttling. It could possibly hit the MaxConcurrentRequestsPerCPU limit, but that limit is really high.

If you’re doing synchronous processing and blocking on web calls for long periods of time, it’s much more likely you run into these limits. MaxConcurrentRequestsPerCPU is something to watch out for before .NET 4 but there’s still the ThreadPool.

Performance testing
I put together a simple test to see how this throttling worked. I have a simple page with a 500ms Thread.Sleep() call. One host machine makes 800 simultaneous async requests and a worker machine running ASP.NET processes them all. The results were interesting:

.NET 3.5, no modifications: 46 seconds. Saw 9 worker threads with process explorer.
.NET 3.5, with MaxConcurrentRequestsPerCPU set to 5000: 46 seconds. 9 worker threads.
.NET 4: 42 seconds, or 13 seconds when running hot. Saw around 35 worker threads gradually get created.
.NET 4, async: 3 seconds

A few observations:

• MaxConcurrentRequestsPerCPU wasn’t getting hit. It looks like it was the limitation of the ThreadPool itself.
• .NET 3.5 seems rather reluctant to make new threads to process synchronous requests. .NET 4 does a much better job of ramping up to handle load.
• Async is still the best by a country mile.