Question

I was running a benchmark on CouchDB when I noticed that even with large bulk inserts, running a few of them in parallel is almost twice as fast. I also know that web browsers use a number of parallel connections to speed up page loading.

What is the reason multiple connections are faster than one? They go over the same wire, or even to localhost.

How do I determine the ideal number of parallel requests? Is there a rule of thumb, like “threadpool size = # cores + 1”?

Answer 1

The gating factor is not the wire itself which, after all, runs pretty quick (ignoring router delays) but the software overhead at each end. Each physical transfer has to be set up, the data sent and stored, and then completely handled before anything can go the other way. So each connection is effectively synchronous, no matter what it claims to be at the socket level: one socket operating asynchronously is still moving data back and forth in a synchronous way because the software demands synchronicity.

A second connection can take advantage of the latency — the dead time on the wire — that arises from the software doing its thing for first connection. So, even though each connection is synchronous, multiple connections let things happen much faster. Things seem (but of course only seem) to happen in parallel.

You might want to take a look at RFC 2616, the HTTP spec. It will tell you about the interchanges that happen to get an HTTP connection going.

I can’t say anything about optimal number of parallel requests, which is a matter between the browser and the server.