Question

We have a webapp serving the requests for one of our services. We have it running in a Tomcat 7 server. As explained below, not much more details are needed as we have similar results with the most basic Servlet we could create.

We run a load test with jmeter with different number of concurrent threads doing requests to our service for a minute. With the results we generate some charts with average response time vs concurrent threads, and number of requests per second vs concurrent threads.

At around 10 concurrent threads in our test we find a limit in the throughput managed by our application. Starting from that number of threads the average response time increases and the number of requests per second is stable.

We have tested several changes to our application, even removing all the application logic and returning an static response. And also run the same tests against a basic Servlet that does nothing, and we got similar results: much better average response time and throughput values, but limit reached at around 10 concurrent threads in the test.

We have also run similar tests with Apache Benchmark to discard problems with the testing tool.

The tests were run in a 2 server pool. When run in a local computer they gave a similar result but the throughput limit was reached at around 5 concurrent threads.

Here’s the throughput chart. Note that the values of each of the lines are not important (for example: some have db access removed, or even most application logic so they take fewer time).

Throughput chart

What we want to understand is where the throughput limit comes from and what we can do to improve it.

Thanks,
Jorge

PS: I’d add the image instead of just a link but it looks like I don’t have enough reputation 🙁

Answer 1

After tuning the jvm to an optimal setup the limitation will get down to the hardware you are running on. Threads need cpu to run and adding more threads only helps if other threads are blocked from execution by something like IO, even then the containers threads add overhead. When you reach that limit you will need to add more hardware.