Question

I tried testing things on a VPS, and came close to 10K requests per second, and that was a simple 'hello world' servlet, let alone making a call to membase.

My VPS was a 2 x Intel Xeon X5570, quad-core “Nehalem” architecture.

Note: I’m not a java expert, nor a tomcat expert, this was on default settings.

Does anyone else deal with such high traffic that could shed some light?

I used apache bench, and I ran it maybe 4-5 times, doing about 100K requests to the server.

original: how to handle 2000+ requests/sec on tomcat?

Answer 1

Turn on NIO (Non-Blocking IO). This is not by default turned on. Without NIO, every HTTP connection is handled by a single thread and the limit is dependent on the amount of threads available. With NIO, multiple HTTP connections can be handled by a single thread and the limit is dependent on amount of heap memory available. With about 2GB you can go up to 20K connections.

Turning on NIO is a matter of changing the protocol attribute of the <Connector> element in Tomcat’s /conf/server.xml to "org.apache.coyote.http11.Http11NioProtocol".

<Connector
    protocol="org.apache.coyote.http11.Http11NioProtocol"
    port="80"
    redirectPort="8443"
    connectionTimeout="20000"
    compression="on" />