Question

I have a web application that our product owner has stated must respond within 2.5 seconds and be usable within 3 seconds.

Right now it’s mostly POJOs on top of a thinly sliced Spring MVC layer which is being used for dependency injection and directing requests to controllers.

The webapp is a very thin slice of meat between a stack of APIs which are being built to provide cohesion to a disparate set of very old, irregular APIs. Some of which can take longer than 3 seconds (though not too often)

What I want is a way to tell spring or tomcat that if it isn’t loading the JSP by within 2500ms I want to load the jsp that tells our user that “Dude, we’re crawlin’ through molasses here! Give us a second and try again. If you get this page multiple times; then you’re really boned!”

Right now it appears I could do this several ways (ordered by my desire)

• In spring I may be able to set up a timeout and load a different jsp
• In the load balancer I may be able to set up a timeout with a redirect
• In tomcat I may be able to set up a timeout and load a different jsp
• In apache I may be able to set up a tomcat iwth a redirect
• in the app I split the architecture to run inside a thread that is polled every 100ms to see if it had stopped running, and after 2500ms it throws an exception

Answer 1

If I remember correctly, all requests to Spring MVC go via a single filter. You can probably override it and program your timeout settings (e.g. have a configurable XML with timeout-enforced URLs or paths). Guava’s TimeLimiter would be a good tool to implement the timeout enforcement itself.

<general-rant>

I don’t know the industry you’re in, but generally in the web world it is a bit too self-demanding to panic if a response doesn’t come back in 2-3s, especially if long taking responses are just a minority (as you say). I have experience with large heaps (Xmx=10g) where just a GC can take several minutes to accomplish. OK, maybe this is an overrated example, but one-two second taking GCs are pretty reasonable.

Even Tomcat manual says that very small timeouts will likely be counterproductive. Your product owner must be choleric 🙂

</general-rant>

Ah, there’s more. Don’t forget you can’t just redirect if you have already posted something back to your servlet response. So you probably need to delay your response until the last bit of computation is done and flush everything at the end. This means you have to wrap your HttpServletResponse with a decorator which has some sort of a latch which delays response output until the full response is generated, or if timeout occurs – redirects to “ouch” page.