I was recently asked this question in an interview. It was open-ended question on web-server design. Herez the gist of the question.
“Queues quite often become a bottle neck for applications like web servers. What changes will you make to your thread handling classes, thread pool and queues to improve any bottle neck?”
I pointed out measures like
1. Thread pool management
2. Using buffers and packets in queues.
But the interviewer was not quite satisfied. What all design measures should I have mentioned? Obviously its an open-ended question, so answer doesn’t have to depend on the underlying technology like Spring, J2EE etc.
In case there is good tutorial on server design, please share it.
I can’t resist posting this:
http://www.engineerguy.com/videos/video-lines.htm
The main problem with a queue is that it blocks waiters deeper down the queue. So the design must either be so fast that queues never fill up, or it must take queue fill-up into consideration. Also see the recent things on bufferbloat Jim Gettys have posted:
http://gettys.wordpress.com/category/bufferbloat/
Where buffering of IP packets in modern routers lead to TCP going seriously wrong (and it has effects very indirectly on queuing theory inside web servers as well).