Question

I had a need to limit the connection rate (in my servlet) to certain external service and I decided to give ScheduledExecutorService a try. Scheduling itself seems to function just fine, but output gets printed only occasionally – in most cases nothing is outputted. Why does such happen? I’m using Tomcat 7 as a test server.

int waitingtimeinmilliseconds = 5000;

ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();

ScheduledFuture scheduledFuture = scheduledExecutorService.schedule() {

    public void run() {

        Fetcher fetcher = new Fetcher(loginname, password);
        List<Item> items = fetcher.fetchItems();
        // do something with the results

        //ServletOutputStream
        out.print("teststring" + items.size());

    }

}, waitingtimeinmilliseconds, TimeUnit.MILLISECONDS);
scheduledExecutorService.shutdown();

Answer 1

You’ll find very exhaustive description of what is causing your problem in: HttpServletResponse seems to periodically send prematurely (also check: starting a new thread in servlet).

Basically you cannot use external threads to write to servlet output. Once you leave doGet()/doPost(), servlet container assumes you are done and discards the output after flushing it to the client. But since you are writing to the stream asynchronously, sometimes the output gets through, while other times gets discarded.

If you want your rate limiting to be very scalable, consider async servlets (from 3.0). If you just want to throttle some clients, RateLimiter from guava will work for you¹.

^{1 – see RateLimiter – discovering Google Guava on my blog.}