Question

The Java ExecutorService framework allows you to delegate a number of tasks to be performed using a managed thread pool so that N tasks can be performed X tasks at a time until complete.

My question is … what if N is a number that is either infinite or so large as to be impractical to allocate/assign/define initially.

How can you leverage the concept of thread pooling in Java (ExecutorService) to handle more tasks than you could reasonably submit without exhausting resources.

For purposes of this answer, assume each task is self-contained and does not depend on any other task and that tasks can be completed in arbitrary order.

My initial attempt at attacking this problem involved feeding the ExecutorService Y threads at a time but I quickly realized that there’s no apparent way to tell when a particular task is complete and therefore submit a new task to be executed.

I know I could write my own “ExecutorService” but I am trying to leverage the bounty of what the Java framework already provides. I’m generally in the “don’t re-invent the wheel” category because greater minds than mine have already made investments for me.

Thanks in advance to anybody that can provide any insight in how to attack this type of problem.

Answer 1

You could use a CompletionService to do it. You can have one thread that seeds the service with a bunch of tasks, then as tasks complete you can add new ones.

A simple example:

final CompletionService service = new ExecutorCompletionService(Executors.newFixedThreadPool(5));
Runnable taskGenerator = new Runnable() {
    public void run() {
        // Seed the service
        for (int i = 0; i < 100; ++i) {
            service.submit(createNewTask());
        }
        // As tasks complete create new ones
        while (true) {
            Future<Something> result = service.take();
            processResult(result.get());
            service.submit(createNewTask());
        }
    }
};
new Thread(taskGenerator).start();

This uses a ThreadPoolExecutor with 5 threads to process tasks and a hand-rolled producer/consumer thread for generating tasks and processing results.

Obviously you’ll want something a little smarter than while (true), you’ll need to have sensible implementations of processResult and createNewTask, and this assumes that task execution is much slower than generating them or processing the results.

Hopefully this will get you on the right track.