Question

I’m faced with a very slow data import console app and I’m trying to speed it up by multithreading it.

Is there an example pattern for starting and managing X number of worker threads in a console app? Ideally I was hoping that there would be something like:

ThreadManager tm = new ThreadManager(maxthreads=10);
while (moreWork = true) {
    tm.addThread(new Thread(new MyClass));
}

The ThreadManager would add threads until the max was reached and then wait patiently until a slot became available for a new thread.

Anything like that out there? I can’t be the first one that’s faced this problem.

Answer 1

@jeshurun’s answer is correct but for posterity, I thought I’d add some more information. If you utilize the great ExecutorService code then your code would turn into:

ExecutorService threadPool = Executors.newFixedThreadPool(10);
while (moreWork) {
    threadPool.submit(new MyClass);
}
// stop pool after you've submitted the last job, submitted jobs will still run
threadPool.shutdown();
// you can wait for the last job to finish if you'd like
threadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);

Your MyClass should either implement Runnable or Callable. If it is a Runnable then the thread pool will execute the MyClass.run() method when it has an available thread. Same with Callable except that the Future<?> returned by the submit() method can be used to get the value returned by your MyClass.call() method. You can also get any exceptions that were thrown by call(). Here’s a decent tutorial.