Question

I’m working on a Java daemon that requires concurrency: an infinite loop that listens on a job queue (redis) and distributes each job to a worker. The worker doesn’t necessarily have to return a value.

I found Executors pretty useful and I am using a ThreadPoolExecutor to maintain a number of worker threads.

Though, those workers run 3rd party code which needs to run as isolated as possible, avoiding sharing static properties.

My question: is there any Java library/framework that provides features similar to Executors, like:

• worker pools
• automatically adjusted pool size

..but spawning processes instead of threads?

Answer 1

I understand that you have 3rd party libraries that you want to run isolated somehow — so they do not have access to static variables for example.

I would run your tasks in another ClassLoader. There are web frameworks that use this mechanism to isolate web requests. Passing data between the classloaders is a bit tricky. I’m not sure how Jetty does it. Maybe using some sort of socket? Here’s an interesting article about using System.out to share objects. Certainly a hack.

---

My question is: is there any Java library/framework that provides features similar to Executors… but spawning processes instead of threads?

Modern operating systems will utilize modern multi-processor architectures to get the most out of Java threads. Spawning another thread inside of your current process will run on another processor if available and will be much more efficient that running that task in a completely separate process, especially something as heavy as another JVM process.