Question

I’m trying to wrap my brain around parallel/concurrent programming (in Java) and am getting hung up on some fundamentals that don’t seem to be covered in any of the tutorials I’ve been reading.

When we talk about “multi-threading”, or “parallel/concurrent programming”, does that mean we’re taking a big problem and spreading it over many threads, or are we first explicitly decomposing it into smaller sub-problems, and passing each sub-problem to its own thread?

For example, let’s say we have EndWorldHungerTask implements Runnable, and task accomplishes some enormous problem. In order to complete its objective, it has to do some really heavy lifting, say, a hundred million times:

public class EndWorldHungerTask implements Runnable {

    public void run() {
        for(int i = 0; i < 100000000; i++)
            someReallyExpensiveOperation();
    }
}

In order to make this “concurrent” or “multi-threaded”, would we pass this EndWorldHungerTask to, say, 100 worker threads (where each of the 100 workers are told by the JVM when to be active and work on the next iteration/someReallyExpensiveOperation() call), or would we refactor it manually/explicitly so that each of the 100 workers is iterating over different parts of the loop/work-to-be-done? In both cases, each of the 100 workers is only iterating a million times.

But, under the first paradigm, Java is telling each Thread when to execute. Under the second, the developer needs to manually (in the code) partition the problem ahead of time, and assign each sub-problem to a new Thread.

I guess I’m asking how its “normally done” in Java land. And, not just for this problem, but in general.

Answer 1

I guess I’m asking how its “normally done” in Java land. And, not just for this problem, but in general.

This is highly dependent on the task at hand.

The standard paradigm in Java is that you have to split the work into chunks yourself. Distributing those chunks across multiple threads/cores is a separate problem, and there exist a variety of patterns for that (queues, thread pools, etc).

It is interesting to note that there exist frameworks that can automatically make use of multiple cores to execute things like for loops in parallel (for example, OpenMP). However, I am not aware of any such frameworks for Java.

Finally, it could be the case that the low-level library that does the bulk of the work can make use of multiple cores. In such a case the higher-level code may be able to remain single-threaded and still benefit from multicore hardware. One example might be numerical code using MKL under the covers.