Question

How do you figure out whether it’s worth parallelizing a particular code block based on its code size? Is the following calculation correct?

Assume:

• Thread pool consisting of one thread per CPU.
• CPU-bound code block with execution time of X milliseconds.
• Y = min(number of CPUs, number of concurrent requests)

Therefore:

• Cost: code complexity, potential bugs
• Benefit: (X * Y) milliseconds

My conclusion is that it isn’t worth parallelizing for small values of X or Y, where “small” depends on how responsive your requests must be.

Answer 1

One thing that will help you figure that out is Amdahl’s Law

The speedup of a program using multiple processors in parallel computing is limited by the time needed for the sequential fraction of the program. For example, if a program needs 20 hours using a single processor core, and a particular portion of 1 hour cannot be parallelized, while the remaining promising portion of 19 hours (95%) can be parallelized, then regardless of how many processors we devote to a parallelized execution of this program, the minimal execution time cannot be less than that critical 1 hour. Hence the speed up is limited up to 20x.

Figure out what you want to achieve in speed up, and how much parallelism you can actually achieve, then see if its worth it.