Question

Not sure what “parallel programing” means… but I have two thoughts of that

1. Process A produces a certain number of child processes. Once process A finishes creating child processes, all the child processes run at the same time.

2. Process A creates child process B, process B create another its child process C, process C create child process D, and so on… Once finished, all the processes with assigned tasks run at the same time.

Which one is the correct thought of parallel programming? Thanks for the help!

EDIT: I assume that running different programs at the same time requires multi-process?

Answer 1

The term “parallel programming” is much more broad than merely “a process spawning child processes or threads”. It can mean:

1. A single process spawning a team of threads to do some work.
2. A single process spawning a team of child processes to do some work.
3. Multiple processes spawned independently to do cooperative computing on the same machine.
4. Multiple processes spawned independently to do cooperative computing on different machines in a network.
5. A single process communicating with a GPU that performs all the parallel computing.
6. Any combination or nesting of the above.

Basically parallel programming is the act of writing applications or groups of applications that solve a problem in parallel. Threads, processes, etc. are just means to achieve this.

Consider this scenario: A large parallel text processing task taking place on a cluster. A master node issues commands to all its slave nodes to spawn processes for computation (global parallelism). Each process in turn spawns multiple threads/child processes to take advantage of local parallelism (multi-core/multi-processor nodes).