Question

I’m currently reading Advanced Programming in the Unix Environment by Stevens/Rago.

Within the process group section of the book, the author discusses how process groups are utilized commonly by shells for pipelining.

For example, the argument shown below could have been generated by
shell commands of the form:

proc1 | proc2 & proc3 | proc4 | proc5

Lots of other resources also discuss the relationship between process groups and pipelining. However, the one thing which I cannot find is an explanation of how the pipelining portion of this is implemented.

I know that in posix/unix shells like Boune-again shell (BASH), processes in a pipeline are executed in parallel — that is, in the previous example showing proc3 | proc4 | proc5, all three of these processes are simultaneously executed. The stdin of proc4 is connected to the stdin of proc3. (I also know that MS-DOS used temporary files and did not execute pipes in parallel, but let’s ignore that for the moment).

So, I’ve got proc3, proc4, proc5 all in a process group. Fantastic. How does this actually help with creating the pipelines between them?

As far as I can tell, I need to do the following to enable pipelining in a shell which I build:

1. Create N-1 pipelines, where N is the number of processes in the
   pipelined statement fork() the shell process N times
2. In each forked process, I need to use dup2 to properly set up the
   shared pipelines
3. Then, after all of the forked processes have confirmation that they
   have all finalized setting up their pipes (likely via some IPC via a
   shared memory space), each can then run exec() and actually launch
   their respective processes.

However, all of the texts which I keep reading act as if process groups provide some magical functionality to create these pipelines — or they are simply neglecting to mention the procedure I outlined above.

Any comments or advice is always appreciated.

Answer 1

Process groups don’t magically provide the pipes. You do need to set the pipes up as you describe (except that you don’t need any kind of confirmations in step 3 – each processes just starts using its pipes, and if the other end isn’t set up yet, it will simply block until it is).

The “magic” that process groups provide is around the handling of signals and sharing a terminal.