Question

I’m using a pipe of several commands in bash. Is there a way of configuring bash to terminate all commands in the whole pipeline immediately should one of the commands fail?

In my case, the first command, say command1, runs for a while until it produces some output. You might substitute command1 by (sleep 5 && echo "Hello"), for instance.

Now, command1 | false does fail after 5 seconds but not immediately.

This behavior seems to have something to do with the amount of output the command produces. For instance, find / | false returns immediately.

In general, I wonder why bash behaves like this. Can anyone imagine any situation where it is useful that code like command1 | non-existing-command does not exit at once?

PS: Using temporary files is not an option for me, as the intermediate results I pipe around are to big to be stored.

PPS: Neither set -e nor set -o pipefail seem to influence this phenomenon.

Answer 1

The bash documentation says in its section about pipelines:

Each command in a pipeline is executed in its own subshell […]

“In its own subshell” means that a new bash process is spawned, which then gets to execute the actual command. Each subshell starts successfully, even when it immediately determines that the command it is asked to execute doesn’t exist.

This explains why the entire pipe can be set up successfully even when one of the commands is nonsense. Bash does not check if each command can be run, it delegates that to the subshells. That also explains why, for example, the command nonexisting-command | touch hello will throw a “command not found” error, but the file hello will be created nonetheless.

In the same section, it also says:

The shell waits for all commands in the pipeline to terminate before returning a value.

In sleep 5 | nonexisting-command, as A.H. pointed out, the sleep 5 terminates after 5 seconds, not immediately, hence the shell will also wait 5 seconds.

I don’t know why the implementation was done this way. In cases like yours, the behavior is surely not as one would expect.

Anyway, one slightly ugly workaround is to use FIFOs:

mkfifo myfifo
./long-running-script.sh > myfifo &
whoops-a-typo < myfifo

Here, the long-running-script.sh is started and then the scripts fails immediately on the next line. Using mutiple FIFOs, this could be extended to pipes with more than two commands.