Question

For testing purposes, I would like to save stdout and stderr separately for inspection by subsequent code. For example, a test run with erroneous input should result in output to stderr, but nothing on stdout, while a test run with correct input should result in output to stdout, but nothing to stderr. The saving has to be synchronous, to avoid race conditions with the tests (so I can’t use process substitution).

To be able to debug the test after the fact, I also need to see stdout and stderr in the sequence they were output. So I have to either save them both to the same file/variable/whatever or send them to the terminal at the same time as saving them separately.

To test which error happened, I also need the exit code of the command.

For reasons of efficiency and accuracy, I of course cannot run each test twice.

Is it for example possible to redirect stdout to stdout.log, stderr to stderr.log, and both of them to output.log in the same command? Or to use a synchronous tee command separately for stdout and stderr? Or to save a copy of stdout and stderr to separate variables?

Update: It looks like tim’s solution almost works (modified to output on terminal instead of logging to all.log):

$ set -o pipefail
$ {
    {
        echo foo | tee stdout.log 2>&3 3>&-
    } 2>&1 >&4 4>&- | tee stderr.log 2>&3 3>&-
} 3>&2 4>&1
foo
$ cat stdout.log
foo
$ cat stderr.log
$ {
    {
        echo foo >&2 | tee stdout.log 2>&3 3>&-
    } 2>&1 >&4 4>&- | tee stderr.log 2>&3 3>&-
} 3>&2 4>&1
foo
$ cat stdout.log
$ cat stderr.log
foo
$ bar=$({
    {
        echo foo | tee stdout.log 2>&3 3>&-
    } 2>&1 >&4 4>&- | tee stderr.log 2>&3 3>&-
} 3>&2 4>&1)
$ echo "$bar"
foo
$ cat stdout.log
foo
$ cat stderr.log
$ bar=$({
    {
        echo foo >&2 | tee stdout.log 2>&3 3>&-
    } 2>&1 >&4 4>&- | tee stderr.log 2>&3 3>&-
} 3>&2 4>&1)
$ cat stdout.log
$ cat stderr.log
foo
$ echo "$bar"
foo

This seems to work except for the last iteration, where the value of bar is set to the contents of stderr. Any suggestions for all of these to work?

Answer 1

Please see BashFAQ/106. It’s not easy to have your cake and eat it, too.

From that page:

But some people won’t accept either the loss of separation between stdout and stderr, or the desynchronization of lines. They are purists, and so they ask for the most difficult form of all — I want to log stdout and stderr together into a single file, BUT I also want them to maintain their original, separate destinations.

In order to do this, we first have to make a few notes:

• If there are going to be two separate stdout and stderr streams, then some process has to write each of them.
• There is no way to write a process in shell script that reads from two separate FDs whenever one of them has input available, because the shell has no poll(2) or select(2) interface.
• Therefore, we’ll need two separate writer processes.
• The only way to keep output from two separate writers from destroying each other is to make sure they both open their output in append mode. A FD that is opened in append mode has the guaranteed property that every time data is written to it, it will jump to the end first.

So:

# Bash
> mylog
exec > >(tee -a mylog) 2> >(tee -a mylog >&2)

echo A >&2
cat file
echo B >&2

This ensures that the log file is correct. It does not guarantee that the writers finish before the next shell prompt:

~$ ./foo
A
hi mom
B
~$ cat mylog
A
hi mom
B
~$ ./foo
A
hi mom
~$ B

Also, see BashFAQ/002 and BashFAQ/047.