Capture separate _and_ combined stdout/stderr
Using BASH, how can I redirect stdout and stderr each, to two separate files, simultaneously?
That's a mouthful, let me clarify a little:
I have a hypothetical script that may/will generate output on both stdout and stderr. I want to run the script and wind up with three output files:
- The contents of stdout only (
>stdout.txt). - The contents of stderr only (
2>stderr.txt). - The combined contents of stdout and stderr, interleaved as they would have been had they been printed to the terminal in real time (
>both.txt 2>both.txt).
With Zsh you can do something like this with MULTIOS enabled:
script.txt >stdout.txt >>both.txt 2>stderr.txt 2>>both.txt
...but I need something that'll work on systems that don't have Zsh installed.
1 answer
Each file descriptor can only point to a single file. File descriptors can be duplicated such that both point to a single file, but a file descriptor cannot point to two files.
Hence, you have to decide whether stdout points to out.txt or to both.txt, and similarly with stderr.
Of course, you could use Tee to replicate output, since it writes its input directly to files (given as arguments) and to stdout, which you can further redirect. One such approach, proposed in "How to redirect stderr and stdout to different files and also display in terminal?", is
((cmd | tee out.txt) 3>&1 1>&2 2>&3 | tee err.txt) &> both.txt
It, however, fails to preserve the order of stdout and stderr output when writing to both.txt.
This is because once you send cmd's stdout and stderr to different roads,
you can't guarantee that they will take the same time to traverse it.
More technically: The two Tees are different, concurrent processes, and as such incur in a race condition; as a result the order of the lines in both.txt is not deterministic.
Your proposed trick with Zsh MULTIOS also fails to address this issue,
since, as documented,
it does essentially the same thing:
the shell opens the file descriptor as a pipe to a process that copies its input to all the specified outputs, similar to tee, provided the MULTIOS option is set, as it is by default.
Unfortunately, although piping both stdout and stderr to another program preserves the line order for both.txt, the program then couldn't tell them apart in order to write out.txt and err.txt. I.e., in
cmd 2>&1 | filter
filter only has a notion of its stdin; the information of whether this or that
line "came from" cmd's stdout or stderr is completely lost.
The only reliable and feasible solution I see is to edit the program's source code so as to insert information in its output that shows which lines are stdout and which are stderr (e.g. by prepending "E:" to stderr and "O:" to stdout). Too bad it is cumbersome and not scalable.
0 comment threads
1 comment thread