Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Notifications
Mark all as read
Q&A

Capture separate _and_ combined stdout/stderr

+5
−0

Using BASH, how can I redirect stdout and stderr each, to two separate files, simultaneously?

That's a mouthful, let me clarify a little:

I have a hypothetical script that may/will generate output on both stdout and stderr. I want to run the script and wind up with three output files:

  1. The contents of stdout only (>stdout.txt).
  2. The contents of stderr only (2>stderr.txt).
  3. The combined contents of stdout and stderr, interleaved as they would have been had they been printed to the terminal in real time (>both.txt 2>both.txt).

With Zsh you can do something like this with MULTIOS enabled:

script.txt >stdout.txt >>both.txt 2>stderr.txt 2>>both.txt

...but I need something that'll work on systems that don't have Zsh installed.

Why does this post require moderator attention?
You might want to add some details to your flag.
Why should this post be closed?

1 comment thread

"interleaved as they would have been had they been printed to the terminal" (4 comments)

1 answer

+1
−0

Each file descriptor can only point to a single file. File descriptors can be duplicated such that both point to a single file, but a file descriptor cannot point to two files.

Hence, you have to decide whether stdout points to out.txt or to both.txt, and similarly with stderr.

Of course, you could use Tee to replicate output, since it writes its input directly to files (given as arguments) and to stdout, which you can further redirect. One such approach, proposed in "How to redirect stderr and stdout to different files and also display in terminal?", is

((cmd | tee out.txt) 3>&1 1>&2 2>&3 | tee err.txt) &> both.txt

It, however, fails to preserve the order of stdout and stderr output when writing to both.txt.

This is because once you send cmd's stdout and stderr to different roads, you can't guarantee that they will take the same time to traverse it.

More technically: The two Tees are different, concurrent processes, and as such incur in a race condition; as a result the order of the lines in both.txt is not deterministic.

Your proposed trick with Zsh MULTIOS also fails to address this issue, since, as documented, it does essentially the same thing:

the shell opens the file descriptor as a pipe to a process that copies its input to all the specified outputs, similar to tee, provided the MULTIOS option is set, as it is by default.

Unfortunately, although piping both stdout and stderr to another program preserves the line order for both.txt, the program then couldn't tell them apart in order to write out.txt and err.txt. I.e., in

cmd 2>&1 | filter

filter only has a notion of its stdin; the information of whether this or that line "came from" cmd's stdout or stderr is completely lost.

The only reliable and feasible solution I see is to edit the program's source code so as to insert information in its output that shows which lines are stdout and which are stderr (e.g. by prepending "E:" to stderr and "O:" to stdout). Too bad it is cumbersome and not scalable.

Why does this post require moderator attention?
You might want to add some details to your flag.

0 comment threads

Sign up to answer this question »