Reusable handle to thread-local replaceable stdout/err

[CAD97]

Lays the groundwork for mitigating:

• Capturing stdout from test code is inconsistent #12309
• libtest captures stdout&stderr only on first thread. #31983
• libtest - capture stderr from tests #40298
• cargo test doesn't capture print from threads #42474

When running tests with logging, logs directed to io::stdout() are not captured by the test runner, and intermingle with the test runner's output. This does not aim to directly solve that problem, but rather provides the infrastructure towards allowing users to opt-in to the same behavior that println! and friends have in regards to std(out|err) for any type generic over Write.

By providing LocalStderr/LocalStdout, when building logging for tests, the logger can be constructed to use io::LocalStdout rather than io::stdout(), which will redirect the output with the exact same mechanism that println! output is captured in the test runner.

The hidden fn _print and _eprint have not been removed to avoid necessitating inlining the panicking code for if stdio/stdout error. However, they have been adjusted to write to LocalStdout/LocalStderr. print_to has been replaced with with_write, allowing the Write impls to defer to the backing Write implementation for all methods.

Is this approach amenable? Is this approach towards allowing well-behaved logs to be captured by test runners one that could in the future potentially be stabilized? Would that require a full RFC? (I can write one up.) (I'm not exactly certain how to add a new feature gate, so for now they're gated by set_stdio, which isn't wrong.) Any bikeshed on naming?

This is mostly a proof-of-concept that this approach would (hopefully) work. Alternately, this switching behavior can be added directly onto io::Stdin/Stdout, in which case the local versions would not be needed, and test output capturing would be closer to avoiding leakage, but spawning a new thread would still avoid the capture, as the approach used to change stdout in tests is inherently thread-local.

(My local ./x.py test also stalled out due to what I assume is an environment error, so I'm hoping Travis will indicate if I messed something up horribly.)

WARNING: this touches the output routine used by print!/eprint!/the default panic handler.
HANDLE WITH CAUTION

[rust-highfive]

r? @kennytm

(rust_highfive has picked a reviewer for you, use r? to override)

[CAD97]

As an example usage, I currently have the following helper function for tests in a workspace using slog for logging:

    fn make_logger() -> ::slog::Logger {
        use slog::{Drain, Logger};
        use std::{io, sync::Mutex};
        Logger::root(
            Mutex::new(
                slog_term::CompactFormat::new(
                    slog_term::PlainSyncDecorator::new(io::stdout())
                ).build().fuse()
            ).fuse(),
            o!(),
        )
    }

With the approach taken in this PR, I should be able to replace io::stdout() directly with io::LocalStdout, and the logging done will go from escaping into the test logger's output to being captured as would any other simple println! call.

The alternate solution here would be to create a new logger type that uses println! rather than writing to a Write sink, but that scales linearly with the number of logger frameworks, instead of this approach's constant amount of work :P

[kennytm]

r? @alexcrichton

[alexcrichton]

Thanks for the PR! This I think is pretty highly related to the work being done on custom test frameworks and so I'm somewhat hesitatnt to land without weighing in with those folks.

In terms of this particular solution I've never been too fond of the ability to override stdout/stderr in libstd and it was basically always just a necessary evil of thet test harness itself. I'd prefer to pursue different avenues to avoid libtest needing to capture entirely first before we add more api surface area to libstd itself.

[CAD97]

Seems reasonable to me! (I'm fine with closing this PR for the time being to reduce clutter if that's desired.)

The stdout/stderr overriding is only necessary as I understand it because stdout/stderr are by-process and the test harness works by spawning threads in-process to run the individual tests.

If you point me in the direction where the test harness discussion is happening, I'd be happy to participate, I'm just not exactly certain where that is, though I've been excited by what bits of it I've stumbled across.

[alexcrichton]

I'm not sure myself exactly where the test framework discussion is happening, but I suspect that @killercup might!

[pietroalbini]

Ping from triage @alexcrichton! If I understood correctly this is blocked on the discussion about custom test frameworks?

[killercup]

@CAD97 the custom test frameworks eRFC was merged two weeks ago, but there is no implementation work ongoing yet. Here's the tracking issue: #50297

[alexcrichton]

Ok I'm gonna close this in favor of that issue, @CAD97 mind writing up your thoughts though on the issue about the problem here with this possible solution?

[CAD97]

It has been done. See #50297 (comment) for my current thoughts. TL;DR: To have sane capture behavior for stdout and stderr for print-family macros, panic, and io::{stdout, stderr}, running each test in its own (child) process is probably required. I'd love to help out wherever I can, but lack confidence.

[alexcrichton]

@CAD97 ok, thanks for the comment!