Writing to stdout with write() changed behavior between version 1.47 and >= 1.48

[sdragic]

I'm not sure this is worth reporting, but it has caused confusion at least once, so maybe it makes sense to be documented somewhere.

When writing to stdout with write(), write will return smaller number bytes than direct write() call would do in some cases. Here is example code:

use std::io::{self, Write};

fn main() {
    let mut a = ['a' as u8; 2048];
    eprintln!("{}", a.len());
    eprintln!("{:?}", io::stdout().write(&a));
    a[50] = '\n' as u8;
    eprintln!("{:?}", io::stdout().write(&a));
}

Rustc 1.47 gives expected result:

#./main > /dev/null
2048
Ok(2048)
Ok(2048)

This is also in accordance with similar programs written in C, Python and Rust using libc::write().

However rustc 1.48 and greater return this:

#./main > /dev/null
2048
Ok(2048)
Ok(1075)

This is technically not a bug, but IMHO it is unexpected behavior. Rust docs says this:

This function will attempt to write the entire contents of buf, but the entire write may not succeed, or the write may also generate an error. A call to write represents at most one attempt to write to any wrapped object.

I'm aware "wrapped object" here is some internal std library structure, but most people would expect it should behave similarly to underlying system write().

man 2 write says:

The number of bytes written may be less than count if, for example, there is insufficient space on the underlying physical medium, or the RLIMIT_FSIZE resource limit is encountered (see setr‐limit(2)), or the call was interrupted by a signal handler after having written less than count bytes.

As I understand this, returning less than count should be affected only by OS state and not by content of data (newline in this case).

This behavior seems to be introduced with PR #72808, and pending PR #78515 seems to revert behavior to expected.

Just a disclaimer, I do not condone use of unchecked write(). Reason for reporting this is inconsistency between io::stdout().write() and similar approaches.

[m-ou-se]

Thanks for your detailed report!

This is technically not a bug, but IMHO it is unexpected behavior.

Agreed. Marking this as low priority.

[Lucretiel]

The basic issue at play here is that, in line buffering logic, the buffer will generally attempt to only forward whole lines to the underlying writer; the only exception is when the line length exceeds the size of the buffer. Once it forwards the line, it can't attempt any more fallible operations, because it must return an Ok result at that point. This means it will:

• Write the line to the underlying device (a[..51])
• Buffer as much as possible of the remaining content (a[51..(51 + 1024)])

However, there's an interesting issue at play here. In the sample code, the write method should be able to tell that even after the line write succeeds, the upcoming content (a[51..]) will on its own exceed the size of the buffer, which means that it probably warrants being written directly.

This is technically unrelated to #72808; that PR will only fix this issue when stdout is in buffered writing mode. However, it's fairly easy to add the logic, and it makes sense that it should be present, so I'm writing a separate PR for it.

[the8472]

@Lucretiel #78515 should provide the necessary control to solve this issue by switching stdout to immediate mode, right?

Edit: silly question, this already is mentioned in OP

This behavior seems to be introduced with PR #72808, and pending PR #78515 seems to revert behavior to expected.

[joshtriplett]

We talked about this in the @rust-lang/libs meeting today.

This is behavior we care about, and we think it's valuable to work on stdout buffering (including the ongoing work in issues like #78515 and similar).

However:

This is technically not a bug, but IMHO it is unexpected behavior.

Given this, we don't plan to track this as a regression. We'd encourage people to get involved with the stdout buffering rework to improve the behavior.