BufReader/Writer extension methods: is_empty, buffer

[fintelia]

Update: this is the tracking issue for:

impl<R: Read> BufReader<R> {
    pub fn buffer(&self) -> &[u8] {…}

    #[rustc_deprecated(since = "1.26.0", reason = "use .buffer().is_empty() instead")]
    pub fn is_empty(&self) -> bool {…}
}

is_empty is both unstable and deprecated, it should be removed.

---

There is currently no way to tell whether there is any data buffered inside a BufReader. This is unfortunate because it means that an application has no way to know whether calls to read() and fill_buffer() will return instantly or trigger a potentially expensive call to the underlying Reader. I propose adding an is_empty() method to BufReader to fill this gap.

[jonhoo]

@kennytm now that #45369 has been merged, I'm guessing this should get the C-tracking-issue label?

[dbrgn]

What's left for this to stabilize?

I'm parsing bytes from a BufReader to MsgPack using read_value. The function will stop parsing once a valid MsgPack value has been found. But in my application the parsing should fail if the buffer hasn't been consumed completely. Checking buf.is_empty() would probably be the cleanest way to do so.

[jonhoo]

@dbrgn the BufReader::buf_bytes method proposed in #48022 may also be useful for that purpose, though that may take even longer to land.

[SimonSapin]

This is the tracking issue for:

impl<R: Read> BufReader<R> {
    pub fn is_empty(&self) -> bool { /*…*/ }
}

Looks good to me to stabilize.

@rfcbot fcp merge

[rfcbot]

Team member @SimonSapin has proposed to merge this. The next step is review by the rest of the tagged teams:

• @Kimundi
• @SimonSapin
• @alexcrichton
• @aturon
• @dtolnay
• @sfackler
• @withoutboats

No concerns currently listed.

Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[sfackler]

This seems fine to stabilize, but it seems like it'd be more useful to know how many bytes are buffered, rather than just that there are some number of bytes.

[SimonSapin]

Why not both? Slices have is_empty even though it’s somewhat redundant with len.

[sfackler]

For sure. Would it be reasonable to try to squeeze that into this stabilization?

[SimonSapin]

Sounds ok to me, though we should probably (re)start FCP after the implementation is merged.

[sfackler]

Naming wise, I don't think len fits since that seems to me like it'd refer to the length of the entire reader rather than just the current buffered data (similarly is_empty is a bit confusing). How about buffered_len and buffer_is_empty?

[SimonSapin]

Agreed. Sounds good to me.

[sfackler]

Alternatively, we could add fn buffer(&self) -> &[u8] to just return the buffer, which is also a thing people want separately from this issue. At that point r.buffer().len() seems not much worse than r.buffer_len() so we could have one method instead of three.

[sfackler]

#49139 adds buffer and deprecates is_empty.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[fintelia]

With @sfackler's alternative (and IMO better) proposal just introduced, should this still be going into its final comment period?

[SimonSapin]

@rfcbot fcp cancel

Let’s start again after buffer() has landed.

[rfcbot]

@SimonSapin proposal cancelled.

[jonhoo]

We should likely also add a .buffer() method to BufWriter for consistency?

[sfackler]

That can't be done backwards compatibly.

[jonhoo]

@sfackler why not? It's just a struct, right? Just like BufReader.

[fintelia]

It has been a while since buffer landed. Is this ready to stabilize?

[SimonSapin]

BufReader::buffer is implemented, but BufWriter::buffer discussed above is not. (I’ve updated the issue’s description with the current status.)

[fintelia]

Wait, we aren't proposing to stabilize the (already deprecated) is_empty method are we?

[SimonSapin]

No, sorry I didn’t clarify. An API marked both unstable and deprecated is to be removed (after something like a couple release cycles of deprecation period, which we’re at in this case).

[zslayton]

I'm excited to see that this is so close to landing. Is this still waiting on an implementation of BufWriter::buffer? Would it be possible to track that feature separately to unblock this one?

[fintelia]

Bump. Is there anything else blocking stabilization?

[edef1c]

Is there any particular reason not to add fn buffer(&self) -> &[u8] { &[] } to BufWrite?
It seems like a useful enough general-purpose API ("get me however many bytes you can easily give me"), and I found myself looking for such a method while working on parser tooling.
Individual implementations could make stricter guarantees about the lower bounds, of which "at least the unconsumed fraction of the last fill_buf" would be a common one.

[fintelia]

Yeah, this has been a while. @SimonSapin @sfackler is this ready for a vote to stabilize?

[fintelia]

This issue seems to have been stuck in a strange limbo state for quite a while now. Pinging more people the rfcbot mentioned... @alexcrichton @aturon @dtolnay @withoutboats

[SimonSapin]

Sorry we dropped the ball here. This is now one method on the BufReader and BufWriter structs.

impl<R> BufReader<R> {
    pub fn buffer(&self) -> &[u8] {…}
}
impl<W: Write> BufWriter<W> {
    pub fn buffer(&self) -> &[u8] {…}
}

@rfcbot fcp merge

[rfcbot]

Team member @SimonSapin has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @Kimundi
• @SimonSapin
• @alexcrichton
• @dtolnay
• @sfackler
• @withoutboats

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[SimonSapin]

@edef1c There is nothing called BufWrite in std. Did you mean the BufRead trait? A downside is that to add it now we would have to make it a default method, which would (at first) have an inconsistent behavior with fill_buf for existing impls. We can update impls in std at the same time, but that leaves impls in other crates.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[edef1c]

@SimonSapin Sorry, yes, I meant BufRead. For my use cases, not having a lower bound is fine. Anything that does can use a marker subtrait that encompasses that guarantee.
I think the only stipulation should be that it not provide more than fill_buf, but less is fine. For example, an implementation might copy existing data around on fill_buf to make a larger contiguous segment, but can't/shouldn't do so in buffer (I think it should be a cheap conversion, as_buffer wouldn't have been an unreasonable name).

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.