-
Notifications
You must be signed in to change notification settings - Fork 12.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tracking issue for Read::initializer
#42788
Comments
I'm personally no longer sold on the infrastructure that we've got here I think. I was initially hoping that we'd have some safe way of working with these APIs and such, but in the end it all turned up unsafe. In light of all that I'd personally advocate for much simpler API: impl Read {
#[inline]
unsafe fn may_read_buffer(&self) -> bool { true }
} I don't think the complexity we have today is worth the cost currently, and I'd imagine that we could start making abstractions like: fn initialize_buffer_for(r: &Read, buf: &mut [u8]) {
// ...
} |
I wrote a tiny library for reading into uninitialized buffers: https://crates.io/crates/buffer/ Maybe its ideas could be adapted for the standard library. The following is safe code using the library:
Adoption of something like this in the standard library would probably also help with adoption by third party crates, see e.g. tokio-rs/mio#628. |
An alternative to the current API is to go along the path that unsafe trait ReadUninit: Read {} With specialization landing, anyone who cares to can take advantage of impl<T: Read> ReadBuf for T {
fn read_buf<B: BufMut>(&mut self, buf: &mut B) {
self.read(buf.zeroed())
}
}
impl<T: ReadUninit> ReadBuf for T {
fn read_buf<B: BufMut>(&mut self, buf: &mut B) {
self.read(buf.bytes())
}
} |
That's incompatible with trait objects which are used pretty heavily with Read in particular. |
Specialization is incompatible with trait objects? |
If I have a |
@RalfJung suggested a function that takes |
@sfackler do you recall if we have a tracking issue for the alternative method of implementing this where we hint to the compiler that a buffer shouldn't be considered as undefined? (the strategy we discussed at the last work week) |
There's a bunch of discussion in the PR adding it: #58363. It turns out that we explicitly decided to not go in that direction in the lead up to 1.0, so I think it's worth an RFC that I haven't gotten around to writing. In particular, that approach can change behavior for user-provided Read implementations that are passed to e.g. |
I second the proposal for // std::mem
impl<T> MaybeUninit<T> {
pub fn init_all(uninit: &mut [MaybeUninit<T>], val: T) -> &mut [T] where T: Copy { // `T` could even be `Clone` since this method is panic-safe except for leaks
... // write `val` to all elements
Self::as_init_all(uninit)
}
pub fn init_copy(uninit: &mut [MaybeUninit<T>], data: &[T]) -> &mut [T] where T: Copy { // `T` could also be `Clone`
assert_eq!(uninit.len(), data.len());
unsafe {
Self::as_first_ptr(uninit).copy(data, uninit.len());
}
Self::as_init_all(uninit)
}
pub unsafe fn as_init_all(uninit: &mut [MaybeUninit<T>] -> &mut [T] {
... // assume-initialized intrinsic
mem::transmute(uninit)
}
}
// std::io
pub trait Read {
// ... existing methods
fn read_uninit(&mut self, uninit: &mut [MaybeUninit<u8>]) -> io::Result<()> {
self.read(MaybeUninit::init_all(uninit, 0))
}
} Then implementations that don't read the buffer can override |
I wonder about having a method like the following in fn read_uninit<'a>(&mut self, buf: &'a mut [MaybeUninit<u8>]) -> Result<&'a mut [u8]> {
for byte in &mut *buf {
unsafe {
*byte.as_mut_ptr() = 0;
}
}
let initialized = unsafe { mem::transmute::<&mut [MaybeUninit<u8>], &mut [u8]>(buf) };
let size = self.read(initialized)?;
Ok(&mut initialized[..size])
} The reason for it returning The choice of Note that such an implementation doesn't ensure that that |
The default implementation will make that pretty hard to use in practice, unless you already know that your Read implementation overrides it. You're commonly calling read with a large buffer and don't expect it to be entirely filled, so writing to the entire thing before the raw read call imposes a very large performance penalty. There's an issue somewhere on Tokio's issue tracker about a similar issue with their AsyncRead::read_buf method. |
Maybe this, together with |
I've been thinking for a few months now that if I could redo them, I wouldn't pass byte slices, but some trait object probably (like |
What examples of new concepts do you have in mind? |
So far, there's 2 that we use extensively: vectored IO, and uninitialized buffers. Using these features require an extra method to be implemented on If there were a single |
Can't we address the issue of wrappers not forwarding with a lint? It's not bulletproof but anyone who cares should see the warning. The bonus of a lint is that it can be applied to other traits with similar situations, like The problem with introducing a new trait/module is that it fragments the ecosystem and early adopters suffer from a lack of widespread support. There's plenty of examples of this. I still don't see a whole lot of use of Java's NIO myself, but then again I don't really work in enterprise software. I'm not entirely opposed to the idea myself, but it would definitely need a concerted push for adoption. Now, if Rust ever decides to bring async I/O into the stdlib, it could definitely make the transition then as there would be no competing implementation. |
Getting rid of all references to uninitialized bytes in
|
…ut_uninit_integers, r=Centril Added warning around code with reference to uninit bytes Officially, uninitialized integers, and therefore, Rust references to them are _invalid_ (note that this may evolve into official defined behavior (_c.f._, rust-lang/unsafe-code-guidelines#71)). However, `::std` uses references to uninitialized integers when working with the `Read::initializer` feature (rust-lang#42788), since it relies on this unstably having defined behavior with the current implementation of the compiler (IIUC). Hence the comment to disincentivize people from using this pattern outside the standard library.
…ut_uninit_integers, r=Centril Added warning around code with reference to uninit bytes Officially, uninitialized integers, and therefore, Rust references to them are _invalid_ (note that this may evolve into official defined behavior (_c.f._, rust-lang/unsafe-code-guidelines#71)). However, `::std` uses references to uninitialized integers when working with the `Read::initializer` feature (rust-lang#42788), since it relies on this unstably having defined behavior with the current implementation of the compiler (IIUC). Hence the comment to disincentivize people from using this pattern outside the standard library.
…ut_uninit_integers, r=Centril Added warning around code with reference to uninit bytes Officially, uninitialized integers, and therefore, Rust references to them are _invalid_ (note that this may evolve into official defined behavior (_c.f._, rust-lang/unsafe-code-guidelines#71)). However, `::std` uses references to uninitialized integers when working with the `Read::initializer` feature (rust-lang#42788), since it relies on this unstably having defined behavior with the current implementation of the compiler (IIUC). Hence the comment to disincentivize people from using this pattern outside the standard library.
Read: adjust a FIXME reference There's already another reference to rust-lang#42788 for basically the same problem, so lets reuse it here: https://github.com/rust-lang/rust/blob/5e208efaa850efaa97495e81c49cf0f5767e8f49/library/std/src/io/mod.rs#L369-L376 r? @Dylan-DPC
Can this be closed now that the reading into unintialized buffers RFC is merged and #78485 exists? Since the API is going to be removed anyway, it seems pointless to keep this open. |
It will be closed when the API is removed. |
Implemented in #42002.
Open questions / blockers
42788
), can we avoid that?The text was updated successfully, but these errors were encountered: