-
Notifications
You must be signed in to change notification settings - Fork 352
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Undefined behavior from poll_fn
+ futures::join!
/ std::future::join!
#3780
Comments
futures::join!
/ std::future::join!
poll_fn
+ futures::join!
/ std::future::join!
Miri honors Please include the error when reporting a bug. Here it is:
|
In the original bug report to Tokio, the UB was happening even though |
It seems adding |
Or making Btw, it is also possible to reproduce the error with a function, such as |
Minimized: use std::{
future::Future,
pin::Pin,
sync::Arc,
task::{Context, Poll, Wake},
};
struct ThingAdder<'a> {
thing: &'a mut String,
}
impl Future for ThingAdder<'_> {
type Output = ();
fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<Self::Output> {
unsafe {
*self.get_unchecked_mut().thing += ", world";
}
Poll::Pending
}
}
fn main() {
let mut thing = "hello".to_owned();
let fut = async move { ThingAdder { thing: &mut thing }.await };
let mut fut = MaybeDone::Future(fut);
let mut fut = unsafe { Pin::new_unchecked(&mut fut) };
let waker = Arc::new(DummyWaker).into();
let mut ctx = Context::from_waker(&waker);
assert_eq!(fut.as_mut().poll(&mut ctx), Poll::Pending);
assert_eq!(fut.as_mut().poll(&mut ctx), Poll::Pending);
}
struct DummyWaker;
impl Wake for DummyWaker {
fn wake(self: Arc<Self>) {}
}
pub enum MaybeDone<F: Future> {
Future(F),
Done,
}
impl<F: Future<Output = ()>> Future for MaybeDone<F> {
type Output = ();
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
unsafe {
match *self.as_mut().get_unchecked_mut() {
MaybeDone::Future(ref mut f) => Pin::new_unchecked(f).poll(cx),
MaybeDone::Done => unreachable!(),
}
}
}
} Removing the edit: was able to get rid of |
The memory that the conflict occurs on is the The In other words, this is the "mutable reference aliasing" version of why |
Since Miri is behaving as intended here, I'm going to close the issue in favor of rust-lang/rust#63818 and rust-lang/rust#125735. I don't see a way that we can work around this akin to the Thanks for bringing this up, and for the very helpful minimization! In retrospect it is somewhat surprising that it took us so long to discover this, given that we already knew about the exact same issue with shared references and Cc @rust-lang/opsem, just an FYI :) |
I'm just as surprised! 😄 Given that the |
If you only need |
Is there anything Tokio should be doing to avoid UB with niche-optimized enums? We have the pin_project! {
/// A future that may have completed.
#[derive(Debug)]
#[project = MaybeDoneProj]
#[project_replace = MaybeDoneProjReplace]
pub enum MaybeDone<Fut: Future> {
/// A not-yet-completed future.
Future { #[pin] future: Fut },
/// The output of the completed future.
Done { output: Fut::Output },
/// The empty variant after the result of a [`MaybeDone`] has been
/// taken using the [`take_output`](MaybeDone::take_output) method.
Gone,
}
} |
I don't know if you should be doing anything, since it seems unlikely that this will lead to actual miscompilations. Though maybe @comex is able to trick LLVM into generating bad code here again. ;) What you could do it mark that enum with a The best thing to do would be to find someone who can implement |
This reproduces a potential UB where a `task::Cell` gets niche-optimized in the `Joined` case. This is based on the test added to Tokio in tokio-rs/tokio#6744
This reproduces a potential UB where a `task::Cell` gets niche-optimized in the `Joined` case. This is based on the test added to Tokio in tokio-rs/tokio#6744
See [this comment][1]. Adding `#[repr(C)]` to this type suppresses niche optimization, resolving potential unsoundness that could occur when the compiler decides to niche-optimize a self-referential future type. In practice this probably doesn't actually happen, but we should make sure it never does. [1]: rust-lang/miri#3780 (comment)
This reproduces a potential UB where a `task::Cell` gets niche-optimized in the `Joined` case. This is based on the test added to Tokio in tokio-rs/tokio#6744
See [this comment][1]. Adding `#[repr(C)]` to this type suppresses niche optimization, resolving potential unsoundness that could occur when the compiler decides to niche-optimize a self-referential future type. In practice this probably doesn't actually happen, but we should make sure it never does. [1]: rust-lang/miri#3780 (comment)
While this is adequate as a workaround for the specific case here, it does not appear to be a reasonable workaround for this problem as a whole. This is because the same issue can be caused by using Option instead of a custom enum like MaybeDone. |
Yeah I did not mean to imply that this is a reasonable work-around for this case, I just pointed out the possibility. I don't think there is a good work-around, this fundamentally needs rustc layout computation to change -- either by special-casing generators, or by implementing rust-lang/rust#125735. |
I thought wrapping a coroutine with a MaybeUninit inside a future returned by an async block/async function (like crossbeam-rs/crossbeam#834) as a potential workaround. In previous async lowing I think it was enough to change the |
Here's a proper work-around: rust-lang/rust#129313. It increases the size of some futures, though... |
…r-errors Supress niches in coroutines to avoid aliasing violations As mentioned [here](rust-lang#63818 (comment)), using niches in fields of coroutines that are referenced by other fields is unsound: the discriminant accesses violate the aliasing requirements of the reference pointing to the relevant field. This issue causes [Miri errors in practice](rust-lang/miri#3780). The "obvious" fix for this is to suppress niches in coroutines. That's what this PR does. However, we have several tests explicitly ensuring that we *do* use niches in coroutines. So I see two options: - We guard this behavior behind a `-Z` flag (that Miri will set by default). There is no known case of these aliasing violations causing miscompilations. But absence of evidence is not evidence of absence... - (What this PR does right now.) We temporarily adjust the coroutine layout logic and the associated tests until the proper fix lands. The "proper fix" here is to wrap fields that other fields can point to in [`UnsafePinned`](rust-lang#125735) and make `UnsafePinned` suppress niches; that would then still permit using niches of *other* fields (those that never get borrowed). However, I know that coroutine sizes are already a problem, so I am not sure if this temporary size regression is acceptable. `@compiler-errors` any opinion? Also who else should be Cc'd here?
…r-errors Supress niches in coroutines to avoid aliasing violations As mentioned [here](rust-lang#63818 (comment)), using niches in fields of coroutines that are referenced by other fields is unsound: the discriminant accesses violate the aliasing requirements of the reference pointing to the relevant field. This issue causes [Miri errors in practice](rust-lang/miri#3780). The "obvious" fix for this is to suppress niches in coroutines. That's what this PR does. However, we have several tests explicitly ensuring that we *do* use niches in coroutines. So I see two options: - We guard this behavior behind a `-Z` flag (that Miri will set by default). There is no known case of these aliasing violations causing miscompilations. But absence of evidence is not evidence of absence... - (What this PR does right now.) We temporarily adjust the coroutine layout logic and the associated tests until the proper fix lands. The "proper fix" here is to wrap fields that other fields can point to in [`UnsafePinned`](rust-lang#125735) and make `UnsafePinned` suppress niches; that would then still permit using niches of *other* fields (those that never get borrowed). However, I know that coroutine sizes are already a problem, so I am not sure if this temporary size regression is acceptable. `@compiler-errors` any opinion? Also who else should be Cc'd here?
…r-errors Supress niches in coroutines to avoid aliasing violations As mentioned [here](rust-lang#63818 (comment)), using niches in fields of coroutines that are referenced by other fields is unsound: the discriminant accesses violate the aliasing requirements of the reference pointing to the relevant field. This issue causes [Miri errors in practice](rust-lang/miri#3780). The "obvious" fix for this is to suppress niches in coroutines. That's what this PR does. However, we have several tests explicitly ensuring that we *do* use niches in coroutines. So I see two options: - We guard this behavior behind a `-Z` flag (that Miri will set by default). There is no known case of these aliasing violations causing miscompilations. But absence of evidence is not evidence of absence... - (What this PR does right now.) We temporarily adjust the coroutine layout logic and the associated tests until the proper fix lands. The "proper fix" here is to wrap fields that other fields can point to in [`UnsafePinned`](rust-lang#125735) and make `UnsafePinned` suppress niches; that would then still permit using niches of *other* fields (those that never get borrowed). However, I know that coroutine sizes are already a problem, so I am not sure if this temporary size regression is acceptable. `@compiler-errors` any opinion? Also who else should be Cc'd here?
…r-errors Supress niches in coroutines to avoid aliasing violations As mentioned [here](rust-lang#63818 (comment)), using niches in fields of coroutines that are referenced by other fields is unsound: the discriminant accesses violate the aliasing requirements of the reference pointing to the relevant field. This issue causes [Miri errors in practice](rust-lang/miri#3780). The "obvious" fix for this is to suppress niches in coroutines. That's what this PR does. However, we have several tests explicitly ensuring that we *do* use niches in coroutines. So I see two options: - We guard this behavior behind a `-Z` flag (that Miri will set by default). There is no known case of these aliasing violations causing miscompilations. But absence of evidence is not evidence of absence... - (What this PR does right now.) We temporarily adjust the coroutine layout logic and the associated tests until the proper fix lands. The "proper fix" here is to wrap fields that other fields can point to in [`UnsafePinned`](rust-lang#125735) and make `UnsafePinned` suppress niches; that would then still permit using niches of *other* fields (those that never get borrowed). However, I know that coroutine sizes are already a problem, so I am not sure if this temporary size regression is acceptable. `@compiler-errors` any opinion? Also who else should be Cc'd here?
Supress niches in coroutines to avoid aliasing violations As mentioned [here](rust-lang/rust#63818 (comment)), using niches in fields of coroutines that are referenced by other fields is unsound: the discriminant accesses violate the aliasing requirements of the reference pointing to the relevant field. This issue causes [Miri errors in practice](rust-lang#3780). The "obvious" fix for this is to suppress niches in coroutines. That's what this PR does. However, we have several tests explicitly ensuring that we *do* use niches in coroutines. So I see two options: - We guard this behavior behind a `-Z` flag (that Miri will set by default). There is no known case of these aliasing violations causing miscompilations. But absence of evidence is not evidence of absence... - (What this PR does right now.) We temporarily adjust the coroutine layout logic and the associated tests until the proper fix lands. The "proper fix" here is to wrap fields that other fields can point to in [`UnsafePinned`](rust-lang/rust#125735) and make `UnsafePinned` suppress niches; that would then still permit using niches of *other* fields (those that never get borrowed). However, I know that coroutine sizes are already a problem, so I am not sure if this temporary size regression is acceptable. `@compiler-errors` any opinion? Also who else should be Cc'd here?
Supress niches in coroutines to avoid aliasing violations As mentioned [here](rust-lang/rust#63818 (comment)), using niches in fields of coroutines that are referenced by other fields is unsound: the discriminant accesses violate the aliasing requirements of the reference pointing to the relevant field. This issue causes [Miri errors in practice](rust-lang/miri#3780). The "obvious" fix for this is to suppress niches in coroutines. That's what this PR does. However, we have several tests explicitly ensuring that we *do* use niches in coroutines. So I see two options: - We guard this behavior behind a `-Z` flag (that Miri will set by default). There is no known case of these aliasing violations causing miscompilations. But absence of evidence is not evidence of absence... - (What this PR does right now.) We temporarily adjust the coroutine layout logic and the associated tests until the proper fix lands. The "proper fix" here is to wrap fields that other fields can point to in [`UnsafePinned`](rust-lang/rust#125735) and make `UnsafePinned` suppress niches; that would then still permit using niches of *other* fields (those that never get borrowed). However, I know that coroutine sizes are already a problem, so I am not sure if this temporary size regression is acceptable. `@compiler-errors` any opinion? Also who else should be Cc'd here?
Recently, when working on a library for observables, I found new miri test failures (I had been getting them on a specific test before, but didn't investigate after ensuring it wasn't my own UB). This lead to a bug report to
tokio
where Alice Ryhl showed that it didn't actually requiretokio::spawn
(which I had failed to get rid of in minimizing the issue), and the same issue was reproducible withjoin!
+futures_executor
.I have now reduced the example much further, to this (works with the nightly
std::future::join!
, orfutures::join!
; playground):Alice suggested that this may be an instance of rust-lang/rust#63818, is that correct? I was under the impression that the compiler for now doesn't implement alias restrictions for
!Unpin
types, but maybe this only applies to the LLVM backend, not miri?The text was updated successfully, but these errors were encountered: