-
Notifications
You must be signed in to change notification settings - Fork 12.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tracking issue for panics in mem::uninitialized/zeroed #66151
Comments
git2-rs needed some adjustment to work even with the minimal checks that have already landed, which is good enough for now but even those adjustments will nut suffice once we properly check enums. The reason for this is that ctest uses |
Will the unstable methods associated with I agree this is a positive change for the language, but with a large amount of the |
@valarauca Which patterns specifically are you talking about? |
…petrochenkov might_permit_raw_init: also check aggregate fields This is the next step for rust-lang#66151: when doing `mem::zeroed`/`mem::uninitialized`, also recursively check fields of aggregates (except for arrays) for whether they permit zero/uninit initialization.
In #77970 (comment), the panic in I think it might make sense to force an abort (with a message), rather than use whatever panic strategy is configured. There's some precedent for this - both One downside is that this test would become more complicated, since we can longer repeatedly trigger the panic in a single test. |
From #66059 (comment):
|
There's ongoing work to make Once that becomes available on nightly, we could use it internally to attempt to print a backtrace, and then abort. |
Yes, that would certainly be worth trying! I ran into such unexpected-unwind segfaults before when analyzing crater runs for this change and they are indeed quite hard to debug. |
It's been about a year since this issue was created. Has enough time passed that PRs adding the check for arrays and enums would be merged? |
We just landed this check for structs and tuples on the stable branch with Rust 1.48. The fallout from that is still rippling through the ecosystem, e.g. here. So I feel we should wait a bit before retching this up another level. @nico-abram would you be interested in preparing such a change for arrays and seeing it through (in particular analyzing the crater report)? The code change for arrays is trivial, and I can provide mentoring, but last time analyzing the crater fallout took forever so I won't do this again anytime soon.^^ |
Yes! I can, assuming there is no set timeline required If I understand correctly, the change would be replacing this comment with a similar check to the The change would just be a straight up PR against this repository, and someone authorized would schedule a crater run? Finally, does analyzing the crater report mean looking into the errors and checking why they happen, analyzing the ecosystem-wide impact of breakage, and contacting maintainers with bug reports? |
No timeline required. :) Thanks!
Yes, that sounds right: first determine if the array has at least one element, then check the layout of the element type. The
Exactly. The PR should also extend the test "src/test/ui/intrinsics/panic-uninitialized-zeroed.rs" to make sure the check behaves as expected.
Basically, yes. You can see my analysis for the last change here. As you can see from the bingbacks above that comment, there were quite a few bugreports to be made for this one. I believe that the fallout will be smaller this time, since I think that we covered most of the "heavy hitters" last time and they wouldn't show up as regressions any more now (since they are already broken on stable). However, there might well be unexpected things showing up (like vast numbers of people still using outdated smallvec, which is known to be broken). |
Here's a good testcase for an enum that we should detect is UB for EDIT: Ah, we already do. |
This commit requires linked-hash-map to include contain-rs/linked-hash-map#100 When linked-hash-map is on v0.5.2, ttl_cache can panic due to the UB checks implemented in Rust 1.48: rust-lang/rust#66151
Another possible avenue that has been discussed is to give up on trying to fix old code (or getting people to fix it, by raising panics) and instead switch to mitigation mode, by reducing the amount of UB. We could make That said, because it would not fully fix UB, I am not sure whether we should rely on this mitigation, or still also go with panics for the cases where we can get away with that. |
Sounds good, but I'd want to not do that if the user is using memory sanitizer or miri, otherwise those tools will get worse at detecting UB. And that would also hide branching on undef if you run valgrind, but that seems more niche. We have those
And I think it's still worth pushing for panics, I definitely think we shouldn't rely on that. But as a "well, it makes it much less likely that LLVM will notice we're lying to it for the time being", eh... I'm fine with that, but not if it means we never get rid of that hack. |
Yeah good point; we should probably pub unsafe fn uninitialized<T>() -> T {
// SAFETY: the caller must guarantee that an uninitialized value is valid for `T`.
unsafe {
intrinsics::assert_uninit_valid::<T>();
#[allow(unused_mut)]
let mut val = MaybeUninit::uninit();
// Fill memory with 0x01, as a mitigation for old code that uses
// this function on bool and nonnull types. But don't do this
// if we actively want to detect UB.
#[cfg(not(any(miri, sanitize = "???")))]
val.as_mut_ptr().write_bytes(0x01, 1);
val.assume_init()
}
}
Ah yeah, there's no single byte that works for |
PR for the |
While we cannot synthesize a known-valid value for references in general, we can for slices. I think that would remove old versions of |
mem::uninitialized: mitigate many incorrect uses of this function Alternative to rust-lang#98966: fill memory with `0x01` rather than leaving it uninit. This is definitely bitewise valid for all `bool` and nonnull types, and also those `Option<&T>` that we started putting `noundef` on. However it is still invalid for `char` and some enums, and on references the `dereferenceable` attribute is still violated, so the generated LLVM IR still has UB -- but in fewer cases, and `dereferenceable` is hopefully less likely to cause problems than clearly incorrect range annotations. This can make using `mem::uninitialized` a lot slower, but that function has been deprecated for years and we keep telling everyone to move to `MaybeUninit` because it is basically impossible to use `mem::uninitialized` correctly. For the cases where that hasn't helped (and all the old code out there that nobody will ever update), we can at least mitigate the effect of using this API. Note that this is *not* in any way a stable guarantee -- it is still UB to call `mem::uninitialized::<bool>()`, and Miri will call it out as such. This is somewhat similar to rust-lang#87032, which proposed to make `uninitialized` return a buffer filled with 0x00. However - That PR also proposed to reduce the situations in which we panic, which I don't think we should do at this time. - The 0x01 bit pattern means that nonnull requirements are satisfied, which (due to references) is the most common validity invariant. `@5225225` I hope I am using `cfg(sanitize)` the right way; I was not sure for which ones to test here. Cc rust-lang#66151 Fixes rust-lang#87675
…alfJung Strengthen invalid_value lint to forbid uninit primitives, adjust docs to say that's UB For context: rust-lang#66151 (comment) This does not make it a FCW, but it does explicitly state in the docs that uninit integers are UB. This also doesn't affect any runtime behavior, uninit u32's will still successfully be created through mem::uninitialized.
I think we should slightly change track here, and rather than trying to detect and root out all Rust-level UB (which at this point I consider a failed experiment), we should do what we can do mitigate the LLVM-level UB that can be caused by Based on that, if #101061 lands, my plan would be to strengthen the checks to also recurse below |
Could we add a codegen test to ensure that we don't do this, leaving a comment back to this issue and saying that we must start recursing into arrays if we start emitting It doesn't need to be done before #101061 lands, but I think it would be good to add at some point. I'll have a shot at it once I'm done with cleaning up the FCW PR. |
Yes, the PR that finishes up the panics and closes this issue (the PR after #101061) should add such a codegen test. |
…fJung Strengthen invalid_value lint to forbid uninit primitives, adjust docs to say that's UB For context: rust-lang#66151 (comment) This does not make it a FCW, but it does explicitly state in the docs that uninit integers are UB. This also doesn't affect any runtime behavior, uninit u32's will still successfully be created through mem::uninitialized.
Strengthen invalid_value lint to forbid uninit primitives, adjust docs to say that's UB For context: rust-lang/rust#66151 (comment) This does not make it a FCW, but it does explicitly state in the docs that uninit integers are UB. This also doesn't affect any runtime behavior, uninit u32's will still successfully be created through mem::uninitialized.
mem::uninitialized: mitigate many incorrect uses of this function Alternative to rust-lang/rust#98966: fill memory with `0x01` rather than leaving it uninit. This is definitely bitewise valid for all `bool` and nonnull types, and also those `Option<&T>` that we started putting `noundef` on. However it is still invalid for `char` and some enums, and on references the `dereferenceable` attribute is still violated, so the generated LLVM IR still has UB -- but in fewer cases, and `dereferenceable` is hopefully less likely to cause problems than clearly incorrect range annotations. This can make using `mem::uninitialized` a lot slower, but that function has been deprecated for years and we keep telling everyone to move to `MaybeUninit` because it is basically impossible to use `mem::uninitialized` correctly. For the cases where that hasn't helped (and all the old code out there that nobody will ever update), we can at least mitigate the effect of using this API. Note that this is *not* in any way a stable guarantee -- it is still UB to call `mem::uninitialized::<bool>()`, and Miri will call it out as such. This is somewhat similar to rust-lang/rust#87032, which proposed to make `uninitialized` return a buffer filled with 0x00. However - That PR also proposed to reduce the situations in which we panic, which I don't think we should do at this time. - The 0x01 bit pattern means that nonnull requirements are satisfied, which (due to references) is the most common validity invariant. `@5225225` I hope I am using `cfg(sanitize)` the right way; I was not sure for which ones to test here. Cc rust-lang/rust#66151 Fixes rust-lang/rust#87675
Strengthen invalid_value lint to forbid uninit primitives, adjust docs to say that's UB For context: rust-lang/rust#66151 (comment) This does not make it a FCW, but it does explicitly state in the docs that uninit integers are UB. This also doesn't affect any runtime behavior, uninit u32's will still successfully be created through mem::uninitialized.
Strengthen invalid_value lint to forbid uninit primitives, adjust docs to say that's UB For context: rust-lang/rust#66151 (comment) This does not make it a FCW, but it does explicitly state in the docs that uninit integers are UB. This also doesn't affect any runtime behavior, uninit u32's will still successfully be created through mem::uninitialized.
Strengthen invalid_value lint to forbid uninit primitives, adjust docs to say that's UB For context: rust-lang/rust#66151 (comment) This does not make it a FCW, but it does explicitly state in the docs that uninit integers are UB. This also doesn't affect any runtime behavior, uninit u32's will still successfully be created through mem::uninitialized.
With #66059,
mem::uninitialized
andmem::zeroed
dynamically detect some misuses (such asmem::zeroed::<&T>()
ormem::uninitialized::<bool>()
) and panic instead of causing UB. Also see this summary for the original FCP. But the check is conservative for now to reduce breakage. This is to track strengthening the check.MaybeUninit
and thus triggers this check (if the type inside the array has invalid bit patterns). Other widely-used crates that trigger the panic as of Dec 2021:Variant::Multiple
enums, which currently we do not just to stay conservative.The text was updated successfully, but these errors were encountered: