-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Guarantee representation of None in NPO #115333
Conversation
(rustbot has picked a reviewer for you, use r? to override) |
IMO it may not be sound due to niche-filling.
|
Would this be categorized as a null-pointer optimization/included in the list of types in this doc section? Presumably if we added a type like that we'd just update the docs to make sure the sentence I'm adding doesn't apply to it, right? |
Oh yeah for the listed types I believe |
I am not sure what the status of "alignment niches" is, but I think that this guarantee could interfere with a case where |
Nominating for lang team discussion. QuestionShould it be guaranteed that the option null pointer optimization is actually a null pointer optimization, choosing the null pointer (0) to represent |
My understanding is that a big selling point of these optimizations is that they make Option FFI-compatible if you assume that None is represented as 0. While I agree that it's technically not the guaranteed behavior today, I wonder how much code is already relying on this. I don't have a citation for it, but I remember folks being excited about the implications for FFI when this was first introduced. |
Do we need to note nested niches? E.g. |
|
https://github.com/rust-lang/rfcs/blob/master/text/3391-result_ffi_guarantees.md has an RFC. Does this also need one? |
library/core/src/option.rs
Outdated
@@ -142,6 +142,9 @@ | |||
//! from `Some::<T>(_)` to `T` (but transmuting `None::<T>` to `T` | |||
//! is undefined behaviour). | |||
//! | |||
//! Finally, it is guaranteed that, for the cases above, `transmute::<_, Option<T>>([0u8; size_of::<T>()])` | |||
//! is sound and produces `Option::<T>::None`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For some of the types, this is actually documented on the inner types instead, like https://doc.rust-lang.org/nightly/std/num/struct.NonZeroU32.html#layout-1
This seems like an obvious yes for NonNull
and &
and &mut
, but I don't know that writing the guarantee here is the right way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NonNull
and NonZeroX
types are an obvious yes by nature of "there's one value that we have to represent and only one bit pattern for it". I think that adding the documentation for those is somewhat recent.
While I think that it's a valid concern to avoid documenting things multiple times, I think that it's important that there's something here. This is the documentation about the NPO generally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have an accepted RFC that this optimization applies not only to Option
but also some cases of Result
. So medium-term Option
isn't really the right place to put this. We'll have to find some central place describing the idea of null-pointer opt, and then each individual involved type should reference that central place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm afraid this might make it sound like we guarantee None is always all zeros, even for Option<char>
or Option<ascii::Char>
, which are actually nonzero.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we guarantee that None::<&T>
and None::<NonNull>
are always represented as all zeros? Could we ever support platforms where NULL isn't 0?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have an accepted RFC that this optimization applies not only to
Option
but also some cases ofResult
. So medium-termOption
isn't really the right place to put this. We'll have to find some central place describing the idea of null-pointer opt, and then each individual involved type should reference that central place.
Is there a harm in putting this here until the Result
optimization is implemented, at which point we can move the docs to a central location?
I'm afraid this might make it sound like we guarantee None is always all zeros, even for
Option<char>
orOption<ascii::Char>
, which are actually nonzero.
Would it help if we explicitly called out that this guarantee does not apply to all types, and maybe name these as examples?
Can we guarantee that
None::<&T>
andNone::<NonNull>
are always represented as all zeros? Could we ever support platforms where NULL isn't 0?
This is a question above my pay grade, but I'd expect ~everything to break if we tried to do this 😛
On a more serious note, if we did this, what would the NPO look like? Would we expect to make guarantees about what value constitutes NULL, or would we just guarantee that some value constituted NULL, and so we could make guarantees about the size of Option<T>
? Basically, I'm wondering if we could still write down the X
in transmute::<_, Option<T>>(X)
, or if we just can't make any guarantees at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a harm in putting this here until the Result optimization is implemented, at which point we can move the docs to a central location?
No, it's fine having this all in Option
for now.
Could we ever support platforms where NULL isn't 0?
I think right now the general thinking is that no, there is no intent to support such a platform.
@rfcbot fcp merge |
Team member @tmandry has proposed to merge this. The next step is review by the rest of the tagged team members: Concerns:
Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns. |
@rfcbot reviewed -- I'm in favor of making guarantees here! No strong opinion about whether this is the right place to write the doc. |
The final comment period, with a disposition to merge, as per the review above, is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. This will be merged soon. |
Is there anything we need to do to get this unstuck, or is it the expected behavior for it to take a few weeks to merge? |
When FCP finishes, it's the job of the reviewer to do the usual kind of review to check that the code/implementation part of the PR is good to land. Reviewers often don't realize they are "on the hook" for this PR though, since they might have forgotten that they are the assigned reviewer and the FCP bot message doesn't remind them. (I'll file an rfcbot feature request, would be good to ping the assigned reviewer to get this back into their queue.) In this case there's no code change so we can just land this. |
☀️ Test successful - checks-actions |
Finished benchmarking commit (1bb6553): comparison URL. Overall result: ❌ regressions - ACTION NEEDEDNext Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression Warning ⚠: The following benchmark(s) failed to build:
cc @rust-lang/wg-compiler-performance Instruction countThis is a highly reliable metric that was used to determine the overall result at the top of this comment.
Max RSS (memory usage)ResultsThis is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
CyclesResultsThis is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
Binary sizeThis benchmark run did not return any relevant results for this metric. Bootstrap: 636.863s -> 635.435s (-0.22%) |
Not sure if I have the permission to do this, and not sure if I'm getting the syntax right, but... @rustbot label: +perf-regression-triaged This is a docs change |
It looks more like an issue during the collection process, with benchmarks timing out. We’ll keep an eye on this for the following PRs, but this PR is especially unlikely to have caused this. |
I've submitted #117591 to follow up on this. |
84: Automated pull from upstream `master` r=Dajamante a=github-actions[bot] This PR pulls the following changes from the upstream repository: * rust-lang/rust#117585 * rust-lang/rust#117576 * rust-lang/rust#96979 * rust-lang/rust#117191 * rust-lang/rust#117179 * rust-lang/rust#117574 * rust-lang/rust#117537 * rust-lang/rust#117608 * rust-lang/rust#117596 * rust-lang/rust#117588 * rust-lang/rust#117524 * rust-lang/rust#116017 * rust-lang/rust#117504 * rust-lang/rust#117469 * rust-lang/rust#116218 * rust-lang/rust#117589 * rust-lang/rust#117581 * rust-lang/rust#117503 * rust-lang/rust#117590 * rust-lang/rust#117583 * rust-lang/rust#117570 * rust-lang/rust#117562 * rust-lang/rust#117534 * rust-lang/rust#116894 * rust-lang/rust#110340 * rust-lang/rust#113343 * rust-lang/rust#117579 * rust-lang/rust#117094 * rust-lang/rust#117566 * rust-lang/rust#117564 * rust-lang/rust#117554 * rust-lang/rust#117550 * rust-lang/rust#117343 * rust-lang/rust#115274 * rust-lang/rust#117540 * rust-lang/rust#116412 * rust-lang/rust#115333 * rust-lang/rust#117507 * rust-lang/rust#117538 * rust-lang/rust#117533 * rust-lang/rust#117523 * rust-lang/rust#117520 * rust-lang/rust#117505 * rust-lang/rust#117434 * rust-lang/rust#117535 * rust-lang/rust#117510 * rust-lang/rust#116439 * rust-lang/rust#117508 Co-authored-by: Ben Wiederhake <BenWiederhake.GitHub@gmx.de> Co-authored-by: SabrinaJewson <sejewson@gmail.com> Co-authored-by: J-ZhengLi <lizheng135@huawei.com> Co-authored-by: koka <koka.code@gmail.com> Co-authored-by: bjorn3 <17426603+bjorn3@users.noreply.github.com> Co-authored-by: Joshua Liebow-Feeser <joshlf@users.noreply.github.com> Co-authored-by: lengyijun <sjtu5140809011@gmail.com> Co-authored-by: Zalathar <Zalathar@users.noreply.github.com> Co-authored-by: Oli Scherer <git-spam-no-reply9815368754983@oli-obk.de> Co-authored-by: Philipp Krones <hello@philkrones.com> Co-authored-by: y21 <30553356+y21@users.noreply.github.com> Co-authored-by: bors <bors@rust-lang.org> Co-authored-by: bohan <bohan-zhang@foxmail.com>
|
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
Pkgsrc changes: * Adjust patches and cargo checksums to new versions. * For an external LLVM, set dependency of llvm >= 16, in accordance with the upstream changes. * Mark that on NetBSD we now need >= 9.0, so 8.x is no longer supported. * On NetBSD/sparc64 10.x, we now need GCC 12 to build the embedded LLVM, which is version 17; apparently GCC 10.4 or 10.5 mis-compiles it, resulting in an illegal instruction fault during the build. Ref. rust-lang/rust#117231 Upstream changes: Version 1.75.0 (2023-12-28) ========================== - [Stabilize `async fn` and return-position `impl Trait` in traits.] (rust-lang/rust#115822) - [Allow function pointer signatures containing `&mut T` in `const` contexts.] (rust-lang/rust#116015) - [Match `usize`/`isize` exhaustively with half-open ranges.] (rust-lang/rust#116692) - [Guarantee that `char` has the same size and alignment as `u32`.] (rust-lang/rust#116894) - [Document that the null pointer has the 0 address.] (rust-lang/rust#116988) - [Allow partially moved values in `match`.] (rust-lang/rust#103208) - [Add notes about non-compliant FP behavior on 32bit x86 targets.] (rust-lang/rust#113053) - [Stabilize ratified RISC-V target features.] (rust-lang/rust#116485) Compiler -------- - [Rework negative coherence to properly consider impls that only partly overlap.] (rust-lang/rust#112875) - [Bump `COINDUCTIVE_OVERLAP_IN_COHERENCE` to deny, and warn in dependencies.] (rust-lang/rust#116493) - [Consider alias bounds when computing liveness in NLL.] (rust-lang/rust#116733) - [Add the V (vector) extension to the `riscv64-linux-android` target spec.] (rust-lang/rust#116618) - [Automatically enable cross-crate inlining for small functions] (rust-lang/rust#116505) - Add several new tier 3 targets: - [`csky-unknown-linux-gnuabiv2hf`] (rust-lang/rust#117049) - [`i586-unknown-netbsd`] (rust-lang/rust#117170) - [`mipsel-unknown-netbsd`] (rust-lang/rust#117356) Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Override `Waker::clone_from` to avoid cloning `Waker`s unnecessarily.] (rust-lang/rust#96979) - [Implement `BufRead` for `VecDeque<u8>`.] (rust-lang/rust#110604) - [Implement `FusedIterator` for `DecodeUtf16` when the inner iterator does.] (rust-lang/rust#110729) - [Implement `Not, Bit{And,Or}{,Assign}` for IP addresses.] (rust-lang/rust#113747) - [Implement `Default` for `ExitCode`.] (rust-lang/rust#114589) - [Guarantee representation of None in NPO] (rust-lang/rust#115333) - [Document when atomic loads are guaranteed read-only.] (rust-lang/rust#115577) - [Broaden the consequences of recursive TLS initialization.] (rust-lang/rust#116172) - [Windows: Support sub-millisecond sleep.] (rust-lang/rust#116461) - [Fix generic bound of `str::SplitInclusive`'s `DoubleEndedIterator` impl] (rust-lang/rust#100806) - [Fix exit status / wait status on non-Unix `cfg(unix)` platforms.] (rust-lang/rust#115108) Stabilized APIs --------------- - [`Atomic*::from_ptr`] (https://doc.rust-lang.org/stable/core/sync/atomic/struct.AtomicUsize.html#method.from_ptr) - [`FileTimes`] (https://doc.rust-lang.org/stable/std/fs/struct.FileTimes.html) - [`FileTimesExt`] (https://doc.rust-lang.org/stable/std/os/windows/fs/trait.FileTimesExt.html) - [`File::set_modified`] (https://doc.rust-lang.org/stable/std/fs/struct.File.html#method.set_modified) - [`File::set_times`] (https://doc.rust-lang.org/stable/std/fs/struct.File.html#method.set_times) - [`IpAddr::to_canonical`] (https://doc.rust-lang.org/stable/core/net/enum.IpAddr.html#method.to_canonical) - [`Ipv6Addr::to_canonical`] (https://doc.rust-lang.org/stable/core/net/struct.Ipv6Addr.html#method.to_canonical) - [`Option::as_slice`] (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.as_slice) - [`Option::as_mut_slice`] (https://doc.rust-lang.org/stable/core/option/enum.Option.html#method.as_mut_slice) - [`pointer::byte_add`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.byte_add) - [`pointer::byte_offset`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.byte_offset) - [`pointer::byte_offset_from`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.byte_offset_from) - [`pointer::byte_sub`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.byte_sub) - [`pointer::wrapping_byte_add`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.wrapping_byte_add) - [`pointer::wrapping_byte_offset`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.wrapping_byte_offset) - [`pointer::wrapping_byte_sub`] (https://doc.rust-lang.org/stable/core/primitive.pointer.html#method.wrapping_byte_sub) These APIs are now stable in const contexts: - [`Ipv6Addr::to_ipv4_mapped`] (https://doc.rust-lang.org/stable/core/net/struct.Ipv6Addr.html#method.to_ipv4_mapped) - [`MaybeUninit::assume_init_read`] (https://doc.rust-lang.org/stable/core/mem/union.MaybeUninit.html#method.assume_init_read) - [`MaybeUninit::zeroed`] (https://doc.rust-lang.org/stable/core/mem/union.MaybeUninit.html#method.zeroed) - [`mem::discriminant`] (https://doc.rust-lang.org/stable/core/mem/fn.discriminant.html) - [`mem::zeroed`] (https://doc.rust-lang.org/stable/core/mem/fn.zeroed.html) Cargo ----- - [Add new packages to `[workspace.members]` automatically.] (rust-lang/cargo#12779) - [Allow version-less `Cargo.toml` manifests.] (rust-lang/cargo#12786) - [Make browser links out of HTML file paths.] (rust-lang/cargo#12889) Rustdoc ------- - [Accept less invalid Rust in rustdoc.] (rust-lang/rust#117450) - [Document lack of object safety on affected traits.] (rust-lang/rust#113241) - [Hide `#[repr(transparent)]` if it isn't part of the public ABI.] (rust-lang/rust#115439) - [Show enum discriminant if it is a C-like variant.] (rust-lang/rust#116142) Compatibility Notes ------------------- - [FreeBSD targets now require at least version 12.] (rust-lang/rust#114521) - [Formally demote tier 2 MIPS targets to tier 3.] (rust-lang/rust#115238) - [Make misalignment a hard error in `const` contexts.] (rust-lang/rust#115524) - [Fix detecting references to packed unsized fields.] (rust-lang/rust#115583) - [Remove support for compiler plugins.] (rust-lang/rust#116412)
This allows users to soundly transmute zeroes into
Option
types subject to the null pointer optimization (NPO). It unblocks google/zerocopy#293.