const-eval: detect more pointers as definitely not-null #133700

RalfJung · 2024-12-01T12:07:30Z

This fixes #133523 by making the scalar_may_be_null check smarter: for instance, an odd offset in any 2-aligned allocation can never be null, even if it is out-of-bounds.

More generally, if an allocation with unknown base address B is aligned to alignment N, and a pointer is at offset X inside that allocation, then we know that (B + X) mod N = B mod N + X mod N = X mod N. Since 0 mod N is definitely 0, if we learn that X mod N is not 0 we can deduce that B + X is not 0.

This is immediately visible on stable, via ptr.is_null() (and, more subtly, by not raising a UB error when such a pointer is used somewhere that a non-null pointer is required). Therefore nominating for @rust-lang/lang.

rustbot · 2024-12-01T12:07:38Z

r? @davidtwco

rustbot has assigned @davidtwco.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2024-12-01T12:07:40Z

Some changes occurred to the CTFE machinery

cc @rust-lang/wg-const-eval

Some changes occurred to the CTFE / Miri interpreter

cc @rust-lang/miri

RalfJung · 2024-12-02T08:54:31Z

r? @lcnr

lcnr · 2024-12-02T09:35:59Z

r=me after lang approval (idk if it needs a full FCP, it is observable by users after all)

tmandry · 2024-12-04T17:06:34Z

@RalfJung @JakobDegen Could you elaborate on the motivation? I agree it would be nice if #133523 compiled but I find myself asking "how clever is clever enough" for these checks. Do you feel comfortable with writing this behavior into the language spec?

nikomatsakis · 2024-12-04T17:07:14Z

Oh dear. @rfcbot cancel

@labels -T-compiler

nikomatsakis · 2024-12-04T17:07:23Z

what am I doing :)

nikomatsakis · 2024-12-04T17:07:35Z

@rustbot labels -T-compiler

@rfcbot fcp cancel

rfcbot · 2024-12-04T17:07:59Z

@nikomatsakis proposal cancelled.

nikomatsakis · 2024-12-04T17:08:02Z

@rfcbot fcp merge

Based on discussion in the lang-team meeting we felt this needed an FCP. We discussed a few points we'd like to see clarified

overall this doesn't seem to be an undue complication on the model itself -- i.e., it's not adding new information into the abstracted form of values that CTFE thinks about, only making better use of the data it already has

but it is still complicating the spec, and it's not obvious when this function (or any other) will be "smart enough", so @tmandry was looking for better motivation than an issue (does this represent a real-world pattern?). The other question came from @pnkfelix who was wondering if the logic could be invalidated by people casting unaligned pointers or doing other things that don't respect alignment.

rfcbot · 2024-12-04T17:08:49Z

Team member @nikomatsakis has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

nikomatsakis · 2024-12-04T17:20:52Z

@rustbot labels +T-compiler

RalfJung · 2025-01-18T07:52:02Z

@bors r=lcnr

bors · 2025-01-18T07:52:04Z

📌 Commit e1dda10 has been approved by lcnr

It is now in the queue for this repository.

bors · 2025-01-18T07:52:05Z

🌲 The tree is currently closed for pull requests below priority 100. This pull request will be tested once the tree is reopened.

…iaskrgr Rollup of 7 pull requests Successful merges: - rust-lang#133700 (const-eval: detect more pointers as definitely not-null) - rust-lang#135290 (Encode constraints that hold at all points as logical edges in location-sensitive polonius) - rust-lang#135478 (Run clippy for rustc_codegen_gcc on CI) - rust-lang#135583 (Move `std::pipe::*` into `std::io`) - rust-lang#135612 (Include x scripts in tarballs) - rust-lang#135624 (ci: mirror buildkit image to ghcr) - rust-lang#135661 (Stabilize `float_next_up_down`) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#133700 - RalfJung:const-non-null, r=lcnr const-eval: detect more pointers as definitely not-null This fixes rust-lang#133523 by making the `scalar_may_be_null` check smarter: for instance, an odd offset in any 2-aligned allocation can never be null, even if it is out-of-bounds. More generally, if an allocation with unknown base address B is aligned to alignment N, and a pointer is at offset X inside that allocation, then we know that `(B + X) mod N = B mod N + X mod N = X mod N`. Since `0 mod N` is definitely 0, if we learn that `X mod N` is *not* 0 we can deduce that `B + X` is not 0. This is immediately visible on stable, via `ptr.is_null()` (and, more subtly, by not raising a UB error when such a pointer is used somewhere that a non-null pointer is required). Therefore nominating for `@rust-lang/lang.`

The "panic in const if CTFE doesn't know the answer" behavior was discussed to be the desired behavior in rust-lang#74939, and is currently how the function actually behaves. I intentionally wrote this documentation to allow for the possibility that a panic might not occur even if the pointer is out of bounds, because of rust-lang#133700 and other potential changes in the future.

Correctly document CTFE behavior of is_null and methods that call is_null. The "panic in const if CTFE doesn't know the answer" behavior was discussed to be the desired behavior in rust-lang#74939, and is currently how the function actually behaves. I intentionally wrote this documentation to allow for the possibility that a panic might not occur even if the pointer is out of bounds, because of rust-lang#133700 and other potential changes in the future. This is beta-nominated since `const fn is_null` stabilization is in beta already but the docs there are wrong, and it seems better to have the docs be correct at the time of stabilization.

Upstream changes relative to 1.85.1: Version 1.86.0 (2025-04-03) ========================== Language -------- - [Stabilize upcasting trait objects to supertraits.] (rust-lang/rust#134367) - [Allow safe functions to be marked with the `#[target_feature]` attribute.] (rust-lang/rust#134090) - [The `missing_abi` lint now warns-by-default.] (rust-lang/rust#132397) - Rust now lints about double negations, to catch cases that might have intended to be a prefix decrement operator (`--x`) as written in other languages. This was previously a clippy lint, `clippy::double_neg`, and is [now available directly in Rust as `double_negations`.] (rust-lang/rust#126604) - [More pointers are now detected as definitely not-null based on their alignment in const eval.] (rust-lang/rust#133700) - [Empty `repr()` attribute applied to invalid items are now correctly rejected.] (rust-lang/rust#133925) - [Inner attributes `#![test]` and `#![rustfmt::skip]` are no longer accepted in more places than intended.] (rust-lang/rust#134276) Compiler -------- - [Debug-assert that raw pointers are non-null on access.] (rust-lang/rust#134424) - [Change `-O` to mean `-C opt-level=3` instead of `-C opt-level=2` to match Cargo's defaults.] (rust-lang/rust#135439) - [Fix emission of `overflowing_literals` under certain macro environments.] (rust-lang/rust#136393) Platform Support ---------------- - [Replace `i686-unknown-redox` target with `i586-unknown-redox`.] (rust-lang/rust#136698) - [Increase baseline CPU of `i686-unknown-hurd-gnu` to Pentium 4.] (rust-lang/rust#136700) - New tier 3 targets: - [`{aarch64-unknown,x86_64-pc}-nto-qnx710_iosock`] (rust-lang/rust#133631). For supporting Neutrino QNX 7.1 with `io-socket` network stack. - [`{aarch64-unknown,x86_64-pc}-nto-qnx800`] (rust-lang/rust#133631). For supporting Neutrino QNX 8.0 (`no_std`-only). - [`{x86_64,i686}-win7-windows-gnu`] (rust-lang/rust#134609). Intended for backwards compatibility with Windows 7. `{x86_64,i686}-win7-windows-msvc` are the Windows MSVC counterparts that already exist as Tier 3 targets. - [`amdgcn-amd-amdhsa`](rust-lang/rust#134740). - [`x86_64-pc-cygwin`](rust-lang/rust#134999). - [`{mips,mipsel}-mti-none-elf`] (rust-lang/rust#135074). Initial bare-metal support. - [`m68k-unknown-none-elf`](rust-lang/rust#135085). - [`armv7a-nuttx-{eabi,eabihf}`, `aarch64-unknown-nuttx`, and `thumbv7a-nuttx-{eabi,eabihf}`] (rust-lang/rust#135757). Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - The type of `FromBytesWithNulError` in `CStr::from_bytes_with_nul(bytes: &[u8]) -> Result<&Self, FromBytesWithNulError>` was [changed from an opaque struct to an enum] (rust-lang/rust#134143), allowing users to examine why the conversion failed. - [Remove `RustcDecodable` and `RustcEncodable`.] (rust-lang/rust#134272) - [Deprecate libtest's `--logfile` option.] (rust-lang/rust#134283) - [On recent versions of Windows, `std::fs::remove_file` will now remove read-only files.] (rust-lang/rust#134679) Stabilized APIs --------------- - [`{float}::next_down`] (https://doc.rust-lang.org/stable/std/primitive.f64.html#method.next_down) - [`{float}::next_up`] (https://doc.rust-lang.org/stable/std/primitive.f64.html#method.next_up) - [`<[_]>::get_disjoint_mut`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.get_disjoint_mut) - [`<[_]>::get_disjoint_unchecked_mut`] (https://doc.rust-lang.org/stable/std/primitive.slice.html#method.get_disjoint_unchecked_mut) - [`slice::GetDisjointMutError`] (https://doc.rust-lang.org/stable/std/slice/enum.GetDisjointMutError.html) - [`HashMap::get_disjoint_mut`] (https://doc.rust-lang.org/std/collections/hash_map/struct.HashMap.html#method.get_disjoint_mut) - [`HashMap::get_disjoint_unchecked_mut`] (https://doc.rust-lang.org/std/collections/hash_map/struct.HashMap.html#method.get_disjoint_unchecked_mut) - [`NonZero::count_ones`] (https://doc.rust-lang.org/stable/std/num/struct.NonZero.html#method.count_ones) - [`Vec::pop_if`] (https://doc.rust-lang.org/std/vec/struct.Vec.html#method.pop_if) - [`sync::Once::wait`] (https://doc.rust-lang.org/stable/std/sync/struct.Once.html#method.wait) - [`sync::Once::wait_force`] (https://doc.rust-lang.org/stable/std/sync/struct.Once.html#method.wait_force) - [`sync::OnceLock::wait`] (https://doc.rust-lang.org/stable/std/sync/struct.OnceLock.html#method.wait) These APIs are now stable in const contexts: - [`hint::black_box`] (https://doc.rust-lang.org/stable/std/hint/fn.black_box.html) - [`io::Cursor::get_mut`] (https://doc.rust-lang.org/stable/std/io/struct.Cursor.html#method.get_mut) - [`io::Cursor::set_position`] (https://doc.rust-lang.org/stable/std/io/struct.Cursor.html#method.set_position) - [`str::is_char_boundary`] (https://doc.rust-lang.org/stable/std/primitive.str.html#method.is_char_boundary) - [`str::split_at`] (https://doc.rust-lang.org/stable/std/primitive.str.html#method.split_at) - [`str::split_at_checked`] (https://doc.rust-lang.org/stable/std/primitive.str.html#method.split_at_checked) - [`str::split_at_mut`] (https://doc.rust-lang.org/stable/std/primitive.str.html#method.split_at_mut) - [`str::split_at_mut_checked`] (https://doc.rust-lang.org/stable/std/primitive.str.html#method.split_at_mut_checked) Cargo ----- - [When merging, replace rather than combine configuration keys that refer to a program path and its arguments.] (rust-lang/cargo#15066) - [Error if both `--package` and `--workspace` are passed but the requested package is missing.] (rust-lang/cargo#15071) This was previously silently ignored, which was considered a bug since missing packages should be reported. - [Deprecate the token argument in `cargo login` to avoid shell history leaks.] (rust-lang/cargo#15057) - [Simplify the implementation of `SourceID` comparisons.] (rust-lang/cargo#14980) This may potentially change behavior if the canonicalized URL compares differently in alternative registries. Rustdoc ----- - [Add a sans-serif font setting.] (rust-lang/rust#133636) Compatibility Notes ------------------- - [The `wasm_c_abi` future compatibility warning is now a hard error.] (rust-lang/rust#133951) Users of `wasm-bindgen` should upgrade to at least version 0.2.89, otherwise compilation will fail. - [Remove long-deprecated no-op attributes `#![no_start]` and `#![crate_id]`.] (rust-lang/rust#134300) - [The future incompatibility lint `cenum_impl_drop_cast` has been made into a hard error.] (rust-lang/rust#135964) This means it is now an error to cast a field-less enum to an integer if the enum implements `Drop`. - [SSE2 is now required for "i686" 32-bit x86 hard-float targets; disabling it causes a warning that will become a hard error eventually.] (rust-lang/rust#137037) To compile for pre-SSE2 32-bit x86, use a "i586" target instead. Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Build the rustc on AArch64 Linux with ThinLTO + PGO.] (rust-lang/rust#133807) The ARM 64-bit compiler (AArch64) on Linux is now optimized with ThinLTO and PGO, similar to the optimizations we have already performed for the x86-64 compiler on Linux. This should make it up to 30% faster.

rustbot assigned davidtwco Dec 1, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Dec 1, 2024

RalfJung mentioned this pull request Dec 1, 2024

False positive const-UB check on NonNull pointer #133523

Closed

RalfJung added T-lang Relevant to the language team, which will review and decide on the PR/issue. I-lang-nominated Nominated for discussion during a lang team meeting. labels Dec 1, 2024

lcnr approved these changes Dec 2, 2024

View reviewed changes

rustbot assigned lcnr and unassigned davidtwco Dec 2, 2024

This comment was marked as outdated.

Sign in to view

rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Dec 4, 2024

rustbot removed the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Dec 4, 2024

rfcbot removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Dec 4, 2024

rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Dec 4, 2024

bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Jan 18, 2025

matthiaskrgr mentioned this pull request Jan 18, 2025

Rollup of 7 pull requests #135682

Merged

bors merged commit b78edd7 into rust-lang:master Jan 18, 2025
6 checks passed

rustbot added this to the 1.86.0 milestone Jan 18, 2025

RalfJung deleted the const-non-null branch January 20, 2025 18:51

cuviper mentioned this pull request Mar 21, 2025

Draft release notes for 1.86.0 #138795

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

const-eval: detect more pointers as definitely not-null #133700

const-eval: detect more pointers as definitely not-null #133700

RalfJung commented Dec 1, 2024

rustbot commented Dec 1, 2024

rustbot commented Dec 1, 2024

RalfJung commented Dec 2, 2024

lcnr commented Dec 2, 2024

tmandry commented Dec 4, 2024

This comment was marked as outdated.

This comment was marked as outdated.

nikomatsakis commented Dec 4, 2024

nikomatsakis commented Dec 4, 2024

nikomatsakis commented Dec 4, 2024

rfcbot commented Dec 4, 2024

nikomatsakis commented Dec 4, 2024

rfcbot commented Dec 4, 2024 •

edited by scottmcm

Loading

nikomatsakis commented Dec 4, 2024

RalfJung commented Jan 18, 2025

bors commented Jan 18, 2025

bors commented Jan 18, 2025

const-eval: detect more pointers as definitely not-null #133700

const-eval: detect more pointers as definitely not-null #133700

Conversation

RalfJung commented Dec 1, 2024

rustbot commented Dec 1, 2024

rustbot commented Dec 1, 2024

RalfJung commented Dec 2, 2024

lcnr commented Dec 2, 2024

tmandry commented Dec 4, 2024

This comment was marked as outdated.

This comment was marked as outdated.

nikomatsakis commented Dec 4, 2024

nikomatsakis commented Dec 4, 2024

nikomatsakis commented Dec 4, 2024

rfcbot commented Dec 4, 2024

nikomatsakis commented Dec 4, 2024

rfcbot commented Dec 4, 2024 • edited by scottmcm Loading

nikomatsakis commented Dec 4, 2024

RalfJung commented Jan 18, 2025

bors commented Jan 18, 2025

bors commented Jan 18, 2025

rfcbot commented Dec 4, 2024 •

edited by scottmcm

Loading