Auto merge of rust-lang#119044 - RalfJung:intern-without-types, r=oli-obk

const-eval interning: get rid of type-driven traversal

This entirely replaces our const-eval interner, i.e. the code that takes the final result of a constant evaluation from the local memory of the const-eval machine to the global `tcx` memory. The main goal of this change is to ensure that we can detect mutable references that sneak into this final value -- this is something we want to reject for `static` and `const`, and while const-checking performs some static analysis to ensure this, I would be much more comfortable stabilizing const_mut_refs if we had a dynamic check that sanitizes the final value. (This is generally the approach we have been using on const-eval: do a static check to give nice errors upfront, and then do a dynamic check to be really sure that the properties we need for soundness, actually hold.)

We can do this now that rust-lang#118324 landed and each pointer comes with a bit (completely independent of its type) storing whether mutation is permitted through this pointer or not.

The new interner is a lot simpler than the old one: previously we did a complete type-driven traversal to determine the mutability of all memory we see, and then a second pass to intern any leftover raw pointers. The new interner simply recursively traverses the allocation holding the final result, and all allocations reachable from it (which can be determined from the raw bytes of the result, without knowing anything about types), and ensures they all get interned. The initial allocation is interned as immutable for `const` and pomoted and non-interior-mutable `static`; all other allocations are interned as immutable for `static`, `const`, and promoted. The main subtlety is justifying that those inner allocations may indeed be interned immutably, i.e., that mutating them later would anyway already be UB:
- for promoteds, we rely on the analysis that does promotion to ensure that this is sound.
- for `const` and `static`, we check that all pointers in the final result that point to things that are new (i.e., part of this const evaluation) are immutable, i.e., were created via `&<expr>` at a non-interior-mutable type. Mutation through immutable pointers is UB so we are free to intern that memory as immutable.

Interning raises an error if it encounters a dangling pointer or a mutable pointer that violates the above rules.

I also extended our type-driven const validity checks to ensure that `&mut T` in the final value of a const points to mutable memory, at least if `T` is not zero-sized. This catches cases of people turning `&i32` into `&mut i32` (which would still be considered a read-only pointer). Similarly, when these checks encounter an `UnsafeCell`, they are checking that it lives in mutable memory. (Both of these only traverse the newly created values; if those point to other consts/promoteds, the check stops there. But that's okay, we don't have to catch all the UB.) I co-developed this with the stricter interner changes but I can split it out into a separate PR if you prefer.

This PR does have the immediate effect of allowing some new code on stable, for instance:
```rust
const CONST_RAW: *const Vec<i32> = &Vec::new() as *const _;
```
Previously that code got rejected since the type-based interner didn't know what to do with that pointer. It's a raw pointer, we cannot trust its type. The new interner does not care about types so it sees no issue with this code; there's an immutable pointer pointing to some read-only memory (storing a `Vec<i32>`), all is good. Accepting this code pretty much commits us to non-type-based interning, but I think that's the better strategy anyway.

This PR also leads to slightly worse error messages when the final value of a const contains a dangling reference. Previously we would complete interning and then the type-based validation would detect this dangling reference and show a nice error saying where in the value (i.e., in which field) the dangling reference is located. However, the new interner cannot distinguish dangling references from dangling raw pointers, so it must throw an error when it encounters either of them. It doesn't have an understanding of the value structure so all it can say is "somewhere in this constant there's a dangling pointer". (Later parts of the compiler don't like dangling pointers/references so we have to reject them either during interning or during validation.) This could potentially be improved by doing validation before interning, but that's a larger change that I have not attempted yet. (It's also subtle since we do want validation to use the final mutability bits of all involved allocations, and currently it is interning that marks a bunch of allocations as immutable -- that would have to still happen before validation.)

`@rust-lang/wg-const-eval` I hope you are okay with this plan. :)
`@rust-lang/lang` paging you in since this accepts new code on stable as explained above. Please let me know if you think FCP is necessary.

Author: bors

compiler/rustc_const_eval/messages.ftl

@@ -46,8 +46,8 @@ const_eval_dangling_int_pointer =
     {$bad_pointer_message}: {$pointer} is a dangling pointer (it has no provenance)
 const_eval_dangling_null_pointer =
     {$bad_pointer_message}: null pointer is a dangling pointer (it has no provenance)
-const_eval_dangling_ptr_in_final = encountered dangling pointer in final constant
 
+const_eval_dangling_ptr_in_final = encountered dangling pointer in final value of {const_eval_intern_kind}
 const_eval_dead_local =
     accessing a dead local variable
 const_eval_dealloc_immutable =
@@ -134,6 +134,14 @@ const_eval_interior_mutable_data_refer =
         This would make multiple uses of a constant to be able to see different values and allow circumventing
         the `Send` and `Sync` requirements for shared mutable data, which is unsound.
 
+const_eval_intern_kind = {$kind ->
+    [static] static
+    [static_mut] mutable static
+    [const] constant
+    [promoted] promoted
+    *[other] {""}
+}
+
 const_eval_invalid_align =
     align has to be a power of 2
 
@@ -205,6 +213,8 @@ const_eval_modified_global =
 const_eval_mut_deref =
     mutation through a reference is not allowed in {const_eval_const_context}s
 
+const_eval_mutable_ptr_in_final = encountered mutable pointer in final value of {const_eval_intern_kind}
+
 const_eval_non_const_fmt_macro_call =
     cannot call non-const formatting macro in {const_eval_const_context}s
 
@@ -327,7 +337,7 @@ const_eval_too_many_caller_args =
 
 const_eval_transient_mut_borrow = mutable references are not allowed in {const_eval_const_context}s
 
-const_eval_transient_mut_borrow_raw = raw mutable references are not allowed in {const_eval_const_context}s
+const_eval_transient_mut_raw = raw mutable pointers are not allowed in {const_eval_const_context}s
 
 const_eval_try_block_from_output_non_const =
     `try` block cannot convert `{$ty}` to the result in {const_eval_const_context}s
@@ -341,21 +351,21 @@ const_eval_unallowed_heap_allocations =
 
 const_eval_unallowed_inline_asm =
     inline assembly is not allowed in {const_eval_const_context}s
-const_eval_unallowed_mutable_refs =
-    mutable references are not allowed in the final value of {const_eval_const_context}s
+const_eval_unallowed_mutable_raw =
+    raw mutable pointers are not allowed in the final value of {const_eval_const_context}s
     .teach_note =
+        References in statics and constants may only refer to immutable values.
+
+
         Statics are shared everywhere, and if they refer to mutable data one might violate memory
         safety since holding multiple mutable references to shared data is not allowed.
 
 
         If you really want global mutable state, try using static mut or a global UnsafeCell.
 
-const_eval_unallowed_mutable_refs_raw =
-    raw mutable references are not allowed in the final value of {const_eval_const_context}s
+const_eval_unallowed_mutable_refs =
+    mutable references are not allowed in the final value of {const_eval_const_context}s
     .teach_note =
-        References in statics and constants may only refer to immutable values.
-
-
         Statics are shared everywhere, and if they refer to mutable data one might violate memory
         safety since holding multiple mutable references to shared data is not allowed.
 
@@ -392,9 +402,6 @@ const_eval_unstable_in_stable =
     .unstable_sugg = if it is not part of the public API, make this function unstably const
     .bypass_sugg = otherwise `#[rustc_allow_const_fn_unstable]` can be used to bypass stability checks
 
-const_eval_unsupported_untyped_pointer = unsupported untyped pointer in constant
-    .note = memory only reachable via raw pointers is not supported
-
 const_eval_unterminated_c_string =
     reading a null-terminated string starting at {$pointer} with no null found before end of allocation
 
@@ -406,7 +413,6 @@ const_eval_upcast_mismatch =
 
 ## The `front_matter`s here refer to either `const_eval_front_matter_invalid_value` or `const_eval_front_matter_invalid_value_with_path`.
 ## (We'd love to sort this differently to make that more clear but tidy won't let us...)
-const_eval_validation_box_to_mut = {$front_matter}: encountered a box pointing to mutable memory in a constant
 const_eval_validation_box_to_static = {$front_matter}: encountered a box pointing to a static variable in a constant
 const_eval_validation_box_to_uninhabited = {$front_matter}: encountered a box pointing to uninhabited type {$ty}
 const_eval_validation_dangling_box_no_provenance = {$front_matter}: encountered a dangling box ({$pointer} has no provenance)
@@ -441,7 +447,8 @@ const_eval_validation_invalid_fn_ptr = {$front_matter}: encountered {$value}, bu
 const_eval_validation_invalid_ref_meta = {$front_matter}: encountered invalid reference metadata: total size is bigger than largest supported object
 const_eval_validation_invalid_ref_slice_meta = {$front_matter}: encountered invalid reference metadata: slice is bigger than largest supported object
 const_eval_validation_invalid_vtable_ptr = {$front_matter}: encountered {$value}, but expected a vtable pointer
-const_eval_validation_mutable_ref_in_const = {$front_matter}: encountered mutable reference in a `const`
+const_eval_validation_mutable_ref_in_const = {$front_matter}: encountered mutable reference in a `const` or `static`
+const_eval_validation_mutable_ref_to_immutable = {$front_matter}: encountered mutable reference or box pointing to read-only memory
 const_eval_validation_never_val = {$front_matter}: encountered a value of the never type `!`
 const_eval_validation_null_box = {$front_matter}: encountered a null box
 const_eval_validation_null_fn_ptr = {$front_matter}: encountered a null function pointer
@@ -451,15 +458,14 @@ const_eval_validation_out_of_range = {$front_matter}: encountered {$value}, but
 const_eval_validation_partial_pointer = {$front_matter}: encountered a partial pointer or a mix of pointers
 const_eval_validation_pointer_as_int = {$front_matter}: encountered a pointer, but {$expected}
 const_eval_validation_ptr_out_of_range = {$front_matter}: encountered a pointer, but expected something that cannot possibly fail to be {$in_range}
-const_eval_validation_ref_to_mut = {$front_matter}: encountered a reference pointing to mutable memory in a constant
 const_eval_validation_ref_to_static = {$front_matter}: encountered a reference pointing to a static variable in a constant
 const_eval_validation_ref_to_uninhabited = {$front_matter}: encountered a reference pointing to uninhabited type {$ty}
 const_eval_validation_unaligned_box = {$front_matter}: encountered an unaligned box (required {$required_bytes} byte alignment but found {$found_bytes})
 const_eval_validation_unaligned_ref = {$front_matter}: encountered an unaligned reference (required {$required_bytes} byte alignment but found {$found_bytes})
 const_eval_validation_uninhabited_enum_variant = {$front_matter}: encountered an uninhabited enum variant
 const_eval_validation_uninhabited_val = {$front_matter}: encountered a value of uninhabited type `{$ty}`
 const_eval_validation_uninit = {$front_matter}: encountered uninitialized memory, but {$expected}
-const_eval_validation_unsafe_cell = {$front_matter}: encountered `UnsafeCell` in a `const`
+const_eval_validation_unsafe_cell = {$front_matter}: encountered `UnsafeCell` in read-only memory
 
 const_eval_write_through_immutable_pointer =
     writing through a pointer that was derived from a shared (immutable) reference

compiler/rustc_const_eval/src/const_eval/eval_queries.rs

@@ -293,6 +293,9 @@ pub fn eval_in_interpreter<'mir, 'tcx>(
     cid: GlobalId<'tcx>,
     is_static: bool,
 ) -> ::rustc_middle::mir::interpret::EvalToAllocationRawResult<'tcx> {
+    // `is_static` just means "in static", it could still be a promoted!
+    debug_assert_eq!(is_static, ecx.tcx.static_mutability(cid.instance.def_id()).is_some());
+
     let res = ecx.load_mir(cid.instance.def, cid.promoted);
     match res.and_then(|body| eval_body_using_ecx(&mut ecx, cid, body)) {
         Err(error) => {
@@ -330,8 +333,7 @@ pub fn eval_in_interpreter<'mir, 'tcx>(
         Ok(mplace) => {
             // Since evaluation had no errors, validate the resulting constant.
             // This is a separate `try` block to provide more targeted error reporting.
-            let validation =
-                const_validate_mplace(&ecx, &mplace, is_static, cid.promoted.is_some());
+            let validation = const_validate_mplace(&ecx, &mplace, cid);
 
             let alloc_id = mplace.ptr().provenance.unwrap().alloc_id();
 
@@ -350,22 +352,26 @@ pub fn eval_in_interpreter<'mir, 'tcx>(
 pub fn const_validate_mplace<'mir, 'tcx>(
     ecx: &InterpCx<'mir, 'tcx, CompileTimeInterpreter<'mir, 'tcx>>,
     mplace: &MPlaceTy<'tcx>,
-    is_static: bool,
-    is_promoted: bool,
+    cid: GlobalId<'tcx>,
 ) -> InterpResult<'tcx> {
     let mut ref_tracking = RefTracking::new(mplace.clone());
     let mut inner = false;
     while let Some((mplace, path)) = ref_tracking.todo.pop() {
-        let mode = if is_static {
-            if is_promoted {
-                // Promoteds in statics are allowed to point to statics.
-                CtfeValidationMode::Const { inner, allow_static_ptrs: true }
-            } else {
-                // a `static`
-                CtfeValidationMode::Regular
+        let mode = match ecx.tcx.static_mutability(cid.instance.def_id()) {
+            Some(_) if cid.promoted.is_some() => {
+                // Promoteds in statics are consts that re allowed to point to statics.
+                CtfeValidationMode::Const {
+                    allow_immutable_unsafe_cell: false,
+                    allow_static_ptrs: true,
+                }
+            }
+            Some(mutbl) => CtfeValidationMode::Static { mutbl }, // a `static`
+            None => {
+                // In normal `const` (not promoted), the outermost allocation is always only copied,
+                // so having `UnsafeCell` in there is okay despite them being in immutable memory.
+                let allow_immutable_unsafe_cell = cid.promoted.is_none() && !inner;
+                CtfeValidationMode::Const { allow_immutable_unsafe_cell, allow_static_ptrs: false }
             }
-        } else {
-            CtfeValidationMode::Const { inner, allow_static_ptrs: false }
         };
         ecx.const_validate_operand(&mplace.into(), path, &mut ref_tracking, mode)?;
         inner = true;

compiler/rustc_const_eval/src/const_eval/machine.rs

@@ -723,7 +723,7 @@ impl<'mir, 'tcx> interpret::Machine<'mir, 'tcx> for CompileTimeInterpreter<'mir,
             && ty.is_freeze(*ecx.tcx, ecx.param_env)
         {
             let place = ecx.ref_to_mplace(val)?;
-            let new_place = place.map_provenance(|p| p.map(CtfeProvenance::as_immutable));
+            let new_place = place.map_provenance(CtfeProvenance::as_immutable);
             Ok(ImmTy::from_immediate(new_place.to_ref(ecx), val.layout))
         } else {
             Ok(val.clone())

compiler/rustc_const_eval/src/errors.rs

@@ -1,3 +1,5 @@
+use std::borrow::Cow;
+
 use rustc_errors::{
     DiagCtxt, DiagnosticArgValue, DiagnosticBuilder, DiagnosticMessage, EmissionGuarantee,
     IntoDiagnostic, Level,
@@ -13,12 +15,24 @@ use rustc_middle::ty::{self, Ty};
 use rustc_span::Span;
 use rustc_target::abi::call::AdjustForForeignAbiError;
 use rustc_target::abi::{Size, WrappingRange};
+use rustc_type_ir::Mutability;
+
+use crate::interpret::InternKind;
 
 #[derive(Diagnostic)]
 #[diag(const_eval_dangling_ptr_in_final)]
 pub(crate) struct DanglingPtrInFinal {
     #[primary_span]
     pub span: Span,
+    pub kind: InternKind,
+}
+
+#[derive(Diagnostic)]
+#[diag(const_eval_mutable_ptr_in_final)]
+pub(crate) struct MutablePtrInFinal {
+    #[primary_span]
+    pub span: Span,
+    pub kind: InternKind,
 }
 
 #[derive(Diagnostic)]
@@ -100,8 +114,8 @@ pub(crate) struct TransientMutBorrowErr {
 }
 
 #[derive(Diagnostic)]
-#[diag(const_eval_transient_mut_borrow_raw, code = "E0658")]
-pub(crate) struct TransientMutBorrowErrRaw {
+#[diag(const_eval_transient_mut_raw, code = "E0658")]
+pub(crate) struct TransientMutRawErr {
     #[primary_span]
     pub span: Span,
     pub kind: ConstContext,
@@ -142,8 +156,8 @@ pub(crate) struct UnallowedMutableRefs {
 }
 
 #[derive(Diagnostic)]
-#[diag(const_eval_unallowed_mutable_refs_raw, code = "E0764")]
-pub(crate) struct UnallowedMutableRefsRaw {
+#[diag(const_eval_unallowed_mutable_raw, code = "E0764")]
+pub(crate) struct UnallowedMutableRaw {
     #[primary_span]
     pub span: Span,
     pub kind: ConstContext,
@@ -194,14 +208,6 @@ pub(crate) struct UnallowedInlineAsm {
     pub kind: ConstContext,
 }
 
-#[derive(Diagnostic)]
-#[diag(const_eval_unsupported_untyped_pointer)]
-#[note]
-pub(crate) struct UnsupportedUntypedPointer {
-    #[primary_span]
-    pub span: Span,
-}
-
 #[derive(Diagnostic)]
 #[diag(const_eval_interior_mutable_data_refer, code = "E0492")]
 pub(crate) struct InteriorMutableDataRefer {
@@ -615,18 +621,16 @@ impl<'tcx> ReportErrorExt for ValidationErrorInfo<'tcx> {
             PtrToStatic { ptr_kind: PointerKind::Box } => const_eval_validation_box_to_static,
             PtrToStatic { ptr_kind: PointerKind::Ref } => const_eval_validation_ref_to_static,
 
-            PtrToMut { ptr_kind: PointerKind::Box } => const_eval_validation_box_to_mut,
-            PtrToMut { ptr_kind: PointerKind::Ref } => const_eval_validation_ref_to_mut,
-
             PointerAsInt { .. } => const_eval_validation_pointer_as_int,
             PartialPointer => const_eval_validation_partial_pointer,
             MutableRefInConst => const_eval_validation_mutable_ref_in_const,
+            MutableRefToImmutable => const_eval_validation_mutable_ref_to_immutable,
             NullFnPtr => const_eval_validation_null_fn_ptr,
             NeverVal => const_eval_validation_never_val,
             NullablePtrOutOfRange { .. } => const_eval_validation_nullable_ptr_out_of_range,
             PtrOutOfRange { .. } => const_eval_validation_ptr_out_of_range,
             OutOfRange { .. } => const_eval_validation_out_of_range,
-            UnsafeCell => const_eval_validation_unsafe_cell,
+            UnsafeCellInImmutable => const_eval_validation_unsafe_cell,
             UninhabitedVal { .. } => const_eval_validation_uninhabited_val,
             InvalidEnumTag { .. } => const_eval_validation_invalid_enum_tag,
             UninhabitedEnumVariant => const_eval_validation_uninhabited_enum_variant,
@@ -772,11 +776,11 @@ impl<'tcx> ReportErrorExt for ValidationErrorInfo<'tcx> {
             }
             NullPtr { .. }
             | PtrToStatic { .. }
-            | PtrToMut { .. }
             | MutableRefInConst
+            | MutableRefToImmutable
             | NullFnPtr
             | NeverVal
-            | UnsafeCell
+            | UnsafeCellInImmutable
             | InvalidMetaSliceTooLarge { .. }
             | InvalidMetaTooLarge { .. }
             | DanglingPtrUseAfterFree { .. }
@@ -905,3 +909,14 @@ impl ReportErrorExt for ResourceExhaustionInfo {
     }
     fn add_args<G: EmissionGuarantee>(self, _: &DiagCtxt, _: &mut DiagnosticBuilder<'_, G>) {}
 }
+
+impl rustc_errors::IntoDiagnosticArg for InternKind {
+    fn into_diagnostic_arg(self) -> DiagnosticArgValue<'static> {
+        DiagnosticArgValue::Str(Cow::Borrowed(match self {
+            InternKind::Static(Mutability::Not) => "static",
+            InternKind::Static(Mutability::Mut) => "static_mut",
+            InternKind::Constant => "const",
+            InternKind::Promoted => "promoted",
+        }))
+    }
+}