Tracking Issue for `const_heap`

[oli-obk]

This is a tracking issue for rust-lang/const-eval#20
The feature gate for the issue is #![feature(const_heap)].
The general design can be found here

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

• Implement Heap allocations in constants const-eval#20 experimentally
• Adjust documentation (see instructions on rustc-dev-guide)
• Stabilization PR (see instructions on rustc-dev-guide)

Unresolved Questions

• Do we want to allow the regular Global allocator? This would give us post-monomorpization errors. More details can be found in https://hackmd.io/h2O2vkj3RimrBTfm9hvZWA#Transient-heap-allocations-via-Global
• If we stabilize this, we fundamentally change the nature of const fn: before this change, a const fn() -> T would inevitably have to always return the same value; after this change, that is no longer the case. With ptr_guaranteed_eq, this can even to some extend be observed during CTFE. So in that sense, this feature is much bigger than anything we ever stabilized for CTFE.
• We can already emulate all this via build.rs scripts or proc macros that generate the appropriate constants/statics and references them. Const heap will "just" make generating static/const allocations more convenient.
• Once we permit references to things that (can) contain interior mutability, it would be possible for users to write a const fn foo() -> &'static Cell<i32>, which could then be used to initialize a const FOO: &'static Cell<i32> which is unsound (and only caught by dynamic checks). The tracking issue for interior mutability behind references is Tracking Issue for const_refs_to_cell #80384
• const_heap feature can be used to leak mutable memory into final value of constant #129233
• ICEs when making self-referential values: dealing with cyclic (self-referential) values with const_heap #143869

Implementation history

• add const_allocate intrinsic #79594 adds the base intrinsic for telling the miri-engine to create a virtual heap allocation
• move interpret::MemoryKind::Heap to const eval #79664 makes sure that code outside of the const evaluator has no knowledge about heap allocations
• add const_make_global; err for const_allocate ptrs if didn't call #143595

[RalfJung]

As discussed here, this feature makes const fn observably impure: let x = f(); (x, x) is no longer always equivalent to (f(), f()). We are at a crossroads for the language here: if we want to support heap allocations in const fn, we have to either introduce some kind of effect system or say goodbye to the idea that const fn denotes a form of purity that would be useful to unsafe code.

[RalfJung]

Btw, given how far-reaching the consequences of this feature are, I wonder if doing this without an RFC is really appropriate. But it could make sense to experiment some more with what can be done for transient vs. non-transient allocations, and how Global comes in, so that we have a better idea of the design space.

@oli-obk something is wrong with "implementation history", it links to the same PR twice.

[oli-obk]

The experimentation that @vn-ki has been doing has exposed various problems that aren't even related to heap stuff itself, but are prerequisites. I believe we should keep on doing this experimentation, just to find out what all the problems are and how a possible implementation could look. Doing this in the abstract has not allowed us to realize what the dependency graph actually is and I will be updating the skill tree to reflect that as we gain more information. Some examples off the top of my head:

• generic types are considered having interior mutability within constants and thus forbid short lived immutable references to them
• trait bounds on methods only work if the bounds are on the method and not if they are on the containing impl block
• the existing alloc code needs support for closures or at least impl const Fn for const fn()
• just needs an impl: use fallible allocations for Allocation to make sure that we report an error instead of crashing rustc with OOM (this can be done on stable via the use of large arrays)

[RalfJung]

The first three points don't seem really related to heap allocations though?

[oli-obk]

They are not, they are prerequisites for Box::new_in and whatever things are called by that.

[RalfJung]

generic types are considered having interior mutability within constants and thus forbid short lived immutable references to them

Is there a separate issue for this? I think this is closely related to some of the other issues that came up over the last year -- where we have a restriction that applies to the final value of all consts, but be unnecessarily apply it even during the course of the computation -- since we have no good way to otherwise ensure that the final value is okay. &mut references have basically the same problem.

[oli-obk]

There is not. I have a local branch that generalizes to all such checks, so we'll be able to just check them on the final value

[RalfJung]

For associated consts, I assume that's a post-monomorphization error then?

(I know this is off-topic here, that's why I asked if there is a better place.^^)