Const generics

[withoutboats]

This is a new const generics RFC which addresses the issues around equality between const variables that were raised in this internals thread.

Rendered

[updated to link to final rendered version]

[eddyb]

My sole nit for this entire RFC is this: "syntactic equality" is, IMO, unactionable.
What I have had in mind is "semantic identity", i.e. what you'd expect from nominal types, where the same node, when used multiple times, unifies with itself.

However, there is another subtlety here: consider unifying two copies of {X / 2}, each with a different inference variable for X. As far as inference is concerned, those variables don't have to be equal. After all, each loses one bit.

cc @nikomatsakis who I believe brought up the same problem with associated types recently.

[withoutboats]

Yea I wasn't sure how to phrase this correctly; what I was trying to get across was that {N + 1} should unify with itself in the same way that T::Assoc does.

[clarfonthey]

I personally would like to avoid the notion of generic equality beyond just X altogether for now. We don't need to add an algebra solver into the compiler, and imo X * 2 and X << 1 and X + X should all be equivalent if we allow this.

[eddyb]

That seems accurate, I wonder how attached @nikomatsakis is to that rule - it's a trade-off.

[eddyb]

@clarcharr There are certainly multiple levels of equivalence we could use.
The worst part IMO is giving more specific results from unification that can be really known.

I do want to eventually treat e.g. {N + 2} and {N + (1 + 1)} as identical, less so have any rules specific to operators or functions, but those future extensions are a bit oit of scope here.

[withoutboats]

@eddyb let's talk out of band about what the right wording for this section is. We're starting out more conservatively than I thought (which is fine with me!).

[gnzlbg]

@withoutboats @eddyb I think that saying that "An expression only unifies with itself" and maybe adding @mark-i-m 's example as a clarification (maybe with some comments) would suffice to make it clear what you exactly mean by "with itself".

EDIT: the RFC still needs to be updated with something like this.

[pnkfelix]

@rfcbot concern expression-unifies-with-itself

Was this thread of discussion ever resolved? On my latest reading (Sept 1), I came away with the impression that two occurrences of the AST {N + 1} (i.e. two different nodes in the AST that both are the subtree {N + 1}) would be considered equal to each other and thus [usize; N + 1] would unify with the type of [0_usize; N + 1].

But @eddyb seems to say that contradicts what he wants to see.

[pnkfelix]

(or is @eddyb's sole point merely that he anticipates this being an initial implementation limitation, but not a problem with the fundamental design here... I remain confused...)

[eddyb]

The initial implementation will consider them distinct, but we can start work on unification strategies after we have anything working at all.

[anp]

nit: diverge?

[hdevalence]

nit: is there an extra > here?

[withoutboats]

No, there's just no type name with it, this should be Foo<T, const N: T>.

[withoutboats]

oh I saw the wrong >. Yes!

[withoutboats]

nits:

1. I didn't actually talk about the compilation model in this RFC, should probably specify that these params are monomorphized.
2. Consts as inputs to traits? Unclear to me what the compilation is for that. Monomorphize the traits into distinct traits?

[Gankra]

I'm perfectly fine with shipping with this rule, but can you elaborate on... why? It seems unfortunate that this doesn't work:

fn foo<const X: usize>() {
  const STACK_CAP: usize = X * 2;
  let stack1 = ArrayVec<u32; STACK_CAP>::new();
  let stack2 = ArrayVec<u32; STACK_CAP>::new();
  // ...
}

[withoutboats]

@gankro The same reason this doesn't work:

fn foo<I: Iterator>(iter: I) {
     fn bar(item: I::Item) { }
}

It would make that internal const a kind of secret associated const of the function, rather than its own item. Obviously this could work someday (even the function example I comment here could work someday) but in the name of incrementalism it's a separate feature.

Possibly we could make an exception for consts (not statics, types, or functions) since they have no representation in the compiled binary. cc @eddyb on this one

[Gankra]

Yeah I was only thinking of consts. Since they're basically just named temporaries, it seems totally fine (no weird codegen implications like statics).

[cramertj]

@withoutboats This doesn't work:

fn foo<I: Iterator>(iter: I) -> fn(I::Item) {
     fn bar(item: I::Item) { }
     bar
}
fn bla<I: Iterator>(iter: I) {
    type Bla = I;
}

But this does:

fn foo<I: Iterator>(iter: I) -> fn(I::Item) {
     fn bar<I: Iterator>(item: I::Item) { }
     bar::<I>
}
fn bla<I: Iterator>(iter: I) {
    type Bla<I> = I;
}

So there's already a way to work around it for functions and types. Can you think of a similar way we could make it work for consts and statics? Like @gankro, I think it makes sense for it to "just work" for consts, but I don't know about statics.

[withoutboats]

The same problem applies with type arguments in consts and statics today, this doesn't work and there's no way to make it work:

fn foo<I: Iterator>() {
    const NUL: Option<I::Item> = None;
}

I think solving this is the same for both const params and type params, so its an orthogonal RFC from this one.

[cramertj]

there's no way to make it work

To clarify, you mean that there's no way to do this in the language right now, correct?

[withoutboats]

Yes! Unlike functions you can't thread a parameter into there. (I think the solution is to make consts Just Work and say sorry about statics).

[gnzlbg]

@withoutboats can you update the RFC with this information? I had exactly this same question.

[Gankra]

typo: "which have" has an extra space in the middle

[Gankra]

typo: diverse -> diverge

[Gankra]

Is the structural match property not intended to be exposed as a trait? (why not?)

[Gankra]

typo: an -> a

[hdevalence]

nit: commutativity

[cramertj]

Nit: how does this interact with default type parameters? Can a struct have default type parameters and const parameters? Edit: I ask because default type parameters are required to be listed at the end of the type parameter list.

[withoutboats]

I don't feel like I've ever properly understood this concern. Can't we determine from the kind of the node you put there and (usually) how many arguments you've supplied whether it is intended to be a const or a type?

The only case that seems ambiguous to me is something like this:

struct Foo<T = i32, const N: usize = 0>([T; N]);

fn foo<T, const T: usize>(_: Foo<T>) { }

That is you have both const and type default parameters, and you have an ident which is a name in both type and const context (bad news in general), and you supply it once to the type. I don't particularly care what we do here since its such an edge case (probably treat it as the type parameter).

Am I missing something? Why wouldn't this Just Work?

[cramertj]

Oh it very well may "Just Work." I'm just wondering what the plan would be. I think this looks a little odd, for example, since it results in "skipping" a type parameter:

struct Foo<A, B=i32, const N: usize>(A, [B; N]);

fn foo(x: Foo<String, 4>) {...} // The default makes this `Foo<String, i32, 4>`

[withoutboats]

I guess I don't think it looks odd because we elide lifetimes all the time (which is problematic, but not in a way that applies here).

[eddyb]

@withoutboats We can only determine whether an identifier is meant to be a type or a constant by checking what its position is declared as - you can right now have both a type and a const defined/imported with the same name in a scope and it disambiguates just fine.

[gnzlbg]

Why is having a type and a const defined/imported with the same name in a scope useful? That is so confusing when talking about const level values that I have to ask whether it wouldn't be better to deprecate it.

[cramertj]

I don't think we ever resolved this point. @withoutboats @eddyb Have either of you had any ideas since we discussed? I think it's necessary to have a backwards-compatible way to add default type parameters to things that already have const parameters.

[withoutboats]

I added an unresolved question about it.

[eddyb]

Consts as inputs to traits? Unclear to me what the compilation is for that. Monomorphize the traits into distinct traits?

What does it mean to "monomorphize" a trait? If const parameters don't match the behavior of type parameters in some context then something has gone wrong with the semantics.

[withoutboats]

I'm not sure what I was thinking, its the same as multiparameter traits - we just need to create a new instance during trans for every product of types and consts. That is <T as Foo<0>>::foo generates a different from from <T as Foo<1>>::foo, but this is no different from <T as Foo<Type0>>::foo and <T as Foo<Type1>>::foo.

[matthewjasper]

In the cases that don't already make this clear, can a const expression involving a const variable also be used in these contexts? And when are they evaluated, e.g. (when) does

impl<const N: usize> SomeType<N> {
     const M: usize = N + usize::MAX
}

error if N > 0?

[eddyb]

I'd say that if it ends up in a type, it can be considered an implement WF requirement for that type, propagating outwards so if it ends up in a concrete type written/inferred, then there would be an error - but if the error comes from monomorphizing a function, it can only be a warning, as per #1229.

@rust-lang/lang might disagree with me, but I think they'd agree we should specify something in this RFC.

[withoutboats]

When / why should we do something different than whatever we do when a user writes const M: usze = 1 + usize::MAX;?

[cramertj]

@withoutboats That's an ICE right now on nightly, although it does emit a const-eval error first:

error[E0080]: constant evaluation error
 --> <anon>:4:22
  |
4 |     const M: usize = 1 + ::std::usize::MAX;
  |                      ^^^^^^^^^^^^^^^^^^^^^ attempt to add with overflow

error: internal compiler error: /checkout/src/librustc_trans/mir/constant.rs:377: _1 not initialized
 --> <anon>:5:20
  |
5 |     println!("{}", M);
  |                    ^

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

thread 'rustc' panicked at 'Box<Any>', /checkout/src/librustc_errors/lib.rs:376
note: Run with `RUST_BACKTRACE=1` for a backtrace.

[withoutboats]

Unfortunate, but I'm trying to get at what needs to be specified by this RFC (trying to keep it as orthogonal as possible from the const eval system.)

[mark-i-m]

It seems like there's const eval and const eval for type unification. The first I agree is orthogonal, but the second I think should be mentioned in the RFC... for example, when are abstract const expressions evaluated (it looks like monomorphization time right now)? do they use the same mechanisms as normal const eval? when are unification errors discovered by the compiler? how does this change the current unification algorithm?

[eddyb]

when are abstract const expressions evaluated

Associated type projections are the analogy here, so: whenever <T as Trait>::Assoc would retry normalizing itself - failure due to dependence on type/const parameters simply results in the projection (abstract expression for constants) not being replaced.

[mark-i-m]

To be clear, this would work for user-defined types, too, right? As long as they have structural equality? How does this work exactly? Do we just refuse to compile if they use a type that overloads equality? Or is operator overloading irrelevant?

[withoutboats]

The RFC for structural_match should answer your questions I think: https://github.com/rust-lang/rfcs/blob/master/text/1445-restrict-constants-in-patterns.md

[mark-i-m]

So arrays are introduced as magic at first?

[withoutboats]

The exact documentation might foremention that you can define your own types with const parameters, but we should avoid bogging users down in a deep understanding of this (or any) feature.

[gnzlbg]

So arrays are introduced as magic at first?

I don't recall what the book says when arrays are introduced, does it say that user defined types can also be parametrized by types? If yes, we should add "and values". Otherwise, I don't see the need.

[petrochenkov]

Minor detail: a resolution ambiguity case is possible:

type X = u8;
const X: u8 = 0;
let _: RectangularArray<i32, X, 0>; // Is `X` a type or a constant?

This needs to be disambiguated in favor of type X for backward compatibility.

(I'm personally mildly against supporting this convenience in the initial implementation, until some experience is gained about how bad RectangularArray<i32, {X}, 0> turns out to be in practice.)

[eddyb]

I believe we can look at the definition to know what to expect from a parameter position.

[withoutboats]

Two things:

1. We could consider disambiguating on the basis of the kinds of the params before falling back to assuming its a type, this is discussed in another comment thread. (Not saying we should, I'm uncertain; there are definitely cons to doing this).
2. Allowing identity expressions is not intended as a convenience per se but to distinguish them visually from the kinds of const expressions we have to treat as projections. Its primarily pedagogical to help users understand when they can expect certain unification results and when they can't.

[petrochenkov]

@eddyb

I believe we can look at the definition to know what to expect from a parameter position.

Not in general case.

type X = u8;
const X: u8 = 0;
Type::method<X>; // We can't look at the definition of `method`, it's only available during type checking.
value.method::<X>(); // Same here.

(I don't think just disambiguating in favor of type X will ever cause problems in practice.)

[withoutboats]

(I don't think just disambiguating in favor of type X will ever cause problems in practice.)

True!

[gnzlbg]

Having a constant and a type with the same identifier is extremely confusing, and even more so if constants can be "types". Why can't this be deprecated?

[withoutboats]

I also think it is confusing. Some projects (including the compiler at one point!) take advantage of these two namespaces to create functions with the same names as types to get "constructor syntax." I don't think this is a good idea, and I would be in favor of warning on it, but that's separate from this RFC probably.

[rkjnsn]

Am I remembering correctly that struct Foo; puts Foo in both namespaces?

[petrochenkov]

Am I remembering correctly that struct Foo; puts Foo in both namespaces?

That't true. Also struct Foo(u8, u8);.
So, the namespace separation is used all the time and cannot be deprecated, this is misunderstanding from the previous commenters.

[rkjnsn]

Fortunately, defaulting to the type when there's an ambiguity seems reasonable in both this case and the "constructor syntax" case.

[mark-i-m]

Also, where does MIRI fit into this? Is it just that when MIRI comes around consts will suddenly gain a lot of functionality?

[petrochenkov]

Does this include reference types (&T/&mut T)?
They are supported in patterns and use semantic equality (reference targets are compared, not addresses themselves), despite being bitwise comparable ("structural match") as well.

[petrochenkov]

C++ supports reference const parameters and uses "structural" bitwise comparison to unify them.
http://coliru.stacked-crooked.com/a/3301c82ba77a2f32

[eddyb]

"structural match" for references isn't pointer equality though, they're considered to be equivalent to a newtype for that purpose.

[withoutboats]

I believe we should get whatever the match semantics over, so it will compare the targets. Seems very important to be certain we unify two identical string literals even if for whatever reason they are allocated separately in rodata.

[withoutboats]

Also, where does MIRI fit into this? Is it just that when MIRI comes around consts will suddenly gain a lot of functionality?

Yes, MIRI is orthogonal. The RFC has a comment in it which says that for the sake of this RFC we just assume integer arithmetic works; we're drawing a distinction between the range of expressions that can be evaluated at compile time (MIRI and const fn) and making types depend on constants (const generics).

[est31]

Note: not just the standard library itself has this limitation, but also libraries like serde have it.

[est31]

👍 🎉 Big support for this RFC and that it tries to not solve every issue related to const generics, but instead goes the slim but faster path, by avoiding to do anything about orthogonal issues.

I too would like to see many of the proposed extensions happen (like unifying {1+N} with {N+1}), but if they were added with this RFC it would increase chance for this RFC to be merged at a later time, and the RFC to hang in unstable limbo for longer, because of some little edge case or because it would require MIRI. I rather have the main functionality merged now, and stabilized fast, and the orthogonal issues looked at separately. Therefore, I'm very glad of the direction this RFC is going!

❤️ Many thanks @withoutboats for championing this RFC!

[clarfonthey]

Question: does N - 1 unify with usize::MAX?

[mark-i-m]

@clarcharr where do you see that?