🔬 Tracking issue for generic associated types (GAT)

[withoutboats]

This is a tracking issue for generic associated types (rust-lang/rfcs#1598)

Repository

• Detailed tracking information is available at the GAT initiative repository

Unresolved questions:

• GATs: Decide whether to have defaults for where Self: 'a #87479 -- defaults for where Self: 'a
• Proposal: Change syntax of where clauses on type aliases #89122 -- where to put where clauses

Blocking bugs

See also the GATs-blocking tag (the list below should be a superset of that, but its good to double-check).

• lifetime-parametric GATs cause ICEs (with feature(generic_associated_types)) #62521
• Associated type projections don't play well with HRTBs and normalization #30472 No longer blocking
• Pretty printing loses associated type generics and where clauses #67509
• Allow generic associate types in trait paths #67510
• Generic associated types can shadow lifetimes #67512
• Implement type- and const-generic associated types #67513
• GAT's arent typechecked, and can cause UB #68641
• GAT's and broken LLVM #68642
• GAT ICE: use of PlaceRef { local: _0, projection: [] } before def #68643
• GAT ICE: Encountered error Unimplemented selecting `Binder #68644
• GAT ICE: FulfillmentError #68645
• GAT ICE: impl item and trait item have different parameter counts #68648
• GAT ICE: place local already assigned to #68649
• GAT ICE: unsize_thin_ptr: called on bad types #68650
• GAT ICE: codegen_argument #68652
• GAT ICE: unelided lifetime in signature #68653
• GAT ICE: cannot directly store unsized values #68656
• Broken LLVM with GAT #74684
• [nightly] ICE when using GATs with const generics #75415
• ICE when combining GATs and HRTBs #76826
• Internal Compiler Error caused by Generic Associated Types #79768
• Enum variant with GAT field fails to derive Sized #80626 Not GATs specific, not blocking
• Error when inferring a lifetime in return type using GATs #81487
• Bad diagnostics for GATs in trait paths #81801
• defaults on GATs emit an error but function correctlyish #99205

Postponed work

• Trait objects do not work with generic associated types #81823

[nikomatsakis]

Here is a kind of implementation plan I will endeavor to keep updated.

• Step one: add support into the AST and pretty-printing
  • Probably the first step is to start parsing the new forms and to add support for them to the AST.
  • See this comment for more detailed thoughts here.
  • We should be able to write some parsing-only tests and also test the pretty-printer hir
  • When we get to HIR lowering, we can error out if any GAT are present
  • We can also do the feature gate then
• More to come

[nikomatsakis]

Let me start by writing about the AST in more detail. First let's discuss how it works today:

A type Foo: Bar [= Baz]; item in a trait definition is defined by this AST variant. That includes the bounds (Bar) and the (optional) default value Baz. The name is defined in the TraitItem struct.

A type Foo = Bar; item in a trait impl is defined by this AST variant -- that only includes the type Bar, because the Foo etc is defined in the ImplItem struct.

Methods are an interesting case because they can already be made generic. Those generic parameters are declared in the field Generics of the MethodSig structure. This is an instance of the Generics struct.

My take is that the best thing to do would be to "lift" Generics out from methods into the TraitItem (and ImplItem) so that it applies equally to all forms of trait and impl items. For now, we will not support generic constants, I guess, but honestly they probably just fall out from the work we are doing in any case, so they would be a small extension. I think the work will go better if we just plan for them now.

Perhaps a decent first PR would be to just do that change, keeping all other existing functionality the same. That is, we would more Generics into TraitItem (and ImplItem) and out of MethodSig. We would supply an empty Generics for non-methods. We would work through the existing code, plumbing the generics along as needed to make it work.

[sunjay]

@nikomatsakis Cool! Thank you so much! I started experimenting with this last night and I'm proud to say that I found the same places you pointed to in your comment about the AST. 😄 (Given that this was my first time in rustc, I count that as an accomplishment!)

I didn't think to lift generics into TraitItem. My approach was to put Generics into TraitItemKind::Type since that's where the type declaration is stored already. Your approach makes sense too so I'll work on implementing that. Since I am still completely new to this codebase, I'm interested to know what the pitfalls of my approach would be if it were used over the one you suggested. Could you give me some insight into your thought process? 😃

Here's the change I would have made:

pub enum TraitItemKind {
    // Generics aren't supported here yet
    Const(P<Ty>, Option<P<Expr>>),
    // `Generics` is already a field in `MethodSig`
    Method(MethodSig, Option<P<Block>>),
    // Added `Generics` here:
    Type(Generics, TyParamBounds, Option<P<Ty>>),
    Macro(Mac),
}

Edit: Answer by nikomatsakis on Gitter

regarding the pitfalls of putting them in type
I think that could work too
the reason that I was reluctant to do that
is that we really want to do the same things (in theory, at least) for methods and types
and -- as I said -- in principle I see no reason we couldn't do the same things for constants
I think if you just move the Generics to the type variant
that would probably work ok, but if you look about, right now we often have to do "one thing for types/consts, one thing for methods" precisely because they are different
so I suspect the code will just get more uniform
I'm not really sure how it will go to be honest =) -- it might be a pain
but a lot of times having things be more generic than they have to be isn't so bad, because you can insert span_bug! calls in the impossible cases for now (and later we come aoround and patch them up)

[nikomatsakis]

All right! Next step is to extend the parser. Here are a few tips. Let's start with trait items.

This routine parses trait items. We want to extend the case that handles associated types to also parse things like type Foo<....> = ...; (also maybe where-clauses). (The <...> is the "generics" that we just added to the AST.)

Currently it is using parse_ty_param, which basically parses something like T: Foo etc. We'll have to stop doing that, because the grammar for associated type declarations no longer matches that for type parameters. So we'll probably want to add something like parse_trait_item_assoc_ty. This can start as a kind of clone of parse_ty_param(), but then we'll want to modify it to invoke parse_generics() right about here. That routine will parse a generics declaration (<...>) if one is present, otherwise it just returns an empty generics. Then we want to add a call to parse where clauses right here -- you can model that on the call that occurs when parsing methods, note that the result is stored into the generics that we parsed earlier.

Once we've done that, we should be able to add some parsing tests. I would do that by making a directory like src/test/run-pass/rfc1598-generic-associated-types/ and adding files that you expect to successfully parse in there. Right now they won't work right, but that doesn't matter. Just add an empty main function. Then we can also add examples that should not parse into src/test/ui/rfc1598-generic-associated-types/ (see COMPILER_TESTS.md for directions on adding UI tests).

Something else -- we need to feature gate this work at this point, to avoid people using this stuff in stable builds. There are some instructions for adding a feature-gate here on forge (see the final section). We should add a visit_trait_item and visit_impl_item to the visitor in feature_gate.rs; if that item is not a method, but it has a non-empty generics, we can invoke gate_feature_post (example).

[nikomatsakis]

To setup name resolution, I think all we have to do is to get the proper "ribs" in place (the name resolution stuff organizes the sets of names that are in scope into ribs; each rib represents one binding level). e.g. for an impl:

impl<A,B> Foo<B> for Vec<A> {
   fn bar<T,U>(x: ...) { 
       for y in ... {
       }
   }
}

we would have the following ribs:

- <A,B> (from the impl)
   - <T,U> (from the `bar` method's generics)
      - `x` (from the parameter list)
          - `y` (from the let)

In general, modeling things on how methods work is not a bad idea. We might also do a bit of "future proofing" here, I suppose.

Here is the code that brings a method's type parameters into scope (this is for a method defined in a trait):

rust/src/librustc_resolve/lib.rs

Lines 1890 to 1892 in a35a3ab

	let type_parameters =
	HasTypeParameters(&trait_item.generics,
	MethodRibKind(!sig.decl.has_self()));

Whereas for a type defined in a trait, we are hardcoded to add an empty type parameter rib (NoTypeParameters):

rust/src/librustc_resolve/lib.rs

Lines 1897 to 1901 in a35a3ab

	TraitItemKind::Type(..) => {
	this.with_type_parameter_rib(NoTypeParameters, |this| {
	visit::walk_trait_item(this, trait_item)
	});
	}

Now that generics are in place on every trait/impl item, I think we probably want to remove the handling for type and extract the method handling so that it occurs at a higher level. For the items (e.g., const) where there are no generics, then the newly introduced rib ought to be empty and hence harmless (I hope).

Other points of interest:

• where impl item methods are setup
• where type in impls are setup

You get the idea.

@petrochenkov -- sound about right?

[petrochenkov]

@nikomatsakis

sound about right?

Everything looks correct.