Move arg/constraint partition check to validation & improve recovery #70261

Centril · 2020-03-22T05:19:15Z

In the first commit, we move the check rejecting e.g., <'a, Item = u8, String> from the parser into AST validation.
We then use this to improve the code for parsing generic arguments.
And we add recovery for e.g., <Item = > (missing), <Item = 42> (constant), and <Item = 'a> (lifetime).

This is also preparatory work for supporting #70256.

petrochenkov · 2020-03-22T09:53:24Z

src/librustc_ast/ast.rs

+    /// Argument for a generic parameter.
+    Arg(GenericArg),
+    /// Constraint for an associated item.
+    Constraint(AssocTyConstraint),


At AST/syntax level this should be a variant of GenericArg, IMO.
It would make everything simpler and would help to avoid all those "angle_args" and the terminology duplication.

Only after lowering to HIR the semantic difference between ty/const/lt args and constraints becomes important, before that they are just components of a single list with different grammars.

Putting naming aside, I'm not sure this would simplify things, at least not in terms of parsing (maybe some of the visitors, but that doesn't seem notable). In particular, I would like to have one way of parsing Lifetime | ConstArg | Type and then use that to the right of ident = . So I think the ideal grammar here would be:

TermArg = Lifetime | ConstArg | Type ; GenericArg = TermArg | Ident "=" TermArg | Ident ":" Bounds ;

It's true that we don't have associated lifetimes, but supporting Foo<Bar = CONST> means we're one variant out. At least in the parser, I would like to have this 3-variant enum equivalent to TermArg so that it can be reused for AssocItem = TermArg for recovery purposes and to support #70256. That could also be achieved with unreachable!() for ::Constraint in your setup, but that doesn't seem ideal?

varkor

Overall, this looks good. I just have a bit of feedback about the diagnostics, and a couple of nits with the code.

src/librustc_ast_lowering/path.rs

src/librustc_ast_passes/ast_validation.rs

src/test/ui/suggestions/suggest-move-types.stderr

src/test/ui/parser/recover-assoc-const-constraint.stderr

varkor · 2020-03-26T21:13:07Z

src/test/ui/parser/recover-assoc-eq-missing-term.stderr

+  --> $DIR/recover-assoc-eq-missing-term.rs:3:11
+   |
+LL |     bar::<Item = >();
+   |           ^^^^^^


Maybe the right span is immediately the right of the =?

varkor · 2020-03-26T21:13:28Z

src/test/ui/parser/recover-assoc-eq-missing-term.stderr

+help: remove the `=` if `Item` is a type
+   |
+LL |     bar::<Item  >();
+   |               --


Can we remove the whitespace too?

Did the best I could, almost there, but not quite. Span hackery is deeply mysterious.

Don't go overboard with span wrangling. It's finicky and ICE prone. We can rely on rustfmt for the last 20%.

Yeah, @estebank I think that's good advice :)

Centril · 2020-03-27T08:33:42Z

@varkor Addressed the comments. :)

varkor · 2020-03-27T12:13:13Z

src/test/ui/suggestions/suggest-move-types.stderr

-   |                                                     |     |     this associated type binding should be moved after the generic parameters
-   |                                                     |     this associated type binding should be moved after the generic parameters
-   |                                                     this associated type binding should be moved after the generic parameters
+   |                                                     ----              ^  ^  ^  ^^  ^^  ^^


Do we not still want the label here?

I thought you wanted to avoid the long label repeated on each generic argument. I'll add a shorter one.

Added a shorter label now.

Sorry, I meant I thought we should have one long label (the original one), but the spans it should be associated with are all the spans of the generic parameters that need to be reordered. That way we don't have many duplicate lines, but we still point all the generic parameters out, with a helpful message.

I don't think that works out visually. If we have A, B = u8, C, D = u8, E, then you would get a span underlining D as well, and that wouldn't really visually separate D from the pack.

Also, I don't suspect that the common case will have more than 1 constraint and one generic argument, so I think the diagnostic as is optimizes well for the common case.

cc @estebank: this seems like a good place to be able to use MultiSpan with span_label.

For these kind of cases I usually end up giving a label to the last of the spans in a MultiSpan, but we could encode that for span_label<S: Into<MultiSpan>>(&self, span: S, label: &str) in the API.

I would probably add a label along these lines

error: generic arguments must come before the first constraint --> $DIR/suggest-move-types.rs:48:71 | LL | struct Bl<'a, 'b, 'c, T, U, V, M: ThreeWithLifetime<A=(), B=(), C=(), T, U, V, 'a, 'b, 'c>> { | ---- ^ ^ ^ ^^ ^^ ^^ the constraint must come after these generic parameters | | | the first constraint is provided here

Also, can we not catch all of A, B and C, like we used to?

@estebank: this is exactly the sort of thing I was thinking 👍

Also, can we not catch all of A, B and C, like we used to?

I suppose we can, but @Centril thought that it's only the first one that's really relevant for repositioning.

Centril · 2020-03-27T23:04:51Z

Per private discussions on Discord, @bors r=varkor

bors · 2020-03-27T23:04:52Z

📌 Commit 42c5cfd has been approved by varkor

estebank · 2020-03-28T02:20:24Z

src/librustc_ast_passes/ast_validation.rs

+        let mut misplaced_args = Vec::new();
+        let mut first = None;
+        for arg in &data.args {
+            match (arg, first) {
+                (AngleBracketedArg::Arg(a), Some(_)) => misplaced_args.push(a.span()),
+                (AngleBracketedArg::Constraint(c), None) => first = Some(c.span),
+                (AngleBracketedArg::Arg(_), None) | (AngleBracketedArg::Constraint(_), Some(_)) => {
+                }
+            }
+        }
+        // ...and then error:
+        self.err_handler()
+            .struct_span_err(
+                misplaced_args.clone(),
+                "generic arguments must come before the first constraint",
+            )
+            .span_label(first.unwrap(), "the first constraint is provided here")
+            .span_labels(misplaced_args, "generic argument")
+            .emit();
+    }


What about?

Suggested change

let mut misplaced_args = Vec::new();

let mut first = None;

for arg in &data.args {

match (arg, first) {

(AngleBracketedArg::Arg(a), Some(_)) => misplaced_args.push(a.span()),

(AngleBracketedArg::Constraint(c), None) => first = Some(c.span),

(AngleBracketedArg::Arg(_), None) | (AngleBracketedArg::Constraint(_), Some(_)) => {

}

}

}

// ...and then error:

self.err_handler()

.struct_span_err(

misplaced_args.clone(),

"generic arguments must come before the first constraint",

)

.span_label(first.unwrap(), "the first constraint is provided here")

.span_labels(misplaced_args, "generic argument")

.emit();

}

let constraint_spans = data.args.iter().filter_map(|arg| match arg {

AngleBracketedArg::Constraint(c) => Some(c.span),

_ => None,

}).collect::<Vec<_>>();

let arg_spans = data.args.iter().filter_map(|arg| match arg {

AngleBracketedArg::Arg(a) => Some(a.span()),

_ => None,

}).collect::<Vec<_>>();

let constraint_len = constraint_spans.len();

// ...and then error:

self.err_handler()

.struct_span_err(

arg_spans.clone(),

"generic arguments must come before the first constraint",

)

.span_labels(constraint_spans, &format!(

"the constraint{} {} provided here",

pluralize!(constraint_len),

if constraint_len == 1 {

"is"

} else {

"are"

}

))

.span_labels(arg_spans, "generic argument")

.emit();

}

Also, we don't provide a suggestion, but we could 😇

varkor · 2020-03-28T02:50:35Z

@estebank: happy for you to r- if you'd like these changes to be made now. (Though I can't speak for @Centril 😅)

bors · 2020-03-28T11:13:15Z

⌛ Testing commit 42c5cfd with merge b9d5ee5...

bors · 2020-03-28T14:15:21Z

☀️ Test successful - checks-azure
Approved by: varkor
Pushing b9d5ee5 to master...

Generic param order restriction was changed in rust-lang/rust#58191. Generic argument order restriction was changed in rust-lang/rust#70261. `for` lifetime argument restriction was changed in rust-lang/rust#48326. Generic parameter parsing: https://github.com/rust-lang/rust/blob/206ee1eea3467fd1d7f1efdbeafe27880897bb2c/compiler/rustc_parse/src/parser/generics.rs#L83-L153 Generic argument parsing: https://github.com/rust-lang/rust/blob/17eec1433c69972844dd228b5fe801f218e118c3/compiler/rustc_parse/src/parser/path.rs#L395-L413 `for` argument parsing: https://github.com/rust-lang/rust/blob/206ee1eea3467fd1d7f1efdbeafe27880897bb2c/compiler/rustc_parse/src/parser/ty.rs#L724-L736 Fixes rust-lang#785

rust-highfive assigned varkor Mar 22, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 22, 2020

Centril added the F-const_generics `#![feature(const_generics)]` label Mar 22, 2020

This comment has been minimized.

Sign in to view

petrochenkov reviewed Mar 22, 2020

View reviewed changes

Centril force-pushed the angle-args-partition branch from 332069e to de89ce9 Compare March 23, 2020 01:06

This comment has been minimized.

Sign in to view

Centril force-pushed the angle-args-partition branch from de89ce9 to 9275172 Compare March 24, 2020 06:23

varkor suggested changes Mar 26, 2020

View reviewed changes

Centril added 6 commits March 27, 2020 07:39

parse: move constraint/arg restriction to ast_validation.

91194f7

split parse_angle_args into loop / single step

abce488

parse_angle_arg: parse constraints first

aa4999e

extract parse_generic_arg

f91de44

parse: improve recovery for assoc eq constraints.

2972bb3

address some review comments

2ddc359

Centril force-pushed the angle-args-partition branch from 9275172 to 2ddc359 Compare March 27, 2020 08:30

varkor reviewed Mar 27, 2020

View reviewed changes

add the label back but make it shorter

42c5cfd

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 27, 2020

estebank reviewed Mar 28, 2020

View reviewed changes

bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 28, 2020

bors merged commit b9d5ee5 into rust-lang:master Mar 28, 2020

bors mentioned this pull request Mar 28, 2020

Use if let instead of match when only matching a single variant (clippy::single_match) #70326

Merged

Centril deleted the angle-args-partition branch March 28, 2020 14:58

ehuss mentioned this pull request Apr 2, 2020

Update grammar for new genaric arg syntax rust-lang/reference#785

Closed

steffahn mentioned this pull request Apr 23, 2020

Simplify rules for GenericArgs nonterminal and other small improvements and fixes rust-lang/reference#805

Closed

varkor mentioned this pull request Jul 22, 2020

Tracking issue for const generics (RFC 2000) #44580

Closed

16 tasks

ehuss mentioned this pull request Jan 6, 2021

Update grammar for order of parameters/arguments. rust-lang/reference#920

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move arg/constraint partition check to validation & improve recovery #70261

Move arg/constraint partition check to validation & improve recovery #70261

Centril commented Mar 22, 2020

This comment has been minimized.

petrochenkov Mar 22, 2020 •

edited

Loading

Centril Mar 23, 2020 •

edited

Loading

This comment has been minimized.

varkor left a comment

varkor Mar 26, 2020

varkor Mar 26, 2020

Centril Mar 27, 2020

estebank Mar 28, 2020

Centril Mar 28, 2020

Centril commented Mar 27, 2020

varkor Mar 27, 2020

Centril Mar 27, 2020

Centril Mar 27, 2020

varkor Mar 27, 2020

Centril Mar 27, 2020

varkor Mar 27, 2020

estebank Mar 28, 2020

estebank Mar 28, 2020

varkor Mar 28, 2020

Centril commented Mar 27, 2020

bors commented Mar 27, 2020

estebank Mar 28, 2020

estebank Mar 28, 2020

varkor commented Mar 28, 2020

bors commented Mar 28, 2020

bors commented Mar 28, 2020

Move arg/constraint partition check to validation & improve recovery #70261

Move arg/constraint partition check to validation & improve recovery #70261

Conversation

Centril commented Mar 22, 2020

This comment has been minimized.

petrochenkov Mar 22, 2020 • edited Loading

Choose a reason for hiding this comment

Centril Mar 23, 2020 • edited Loading

Choose a reason for hiding this comment

This comment has been minimized.

varkor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Centril commented Mar 27, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Centril commented Mar 27, 2020

bors commented Mar 27, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

varkor commented Mar 28, 2020

bors commented Mar 28, 2020

bors commented Mar 28, 2020

petrochenkov Mar 22, 2020 •

edited

Loading

Centril Mar 23, 2020 •

edited

Loading