go/parser: audit error recovery for go1.17

[findleyr]

With all the changes to go/parser merged from dev.typeparams, at least some error recovery behavior has changed in go 1.17. For example, a[] parses to a[BadExpr] on go1.16, but just BadExpr on go1.17.

Error recovery in the parser is best-effort, but we should audit changes to ensure they are acceptable. We should probably avoid changes such as the example above that lose information.

CC @griesemer

[toothrot]

@findleyr We're close to RC1, and this issue is a release blocker. Any updates?

[findleyr]

Thanks.

This is mostly procedural, though I'm aware of one minor parser change that must be made. I can make sure to close this out by this Monday the 21st, does that work?

[ianlancetaylor]

Sure, next Monday is fine. Thanks.

[gopherbot]

Change https://golang.org/cl/329791 mentions this issue: go/parser: parse an ast.IndexExpr for _[]

[findleyr]

In addition to #46855, and _[], fuzzing has found a number of test cases that differ between 1.16 and 1.17 related to the parsing of function arguments or index expressions. For example:

• package p; func f\xado() {}: func f(o BadExpr, _ BadExpr) on 1.16, func f(o BadExpr) on 1.17
• package p;var x=a[[": a[BadExpr] on 1.16, a[[BadExpr]BadExpr] on 1.17

These seem to be reasonable changes in behavior related to the refactoring of function signature parsing (to allow for type parameter lists), and index expressions (to allow for type instantiation). In the cases I've seen, the 1.17 recovery is generally equivalent to or better than the 1.16 error recovery, though of course this is subjective. It's also hard to filter out the noise of these classes of errors without rewriting the 1.16 parser.

Therefore, I think we should close this bug after CL 329791 is merged. There may be more classes of differences, but I don't have an easy way to detect them.