Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Account for missing keyword in fn/struct definition #45997

Merged
merged 10 commits into from
Dec 1, 2017

Conversation

estebank
Copy link
Contributor

Fix #38911.

@estebank estebank changed the title Pub ident Account for missing keyword in fn/struct definition Nov 15, 2017
@estebank
Copy link
Contributor Author

r? @nikomatsakis

@kennytm kennytm added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 15, 2017
@petrochenkov petrochenkov self-assigned this Nov 15, 2017
Copy link
Contributor

@nikomatsakis nikomatsakis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, looking more closely, I think this is overkill. I'll leave a more detailed comment with an alternative shortly.

@nikomatsakis
Copy link
Contributor

Currently, after failing to parse, the parser invokes this commonly used method:

/// This is the fall-through for parsing items.
fn parse_macro_use_or_failure(
&mut self,
attrs: Vec<Attribute> ,
macros_allowed: bool,
attributes_allowed: bool,
lo: Span,
visibility: Visibility
) -> PResult<'a, Option<P<Item>>> {

This is intended to look for a macro use like path! -- but right now it decides it has seen a macro quite early, based only on whether the next token can start a path. Presuming it can, we error if we saw a pub, parse the path, and then expect a !. This is where the error will occur in our bad examples.

I think we should customize what happens when we "expect" this ! but do not find it:

To start, we can look at the path. If it was multipart, like a::b, then I think this could only be a macro-use, perhaps one where the ! was forgotten, and we could diagnose it that way. (Or, at least, we could do this if the next token is (.)

If the path is a single identifier, we can then examine the next character: if it is (, we could suggest a function. If it is {, we could suggest a struct. Or, we could just issue something generic like "expected an item like a fn or a macro use like foo!".

@estebank
Copy link
Contributor Author

It is reasonable to go ahead and do that change, but I also feel there's value in modifying the current PR to keep doing what it is doing and returning Ok with the fn or struct that could be parsed, instead of the Err. That way parsing can continue and present other errors down the line, instead of the current behavior of having a failure parsing a macro and then a failure parsing the opening brace or paren. What do you think?

@nikomatsakis
Copy link
Contributor

@estebank that's a good point. On the other hand, I wonder, could we get a similar effect by just skipping over entire token trees?

I feel like only being able to recover when the input is perfect is perhaps not ideal.

@nikomatsakis
Copy link
Contributor

though man but mismatched braces are the worst sort of parse errors =)

@nikomatsakis
Copy link
Contributor

In any case, I could go either way. But I'd like to see us able to recover even from something like this:

pub foo(x: u32) {
    x->foo(); // whoops, too much C++
}

@petrochenkov
Copy link
Contributor

#41282 already implemented this recovery for trait and impl items, it can be adapted to normal items as well.
Saving/restoring parser state should be avoided if possible.

@petrochenkov petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 18, 2017
@estebank
Copy link
Contributor Author

I have modified the code to not perform the rewind. The outwards behavior now is similar to what it already was (the parser will not let the compiler to continue, but it will try to consume the continuing code in order to identify what it was meant (fn or struct), regardless of code errors (just looks for enclosed (...) and {...} sections).

@kennytm kennytm added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 22, 2017
Copy link
Contributor

@nikomatsakis nikomatsakis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks quite cool. I do have one concern about the recovery condition I'd like to get resolved though.

|
help: add `fn` here to parse `foo` as a public method
|
11 | pub fn foo(s: usize) { bar() }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is pretty slick =)

// Verify wether we have encountered a struct or method definition where the user forgot to
// add the `struct` or `fn` keyword after writing `pub`: `pub S {}`
if visibility == Visibility::Public &&
self.check_ident() &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it possible these days to have paths to macros? e.g., foo::bar!?

cc @jseyfried @petrochenkov

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way the fall back happens, if the code isn't either pub foo ( or pub foo {, the current macro handling will happen. In the case of pub foo::bar this new branch will not be executed. As long as parse_macro_use_or_failure does the right thing with paths, this should be fine :)

@estebank estebank force-pushed the pub-ident branch 3 times, most recently from 610cf72 to 35fbdd4 Compare November 23, 2017 15:31
When encountering `pub ident`, attempt to identify the code that comes
afterwards, wether it is a brace block (assume it is a struct), a paren
list followed by a colon (assume struct) or a paren list followed by a
block (assume a fn). Consume those blocks to avoid any further parser
errors and return a `Placeholder` item in order to allow the parser to
continue. In the case of unenclosed blocks, the behavior is the same as
it is currently: no further errors are processed.
Try to identify the following code in order to provide better
diagnostics, but return the error to bail out early during the parse.
@estebank
Copy link
Contributor Author

@bors try

@bors
Copy link
Contributor

bors commented Nov 25, 2017

⌛ Trying commit f103342 with merge 1026513cc276e201297365f43014ee01b2df9b06...

@estebank
Copy link
Contributor Author

estebank commented Nov 25, 2017

@nikomatsakis this PR is ballooning in size with multiple independent changes that are kind or related but not entirely. There's a very small one liner suggestion underline fix, emitting inappropriate DocComment location error but allow it for the parser to continue parsing the struct, making block parsing errors recoverable, other than the original intention of the PR which was to add a suggestion if the parser encounters pub ident ( or pub ident {.

Because of these changes, in the case of compile-fail-fulldeps/proc-macro/derive-bad.rs (which uses compile-fail-fulldeps/proc-macro/auxiliary/derive-bad.rs) the parsing errors from the derive are emitted and not seen by libproc_macro as a failure, so we get the internal parsing errors, that somebody who can't modify the proc macro can't do anything about, and we don't cause the actual expected message. For the second issue I'll try to find a way to emit it, for the first, is it ok to leak these messages?

As for the first paragraph, should I try to split this in a bunch of different dependent PRs?


Edit: I have fixed the issue for all proc macros now: made it check the error count before and after and emit the appropriate diagnostic if it has increased. This should be resilient to further changes in the parser, although it will leak parsing errors from the proc macro to the user. I feel that this is not a bad thing, even if it was an unintended effect originally.

@petrochenkov petrochenkov removed their assignment Nov 26, 2017
@estebank
Copy link
Contributor Author

@nikomatsakis let me know if the output changes are reasonable or if I should keep working on this PR to get to minimal changes from the status quo.

@nikomatsakis
Copy link
Contributor

@bors r+

@estebank seems good to me.

@bors
Copy link
Contributor

bors commented Nov 30, 2017

📌 Commit cf9283e has been approved by nikomatsakis

@bors
Copy link
Contributor

bors commented Dec 1, 2017

⌛ Testing commit cf9283e with merge d1364a6...

bors added a commit that referenced this pull request Dec 1, 2017
Account for missing keyword in fn/struct definition

Fix #38911.
@bors
Copy link
Contributor

bors commented Dec 1, 2017

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing d1364a6 to master...

@bors bors merged commit cf9283e into rust-lang:master Dec 1, 2017
Centril added a commit to Centril/rust that referenced this pull request Dec 8, 2018
…henkov

Suggest an appropriate token when encountering `pub Ident<'a>`

Fix rust-lang#55403. Follow up to rust-lang#45997.
self.check(&token::OpenDelim(token::Brace))
{
("fn", "method", false)
} else if self.check(&token::Colon) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@estebank sorry for the enormous necropost, but do you know what the idea behind this was? It matches pub foo(stuff): and suggests adding struct, but where does that colon come from? Is it possible this was intended to be token::Semi instead? The behaviour isn't covered by any tests, unfortunately.

@estebank estebank deleted the pub-ident branch November 9, 2023 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants