Pretty-printing/tokenization of final has changed to r#final

[alexcrichton]

Given a procedural macro like:

extern crate proc_macro;

use proc_macro::*;

#[proc_macro_attribute]
pub fn foo(a: TokenStream, b: TokenStream) -> TokenStream {
    assert_eq!(a.to_string(), "final");
    return b.into_iter().collect();
}

and a consumer like:

#[foo::foo(final)]
fn main() {
    #[foo::foo(final)]
    fn foo() {}
}

This previously compiled and succeeded on nightly-2019-07-10 but fails on nightly-2017-07-11. Bisection yields #62555 as the culprit of which #62393 is the likely true culprit.

When running if you set RUSTC_LOG=syntax you get:

 INFO 2019-07-12T16:54:49Z: syntax::parse::token: cached tokens found, but they're not "probably equal", going with stringified version

which I believe means that the compiler is pretty-printing the identifier in macros as r#final instead of final.

For reference this is a pretty minimized scenario but it has broken wasm-bindgen's test suite (which uses #[wasm_bindgen(final)] throughout). @petrochenkov is this intentional? Is this something that should be fixed or "the macro parser needs to be update"?

(I'm gonna update the parser anyway but wanted to be sure to file an issue as well)

[Mark-Simulacrum]

I think we probably threaded through a is_raw_guess somewhere we didn't before. I think it's this code: https://github.com/rust-lang/rust/pull/62393/files#diff-c82811a081b5fb219ba68dd88b49d0b1R670. Previously we'd just directly stringify so r# wasn't added but now we try to guess.

[petrochenkov]

This looks like a bug, I assumed that print_attribute_path is used only for the attribute path foo::foo, but not for the attribute arguments (final) which are treated as tokens, but somehow it gets to the arguments as well, which is not right.

[petrochenkov]

I'll look into more detail today.
(Perhaps some attributes go through a conversion to MetaItem and back unnecessarily?)

[petrochenkov]

Perhaps some attributes go through a conversion to MetaItem and back unnecessarily?

This is indeed the case and it's trivially fixable, but the trivial fix regresses formatting of the pretty-printed code.
I'm currently trying to pick the right combination of iboxes and cboxes for the things to look good again, will submit a PR soon.

[petrochenkov]

Fixed in #62667.

[alexcrichton]

Thanks for the quick fix @petrochenkov!