macro_rules: Remove NtIdent nonterminal token

[petrochenkov]

It only exists for keeping the second span (the metavariable span), but after #119673 such spans can be kept using a more general mechanism.

So after this PR spans from ident variables are treated as spans from tt variables.
To avoid regressions, it required migrating some diagnostics in the compiler to the new scheme used to account for tt variables.

This PR should also unblock removal of the uninterpolate stuff from the parser.

Continuation of #119673.

[rustbot]

r? @compiler-errors

(rustbot has picked a reviewer for you, use r? to override)

[compiler-errors]

unassigning myself for now since you said it's a WIP, feel free to reassign when there's stuff to review

[rustbot]

Some changes occurred in src/tools/clippy

cc @rust-lang/clippy

[rustbot]

The Miri subtree was changed

cc @rust-lang/miri

[petrochenkov]

• mdg.leema.ea189835dd2d248699449839be4aed54ad7cd059 - bare_trait_objects lint + deny(warnings)
• async-imap-0.9.7 - elided_lifetimes_in_paths lint + deny(rust_2018_idioms)
• all other regressions are spurious

[petrochenkov]

@bors try @rust-timer queue

[bors]

⌛ Trying commit d9aa80b with merge 1243fc5...

[bors]

☀️ Try build successful - checks-actions
Build commit: 1243fc5 (1243fc549f5262f7c1e3a70c7d45790b52531962)

[rust-timer]

Finished benchmarking commit (1243fc5): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.8%	[0.2%, 27.8%]	22
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.7%	[-0.8%, -0.6%]	6
Improvements ✅ (secondary)	-1.2%	[-1.5%, -0.9%]	8
All ❌✅ (primary)	2.1%	[-0.8%, 27.8%]	28

Max RSS (memory usage)

Results (primary 10.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	10.0%	[9.9%, 10.0%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	10.0%	[9.9%, 10.0%]	2

Cycles

Results (primary 22.9%, secondary -2.9%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	22.9%	[8.8%, 35.1%]	3
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.9%	[-3.5%, -2.0%]	5
All ❌✅ (primary)	22.9%	[8.8%, 35.1%]	3

Binary size

Results (primary -0.2%, secondary -0.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.3%, 0.4%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.3%	[-0.4%, -0.0%]	10
Improvements ✅ (secondary)	-0.0%	[-0.0%, -0.0%]	5
All ❌✅ (primary)	-0.2%	[-0.4%, 0.4%]	12

Bootstrap: 698.422s -> 697.818s (-0.09%)
Artifact size: 324.47 MiB -> 324.44 MiB (-0.01%)

[petrochenkov]

So the syn anomaly from the old perf run still reproduces, I'll look what happens.

[petrochenkov]

I don't understand the syn results at all.

The regression only reproduces in the incr-patched: println scenarions.
For some reason adding that println invalidates insane amount of code, and a large part of typeck and codegen needs to be performed anew.
So the perf diff comes from all parts of the compilation because things needs to be recompiled, and not from some specific place because it's slow.

I'm not sure what to do about this, none of the other crates invalidates this much after adding a println.
The second place is html5ever-0.26.0 with 0.94% regression.

@cjgillot maybe you have some idea what can cause so much recompilation (and how to determine the root cause)?

[petrochenkov]

The dependency tracking story for the metavar span table is not clear to me in general.

• Fix (?) dependency tracking for the metavar span table #127218

[petrochenkov]

Besides the issue with incremental invalidation in syn this should be ready.
r? @compiler-errors

I migrated some of the clippy lints to the post-#119673 scheme, but not all of them, I can make an issue for that after this is merged.
I should also try to write some guideline for code insertion spans for suggestions, when which span should be used, and when inserting code is unreliable and may result in errors (in progress).

[cjgillot]

@petrochenkov I tried to look at the logs from rustc_query_system::dep_graph::graph and to the dumped hir-tree.

The main thing that puzzles me is that we appear to have recompilations due to more HIR changing. I suppose that's due to some spans missing their lower_span call when building HIR. That would be consistent with the effect on a incr-patched println test, which shifts many spans one line down.

[alex-semenyuk]

@petrochenkov
From wg-triage. Does it wait for review from @compiler-errors?
Meantime might make sense to rebase it

[compiler-errors]

Ah -- this has been S-waiting-on-author, hence I never saw it. I can review after it has been rebased.

[petrochenkov]

It's still waiting on author.
After starting writing the design text I started thinking that maybe the whole span combining thing needs to be reconsidered.
I didn't have time to work on this in the recent months though.