This commit applies the existing 'extra angle bracket recovery' logic
when parsing fields in struct definitions. This allows us to continue
parsing the struct's fields, avoiding spurious 'missing field' errors in
code that tries to use the struct.

Use back-ticks instead of quotation marks in docs for the block comment
variant of TokenKind.

Also adds back-ticks when referring to the contents of this collection.

Emscripten's SDK recently bumped the version of NodeJS they shipped, but
our Dockerfile for the wasm32 builder hardcoded the version number. This
will cause consistent CI failures once the currently cached image is
rebuilt (either due to a change or due to the cache expiring).

This commit fixes the problem by finding the latest version of NodeJS in
the Emscripten SDK and symlinking it to a "latest" directory, which is
then added to the PATH.

And final part!!!

…erive macros

When a `macro_rules!` macro expands to another `macro_rules!` macro, we
may see `None`-delimited groups in odd places when another crate
deserializes the 'inner' macro. This commit 'unwraps' an outer
`None`-delimited group to avoid breaking existing code.

See rust-lang#73569 (comment)
for more details.

The proper fix is to handle `None`-delimited groups systematically
throughout the parser, but that will require significant work. In the
meantime, this hack lets us fix important hygiene bugs in macros

Rename TypeckTables to TypeckResults.

Originally suggested by @eddyb.

…trait-soundness, r=nikomatsakis

Don't implement Fn* traits for #[target_feature] functions

Closes rust-lang#72012.

expand: Stop using nonterminals for passing tokens to attribute and derive macros

Make one more step towards fully token-based expansion and fix issues described in rust-lang#72545 (comment).

Now `struct S;` is passed to `foo!(struct S;)` and `#[foo] struct S;` in the same way - as a token stream `struct S ;`, rather than a single non-terminal token `NtItem` which is then broken into parts later.

The cost is making pretty-printing of token streams less pretty.
Some of the pretty-printing regressions will be recovered by keeping jointness with each token, which we will need to do anyway.

Unfortunately, this is not exactly the same thing as rust-lang#73102.
One more observable effect is how `$crate` is printed in the attribute input.
Inside `NtItem` was printed as `crate` or `that_crate`, now as a part of a token stream it's printed as `$crate` (there are good reasons for these differences, see rust-lang#62393 and related PRs).
This may break old proc macros (custom derives) written before the main portion of the proc macro API (macros 1.2) was stabilized, those macros did `input.to_string()` and reparsed the result, now that result can contain `$crate` which cannot be reparsed.

So, I think we should do this regardless, but we need to run crater first.
r? @Aaron1011

…wjasper

Provide more information on duplicate lang item error.

This gives some notes on the location of the files where the lang items were loaded from. Some duplicate lang item errors can be a little confusing, and this might help in diagnosing what has happened.

Here's an example when hitting a bug with Cargo's build-std:

```
error: duplicate lang item in crate `core` (which `rustc_std_workspace_core` depends on): `try`.
  |
  = note: the lang item is first defined in crate `core` (which `z10` depends on)
  = note: first definition in `core` loaded from /Users/eric/Proj/rust/cargo/scratch/z10/target/target/debug/deps/libcore-a764da499c7385f4.rmeta
  = note: second definition in `core` loaded from /Users/eric/Proj/rust/cargo/scratch/z10/target/target/debug/deps/libcore-5b082675aea34986.rmeta
```

…petrochenkov

Handle `macro_rules!` tokens consistently across crates

When we serialize a `macro_rules!` macro, we used a 'lowered' `TokenStream` for its body, which has all `Nonterminal`s expanded in-place via `nt_to_tokenstream`. This matters when an 'outer' `macro_rules!` macro expands to an 'inner' `macro_rules!` macro - the inner macro may use tokens captured from the 'outer' macro in its definition.

This means that invoking a foreign `macro_rules!` macro may use a different body `TokenStream` than when the same `macro_rules!` macro is invoked in the same crate. This difference is observable by proc-macros invoked by a `macro_rules!` macro - a `None`-delimited group will be seen in the same-crate case (inserted when convering `Nonterminal`s to the `proc_macro` crate's structs), but no `None`-delimited group in the cross-crate case.

To fix this inconsistency, we now insert `None`-delimited groups when 'lowering' a `Nonterminal` `macro_rules!` body, just as we do in `proc_macro_server`. Additionally, we no longer print extra spaces for `None`-delimited groups - as far as pretty-printing is concerned, they don't exist (only their contents do). This ensures that `Display` output of a `TokenStream` does not depend on which crate a `macro_rules!` macro was invoked from.

This PR is necessary in order to patch the `solana-genesis-programs` for the upcoming hygiene serialization breakage (rust-lang#72121 (comment)). The `solana-genesis-programs` crate will need to use a proc macro to re-span certain tokens in a nested `macro_rules!`, which requires us to consistently use a `None`-delimited group.

See `src/test/ui/proc-macro/nested-macro-rules.rs` for an example of the kind of nested `macro_rules!` affected by this crate.

…ikomatsakis

Deny unsafe ops in unsafe fns in libcore

After `liballoc`, It's time for `libcore` :D

I planned to do this bit by bit to avoid having a big chunk of diffs, so to make reviews easier, and to make the unsafe blocks narrower and take the time to document them properly.

r? @nikomatsakis cc @RalfJung

…ery, r=matthewjasper

Recover extra trailing angle brackets in struct definition

This commit applies the existing 'extra angle bracket recovery' logic
when parsing fields in struct definitions. This allows us to continue
parsing the struct's fields, avoiding spurious 'missing field' errors in
code that tries to use the struct.

ast_pretty: Pass some token streams and trees by reference

Salvaged from an intermediate version of rust-lang#73345.

…-morse

Some refactoring around intrinsic type checking

So... This PR went a bit overboard. I wanted to make the `rustc_peek` intrinsic safe (cc @ecstatic-morse ), and remembered a long-standing itch of mine. So I made that huge `&str` match for the intrinsic name a match on `Symbol`s (so basically `u32`s). This is unlikely to have a positive perf effect, even if it likely has better codegen (intrinsics are used rarely, mostly once in their wrapper), so it's mostly a consistency thing since other places actually match on the symbol name of the intrinsics.

…nishearth

Split and expand nonstandard-style lints unicode unit test.

RFC 2457 requested that the `nonstandard_style` series of linted be adjusted to cover the non_ascii_identifier case. However when i read the code of those implementations, it seems they're already supporting non_ascii_identifiers. But the exact rules is a little different than what's proposed in RFC 2457.

So I splitted and expanded the existing test case to try to exercise every branch in the code. I think it'll also be easier to examine the cases in these unit tests to see whether it's ok to just leave them as is, or some adjustments are needed.

r? @Manishearth

…Simulacrum

Remove defunct `-Z print-region-graph`

…r=jonas-schievink

Fix markdown rendering in librustc_lexer docs

Use back-ticks instead of quotation marks in docs for the block comment variant of TokenKind.

## [Before](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lexer/enum.TokenKind.html#variant.BlockComment) and after

<img width="1103" alt="Screen Shot 2020-06-28 at 1 22 30 PM" src="https://user-images.githubusercontent.com/19642016/85957562-446a8380-b943-11ea-913a-442cf7744083.png">

<img width="1015" alt="Screen Shot 2020-06-28 at 1 28 29 PM" src="https://user-images.githubusercontent.com/19642016/85957566-4af8fb00-b943-11ea-8fef-a09c1d586772.png">

## Question

For visual consistency, should we use back-ticks throughout the docs for these enum variants?

…jonas-schievink

Add newline to rustc MultiSpan docs

Also adds back-ticks when referring to the contents of this collection.

Fix wasm32 being broken due to a NodeJS version bump

Emscripten's SDK [recently bumped the version of NodeJS they shipped](emscripten-core/emsdk#529), but our Dockerfile for the wasm32 builder hardcoded the version number. This will cause consistent CI failures once the currently cached image is rebuilt (either due to a change or due to the cache expiring).

This PR fixes the problem by finding the latest version of NodeJS in the Emscripten SDK and symlinking it to a "latest" directory, which is then added to the `PATH`.

…an-DPC

Clean up E0712 explanation

r? @Dylan-DPC

remove duplicate test for rust-lang#61935

Apparently I somehow messed up the issue number in rust-lang#67890
which caused us to add this test twice, both as https://github.com/rust-lang/rust/blob/master/src/test/ui/const-generics/issues/issue-61935.rs and https://github.com/rust-lang/rust/blob/master/src/test/ui/const-generics/lazy-normalization/issue-71922.rs

rust-lang#61935 is the actually fixed issue while rust-lang#71922 is still not working, as it depends on lazy norm of repeat expressions