Store HIR attributes in a side table #79519

cjgillot · 2020-11-28T20:51:03Z

Same idea as #72015 but for attributes.
The objective is to reduce incr-comp invalidations due to modified attributes.
Notably, those due to modified doc comments.

Implementation:

collect attributes during AST->HIR lowering, in LocalDefId -> ItemLocalId -> &[Attributes] nested tables;
access the attributes through a hir_owner_attrs query;
local refactorings to use this access;
remove attrs from HIR data structures one-by-one.

Change in behaviour:

the HIR visitor traverses all attributes at once instead of parent-by-parent;
attribute arrays are sometimes duplicated: for statements and variant constructors;
as a consequence, attributes are marked as used after unused-attribute lint emission to avoid duplicate lints.

~~Current bug: the lint level is not correctly applied in std::backtrace_rs, triggering an unused attribute warning on #![no_std]. I welcome suggestions.~~

rust-highfive · 2020-11-28T20:51:07Z

r? @matthewjasper

(rust-highfive has picked a reviewer for you, use r? to override)

jonas-schievink · 2020-12-05T23:19:56Z

@bors try @rust-timer queue

rust-timer · 2020-12-05T23:19:57Z

Awaiting bors try build completion

bors · 2020-12-05T23:20:08Z

⌛ Trying commit 3b47e6a134e12e60f2357b2a31f74315a1cacdf4 with merge b7675636fff6dc80b3b4d552a8c497ce56522491...

bors · 2020-12-06T00:02:29Z

☀️ Try build successful - checks-actions
Build commit: b7675636fff6dc80b3b4d552a8c497ce56522491 (b7675636fff6dc80b3b4d552a8c497ce56522491)

rust-timer · 2020-12-06T00:02:31Z

Queued b7675636fff6dc80b3b4d552a8c497ce56522491 with parent e792288, future comparison URL.

rust-timer · 2020-12-06T03:43:27Z

Finished benchmarking try commit (b7675636fff6dc80b3b4d552a8c497ce56522491): comparison url.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. Please note that if the perf results are neutral, you should likely undo the rollup=never given below by specifying rollup- to bors.

Importantly, though, if the results of this run are non-neutral do not roll this PR up -- it will mask other regressions or improvements in the roll up.

@bors rollup=never
@rustbot modify labels: +S-waiting-on-review -S-waiting-on-perf

bjorn3 · 2020-12-06T06:10:09Z

A few crates got huge incr patched improvements of up to 65%. Some others got a regression of up to ~7%.

cjgillot · 2020-12-08T21:40:16Z

This PR is prone to bitrot. I wait for an initial review before rebasing again.

cjgillot · 2020-12-11T20:17:49Z

@rustbot modify labels: +S-waiting-on-review -S-waiting-on-author

cjgillot · 2021-03-09T19:24:07Z

@wesleywiser : just rebased.

wesleywiser · 2021-03-09T20:19:25Z

@bors r+ rollup=never

Mixed perf results.

bors · 2021-03-09T20:19:27Z

📌 Commit 77c0f21 has been approved by wesleywiser

bors · 2021-03-09T21:07:28Z

⌛ Testing commit 77c0f21 with merge 0eeeb96aa48c8f2d187b92bee2b37525bd5e9134...

bors · 2021-03-10T01:22:28Z

💥 Test timed out

rust-log-analyzer · 2021-03-10T01:28:44Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

cjgillot · 2021-03-10T07:38:42Z

@bors retry

bors · 2021-03-10T08:40:58Z

⌛ Testing commit 77c0f21 with merge dff1edf...

bors · 2021-03-10T11:21:41Z

☀️ Test successful - checks-actions
Approved by: wesleywiser
Pushing dff1edf to master...

Store HIR attributes in a side table Same idea as rust-lang#72015 but for attributes. The objective is to reduce incr-comp invalidations due to modified attributes. Notably, those due to modified doc comments. Implementation: - collect attributes during AST->HIR lowering, in `LocalDefId -> ItemLocalId -> &[Attributes]` nested tables; - access the attributes through a `hir_owner_attrs` query; - local refactorings to use this access; - remove `attrs` from HIR data structures one-by-one. Change in behaviour: - the HIR visitor traverses all attributes at once instead of parent-by-parent; - attribute arrays are sometimes duplicated: for statements and variant constructors; - as a consequence, attributes are marked as used after unused-attribute lint emission to avoid duplicate lints. ~~Current bug: the lint level is not correctly applied in `std::backtrace_rs`, triggering an unused attribute warning on `#![no_std]`. I welcome suggestions.~~

Package changes: * bump bootstraps to 1.51.0. * adjust patches and cargo checksums as required * 1.51 failed to build natively on 32-bit armv7, there is hope that this is fixed with 1.52. (1.51 can be built with netbsd32 emulation on a aarch64 system). Upsteream changes: Version 1.52.0 (2021-05-06) ============================ Language -------- - [Added the `unsafe_op_in_unsafe_fn` lint, which checks whether the unsafe code in an `unsafe fn` is wrapped in a `unsafe` block.][79208] This lint is allowed by default, and may become a warning or hard error in a future edition. - [You can now cast mutable references to arrays to a pointer of the same type as the element.][81479] Compiler -------- - [Upgraded the default LLVM to LLVM 12.][81451] Added tier 3\* support for the following targets. - [`s390x-unknown-linux-musl`][82166] - [`riscv32gc-unknown-linux-musl` & `riscv64gc-unknown-linux-musl`][82202] - [`powerpc-unknown-openbsd`][82733] \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [`OsString` now implements `Extend` and `FromIterator`.][82121] - [`cmp::Reverse` now has `#[repr(transparent)]` representation.][81879] - [`Arc<impl Error>` now implements `error::Error`.][80553] - [All integer division and remainder operations are now `const`.][80962] Stabilised APIs ------------- - [`Arguments::as_str`] - [`char::MAX`] - [`char::REPLACEMENT_CHARACTER`] - [`char::UNICODE_VERSION`] - [`char::decode_utf16`] - [`char::from_digit`] - [`char::from_u32_unchecked`] - [`char::from_u32`] - [`slice::partition_point`] - [`str::rsplit_once`] - [`str::split_once`] The following previously stable APIs are now `const`. - [`char::len_utf8`] - [`char::len_utf16`] - [`char::to_ascii_uppercase`] - [`char::to_ascii_lowercase`] - [`char::eq_ignore_ascii_case`] - [`u8::to_ascii_uppercase`] - [`u8::to_ascii_lowercase`] - [`u8::eq_ignore_ascii_case`] Rustdoc ------- - [Rustdoc lints are now treated as a tool lint, meaning that lints are now prefixed with `rustdoc::` (e.g. `#[warn(rustdoc::non_autolinks)]`).][80527] Using the old style is still allowed, and will become a warning in a future release. - [Rustdoc now supports argument files.][82261] - [Rustdoc now generates smart punctuation for documentation.][79423] - [You can now use "task lists" in Rustdoc Markdown.][81766] E.g. ```markdown - [x] Complete - [ ] Todo ``` Misc ---- - [You can now pass multiple filters to tests.][81356] E.g. `cargo test -- foo bar` will run all tests that match `foo` and `bar`. - [Rustup now distributes PDB symbols for the `std` library on Windows, allowing you to see `std` symbols when debugging.][82218] Internal Only ------------- These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools. - [Check the result cache before the DepGraph when ensuring queries][81855] - [Try fast_reject::simplify_type in coherence before doing full check][81744] - [Only store a LocalDefId in some HIR nodes][81611] - [Store HIR attributes in a side table][79519] Compatibility Notes ------------------- - [Cargo build scripts are now forbidden from setting `RUSTC_BOOTSTRAP`.] [cargo/9181] - [Removed support for the `x86_64-rumprun-netbsd` target.][82594] - [Deprecated the `x86_64-sun-solaris` target in favor of `x86_64-pc-solaris`.] [82216] - [Rustdoc now only accepts `,`, ` `, and `\t` as delimiters for specifying languages in code blocks.][78429] - [Rustc now catches more cases of `pub_use_of_private_extern_crate`][80763] - [Changes in how proc macros handle whitespace may lead to panics when used with older `proc-macro-hack` versions. A `cargo update` should be sufficient to fix this in all cases.][84136] [84136]: rust-lang/rust#84136 [80763]: rust-lang/rust#80763 [82166]: rust-lang/rust#82166 [82121]: rust-lang/rust#82121 [81879]: rust-lang/rust#81879 [82261]: rust-lang/rust#82261 [82218]: rust-lang/rust#82218 [82216]: rust-lang/rust#82216 [82202]: rust-lang/rust#82202 [81855]: rust-lang/rust#81855 [81766]: rust-lang/rust#81766 [81744]: rust-lang/rust#81744 [81611]: rust-lang/rust#81611 [81479]: rust-lang/rust#81479 [81451]: rust-lang/rust#81451 [81356]: rust-lang/rust#81356 [80962]: rust-lang/rust#80962 [80553]: rust-lang/rust#80553 [80527]: rust-lang/rust#80527 [79519]: rust-lang/rust#79519 [79423]: rust-lang/rust#79423 [79208]: rust-lang/rust#79208 [78429]: rust-lang/rust#78429 [82733]: rust-lang/rust#82733 [82594]: rust-lang/rust#82594 [cargo/9181]: rust-lang/cargo#9181 [`char::MAX`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.MAX [`char::REPLACEMENT_CHARACTER`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.REPLACEMENT_CHARACTER [`char::UNICODE_VERSION`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.UNICODE_VERSION [`char::decode_utf16`]: https://doc.rust-lang.org/std/primitive.char.html#method.decode_utf16 [`char::from_u32`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32 [`char::from_u32_unchecked`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32_unchecked [`char::from_digit`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_digit [`Peekable::next_if`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if [`Peekable::next_if_eq`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if_eq [`Arguments::as_str`]: https://doc.rust-lang.org/stable/std/fmt/struct.Arguments.html#method.as_str [`str::split_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.split_once [`str::rsplit_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.rsplit_once [`slice::partition_point`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.partition_point [`char::len_utf8`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf8 [`char::len_utf16`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf16 [`char::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_uppercase [`char::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_lowercase [`char::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.eq_ignore_ascii_case [`u8::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_uppercase [`u8::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_lowercase [`u8::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.eq_ignore_ascii_case

Pkgsrc changes: * Bump bootstrap kit version to 1.51.0. * Adjust patches as needed. * Update checksum adjustments. * Fix syntax error in commands adjusting libserde_derive for Darwin Upstream changes: Version 1.52.1 (2021-05-10) ============================ This release disables incremental compilation, unless the user has explicitly opted in via the newly added RUSTC_FORCE_INCREMENTAL=1 environment variable. This is due to the widespread, and frequently occuring, breakage encountered by Rust users due to newly enabled incremental verification in 1.52.0. Notably, Rust users **should** upgrade to 1.52.0 or 1.52.1: the bugs that are detected by newly added incremental verification are still present in past stable versions, and are not yet fixed on any channel. These bugs can lead to miscompilation of Rust binaries. These problems only affect incremental builds, so release builds with Cargo should not be affected unless the user has explicitly opted into incremental. Debug and check builds are affected. See [84970] for more details. [84970]: rust-lang/rust#84970 Version 1.52.0 (2021-05-06) ============================ Language -------- - [Added the `unsafe_op_in_unsafe_fn` lint, which checks whether the unsafe code in an `unsafe fn` is wrapped in a `unsafe` block.][79208] This lint is allowed by default, and may become a warning or hard error in a future edition. - [You can now cast mutable references to arrays to a pointer of the same type as the element.][81479] Compiler -------- - [Upgraded the default LLVM to LLVM 12.][81451] Added tier 3\* support for the following targets. - [`s390x-unknown-linux-musl`][82166] - [`riscv32gc-unknown-linux-musl` & `riscv64gc-unknown-linux-musl`][82202] - [`powerpc-unknown-openbsd`][82733] \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [`OsString` now implements `Extend` and `FromIterator`.][82121] - [`cmp::Reverse` now has `#[repr(transparent)]` representation.][81879] - [`Arc<impl Error>` now implements `error::Error`.][80553] - [All integer division and remainder operations are now `const`.][80962] Stabilised APIs ------------- - [`Arguments::as_str`] - [`char::MAX`] - [`char::REPLACEMENT_CHARACTER`] - [`char::UNICODE_VERSION`] - [`char::decode_utf16`] - [`char::from_digit`] - [`char::from_u32_unchecked`] - [`char::from_u32`] - [`slice::partition_point`] - [`str::rsplit_once`] - [`str::split_once`] The following previously stable APIs are now `const`. - [`char::len_utf8`] - [`char::len_utf16`] - [`char::to_ascii_uppercase`] - [`char::to_ascii_lowercase`] - [`char::eq_ignore_ascii_case`] - [`u8::to_ascii_uppercase`] - [`u8::to_ascii_lowercase`] - [`u8::eq_ignore_ascii_case`] Rustdoc ------- - [Rustdoc lints are now treated as a tool lint, meaning that lints are now prefixed with `rustdoc::` (e.g. `#[warn(rustdoc::non_autolinks)]`).][80527] Using the old style is still allowed, and will become a warning in a future release. - [Rustdoc now supports argument files.][82261] - [Rustdoc now generates smart punctuation for documentation.][79423] - [You can now use "task lists" in Rustdoc Markdown.][81766] E.g. ```markdown - [x] Complete - [ ] Todo ``` Misc ---- - [You can now pass multiple filters to tests.][81356] E.g. `cargo test -- foo bar` will run all tests that match `foo` and `bar`. - [Rustup now distributes PDB symbols for the `std` library on Windows, allowing you to see `std` symbols when debugging.][82218] Internal Only ------------- These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools. - [Check the result cache before the DepGraph when ensuring queries][81855] - [Try fast_reject::simplify_type in coherence before doing full check][81744] - [Only store a LocalDefId in some HIR nodes][81611] - [Store HIR attributes in a side table][79519] Compatibility Notes ------------------- - [Cargo build scripts are now forbidden from setting `RUSTC_BOOTSTRAP`.][cargo/9181] - [Removed support for the `x86_64-rumprun-netbsd` target.][82594] - [Deprecated the `x86_64-sun-solaris` target in favor of `x86_64-pc-solaris`.][82216] - [Rustdoc now only accepts `,`, ` `, and `\t` as delimiters for specifying languages in code blocks.][78429] - [Rustc now catches more cases of `pub_use_of_private_extern_crate`][80763] - [Changes in how proc macros handle whitespace may lead to panics when used with older `proc-macro-hack` versions. A `cargo update` should be sufficient to fix this in all cases.][84136] [84136]: rust-lang/rust#84136 [80763]: rust-lang/rust#80763 [82166]: rust-lang/rust#82166 [82121]: rust-lang/rust#82121 [81879]: rust-lang/rust#81879 [82261]: rust-lang/rust#82261 [82218]: rust-lang/rust#82218 [82216]: rust-lang/rust#82216 [82202]: rust-lang/rust#82202 [81855]: rust-lang/rust#81855 [81766]: rust-lang/rust#81766 [81744]: rust-lang/rust#81744 [81611]: rust-lang/rust#81611 [81479]: rust-lang/rust#81479 [81451]: rust-lang/rust#81451 [81356]: rust-lang/rust#81356 [80962]: rust-lang/rust#80962 [80553]: rust-lang/rust#80553 [80527]: rust-lang/rust#80527 [79519]: rust-lang/rust#79519 [79423]: rust-lang/rust#79423 [79208]: rust-lang/rust#79208 [78429]: rust-lang/rust#78429 [82733]: rust-lang/rust#82733 [82594]: rust-lang/rust#82594 [cargo/9181]: rust-lang/cargo#9181 [`char::MAX`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.MAX [`char::REPLACEMENT_CHARACTER`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.REPLACEMENT_CHARACTER [`char::UNICODE_VERSION`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.UNICODE_VERSION [`char::decode_utf16`]: https://doc.rust-lang.org/std/primitive.char.html#method.decode_utf16 [`char::from_u32`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32 [`char::from_u32_unchecked`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32_unchecked [`char::from_digit`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_digit [`Peekable::next_if`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if [`Peekable::next_if_eq`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if_eq [`Arguments::as_str`]: https://doc.rust-lang.org/stable/std/fmt/struct.Arguments.html#method.as_str [`str::split_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.split_once [`str::rsplit_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.rsplit_once [`slice::partition_point`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.partition_point [`char::len_utf8`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf8 [`char::len_utf16`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf16 [`char::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_uppercase [`char::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_lowercase [`char::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.eq_ignore_ascii_case [`u8::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_uppercase [`u8::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_lowercase [`u8::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.eq_ignore_ascii_case

rust-highfive assigned matthewjasper Nov 28, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 28, 2020

cjgillot mentioned this pull request Dec 3, 2020

Store HIR ForeignItem in a side table #79318

Merged

cjgillot force-pushed the noattr branch 2 times, most recently from b80b682 to 3b47e6a Compare December 5, 2020 20:41

cjgillot marked this pull request as ready for review December 5, 2020 23:11

cjgillot force-pushed the noattr branch from d1d2c6b to e717f06 Compare December 6, 2020 22:02