Move format_args!() into AST (and expand it during AST lowering) #106745

m-ou-se · 2023-01-11T20:54:00Z

This moves FormatArgs from rustc_builtin_macros to rustc_ast_lowering. For now, the end result is the same. But this allows for future changes to do smarter things with format_args!(). It also allows Clippy to directly access the ast::FormatArgs, making things a lot easier.

This change turns the format args types into lang items. The builtin macro used to refer to them by their path. After this change, the path is no longer relevant, making it easier to make changes in core.

This updates clippy to use the new language items, but this doesn't yet make clippy use the ast::FormatArgs structure that's now available. That should be done after this is merged.

Rustfmt doesn't expand macros, so that's easy: FormatArgs nodes do not occur in the unexpanded AST.

m-ou-se · 2023-01-24T17:13:50Z

Oli said this workaround is acceptable for landing this PR while we figure out the underlying issue.

@bors r=oli-obk

bors · 2023-01-24T17:13:52Z

📌 Commit db731e4 has been approved by oli-obk

It is now in the queue for this repository.

bors · 2023-01-26T12:44:51Z

⌛ Testing commit db731e4 with merge 3e97763...

bors · 2023-01-26T15:58:01Z

☀️ Test successful - checks-actions
Approved by: oli-obk
Pushing 3e97763 to master...

m-ou-se · 2023-01-26T16:10:56Z

🎉!

rust-timer · 2023-01-26T17:16:19Z

Finished benchmarking commit (3e97763): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.6%	[0.2%, 0.8%]	4
Regressions ❌ (secondary)	0.8%	[0.3%, 1.3%]	3
Improvements ✅ (primary)	-0.6%	[-2.3%, -0.2%]	51
Improvements ✅ (secondary)	-0.7%	[-0.8%, -0.7%]	6
All ❌✅ (primary)	-0.6%	[-2.3%, 0.8%]	55

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.4%, 0.4%]	1
Regressions ❌ (secondary)	3.3%	[2.0%, 7.9%]	7
Improvements ✅ (primary)	-2.9%	[-3.8%, -2.1%]	4
Improvements ✅ (secondary)	-3.1%	[-3.9%, -1.8%]	3
All ❌✅ (primary)	-2.2%	[-3.8%, 0.4%]	5

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.6%	[1.5%, 1.8%]	9
Regressions ❌ (secondary)	3.0%	[1.5%, 4.9%]	18
Improvements ✅ (primary)	-2.2%	[-3.1%, -1.3%]	5
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.3%	[-3.1%, 1.8%]	14

Move format_args!() into AST (and expand it during AST lowering) Implements rust-lang/compiler-team#541 This moves FormatArgs from rustc_builtin_macros to rustc_ast_lowering. For now, the end result is the same. But this allows for future changes to do smarter things with format_args!(). It also allows Clippy to directly access the ast::FormatArgs, making things a lot easier. This change turns the format args types into lang items. The builtin macro used to refer to them by their path. After this change, the path is no longer relevant, making it easier to make changes in `core`. This updates clippy to use the new language items, but this doesn't yet make clippy use the ast::FormatArgs structure that's now available. That should be done after this is merged.

nnethercote · 2023-01-29T23:04:35Z

The perf improvements greatly outweigh the regressions.

@rustbot label: +perf-regression-triaged

Pkgsrc changes: * Adjust patches and cargo checksums to new versions. * Sadly, the patch to reduce the cargo verbosity no longer applies, so I've asked upstream about the proper way to get the old result. (so the build log becomes Quite Bloated for now). Upstream changes: Version 1.69.0 (2023-04-20) ========================== Language -------- - [Deriving built-in traits on packed structs works with `Copy` fields.] (rust-lang/rust#104429) - [Stabilize the `cmpxchg16b` target feature on x86 and x86_64.] (rust-lang/rust#106774) - [Improve analysis of trait bounds for associated types.] (rust-lang/rust#103695) - [Allow associated types to be used as union fields.] (rust-lang/rust#106938) - [Allow `Self: Autotrait` bounds on dyn-safe trait methods.] (rust-lang/rust#107082) - [Treat `str` as containing `[u8]` for auto trait purposes.] (rust-lang/rust#107941) Compiler -------- - [Upgrade `*-pc-windows-gnu` on CI to mingw-w64 v10 and GCC 12.2.] (rust-lang/rust#100178) - [Rework min_choice algorithm of member constraints.] (rust-lang/rust#105300) - [Support `true` and `false` as boolean flags in compiler arguments.] (rust-lang/rust#107043) - [Default `repr(C)` enums to `c_int` size.] (rust-lang/rust#107592) Libraries --------- - [Implement the unstable `DispatchFromDyn` for cell types, allowing downstream experimentation with custom method receivers.] (rust-lang/rust#97373) - [Document that `fmt::Arguments::as_str()` may return `Some(_)` in more cases after optimization, subject to change.] (rust-lang/rust#106823) - [Implement `AsFd` and `AsRawFd` for `Rc`.] (rust-lang/rust#107317) Stabilized APIs --------------- - [`CStr::from_bytes_until_nul`] (https://doc.rust-lang.org/stable/core/ffi/struct.CStr.html#method.from_bytes_until_nul) - [`core::ffi::FromBytesUntilNulError`] (https://doc.rust-lang.org/stable/core/ffi/struct.FromBytesUntilNulError.html) These APIs are now stable in const contexts: - [`SocketAddr::new`] (https://doc.rust-lang.org/stable/std/net/enum.SocketAddr.html#method.new) - [`SocketAddr::ip`] (https://doc.rust-lang.org/stable/std/net/enum.SocketAddr.html#method.ip) - [`SocketAddr::port`] (https://doc.rust-lang.org/stable/std/net/enum.SocketAddr.html#method.port) - [`SocketAddr::is_ipv4`] (https://doc.rust-lang.org/stable/std/net/enum.SocketAddr.html#method.is_ipv4) - [`SocketAddr::is_ipv6`] (https://doc.rust-lang.org/stable/std/net/enum.SocketAddr.html#method.is_ipv6) - [`SocketAddrV4::new`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV4.html#method.new) - [`SocketAddrV4::ip`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV4.html#method.ip) - [`SocketAddrV4::port`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV4.html#method.port) - [`SocketAddrV6::new`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV6.html#method.new) - [`SocketAddrV6::ip`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV6.html#method.ip) - [`SocketAddrV6::port`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV6.html#method.port) - [`SocketAddrV6::flowinfo`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV6.html#method.flowinfo) - [`SocketAddrV6::scope_id`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV6.html#method.scope_id) Cargo ----- - [Cargo now suggests `cargo fix` or `cargo clippy --fix` when compilation warnings are auto-fixable.] (rust-lang/cargo#11558) - [Cargo now suggests `cargo add` if you try to install a library crate.] (rust-lang/cargo#11410) - [Cargo now sets the `CARGO_BIN_NAME` environment variable also for binary examples.] (rust-lang/cargo#11705) Rustdoc ----- - [Vertically compact trait bound formatting.] (rust-lang/rust#102842) - [Only include stable lints in `rustdoc::all` group.] (rust-lang/rust#106316) - [Compute maximum Levenshtein distance based on the query.] (rust-lang/rust#107141) - [Remove inconsistently-present sidebar tooltips.] (rust-lang/rust#107490) - [Search by macro when query ends with `!`.] (rust-lang/rust#108143) Compatibility Notes ------------------- - [The `rust-analysis` component from `rustup` now only contains a warning placeholder.] (rust-lang/rust#101841) This was primarily intended for RLS, and the corresponding `-Zsave-analysis` flag has been removed from the compiler as well. - [Unaligned references to packed fields are now a hard error.] (rust-lang/rust#102513) This has been a warning since 1.53, and denied by default with a future-compatibility warning since 1.62. - [Update the minimum external LLVM to 14.] (rust-lang/rust#107573) - [Cargo now emits errors on invalid characters in a registry token.] (rust-lang/cargo#11600) - [When `default-features` is set to false of a workspace dependency, and an inherited dependency of a member has `default-features = true`, Cargo will enable default features of that dependency.] (rust-lang/cargo#11409) - [Cargo denies `CARGO_HOME` in the `[env]` configuration table. Cargo itself doesn't pick up this value, but recursive calls to cargo would, which was not intended.] (rust-lang/cargo#11644) - [Debuginfo for build dependencies is now off if not explicitly set. This is expected to improve the overall build time.] (rust-lang/cargo#11252) Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Move `format_args!()` into AST (and expand it during AST lowering)] (rust-lang/rust#106745)

Pkgsrc changes: * Adjust patches and cargo checksums to new versions. Upstream changes: Version 1.69.0 (2023-04-20) ========================== Language -------- - [Deriving built-in traits on packed structs works with `Copy` fields.] (rust-lang/rust#104429) - [Stabilize the `cmpxchg16b` target feature on x86 and x86_64.] (rust-lang/rust#106774) - [Improve analysis of trait bounds for associated types.] (rust-lang/rust#103695) - [Allow associated types to be used as union fields.] (rust-lang/rust#106938) - [Allow `Self: Autotrait` bounds on dyn-safe trait methods.] (rust-lang/rust#107082) - [Treat `str` as containing `[u8]` for auto trait purposes.] (rust-lang/rust#107941) Compiler -------- - [Upgrade `*-pc-windows-gnu` on CI to mingw-w64 v10 and GCC 12.2.] (rust-lang/rust#100178) - [Rework min_choice algorithm of member constraints.] (rust-lang/rust#105300) - [Support `true` and `false` as boolean flags in compiler arguments.] (rust-lang/rust#107043) - [Default `repr(C)` enums to `c_int` size.] (rust-lang/rust#107592) Libraries --------- - [Implement the unstable `DispatchFromDyn` for cell types, allowing downstream experimentation with custom method receivers.] (rust-lang/rust#97373) - [Document that `fmt::Arguments::as_str()` may return `Some(_)` in more cases after optimization, subject to change.] (rust-lang/rust#106823) - [Implement `AsFd` and `AsRawFd` for `Rc`.] (rust-lang/rust#107317) Stabilized APIs --------------- - [`CStr::from_bytes_until_nul`] (https://doc.rust-lang.org/stable/core/ffi/struct.CStr.html#method.from_bytes_until_nul) - [`core::ffi::FromBytesUntilNulError`] (https://doc.rust-lang.org/stable/core/ffi/struct.FromBytesUntilNulError.html) These APIs are now stable in const contexts: - [`SocketAddr::new`] (https://doc.rust-lang.org/stable/std/net/enum.SocketAddr.html#method.new) - [`SocketAddr::ip`] (https://doc.rust-lang.org/stable/std/net/enum.SocketAddr.html#method.ip) - [`SocketAddr::port`] (https://doc.rust-lang.org/stable/std/net/enum.SocketAddr.html#method.port) - [`SocketAddr::is_ipv4`] (https://doc.rust-lang.org/stable/std/net/enum.SocketAddr.html#method.is_ipv4) - [`SocketAddr::is_ipv6`] (https://doc.rust-lang.org/stable/std/net/enum.SocketAddr.html#method.is_ipv6) - [`SocketAddrV4::new`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV4.html#method.new) - [`SocketAddrV4::ip`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV4.html#method.ip) - [`SocketAddrV4::port`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV4.html#method.port) - [`SocketAddrV6::new`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV6.html#method.new) - [`SocketAddrV6::ip`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV6.html#method.ip) - [`SocketAddrV6::port`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV6.html#method.port) - [`SocketAddrV6::flowinfo`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV6.html#method.flowinfo) - [`SocketAddrV6::scope_id`] (https://doc.rust-lang.org/stable/std/net/struct.SocketAddrV6.html#method.scope_id) Cargo ----- - [Cargo now suggests `cargo fix` or `cargo clippy --fix` when compilation warnings are auto-fixable.] (rust-lang/cargo#11558) - [Cargo now suggests `cargo add` if you try to install a library crate.] (rust-lang/cargo#11410) - [Cargo now sets the `CARGO_BIN_NAME` environment variable also for binary examples.] (rust-lang/cargo#11705) Rustdoc ----- - [Vertically compact trait bound formatting.] (rust-lang/rust#102842) - [Only include stable lints in `rustdoc::all` group.] (rust-lang/rust#106316) - [Compute maximum Levenshtein distance based on the query.] (rust-lang/rust#107141) - [Remove inconsistently-present sidebar tooltips.] (rust-lang/rust#107490) - [Search by macro when query ends with `!`.] (rust-lang/rust#108143) Compatibility Notes ------------------- - [The `rust-analysis` component from `rustup` now only contains a warning placeholder.] (rust-lang/rust#101841) This was primarily intended for RLS, and the corresponding `-Zsave-analysis` flag has been removed from the compiler as well. - [Unaligned references to packed fields are now a hard error.] (rust-lang/rust#102513) This has been a warning since 1.53, and denied by default with a future-compatibility warning since 1.62. - [Update the minimum external LLVM to 14.] (rust-lang/rust#107573) - [Cargo now emits errors on invalid characters in a registry token.] (rust-lang/cargo#11600) - [When `default-features` is set to false of a workspace dependency, and an inherited dependency of a member has `default-features = true`, Cargo will enable default features of that dependency.] (rust-lang/cargo#11409) - [Cargo denies `CARGO_HOME` in the `[env]` configuration table. Cargo itself doesn't pick up this value, but recursive calls to cargo would, which was not intended.] (rust-lang/cargo#11644) - [Debuginfo for build dependencies is now off if not explicitly set. This is expected to improve the overall build time.] (rust-lang/cargo#11252) Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Move `format_args!()` into AST (and expand it during AST lowering)] (rust-lang/rust#106745)

Move format_args!() into AST (and expand it during AST lowering) Implements rust-lang/compiler-team#541 This moves FormatArgs from rustc_builtin_macros to rustc_ast_lowering. For now, the end result is the same. But this allows for future changes to do smarter things with format_args!(). It also allows Clippy to directly access the ast::FormatArgs, making things a lot easier. This change turns the format args types into lang items. The builtin macro used to refer to them by their path. After this change, the path is no longer relevant, making it easier to make changes in `core`. This updates clippy to use the new language items, but this doesn't yet make clippy use the ast::FormatArgs structure that's now available. That should be done after this is merged.

m-ou-se added 4 commits January 11, 2023 21:44

Add some helper functions to LoweringContext.

43105c1

Expose some LoweringContext methods.

3c45176

Turn format arguments types into lang items.

bebf9fe

Add new fn to core::fmt::rt::v1::Argument.

e839451

m-ou-se added A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. A-HIR Area: The high-level intermediate representation (HIR) A-fmt Area: `std::fmt` labels Jan 11, 2023

m-ou-se self-assigned this Jan 11, 2023

rustbot added T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 11, 2023

This comment was marked as outdated.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 11, 2023

This comment was marked as outdated.

Sign in to view

m-ou-se added 6 commits January 12, 2023 00:25

Expand format_args!() in rust_ast_lowering.

a4dbcb5

Bless tests.

525b0bb

Make clippy compile.

c878caf

Update rustfmt for ast::ExprKind::FormatArgs.

a1725fa

Rustfmt doesn't expand macros, so that's easy: FormatArgs nodes do not occur in the unexpanded AST.

Bless pretty tests.

7a3d5fe

Update clippy for new format_args!() lang items.

36fe3b9

m-ou-se force-pushed the format-args-ast branch from 4552f7f to 36fe3b9 Compare January 11, 2023 23:25

This comment was marked as outdated.

Sign in to view

Make core::fmt::rt::v1::Argument::new const+inline.

9e67854

m-ou-se force-pushed the format-args-ast branch from b7f2b61 to 9e67854 Compare January 11, 2023 23:38

Update outdated comment.

bcf388f

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 24, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 26, 2023

bors merged commit 3e97763 into rust-lang:master Jan 26, 2023

rustbot added this to the 1.69.0 milestone Jan 26, 2023

m-ou-se deleted the format-args-ast branch January 26, 2023 16:10

bors mentioned this pull request Jan 26, 2023

New format args template #101272

Closed

m-ou-se mentioned this pull request Jan 26, 2023

Migrate format_args!() lints to to ast::FormatArgs rust-lang/rust-clippy#10233

Closed

24 tasks

This was referenced Jan 26, 2023

Shorten lifetime of panic temporaries in panic_fmt case #104134

Merged

Add offset_of! macro (RFC 3308) #106934

Merged

Coalesce unreachable_patterns lint messages #107262

Closed

rustbot added the perf-regression Performance regression. label Jan 26, 2023

nyurik mentioned this pull request Jan 26, 2023

Support format_args_capture rust-lang/rust-analyzer#11260

Closed

rustbot added the perf-regression-triaged The performance regression has been triaged. label Jan 29, 2023

nyurik mentioned this pull request Jan 30, 2023

non_ascii_literal should not be machine-applicable to raw strings rust-lang/rust-clippy#9805

Open

dtolnay mentioned this pull request Mar 26, 2023

make format! accept const vars as format-strings #69133

Closed

piegamesde mentioned this pull request Apr 21, 2023

Allow storing format_args! in a let binding #92698

Open

SergioBenitez mentioned this pull request Dec 12, 2023

Build fails with rustc<=1.68.2 rwf2/Rocket#2670

Closed

4 tasks

dtolnay mentioned this pull request Feb 13, 2024

A few minor write_str optimizations serde-rs/serde#2697

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move format_args!() into AST (and expand it during AST lowering) #106745

Move format_args!() into AST (and expand it during AST lowering) #106745

m-ou-se commented Jan 11, 2023 •

edited

Loading

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

m-ou-se commented Jan 24, 2023

bors commented Jan 24, 2023

bors commented Jan 26, 2023

bors commented Jan 26, 2023

m-ou-se commented Jan 26, 2023

rust-timer commented Jan 26, 2023

nnethercote commented Jan 29, 2023

Move format_args!() into AST (and expand it during AST lowering) #106745

Move format_args!() into AST (and expand it during AST lowering) #106745

Conversation

m-ou-se commented Jan 11, 2023 • edited Loading

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

m-ou-se commented Jan 24, 2023

bors commented Jan 24, 2023

bors commented Jan 26, 2023

bors commented Jan 26, 2023

m-ou-se commented Jan 26, 2023

rust-timer commented Jan 26, 2023

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Instruction count

Max RSS (memory usage)

Cycles

nnethercote commented Jan 29, 2023

m-ou-se commented Jan 11, 2023 •

edited

Loading