Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid adding builtin functions to symbols.o #118568

Merged
merged 2 commits into from
Dec 7, 2023

Conversation

DianQK
Copy link
Member

@DianQK DianQK commented Dec 3, 2023

We found performance regressions in #113923. The problem seems to be that --gc-sections does not remove these symbols. I tested that lld removes these symbols, but ld and gold do not.

I found that used adds symbols to symbols.o at

if info.level.is_below_threshold(export_threshold) || info.used {
symbols.push((
symbol_export::linking_symbol_name_for_instance_in_crate(tcx, symbol, cnum),
info.kind,
));
}
.
The PR removes builtin functions.

Note that under LTO, ld still preserves these symbols. (lld will still remove them.)

The first commit also fixes #118559. But I think the second commit also makes sense.

@rustbot
Copy link
Collaborator

rustbot commented Dec 3, 2023

r? @davidtwco

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Dec 3, 2023
@DianQK
Copy link
Member Author

DianQK commented Dec 3, 2023

r? @bjorn3

cc @nnethercote

We probably can perform a performance test.

@rustbot rustbot assigned bjorn3 and unassigned davidtwco Dec 3, 2023
@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@bjorn3
Copy link
Member

bjorn3 commented Dec 3, 2023

symbols.o is necessary to ensure cyclic dependencies work correctly by forcing that the object file that defines them is pulled in and that the linker will consider this object file as a way to resolve uses of this symbol. This applies not just to linker-used symbols. If not including all these in symbols.o, we need some other way to make the linker behave in this way.

@rustbot
Copy link
Collaborator

rustbot commented Dec 3, 2023

The Miri subtree was changed

cc @rust-lang/miri

@DianQK
Copy link
Member Author

DianQK commented Dec 3, 2023

symbols.o is necessary to ensure cyclic dependencies work correctly by forcing that the object file that defines them is pulled in and that the linker will consider this object file as a way to resolve uses of this symbol. This applies not just to linker-used symbols. If not including all these in symbols.o, we need some other way to make the linker behave in this way.

I moved #[used(compiler)] back to used. This should keep the original behavior.

@DianQK DianQK changed the title Avoid adding compiler-used functions to symbols.o Avoid adding builtin functions to symbols.o Dec 3, 2023
@@ -110,7 +110,8 @@ fn reachable_non_generics_provider(tcx: TyCtxt<'_>, _: LocalCrate) -> DefIdMap<S
// We have to preserve the symbols of the built-in functions during LTO.
let is_builtin_fn = is_compiler_builtins
&& symbol_export_level(tcx, def_id.to_def_id())
.is_below_threshold(SymbolExportLevel::C);
.is_below_threshold(SymbolExportLevel::C)
&& name.contains("compiler_builtins");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should check the name here. The compiler-builtins crate may have a different name. Maybe check for #[no_mangle] instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's better.

@nnethercote
Copy link
Contributor

It would be good to do a perf run on CI before this is merged, to confirm the expected perf effects.

@rust-log-analyzer

This comment has been minimized.

@DianQK
Copy link
Member Author

DianQK commented Dec 4, 2023

It would be good to do a perf run on CI before this is merged, to confirm the expected perf effects.

Could you do this for me? I don't have perf permissions.

@saethlin
Copy link
Member

saethlin commented Dec 5, 2023

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Dec 5, 2023
@bors
Copy link
Contributor

bors commented Dec 5, 2023

⌛ Trying commit ca0738f with merge 3a9fb17...

bors added a commit to rust-lang-ci/rust that referenced this pull request Dec 5, 2023
Avoid adding builtin functions to `symbols.o`

We found performance regressions in rust-lang#113923. The problem seems to be that `--gc-sections` does not remove these symbols. I tested that lld removes these symbols, but ld and gold do not.

I found that `used` adds symbols to `symbols.o` at https://github.com/rust-lang/rust/blob/3e202ead604be31f4c1a5798a296953d3159da7e/compiler/rustc_codegen_ssa/src/back/linker.rs#L1786-L1791.
The PR removes builtin functions.

Note that under LTO, ld still preserves these symbols. (lld will still remove them.)

The first commit also fixes rust-lang#118559. But I think the second commit also makes sense.
@saethlin
Copy link
Member

saethlin commented Dec 5, 2023

@DianQK You can always ask for a perf run https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/perf.20run which will probably be faster than hoping someone sees a GitHub comment. I mean, maybe it won't for you, since your work attracts so many eyes. But in general, the Zulip topic is pretty quck.

@DianQK
Copy link
Member Author

DianQK commented Dec 5, 2023

@DianQK You can always ask for a perf run https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/perf.20run which will probably be faster than hoping someone sees a GitHub comment. I mean, maybe it won't for you, since your work attracts so many eyes. But in general, the Zulip topic is pretty quck.

Thanks. I saved this stream. :)

@bors
Copy link
Contributor

bors commented Dec 5, 2023

☀️ Try build successful - checks-actions
Build commit: 3a9fb17 (3a9fb172c5d337b0436aa5f0785c97d3cbb973b9)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (3a9fb17): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-5.8% [-10.9%, -0.7%] 16
Improvements ✅
(secondary)
-5.0% [-10.6%, -0.6%] 66
All ❌✅ (primary) -5.8% [-10.9%, -0.7%] 16

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
2.5% [1.4%, 3.6%] 2
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-2.1% [-2.8%, -1.5%] 2
Improvements ✅
(secondary)
-3.5% [-3.5%, -3.5%] 1
All ❌✅ (primary) 0.2% [-2.8%, 3.6%] 4

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-7.5% [-11.7%, -1.3%] 13
Improvements ✅
(secondary)
-6.4% [-11.6%, -1.2%] 52
All ❌✅ (primary) -7.5% [-11.7%, -1.3%] 13

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.0% [0.0%, 0.0%] 27
Regressions ❌
(secondary)
0.0% [0.0%, 0.0%] 4
Improvements ✅
(primary)
-3.1% [-5.4%, -0.4%] 20
Improvements ✅
(secondary)
-5.1% [-5.4%, -3.2%] 74
All ❌✅ (primary) -1.3% [-5.4%, 0.0%] 47

Bootstrap: 672.674s -> 673.359s (0.10%)
Artifact size: 314.14 MiB -> 313.88 MiB (-0.08%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Dec 5, 2023
@pnkfelix
Copy link
Member

pnkfelix commented Dec 7, 2023

@bors r+

@bors
Copy link
Contributor

bors commented Dec 7, 2023

📌 Commit ca0738f has been approved by pnkfelix

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 7, 2023
@bors
Copy link
Contributor

bors commented Dec 7, 2023

⌛ Testing commit ca0738f with merge 503e129...

@bors
Copy link
Contributor

bors commented Dec 7, 2023

☀️ Test successful - checks-actions
Approved by: pnkfelix
Pushing 503e129 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Dec 7, 2023
@bors bors merged commit 503e129 into rust-lang:master Dec 7, 2023
12 checks passed
@rustbot rustbot added this to the 1.76.0 milestone Dec 7, 2023
@DianQK DianQK deleted the no-builtins-symbols branch December 7, 2023 23:03
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (503e129): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-6.6% [-11.0%, -0.7%] 14
Improvements ✅
(secondary)
-5.0% [-10.7%, -0.3%] 66
All ❌✅ (primary) -6.6% [-11.0%, -0.7%] 14

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.0% [0.5%, 2.2%] 5
Regressions ❌
(secondary)
2.4% [2.4%, 2.4%] 1
Improvements ✅
(primary)
-2.2% [-4.1%, -0.5%] 6
Improvements ✅
(secondary)
-3.4% [-3.7%, -3.2%] 2
All ❌✅ (primary) -0.8% [-4.1%, 2.2%] 11

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-7.1% [-11.7%, -0.9%] 14
Improvements ✅
(secondary)
-6.5% [-11.6%, -0.8%] 53
All ❌✅ (primary) -7.1% [-11.7%, -0.9%] 14

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.0% [0.0%, 0.0%] 34
Regressions ❌
(secondary)
0.0% [0.0%, 0.0%] 8
Improvements ✅
(primary)
-3.1% [-5.5%, -0.4%] 20
Improvements ✅
(secondary)
-5.2% [-5.5%, -3.3%] 74
All ❌✅ (primary) -1.1% [-5.5%, 0.0%] 54

Bootstrap: 675.63s -> 675.646s (0.00%)
Artifact size: 314.19 MiB -> 313.94 MiB (-0.08%)

DianQK added a commit to DianQK/rust that referenced this pull request Jan 12, 2024
…r=pnkfelix"

This reverts commit 503e129, reversing
changes made to 0e7f91b.
DianQK added a commit to DianQK/rust that referenced this pull request Jan 12, 2024
…r=pnkfelix"

This reverts commit 503e129, reversing
changes made to 0e7f91b.
This was referenced Jan 12, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Jan 12, 2024
Rollup merge of rust-lang#119885 - DianQK:revert-pr-113923, r=petrochenkov

Revert rust-lang#113923

Per [#t-compiler/meetings > [weekly] 2024-01-11](https://rust-lang.zulipchat.com/#narrow/stream/238009-t-compiler.2Fmeetings/topic/.5Bweekly.5D.202024-01-11) discussion, revert rust-lang#113923. Also revert associated rust-lang#118568.

The PR rust-lang#113923 causes the regression issue rust-lang#118609. We need more time to find a proper solution.

Discussions start at [412365838](https://rust-lang.zulipchat.com/#narrow/stream/238009-t-compiler.2Fmeetings/topic/.5Bweekly.5D.202024-01-11/near/412365838) and continue to [412369643](https://rust-lang.zulipchat.com/#narrow/stream/238009-t-compiler.2Fmeetings/topic/.5Bweekly.5D.202024-01-11/near/412369643).

Fixes rust-lang#118609.

r? compiler
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Windows x64 no-std build broken with errors related to compiler-builtins