Skip to content

Only obey optimize-tests flag on UI tests that are run-pass #98817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jul 4, 2022
Merged

Only obey optimize-tests flag on UI tests that are run-pass #98817

merged 2 commits into from
Jul 4, 2022

Conversation

the8472
Copy link
Member

@the8472 the8472 commented Jul 2, 2022

stage1 UI tests walltime on my machine:

optimize-tests = false, master
25.98s

optimize-tests = true, master
34.69s

optimize-tests = true, patched
28.79s

Effects:

  • faster UI tests
  • llvm asserts get exercised less on build-pass tests
  • the difference between opt and nopt builds shrinks a bit
  • aux libs don't get optimized since they don't have a pass mode and almost never have explicit compile flags

@the8472
Only obey optimize-tests flag on UI tests that are run-pass
125f33a 
```
optimize-tests = false, master
25.98s

optimize-tests = true, master
34.69s

optimize-tests = true, patched
28.79s
```

Effects:

- faster UI tests
- llvm asserts get exercised less on build-pass tests
- the difference between opt and nopt builds shrinks a bit
- aux libs don't get optimized since they don't have a pass mode and almost never have explicit compile flags
@rust-highfive
Copy link
Contributor

r? @Mark-Simulacrum

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 2, 2022
@Mark-Simulacrum
Copy link
Member

I think this makes sense - we can probably adjust nopt (or opt) builders to have different behavior here, but ultimately I suspect impact is pretty minimal.

@bors r+ rollup=iffy

@bors
Copy link
Collaborator

bors commented Jul 2, 2022

📌 Commit 125f33a has been approved by Mark-Simulacrum

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 2, 2022
jyn514
jyn514 reviewed Jul 2, 2022
View reviewed changes
@the8472
Copy link
Member Author

the8472 commented Jul 2, 2022

@bors r-

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 2, 2022
@the8472 the8472 requested a review from jyn514 July 2, 2022 20:20
@the8472 the8472 force-pushed the dont-optimize-ui-tests branch from a9972ca to f719239 Compare July 2, 2022 20:49
@the8472
Copy link
Member Author

the8472 commented Jul 3, 2022

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 3, 2022
@jyn514
Copy link
Member

jyn514 commented Jul 3, 2022

@bors r=Mark-Simulacrum

@bors
Copy link
Collaborator

bors commented Jul 3, 2022

📌 Commit f719239 has been approved by Mark-Simulacrum

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 3, 2022
@bors
Copy link
Collaborator

bors commented Jul 4, 2022

⌛ Testing commit f719239 with merge d2074cb...

@bors
Copy link
Collaborator

bors commented Jul 4, 2022

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing d2074cb to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jul 4, 2022
@bors bors merged commit d2074cb into rust-lang:master Jul 4, 2022
@rustbot rustbot added this to the 1.64.0 milestone Jul 4, 2022
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (d2074cb): comparison url.

Instruction count

  • Primary benchmarks: 🎉 relevant improvements found
  • Secondary benchmarks: 🎉 relevant improvements found
mean1 max count2
Regressions 😿
(primary)		 N/A N/A 0
Regressions 😿
(secondary)		 N/A N/A 0
Improvements 🎉
(primary)		 -0.7% -0.9% 9
Improvements 🎉
(secondary)		 -1.3% -2.3% 12
All 😿🎉 (primary) -0.7% -0.9% 9

Max RSS (memory usage)

Results
  • Primary benchmarks: 🎉 relevant improvement found
  • Secondary benchmarks: mixed results
mean1 max count2
Regressions 😿
(primary)		 N/A N/A 0
Regressions 😿
(secondary)		 2.6% 2.6% 1
Improvements 🎉
(primary)		 -2.0% -2.0% 1
Improvements 🎉
(secondary)		 -2.1% -2.1% 1
All 😿🎉 (primary) -2.0% -2.0% 1

Cycles

Results
  • Primary benchmarks: no relevant changes found
  • Secondary benchmarks: 🎉 relevant improvements found
mean1 max count2
Regressions 😿
(primary)		 N/A N/A 0
Regressions 😿
(secondary)		 2.3% 2.3% 1
Improvements 🎉
(primary)		 N/A N/A 0
Improvements 🎉
(secondary)		 -3.4% -4.7% 9
All 😿🎉 (primary) N/A N/A 0

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

Footnotes

  1. the arithmetic mean of the percent change ↩ ↩2 ↩3

  2. number of relevant changes ↩ ↩2 ↩3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jyn514 jyn514 Awaiting requested review from jyn514

Assignees

@Mark-Simulacrum Mark-Simulacrum

Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Milestone
1.64.0
Development

Successfully merging this pull request may close these issues.
7 participants
@the8472 @rust-highfive @Mark-Simulacrum @bors @jyn514 @rust-timer @rustbot