Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use ReadCache for archive reading in bootstrap #133462

Merged
merged 1 commit into from
Nov 26, 2024
Merged

Use ReadCache for archive reading in bootstrap #133462

merged 1 commit into from
Nov 26, 2024

Conversation

mustartt
Copy link
Contributor

Address expensive archive reading in bootstrap. This fixes #133268

Enable the std feature of object to use ReadCache instead of reading the entire archive file into memory to check for headers. This takes minimal extra time to compile compared to introducing other expensive dependencies to bootstrap.

r? jieyouxu

@rustbot
Copy link
Collaborator

rustbot commented Nov 25, 2024

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @jieyouxu (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

  • @rustbot author: the review is finished, PR author should check the comments and take action accordingly
  • @rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 25, 2024
jieyouxu
jieyouxu approved these changes Nov 25, 2024
View reviewed changes
Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this LGTM, one tiny nit.

compiler/rustc_metadata/src/creader.rs Show resolved Hide resolved
@jieyouxu jieyouxu added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 25, 2024
@jieyouxu
Copy link
Member

Thanks!
@bors r+ rollup

@bors
Copy link
Contributor

bors commented Nov 25, 2024

📌 Commit 9f1cfec has been approved by jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 25, 2024
@jieyouxu jieyouxu added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. O-aix OS: Big Blue's Advanced Interactive eXecutive.. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 25, 2024
jhpratt added a commit to jhpratt/rust that referenced this pull request Nov 26, 2024
@jhpratt
Rollup merge of rust-lang#133462 - mustartt:aix-improve-bootstrap-loa…
265caaf 
…ding, r=jieyouxu

Use ReadCache for archive reading in bootstrap

Address expensive archive reading in bootstrap. This fixes rust-lang#133268

Enable the `std` feature of `object` to use `ReadCache` instead of reading the entire archive file into memory to check for headers. This takes minimal extra time to compile compared to introducing other expensive dependencies to `bootstrap`.

r? jieyouxu
@jhpratt jhpratt mentioned this pull request Nov 26, 2024
Closed
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 26, 2024
@bors
Auto merge of rust-lang#133490 - jhpratt:rollup-f4ao5vz, r=jhpratt
33c3a5e 
Rollup of 28 pull requests

Successful merges:

 - rust-lang#132605 (CI: increase timeout from 4h to 6h)
 - rust-lang#133042 (btree: add `{Entry,VacantEntry}::insert_entry`)
 - rust-lang#133070 (Lexer tweaks)
 - rust-lang#133136 (Support ranges in `<[T]>::get_many_mut()`)
 - rust-lang#133140 (Inline ExprPrecedence::order into Expr::precedence)
 - rust-lang#133248 (CI: split x86_64-msvc-ext job)
 - rust-lang#133282 (Shorten the `MaybeUninit` `Debug` implementation)
 - rust-lang#133304 (Revert diagnostics hack to fix ICE 132920)
 - rust-lang#133326 (Remove the `DefinitelyInitializedPlaces` analysis.)
 - rust-lang#133362 (No need to re-sort existential preds in relate impl)
 - rust-lang#133367 (Simplify array length mismatch error reporting (to not try to turn consts into target usizes))
 - rust-lang#133394 (Bail on more errors in dyn ty lowering)
 - rust-lang#133410 (target check_consistency: ensure target feature string makes some basic sense)
 - rust-lang#133411 (the emscripten OS no longer exists on non-wasm targets)
 - rust-lang#133419 (Added a doc test for std::path::strip_prefix)
 - rust-lang#133430 (Tweak parameter mismatch explanation to not say `{unknown}`)
 - rust-lang#133435 (miri: disable test_downgrade_observe test on macOS)
 - rust-lang#133443 (Remove dead code stemming from the old effects desugaring (II))
 - rust-lang#133449 (std: expose `const_io_error!` as `const_error!`)
 - rust-lang#133450 (remove "onur-ozkan" from  users_on_vacation)
 - rust-lang#133454 (Update test expectations to accept LLVM 'initializes' attribute)
 - rust-lang#133458 (Fix `Result` and `Option` not getting a jump to def link generated)
 - rust-lang#133462 (Use ReadCache for archive reading in bootstrap)
 - rust-lang#133464 (std::thread: avoid leading whitespace in some panic messages)
 - rust-lang#133467 (tests: Add recursive associated type bound regression tests)
 - rust-lang#133470 (Cleanup: delete `//@ pretty-expanded` directive)
 - rust-lang#133473 (tests: Add regression test for recursive enum with Cow and Clone)
 - rust-lang#133481 (Disable `avr-rjmp-offset` on Windows for now)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 26, 2024
@bors
Auto merge of rust-lang#133490 - jhpratt:rollup-f4ao5vz, r=jhpratt
199955e 
Rollup of 28 pull requests

Successful merges:

 - rust-lang#132605 (CI: increase timeout from 4h to 6h)
 - rust-lang#133042 (btree: add `{Entry,VacantEntry}::insert_entry`)
 - rust-lang#133070 (Lexer tweaks)
 - rust-lang#133136 (Support ranges in `<[T]>::get_many_mut()`)
 - rust-lang#133140 (Inline ExprPrecedence::order into Expr::precedence)
 - rust-lang#133248 (CI: split x86_64-msvc-ext job)
 - rust-lang#133282 (Shorten the `MaybeUninit` `Debug` implementation)
 - rust-lang#133304 (Revert diagnostics hack to fix ICE 132920)
 - rust-lang#133326 (Remove the `DefinitelyInitializedPlaces` analysis.)
 - rust-lang#133362 (No need to re-sort existential preds in relate impl)
 - rust-lang#133367 (Simplify array length mismatch error reporting (to not try to turn consts into target usizes))
 - rust-lang#133394 (Bail on more errors in dyn ty lowering)
 - rust-lang#133410 (target check_consistency: ensure target feature string makes some basic sense)
 - rust-lang#133411 (the emscripten OS no longer exists on non-wasm targets)
 - rust-lang#133419 (Added a doc test for std::path::strip_prefix)
 - rust-lang#133430 (Tweak parameter mismatch explanation to not say `{unknown}`)
 - rust-lang#133435 (miri: disable test_downgrade_observe test on macOS)
 - rust-lang#133443 (Remove dead code stemming from the old effects desugaring (II))
 - rust-lang#133449 (std: expose `const_io_error!` as `const_error!`)
 - rust-lang#133450 (remove "onur-ozkan" from  users_on_vacation)
 - rust-lang#133454 (Update test expectations to accept LLVM 'initializes' attribute)
 - rust-lang#133458 (Fix `Result` and `Option` not getting a jump to def link generated)
 - rust-lang#133462 (Use ReadCache for archive reading in bootstrap)
 - rust-lang#133464 (std::thread: avoid leading whitespace in some panic messages)
 - rust-lang#133467 (tests: Add recursive associated type bound regression tests)
 - rust-lang#133470 (Cleanup: delete `//@ pretty-expanded` directive)
 - rust-lang#133473 (tests: Add regression test for recursive enum with Cow and Clone)
 - rust-lang#133481 (Disable `avr-rjmp-offset` on Windows for now)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 26, 2024
@bors
Auto merge of rust-lang#133490 - jhpratt:rollup-f4ao5vz, r=jhpratt
6ce8a37 
Rollup of 28 pull requests

Successful merges:

 - rust-lang#132605 (CI: increase timeout from 4h to 6h)
 - rust-lang#133042 (btree: add `{Entry,VacantEntry}::insert_entry`)
 - rust-lang#133070 (Lexer tweaks)
 - rust-lang#133136 (Support ranges in `<[T]>::get_many_mut()`)
 - rust-lang#133140 (Inline ExprPrecedence::order into Expr::precedence)
 - rust-lang#133248 (CI: split x86_64-msvc-ext job)
 - rust-lang#133282 (Shorten the `MaybeUninit` `Debug` implementation)
 - rust-lang#133304 (Revert diagnostics hack to fix ICE 132920)
 - rust-lang#133326 (Remove the `DefinitelyInitializedPlaces` analysis.)
 - rust-lang#133362 (No need to re-sort existential preds in relate impl)
 - rust-lang#133367 (Simplify array length mismatch error reporting (to not try to turn consts into target usizes))
 - rust-lang#133394 (Bail on more errors in dyn ty lowering)
 - rust-lang#133410 (target check_consistency: ensure target feature string makes some basic sense)
 - rust-lang#133411 (the emscripten OS no longer exists on non-wasm targets)
 - rust-lang#133419 (Added a doc test for std::path::strip_prefix)
 - rust-lang#133430 (Tweak parameter mismatch explanation to not say `{unknown}`)
 - rust-lang#133435 (miri: disable test_downgrade_observe test on macOS)
 - rust-lang#133443 (Remove dead code stemming from the old effects desugaring (II))
 - rust-lang#133449 (std: expose `const_io_error!` as `const_error!`)
 - rust-lang#133450 (remove "onur-ozkan" from  users_on_vacation)
 - rust-lang#133454 (Update test expectations to accept LLVM 'initializes' attribute)
 - rust-lang#133458 (Fix `Result` and `Option` not getting a jump to def link generated)
 - rust-lang#133462 (Use ReadCache for archive reading in bootstrap)
 - rust-lang#133464 (std::thread: avoid leading whitespace in some panic messages)
 - rust-lang#133467 (tests: Add recursive associated type bound regression tests)
 - rust-lang#133470 (Cleanup: delete `//@ pretty-expanded` directive)
 - rust-lang#133473 (tests: Add regression test for recursive enum with Cow and Clone)
 - rust-lang#133481 (Disable `avr-rjmp-offset` on Windows for now)

r? `@ghost`
`@rustbot` modify labels: rollup
@GuillaumeGomez GuillaumeGomez mentioned this pull request Nov 26, 2024
Merged
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 26, 2024
@bors
Auto merge of rust-lang#133500 - GuillaumeGomez:rollup-4vcthwo, r=Gui…
dff3e7c 
…llaumeGomez

Rollup of 13 pull requests

Successful merges:

 - rust-lang#133411 (the emscripten OS no longer exists on non-wasm targets)
 - rust-lang#133419 (Added a doc test for std::path::strip_prefix)
 - rust-lang#133430 (Tweak parameter mismatch explanation to not say `{unknown}`)
 - rust-lang#133443 (Remove dead code stemming from the old effects desugaring (II))
 - rust-lang#133450 (remove "onur-ozkan" from  users_on_vacation)
 - rust-lang#133454 (Update test expectations to accept LLVM 'initializes' attribute)
 - rust-lang#133462 (Use ReadCache for archive reading in bootstrap)
 - rust-lang#133464 (std::thread: avoid leading whitespace in some panic messages)
 - rust-lang#133467 (tests: Add recursive associated type bound regression tests)
 - rust-lang#133470 (Cleanup: delete `//@ pretty-expanded` directive)
 - rust-lang#133473 (tests: Add regression test for recursive enum with Cow and Clone)
 - rust-lang#133481 (Disable `avr-rjmp-offset` on Windows for now)
 - rust-lang#133495 (add test for alias-bound shadowing, rename folder)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 7e3422f into rust-lang:master Nov 26, 2024
6 checks passed
@rustbot rustbot added this to the 1.85.0 milestone Nov 26, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Nov 26, 2024
@rust-timer
Unrolled build for rust-lang#133462
9149c4d 
Rollup merge of rust-lang#133462 - mustartt:aix-improve-bootstrap-loading, r=jieyouxu

Use ReadCache for archive reading in bootstrap

Address expensive archive reading in bootstrap. This fixes rust-lang#133268

Enable the `std` feature of `object` to use `ReadCache` instead of reading the entire archive file into memory to check for headers. This takes minimal extra time to compile compared to introducing other expensive dependencies to `bootstrap`.

r? jieyouxu
@mustartt
Copy link
Contributor Author

mustartt commented Nov 26, 2024 •
edited
Loading

Probably a good idea to have this patch to speed up the boostrap for 1.84 builds. This also fixes a small build error on AIX.

@rustbot label beta-nominated

@rustbot rustbot added the beta-nominated Nominated for backporting to the compiler in the beta channel. label Nov 26, 2024
@apiraino
Copy link
Contributor

Beta backport declined as per compiler team on Zulip, as far as we can see this is mostly about perf. improvements.

@rustbot label -beta-nominated

@rustbot rustbot removed the beta-nominated Nominated for backporting to the compiler in the beta channel. label Nov 28, 2024
@davidtwco
Copy link
Member

Similarly to #133217 (comment):

Thanks for taking the time to fix this bug. Just to add to what @apiraino has said, the project's backport policy describes how we typically only backport regressions, and as far as we can tell, this isn't a regression, it's always been an issue. We also weigh up the risk vs reward for a backport, how likely a change is to break something vs how critical the issue is (can it be worked around, is it on a tier one platform, etc).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jieyouxu jieyouxu jieyouxu approved these changes

Assignees

@jieyouxu jieyouxu

Labels
O-aix OS: Big Blue's Advanced Interactive eXecutive.. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
1.85.0
Development

Successfully merging this pull request may close these issues.

AIX: bootstrap investigate efficient way of reading archive
6 participants
@mustartt @rustbot @jieyouxu @bors @apiraino @davidtwco