Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only include metadata for non-dynamic libraries in rustc-dev #105609

Merged
merged 3 commits into from
Jan 3, 2023

Conversation

bjorn3
Copy link
Member

@bjorn3 bjorn3 commented Dec 12, 2022

The actual object code should be linked from librustc_driver.so, which is still included in rustc-dev. This saves on download time and disk usage.

Fixes #103538

@bjorn3 bjorn3 added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Dec 12, 2022
@rustbot
Copy link
Collaborator

rustbot commented Dec 12, 2022

r? @jyn514

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 12, 2022
@bjorn3
Copy link
Member Author

bjorn3 commented Dec 12, 2022

@bors try

@bors
Copy link
Contributor

bors commented Dec 12, 2022

⌛ Trying commit 1103d021d3c253ab0885497995fb4d9c06d3eb96 with merge 143447c08dc7c029d9b9f4c5905aabb7aa218e68...

@bors
Copy link
Contributor

bors commented Dec 12, 2022

💔 Test failed - checks-actions

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 12, 2022
@bjorn3
Copy link
Member Author

bjorn3 commented Dec 12, 2022

@bors try

@bors
Copy link
Contributor

bors commented Dec 12, 2022

⌛ Trying commit 6662d1eb4992f351ab434dc52eb9be245d366bbf with merge 68167fc36cc9c280dd8942dc97107334abaee4aa...

@bors
Copy link
Contributor

bors commented Dec 12, 2022

💔 Test failed - checks-actions

@bjorn3
Copy link
Member Author

bjorn3 commented Dec 12, 2022

@bors try

@bors
Copy link
Contributor

bors commented Dec 12, 2022

⌛ Trying commit 61bc7a640cc5b2f0bbe55274910419c3cdb37ab0 with merge 6d5b237d87d74959bd7608dfc370ef9feda36b2c...

@bors
Copy link
Contributor

bors commented Dec 12, 2022

☀️ Try build successful - checks-actions
Build commit: 6d5b237d87d74959bd7608dfc370ef9feda36b2c (6d5b237d87d74959bd7608dfc370ef9feda36b2c)

@rust-log-analyzer

This comment has been minimized.

@bjorn3
Copy link
Member Author

bjorn3 commented Dec 13, 2022

Works as intended. It cuts both download and disk size in half.

Before

detecting the channel of the `37d7de337903a558dbeb1e82c844fe915ab8ff25` toolchain...
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/37d7de337903a558dbeb1e82c844fe915ab8ff25/rustc-nightly-x86_64-unknown-linux-gnu.tar.xz>...
67.75 MB / 67.75 MB [==================================================================================================================================================] 100.00 % 13.27 MB/s 
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/37d7de337903a558dbeb1e82c844fe915ab8ff25/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz>...
220.29 MB / 220.29 MB [================================================================================================================================================] 100.00 % 12.68 MB/s 
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/37d7de337903a558dbeb1e82c844fe915ab8ff25/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz>...
29.74 MB / 29.74 MB [==================================================================================================================================================] 100.00 % 12.33 MB/s 
toolchain `37d7de337903a558dbeb1e82c844fe915ab8ff25` is successfully installed!
$ du -sh ~/.rustup/toolchains/37d7de337903a558dbeb1e82c844fe915ab8ff25/lib/rustlib/x86_64-unknown-linux-gnu/lib/
1.6G    /home/gh-bjorn3/.rustup/toolchains/37d7de337903a558dbeb1e82c844fe915ab8ff25/lib/rustlib/x86_64-unknown-linux-gnu/lib/

After

detecting the channel of the `6d5b237d87d74959bd7608dfc370ef9feda36b2c` toolchain...
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/6d5b237d87d74959bd7608dfc370ef9feda36b2c/rustc-nightly-x86_64-unknown-linux-gnu.tar.xz>...
67.71 MB / 67.71 MB [==================================================================================================================================================] 100.00 % 11.51 MB/s 
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/6d5b237d87d74959bd7608dfc370ef9feda36b2c/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz>...
106.88 MB / 106.88 MB [=================================================================================================================================================] 100.00 % 6.31 MB/s 
downloading <https://ci-artifacts.rust-lang.org/rustc-builds/6d5b237d87d74959bd7608dfc370ef9feda36b2c/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz>...
29.73 MB / 29.73 MB [===================================================================================================================================================] 100.00 % 9.17 MB/s 
toolchain `6d5b237d87d74959bd7608dfc370ef9feda36b2c` is successfully installed!
$ du -sh ~/.rustup/toolchains/6d5b237d87d74959bd7608dfc370ef9feda36b2c/lib/rustlib/x86_64-unknown-linux-gnu/lib/
729M    /home/gh-bjorn3/.rustup/toolchains/6d5b237d87d74959bd7608dfc370ef9feda36b2c/lib/rustlib/x86_64-unknown-linux-gnu/lib/

@rust-log-analyzer

This comment has been minimized.

@rustbot rustbot added A-translation Area: Translation infrastructure, and migrating existing diagnostics to SessionDiagnostic T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Dec 13, 2022
@bjorn3 bjorn3 marked this pull request as ready for review December 13, 2022 11:31
@rustbot
Copy link
Collaborator

rustbot commented Dec 13, 2022

rustc_error_messages was changed

cc @davidtwco, @compiler-errors, @JohnTitor, @estebank, @TaKO8Ki

Some changes occurred in src/tools/rustfmt

cc @rust-lang/rustfmt

@bjorn3
Copy link
Member Author

bjorn3 commented Dec 13, 2022

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Dec 13, 2022
@rustbot
Copy link
Collaborator

rustbot commented Dec 13, 2022

The Miri subtree was changed

cc @rust-lang/miri

The actual object code should be linked from librustc_driver.so,
which is still included in rustc-dev. This saves on download time and
disk usage.
@bjorn3
Copy link
Member Author

bjorn3 commented Dec 31, 2022

Done. There is a CI failure that doesn't make any sense though.

@jyn514
Copy link
Member

jyn514 commented Jan 2, 2023

@bors r+

@bors
Copy link
Contributor

bors commented Jan 2, 2023

📌 Commit 8c000a6 has been approved by jyn514

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 2, 2023
@bors
Copy link
Contributor

bors commented Jan 3, 2023

⌛ Testing commit 8c000a6 with merge 3b1c8a9...

@bors
Copy link
Contributor

bors commented Jan 3, 2023

☀️ Test successful - checks-actions
Approved by: jyn514
Pushing 3b1c8a9 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 3, 2023
@bors bors merged commit 3b1c8a9 into rust-lang:master Jan 3, 2023
@rustbot rustbot added this to the 1.68.0 milestone Jan 3, 2023
@bjorn3 bjorn3 deleted the shrink_rustc_dev branch January 3, 2023 11:08
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (3b1c8a9): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
4.3% [4.3%, 4.3%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-1.1% [-1.1%, -1.1%] 1
All ❌✅ (primary) - - 0

Cycles

This benchmark run did not return any relevant results for this metric.

@dtolnay
Copy link
Member

dtolnay commented Jan 4, 2023

For anyone landing here with an error like this:

error: crate `rustc_ast_pretty` required to be available in rlib format, but was not found in this form
  |
  = help: try adding `extern crate rustc_driver;` at the top level of this crate

If you add extern crate rustc_driver and then get an error like this:

error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "/tmp/rustcJ6ZtcA/symbols.o" "target/debug/deps/foo-71b8be8abb28cf36.y793e40jtmbzz9q.rcgu.o" "-Wl,--as-needed" "-L" "target/debug/deps" "-L" ".rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-L" ".rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bdynamic" "-lrustc_driver-eaaa85fc498d717c" "-L" ".rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lstd-62c4894b82797b30" "-Wl,-Bstatic" ".rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-66b9c3ae5ff29c13.rlib" "-Wl,-Bdynamic" "-lLLVM-15-rust-1.68.0-nightly" "-ldl" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" ".rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow" "-nodefaultlibs"
  = note: /usr/bin/ld: cannot find -lLLVM-15-rust-1.68.0-nightly: No such file or directory
          collect2: error: ld returned 1 exit status

then you need to do rustup component add llvm-tools.

Aaron1011 pushed a commit to Aaron1011/rust that referenced this pull request Jan 6, 2023
Only include metadata for non-dynamic libraries in rustc-dev

The actual object code should be linked from librustc_driver.so, which is still included in rustc-dev. This saves on download time and disk usage.

Fixes rust-lang#103538
winterqt pushed a commit to PuercoPop/nixpkgs that referenced this pull request May 12, 2023
Reading this [rust-issue] it seems that we have to link against
rustc_driver. The [clippy.nix] expression already does something similar

Of the 4 executables found in the result of building rustfmt only
rustfmt and git-rustfmt needed to be linked. The other worked without
linking to rustc_driver.

[rust-issue]: rust-lang/rust#105609
[clippy.nix]: https://github.com/NixOS/nixpkgs/blob/c8cf570dae9b41f30395b71f9b432418b4ff0ebc/pkgs/development/compilers/rust/clippy.nix#L27-L36
winterqt pushed a commit to NixOS/nixpkgs that referenced this pull request May 12, 2023
Reading this [rust-issue] it seems that we have to link against
rustc_driver. The [clippy.nix] expression already does something similar

Of the 4 executables found in the result of building rustfmt only
rustfmt and git-rustfmt needed to be linked. The other worked without
linking to rustc_driver.

[rust-issue]: rust-lang/rust#105609
[clippy.nix]: https://github.com/NixOS/nixpkgs/blob/c8cf570dae9b41f30395b71f9b432418b4ff0ebc/pkgs/development/compilers/rust/clippy.nix#L27-L36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-translation Area: Translation infrastructure, and migrating existing diagnostics to SessionDiagnostic merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rustc-dev component recently became a lot bigger
7 participants