-
-
Notifications
You must be signed in to change notification settings - Fork 14.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rustup: addressing a series of issues encountered while using Rust's self-contained ld.lld
#314268
Conversation
ping? @Mic92 |
ld.lld
Result of 3 packages built:
|
What command can I use for testing, which did not work before? i.e. in which rust version does this first break? |
Looks like |
self-contained `ld.lld`
The latest nightly rustc has switched the linker to the self-contained lld. This is an example for testing:
[package]
name = "testlld"
version = "0.1.0"
edition = "2021"
[build-dependencies]
cc = "1.0"
extern "C" {
fn add(a: i32, b: i32) -> i32;
}
fn main() {
unsafe { add(1, 2) };
}
fn main() {
cc::Build::new()
.cpp(true)
.file("foo.cpp")
.cpp_link_stdlib("stdc++")
.compile("foo");
}
#include <iostream>
#include <vector>
extern "C" {
int add(int a, int b) {
std::vector<int> numbers = {1, 2, 3, 4, 5};
numbers.push_back(6);
for (int number : numbers) {
std::cout << number << " ";
}
std::cout << std::endl;
return a + b;
}
} Then run the script: rustup uninstall nightly-2024-05-20
rustup install nightly-2024-05-20
cargo clean
unset LD_LIBRARY_PATH
cargo +nightly-2024-05-20 run We will get the following error:
After using $ patchelf --set-interpreter $(patchelf --print-interpreter $(which patchelf)) ~/.rustup/toolchains/nightly-2024-05-20-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld/ld.lld
$ unset LD_LIBRARY_PATH
$ cargo +nightly-2024-05-20 run
target/debug/testlld: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory |
Reading the ELF file format on 32-bit systems is different. I don't have the environment to test it. :'( |
Testing 32-bit is hopefully as easy as building |
@ofborg build pkgsi686Linux.rustup |
Interestingly on x86_64-linux I get the following build error:
However this might be a different issue, i.e. maybe too many build cores on the system I am building from. UPDATE Yes. When building with less cores, it becomes better. |
I can't reproduce it on my 16-core, 32-thread system. Could you test the upstream repository? I'm curious whether the issue lies with You can test it with the following script: git clone https://github.com/rust-lang/rustup.git --branch 1.26.0 --depth 1
cd rustup
rustup install 1.77.2
cargo +1.77.2 test |
I don't think we need to fix this in this PR. It's a AMD EPYC 7713P 64-Core, I often run into the problem that build systems and test suite are not exercised with this core count. |
I mainly want to test whether $ env RUSTUP_DEBUG=true rustup toolchain install nightly-i686-unknown-linux-gnu --force-non-host
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/lib/rustlib/i686-unknown-linux-gnu/lib/libstd-4ebf06893886ecff.so" using patchelf
...
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/bin/rustc" using patchelf
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/bin/rustdoc" using patchelf
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/lib/librustc_driver-370689965a44b509.so" using patchelf
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/lib/libstd-4ebf06893886ecff.so" using patchelf
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/lib/rustlib/i686-unknown-linux-gnu/bin/gcc-ld/ld.lld" using patchelf
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/lib/rustlib/i686-unknown-linux-gnu/bin/gcc-ld/ld64.lld" using patchelf
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/lib/rustlib/i686-unknown-linux-gnu/bin/gcc-ld/lld-link" using patchelf
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/lib/rustlib/i686-unknown-linux-gnu/bin/gcc-ld/wasm-ld" using patchelf
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/lib/rustlib/i686-unknown-linux-gnu/bin/rust-lld" using patchelf
debug: patching "/home/dianqk/.rustup/toolchains/nightly-i686-unknown-linux-gnu/libexec/rust-analyzer-proc-macro-srv" using patchelf |
I pushed a commit for this patch file: DianQK/rustup@f14f5f2, which should help in reviewing the changes. |
Looks like the 32bit rustup build is broken anyway... So let's focus on 32-bit rust toolchains instead. |
So far the 32-bit headers still seem to be corrupted:
I now actually doubt, this ever worked because we are trying to use the 64-bit link-loader for a 32-bit executable. |
What seems to work is
|
env = lib.optionalAttrs (pname == "rustup") { | ||
inherit (stdenv.cc.bintools) expandResponseParams shell suffixSalt wrapperName coreutils_bin; | ||
hardening_unsupported_flags = ""; | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if we could make a better regression test here, given the growing complexity of machinery that gets added here? If we cannot make it pure, even a self-contained test script would be nice i.e. based on the example project that you already provided.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, we do need a test case. I plan to add an internal test case for the rustup project in a subsequent PR.
Is |
I'm not quite sure what you're expressing here. Are you referring to results entirely on i686? |
I can use |
If you can fix it, it's nice, but it would be also not a new regression because it's broken just now. |
I thought, this would also require patchelf, but actually it doesn't because everything is compiled afterwards. |
I found that |
Only pkgsi686Linux.rustup failed but we also have |
This seems reasonable as well. I'm not currently considering dealing with this particular cross-compilation, but I'll try to add a test case. BTW, we should backport this to 24.05. The backport note could be: fixed rustc's self-contained lld not functioning. |
Should we use nix-ld instead of patchelf in rustup for this? I've seen some people (from nixos_zhcn) complaining about rustup's toolchains break constantly on each system updates, possibly because it's not added to GC roots? I'm not sure if this PR is a good place to discuss it, but this PR does make patchelf in rustup more and more complicated. |
I think we should remain the behavior of rustc itself.
I think we could add a sub-command like
I don't have any good ideas here. |
We could also add a gcroot into the rustup directory. |
Writing such test cases on rustup isn't easy. I read through some test code, and it seems like rustup's internal tests only focus on rustup's own behavior. I can only add this test in nixpkgs, but I'm not familiar with nix. Perhaps we should create an additional package or add a test script directly in the |
I think this might confuse many people as to why certain stores can't be cleaned up. |
Maybe write a test that just uses nix for dependencies but has internet access. Than we can at least test manually when we change things. |
Successfully created backport PR for |
Description of changes
Fixes #312661.
This PR primarily makes the following changes:
ld.lld
and other files by determining whether they are ELF files.ld-wrapped.sh
told.lld
(inspired entirely by @oxalica's work in mk-component-set: wrap gcc-ld/ld.lld instead of multicall rust-lld oxalica/rust-overlay#174).rpath
torust-lld
.Determining whether to patch based on the pathname is inaccurate, which results in the loss of lld under
bin/gcc-ld
and attempts to patch*.rlib
files.I determine the patching method by looking up
.interp
, which should be simpler than the code fromreadelf
.Things done
nix.conf
? (See Nix manual)sandbox = relaxed
sandbox = true
nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"
. Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/
)Add a 👍 reaction to pull requests you find important.