Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lld doesn't work on NixOS #125321

Closed
DaniD3v opened this issue May 20, 2024 · 23 comments
Closed

lld doesn't work on NixOS #125321

DaniD3v opened this issue May 20, 2024 · 23 comments
Labels
A-linkage Area: linking into static, shared libraries and binaries C-discussion Category: Discussion or questions that doesn't represent real issues. O-NixOS Operating system: NixOS, https://nixos.org/

Comments

@DaniD3v
Copy link

DaniD3v commented May 20, 2024

Code

Compilation for the cargo hello-world template with the new nightly failed.

cargo build
   Compiling abc v0.1.0 (/tmp/abc)
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/self-contained:/home/notyou/.cargo/bin:/home/notyou/.local/bin:/run/wrappers/bin:/home/notyou/.nix-profile/bin:/nix/profile/bin:/home/notyou/.local/state/nix/profile/bin:/etc/profiles/per-user/notyou/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin:.local/bin:/nix/store/m3gq2f6iwh3g67zrs27m7w82b37l8gf8-binutils-wrapper-2.41/bin:/nix/store/a4qsjxipd9jgjdy89fggzq9nlkf2m8jq-pciutils-3.12.0/bin:/nix/store/md6hh4rrcrf99nssvcam3qaqs3skj086-gcc-wrapper-13.2.0/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustcZqYSP3/symbols.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.15uilqnmz9fl0pdsf1pp09837.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.2dblsqlv26pnrb2gnxxzoafd0.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.6ocuuw51mfoea9wtaypp2vi8j.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.72b8ipzr9dmhbx6yeplx0a04m.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.9ewb0tvybbd504u6turswtxvi.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.dn1mnlgzn93y0wymda22rajod.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.3ct60f9m81ggqnk1jmipk7pg8.rcgu.o" "-Wl,--as-needed" "-L" "/tmp/abc/target/debug/deps" "-L" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-de48b8168d6cf4fa.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-9f3477fb95a0bba7.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-210d920812faea91.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-f3d3451767410a17.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-1a79dd36d08251de.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-886230e7120831b2.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-2caea079085a58a2.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-a9d7d97cdc65a449.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-5727477b0a78105a.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-7e555563aa211118.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-c8f13465f1a795b2.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-ee5b5774583426df.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-14df174c91007922.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-e8bfe52be756260a.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-6275035a459b3ada.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-b6892f3c52c68f01.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-f72b956e24d1de70.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-632ae0f28c5e55ff.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-e8b7e96e438f08f6.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-B/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/self-contained" "-o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: collect2: fatal error: execvp: No such file or directory
          compilation terminated.


error: could not compile `abc` (bin "abc") due to 1 previous error

It compiles when I either switch to the stable toolchain,
or set the RUSTFLAGS="-Z linker-features=-lld" env variable

Version it worked on

It most recently worked on: nightly-2024-05-17

Version with regression

rustc --version --verbose:

rustc --version --verbose
rustc 1.80.0-nightly (d84b90375 2024-05-19)
binary: rustc
commit-hash: d84b9037541f45dc2c52a41d723265af211c0497
commit-date: 2024-05-19
host: x86_64-unknown-linux-gnu
release: 1.80.0-nightly
LLVM version: 18.1.4

Backtrace

Backtrace

   Compiling abc v0.1.0 (/tmp/abc)
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/self-contained:/home/notyou/.cargo/bin:/home/notyou/.local/bin:/run/wrappers/bin:/home/notyou/.nix-profile/bin:/nix/profile/bin:/home/notyou/.local/state/nix/profile/bin:/etc/profiles/per-user/notyou/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin:.local/bin:/nix/store/m3gq2f6iwh3g67zrs27m7w82b37l8gf8-binutils-wrapper-2.41/bin:/nix/store/a4qsjxipd9jgjdy89fggzq9nlkf2m8jq-pciutils-3.12.0/bin:/nix/store/md6hh4rrcrf99nssvcam3qaqs3skj086-gcc-wrapper-13.2.0/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustcJFepHM/symbols.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.15uilqnmz9fl0pdsf1pp09837.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.2dblsqlv26pnrb2gnxxzoafd0.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.6ocuuw51mfoea9wtaypp2vi8j.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.72b8ipzr9dmhbx6yeplx0a04m.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.9ewb0tvybbd504u6turswtxvi.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.dn1mnlgzn93y0wymda22rajod.rcgu.o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e.3ct60f9m81ggqnk1jmipk7pg8.rcgu.o" "-Wl,--as-needed" "-L" "/tmp/abc/target/debug/deps" "-L" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-de48b8168d6cf4fa.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-9f3477fb95a0bba7.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-210d920812faea91.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-f3d3451767410a17.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-1a79dd36d08251de.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-886230e7120831b2.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-2caea079085a58a2.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-a9d7d97cdc65a449.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-5727477b0a78105a.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-7e555563aa211118.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-c8f13465f1a795b2.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-ee5b5774583426df.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-14df174c91007922.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-e8bfe52be756260a.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-6275035a459b3ada.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-b6892f3c52c68f01.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-f72b956e24d1de70.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-632ae0f28c5e55ff.rlib" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-e8b7e96e438f08f6.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-B/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/self-contained" "-o" "/tmp/abc/target/debug/deps/abc-76f7a6c126880f8e" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: collect2: fatal error: execvp: No such file or directory
          compilation terminated.
          

error: could not compile `abc` (bin "abc") due to 1 previous error

@DaniD3v DaniD3v added C-bug Category: This is a bug. regression-untriaged Untriaged performance or correctness regression. labels May 20, 2024
@rustbot rustbot added I-prioritize Issue: Indicates that prioritization has been requested for this issue. needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels May 20, 2024
@DaniD3v
Copy link
Author

DaniD3v commented May 20, 2024

related: (same error)
#71515

@pnkfelix
Copy link
Member

pnkfelix commented May 20, 2024

cc @lqd (at least, I know they were driving the "use lld by default" effort a la rust-lang/compiler-team#510 )

presumably this was injected by #124129

@lqd
Copy link
Member

lqd commented May 20, 2024

I don't know NixOS, so I'm not going to be of much help. I believe it has issues with lld in general, for example NixOS/nixpkgs#24744 that could be related, or this could be a duplicate of that issue. Changes such as NixOS/nixpkgs#24744 (comment) could be needed to make both lld and rust-lld work there.

cc may not be able to find lld because the toolchain is incomplete (the "No such file or directory" part of the error) or because nix's configuration prevents it (the execvp part of the error, I guess).

To see if we can rule out the first case, I'd like to know the contents of the directories we tell cc to look for the lld wrappers calling rust-lld:

  • /home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin: it should at least contain the rust-lld binary, and the following gcc-ld subfolder.
  • and /home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld: it should contain the 3/4 target-specific lld executables.

If the contents of the directories match what we expect, I would guess the issue is more NixOS/nixpkgs#24744.

@DaniD3v
Copy link
Author

DaniD3v commented May 20, 2024

  • /home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin: it should at least contain the rust-lld binary, and the following gcc-ld subfolder.

It does:

ls /home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin

gcc-ld  rust-lld
  • and /home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld: it should contain the 3/4 target-specific lld executables.

Don't know if that requirement is satisfied:

ls /home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld

ld64.lld  ld.lld  lld-link  wasm-ld

If this issue is nix-specific we should probably make it extra clear what the work-around is until the issue is resolved.

I added this to my ~/.cargo/config.toml:

[target.x86_64-unknown-linux-gnu]
rustflags = ["-Zlinker-features=-lld"] # fix nightly bug

@lqd
Copy link
Member

lqd commented May 20, 2024

The expected executables are there, the toolchain looks complete AFAICT. Again I don't know NixOs, but there are wasm and aarch64 targets that already use rust-lld by default, so I would expect these to fail there, regardless of #124129.

Similarly, we now just do what users did to change the linker, with regular link args like -Clink-arg=-fuse-ld=lld, as well as adding a -B path like /home/notyou/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld. One can emulate what rustc does, and that needs to work, does it?

The way to revert to the default linker like you did is indeed explicitly mentioned in the blog post announcing the change.

It seems less like a nightly bug to me than a known nix-specific issue.

@Nemo157
Copy link
Member

Nemo157 commented May 20, 2024

How are you installing the nightly? It erroring out at that stage seems like it's not "fixing" the ld.lld binary's link paths. Using oxalica/rust-overlay it manages to correctly execute ld.lld, but then if the binary it builds uses any native dependencies it fails because the linker isn't "fixing" the binaries it builds (which is NixOS/nixpkgs#24744 related, oxalica/rust-overlay#168).

nbdd0121 added a commit to Rust-for-Linux/klint that referenced this issue May 20, 2024
With rust-lang/rust#124129, x64 linux switches
to use LLD by default, but this causes dependency issues on NixOS.
Disable LLD for now.

This is tracked by rust-lang/rust#125321.
@Nemo157
Copy link
Member

Nemo157 commented May 20, 2024

If it's via nixpkgs-rustup, I assume this patch needs updating as it doesn't match these new file paths: https://github.com/NixOS/nixpkgs/blob/4cc02342ff90b8484f11e68bdac985837bb956f9/pkgs/development/tools/rust/rustup/0001-dynamically-patchelf-binaries.patch (but that will just get it to the same state as oxalica/rust-overlay, the self-contained lld will execute but not fully work correctly).

@Nemo157
Copy link
Member

Nemo157 commented May 20, 2024

Nixpkgs issue: NixOS/nixpkgs#312661

@DaniD3v
Copy link
Author

DaniD3v commented May 20, 2024

How are you installing the nightly?

Using rustup.

@jieyouxu jieyouxu added the O-NixOS Operating system: NixOS, https://nixos.org/ label May 22, 2024
@saethlin saethlin removed the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label May 22, 2024
@apiraino apiraino removed the I-prioritize Issue: Indicates that prioritization has been requested for this issue. label May 23, 2024
@lqd
Copy link
Member

lqd commented May 23, 2024

Summarizing the discussion from this zulip thread. It's unfortunate but there's nothing we can really do on our side: NixOS' packaging needs to be updated for lld/rust-lld to be executed and to produce working binaries.

This issue is technically a duplicate of open issues in https://github.com/NixOS/nixpkgs/ but we won't close it yet for visibility, so that people can see this summary if they encounter the issue themselves.

@DianQK mentions a possible temporary workaround by patching rustc's lld wrapper: patchelf --set-interpreter $(patchelf --print-interpreter $(which patchelf)) ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld/ld.lld . But our understanding is that while that would fix running rust-lld, it may sometimes still not be enough: lld itself may not always produce working binaries, as mentioned by Nemo157.

Disabling rust-lld with -Zlinker-features=-lld via RUSTFLAGS or .cargo/config.toml, will fall back to the previous default linker. This works and will unblock people temporarily until the packaging issue is fixed downstream.

@lqd lqd removed C-bug Category: This is a bug. regression-untriaged Untriaged performance or correctness regression. labels May 23, 2024
@dj8yfo
Copy link

dj8yfo commented May 29, 2024

@lqd , is there a way to override globally and tell cargo which ld.lld to use?

After installing ld.lld with method proposed in NixOS/nixpkgs#24744 (comment),
cargo continues to use ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld/ld.lld .

Creating a symlink helps:

rm ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld/ld.lld
ln -s $(which ld.lld) ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld/ld.lld	

@lqd
Copy link
Member

lqd commented May 29, 2024

The symlink may be broken the next you time do a rustup update, but .cargo/config.toml profiles should help.

@Nemo157
Copy link
Member

Nemo157 commented May 29, 2024

is there a way to override globally and tell cargo which ld.lld to use?

You can tell rustc to not pass the path to the self-contained linker with RUSTFLAGS='-Clink-self-contained=-linker -Zunstable-options', that leaves the rest of the new defaults active so that it will still pass -fuse-ld=lld and expect it to be on the PATH. Or you can add it explicitly to the paths given to gcc: RUSTFLAGS="-Clink-arg=-B$(nix-build '<nixpkgs>' --no-out-link -A pkgs.llvmPackages.bintools)/bin"; I expected this might be enough on its own, but because of the order of the flags passed to gcc the self-contained linker is still given priority.

There's two useful debugging flags when testing this: first enable logging the linker command RUSTC_LOG=rustc_codegen_ssa::back::link=info, second ask the linker to log its version RUSTFLAGS='-Clink-arg=-Wl,-v' which also causes collect2 to log the full path to which ld.lld it has chosen (needs the first enabled to actually see the output).

@lqd
Copy link
Member

lqd commented May 29, 2024

also readelf -p .comment, e.g.

$ readelf -p .comment target/debug/helloworld

String dump of section '.comment':
  [     0]  Linker: LLD 18.1.6
  [    13]  rustc version 1.80.0-nightly (84b40fc90 2024-05-27)
  [    48]  GCC: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

and to add a bit to Nemo's explanation, nightly has two things enabled on x64 linux:

  • -Zlinker-features=+lld, this instructs rustc to try to use lld in known contexts: when using a c/c++ compiler as the linker driver, we add -fuse-ld=lld. Disabling that deactivates using lld altogether of course.
  • -Clink-self-contained=+linker, this instructs rustc to pass the path to "our rust-lld in the sysroot". These are -B paths to target-specific copies of lld-wrapper binaries that call rust-lld (let's gloss over flavors and arguments and more importantly, why). Deactivating only that, can be seen as a fallback to system lld.

@oxalica
Copy link
Contributor

oxalica commented May 31, 2024

Using oxalica/rust-overlay it manages to correctly execute ld.lld, but then if the binary it builds uses any native dependencies it fails because the linker isn't "fixing" the binaries it builds (which is NixOS/nixpkgs#24744 related, oxalica/rust-overlay#168).

As a note, the mentioned rust-overlay issue is fixed. rust-overlay now wraps the shipped ld.lld with ld-wrapper from nixpkgs, and the nightly rustc with default flags (invoking bundled ld.lld) should work out-of-box.

@DianQK
Copy link
Member

DianQK commented May 31, 2024

@oxalica I have added ld-wrapped.sh to rustup based on your PR: NixOS/nixpkgs#314268. I'm not sure if my patch is reasonable.

@samueltardieu
Copy link
Contributor

@oxalica I have added ld-wrapped.sh to rustup based on your PR: NixOS/nixpkgs#314268. I'm not sure if my patch is reasonable.

Indeed it works for me.

@DianQK
Copy link
Member

DianQK commented Jun 12, 2024

NixOS/nixpkgs#314268 is now available in nixos-unstable. It will soon be available in nixos-24.05 or release-24.05: https://nixpk.gs/pr-tracker.html?pr=319172.

@spearman
Copy link

Btw setting -Zlinker-features=-lld doesn't seem to work when building doctests, it still uses the bundled linker

@lqd
Copy link
Member

lqd commented Jun 17, 2024

@spearman how are you passing the flag? Remember that rustdoc uses RUSTDOCFLAGS, not RUSTFLAGS.

@dj8yfo
Copy link

dj8yfo commented Jun 19, 2024

NixOS/nixpkgs#314268 is now available in nixos-unstable. It will soon be available in nixos-24.05 or release-24.05: https://nixpk.gs/pr-tracker.html?pr=319172.

@DianQK can you please share an absolutely minimal recipe on how to use this? I would really interested in how to couple this with home-manager

@DianQK
Copy link
Member

DianQK commented Jun 19, 2024

NixOS/nixpkgs#314268 is now available in nixos-unstable. It will soon be available in nixos-24.05 or release-24.05: https://nixpk.gs/pr-tracker.html?pr=319172.

@DianQK can you please share an absolutely minimal recipe on how to use this? I would really interested in how to couple this with home-manager

I think you can just check this: https://github.com/Nilstrieb/nixos/blob/d16cacb11f7f08d42bf5ed3ffcdc4d5ed9dccc6e/home-manager/common.nix#L12.

Just change your channel to nixos-unstable. You can also fork this repo and cherry-pick this commit. You can also use a custom pkg in local.

@jieyouxu jieyouxu added A-linkage Area: linking into static, shared libraries and binaries C-discussion Category: Discussion or questions that doesn't represent real issues. labels Jun 19, 2024
@DianQK
Copy link
Member

DianQK commented Jul 1, 2024

Although rustup on NixOS isn't very user-friendly, this issue has been resolved in version 24.05. We can close it now.

@lqd lqd closed this as completed Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-linkage Area: linking into static, shared libraries and binaries C-discussion Category: Discussion or questions that doesn't represent real issues. O-NixOS Operating system: NixOS, https://nixos.org/
Projects
None yet
Development

No branches or pull requests