Skew between clang and rust's compiler-rt causes linker errors

[mk12]

We're seeing this failure when linking stage0 rustc-main on aarch64-unknown-linux-gnu, with a recent clang built from tip-of-tree:

undefined reference to __aarch64_ldadd8_acq_rel

More information, with some paths elided as /.../:

Building stage0 compiler artifacts (aarch64-unknown-linux-gnu -> aarch64-unknown-linux-gnu)
...
   Compiling rustc-main v0.0.0 (/.../rust/compiler/rustc)
error: linking with `/.../clang++` failed: exit code: 1
  |
  = note: "/.../clang++" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-Wl,--eh-frame-hdr" "-L" "/.../stage0-sysroot/lib/rustlib/aarch64-unknown-linux-gnu/lib" "/.../stage0-rustc/aarch64-unknown-linux-gnu/release/deps/rustc_main-9d27db1d70883821.rustc_main.bdhmewtr-cgu.0.rcgu.o" "/.../stage0-rustc/aarch64-unknown-linux-gnu/release/deps/rustc_main-9d27db1d70883821.rustc_main.bdhmewtr-cgu.1.rcgu.o" "/.../stage0-rustc/aarch64-unknown-linux-gnu/release/deps/rustc_main-9d27db1d70883821.rustc_main.bdhmewtr-cgu.2.rcgu.o" "-o" "/.../stage0-rustc/aarch64-unknown-linux-gnu/release/deps/rustc_main-9d27db1d70883821" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/.../stage0-rustc/aarch64-unknown-linux-gnu/release/deps" "-L" "/.../stage0-rustc/release/deps" "-L" "/.../stage0-rustc/aarch64-unknown-linux-gnu/release/build/psm-a7f05eacf47d934f/out" "-L" "/.../stage0-rustc/aarch64-unknown-linux-gnu/release/build/rustc_llvm-e6d15703f36aa661/out" "-L" "/.../llvm/build/lib" "-L" "/.../stage0-sysroot/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-L" "/.../stage0-rustc/aarch64-unknown-linux-gnu/release/deps" "-lrustc_driver-00d9b1ea3dbe5cad" "-Wl,--start-group" "-L" "/.../stage0-sysroot/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-lstd-7e24c4f6cf2d2fe2" "-Wl,--end-group" "-Wl,-Bstatic" "/.../stage0-sysroot/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcompiler_builtins-61a5967ebc23b832.rlib" "-Wl,-Bdynamic" "-lLLVM-11-rust-1.51.0-nightly" "-lc++" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "--target=aarch64-unknown-linux-gnu" "--sysroot=/.../linux-arm64" "-fuse-ld=lld" "-Wl,-rpath,$ORIGIN/../lib"
  = note: ld.lld: error: /.../stage0-rustc/aarch64-unknown-linux-gnu/release/deps/librustc_driver-00d9b1ea3dbe5cad.so: undefined reference to __aarch64_ldadd8_acq_rel [--no-allow-shlib-undefined]
          clang-13: error: linker command failed with exit code 1 (use -v to see invocation)


error: aborting due to previous error

error: could not compile `rustc-main`

It seems rust-lang/llvm-project's compiler-rt doesn't include the out-of-line atomic functions that clang is now emitting by default since D93585.

For now we're going to try to fix this by adding -mno-outline-atomics to CFLAGS in config.toml.

To prevent this sort of issue in the future, it would be nice if there were an option to drop the -nodefaultlibs option to clang and replace it with -nostdlib++ -nolibc. This would cause "system" clang to use its own compiler runtime instead of the one shipped with rust-lang/llvm-project.

It may even be a good idea to make this the default. It’s unclear why Rust needs to link the compiler-rt in its own LLVM tree with code built by the system compiler.

cc @tmandry

[tmandry]

It’s unclear why Rust needs to link the compiler-rt in its own LLVM tree with code built by the system compiler.

This is the big question I have.. presumably during bootstrapping, we link

• LLVM backend, built with system C++ compiler
• Rust code, built with beta Rust compiler

Does the Rust compiler emit code which depends on compiler-rt? EDIT: yes

If not, why can't we use the system's?

[tmandry]

To answer my own question.. the LLVM backend emits code that depends on compiler-rt. It's not just the clang runtime.

Looking forward, I don't think we're guaranteed to be able to do this with any version mismatch between the compiler and runtime.

I think one way to fix this would be to build LLVM in two stages.. once with the system compiler (including building clang), and then have clang LLVM a second time. If I'm thinking about this correctly, it would prevent a mismatch like this.

For now the build option described above would work, which allows falling back to the system compiler runtime for any missing symbols.