Nightly rustc cannot load LLVM plugins on ARM64 platforms (impacts afl.rs crate)

[vanhauser-thc]

This issue prevents using the effective afl.rs (AFL++) plugins for fuzzing in Rust on ARM64 platforms.

The nightly rustc allows for loading LLVM plugins with -Z llvm-plugins=/path/to/plugin.

This works without problems on Linux x86_x64, however on ARM64 - both Linux and MacOS - this fails with symbols not being resolvable.

How to reproduce:

• this requires LLVM in the same version that is used in nightly (currently llvm 18):
  Linux: wget http://apt.llvm.org/llvm.sh ; sudo bash ./llvm.sh 18
  Macos: brew install llvm + setting up the PATH to prefer clang/clang++ from brew and not xlang

# Install cargo-afl because this is where we need LLVM plugins from
cargo install cargo-afl
# Compile the AFL++ plugins for cargo-afl - this requires LLVM installed
cargo afl config --plugins --force
# Now get a simple test example
git clone --depth=1 https://github.com/rust-fuzz/afl.rs
# Build any example
cd afl.rs/afl
cargo afl build --example arbitrary

On Linux x86_x64 this works:

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 8.05s

On ARM64 Ubuntu and MacOS this both fails:

error: failed to run LLVM passes: Could not load library '/Users/ssh_debug/.local/share/afl.rs/rustc-1.81.0-nightly-6be96e3/afl.rs-0.15.8/afl-llvm/cmplog-instructions-pass.so': dlopen(/Users/ssh_debug/.local/share/afl.rs/rustc-1.81.0-nightly-6be96e3/afl.rs-0.15.8/afl-llvm/cmplog-instructions-pass.so, 0x0009): symbol not found in flat namespace '__ZN4llvm17PreservedAnalyses14AllAnalysesKeyE'

After a lot of debugging, the reason seems to be that on Linux x86_x64 the following library is present:
libLLVM-18-rust-1.81.0-nightly.so
But on ARM64 rust nightly installation this is missing.
So this might be the reason for this issue.

[vanhauser-thc]

FYI @smoelius (maybe you can point a knowledge dev to this issue, with this many issues in this tracker maybe this one is buried fast otherwise)

[smoelius]

Nice debugging!

maybe you can point a knowledge dev to this issue

@nikic Please forgive me for pinging you, but this issue reminds me of #121889, which I think you fixed. Is there any insight you could offer here?

[nikic]

LLVM plugins are currently only supported on platforms that use the LLVM dylib, rather than linking LLVM statically into librustc_driver. I believe this is currently only the case on x86_64-unknown-linux-gnu.

[vanhauser-thc]

@nikic thanks for the clarification. So this limitation is known. Is there an issue or project item for that that we can follow to see once this also works for non intel Linux?

[nikic]

It looks like we currently don't have a tracking issue for this option, so I filed #127577.

[smoelius]

Thank you, @nikic!