GNU linker warns “corrupt .drectve” on staticlib binary generated with Rust 1.70 on Windows

[yutannihilation]

It seems, as of version 16, LLVM generates a binary containing incompatible directive with GNU linker. I believe this is the cause:

commit: llvm/llvm-project@c5b3de6

Since Rust 1.70 upgraded LLVM to version 16, we started to see a lot of warnings by GNU linker when I link against staticlibs compiled by Rust. The warnings are like this:

Warning: .drectve `-exclude-symbols:_ZN7testpkg11hello_world17ha68eef8a416aa303E ' unrecognized
Warning: corrupt .drectve at end of def file

It seems I can just ignore this warning (while the concern described in the LLVM's commit can be a real problem in future), but still I feel this is something that should be fixed if possible. I know, from LLVM's viewpoint, it's reasonable to assume the linker is lld and embed the directives that only lld can understand. But, on the other hand, from Rust's viewpoint, is it intentional that the "GNU" target produces a binary that is incompatible with "GNU" linker? I'm honestly not sure about the answer. I'm yet to figure out what the "GNU" in Rust's GNU toolchain/target actually means.

Code

A minimal reproducible code: https://github.com/yutannihilation/rust170_gnu_warning

# compile Rust code to create a staticlib
cargo build --target=x86_64-pc-windows-gnu --lib --release

# compile the C code
gcc -c main.c -o main.o

# link to the staticlib
gcc -o testpkgrust170_gnu_warning.dll main.o -L./target/x86_64-pc-windows-gnu/release -lrust170_gnu_warning -lws2_32 -ladvapi32 -luserenv -lbcrypt -lntdll 

In case of Rust 1.69, we don't see any warnings. However, if it's compiled using Rust 1.70, GNU linker warns:

Warning: corrupt .drectve at end of def file
Warning: corrupt .drectve at end of def file

(This can be reproducible on GHA: result)

Version it worked on

It most recently worked on: Rust 1.69

Version with regression

rustc --version --verbose:

rustc 1.70.0 (90c541806 2023-05-31)
binary: rustc
commit-hash: 90c541806f23a127002de5b4038be731ba1458ca
commit-date: 2023-05-31
host: x86_64-pc-windows-msvc
release: 1.70.0
LLVM version: 16.0.2

Backtrace

References

• the original context: Fix compile error on Windows with Rust 1.70 extendr/rextendr#285 (comment)
• post on the Rust user forum before reporting here: https://users.rust-lang.org/t/corrupt-drectve-warning-on-x86-64-pc-windows-gnu-target-with-rust-1-70/94912

[erikdesjardins]

Looks like support for the -exclude-symbols .drectve was added to GNU ld in this commit, and then added to lld in these two commits.

Those changes were released in Binutils 2.40 (the latest) and LLVM 15, both fairly new versions. GNU ld < 2.40 will emit a warning if it sees this directive, and lld < 15 will fail to link.

[yutannihilation]

Thank you so much! It really helps.

In our particular use case, we use a toolchain called Rtools, which is provided for compiling R packages, and, fortunately, the latest version of it uses binutils 2.40.

These packages have been updated:

(...snip...)
binutils 2.39 to 2.40
(https://cran.r-project.org/bin/windows/Rtools/rtools43/news.html)

Actually, I confirmed I don't see these warnings after I updated the toolchain to the latest version (I didn't notice there's a newer version). So, this might be a less serious problem than I thought at first. I'm not sure if it's fine in general to require such a new version of binutils, though.

[Jules-Bertholet]

@rustbot label -regression-untriaged +regression-from-stable-to-stable

[saurik]

FWIW, I'm running into this issue with lld 14 and I'm definitely not a good position to trivially upgrade to lld 15+ (though I guess if I stick with 1.69.0 for another year or so the maintainers of my toolchain packages will manage to release one based on lld 15).

[hydra]

I'm using the GNU LD linker for two reasons:

1. Because the normal linker hangs 9/10 times - See Linker hangs with no output 9/10 times. #88704
2. Because the normal linker doesn't have the ability to do some things in the way I want for embedded development, so I use arm-none-eabi-ld.

I too started seeing the warnings detailed above after upgrading the nightly version of rust I'm using.

Example warnings from gnu ld 2.34. (arm-none-eabi) and gnu ld 2.39.

Warning: .drectve `-exclude-symbols:"_ZN77_$LT$alloc..raw_vec..RawVec$LT$T$C$A$GT$$u20$as$u20$core..ops..drop..Drop$GT$4drop17hb89ba7c74cb66113E" ' unrecognized
Warning: .drectve `-exclude-symbols:"_ZN5alloc7raw_vec19RawVec$LT$T$C$A$GT$7reserve21do_reserve_and_handle17heb762ccf93d8463bE" ' unrecognized
Warning: corrupt .drectve at end of def file

At the time of writing the latest official GNU Arm Tools is 12.2.MPACBTI-Rel1 https://developer.arm.com/downloads/-/arm-gnu-toolchain-downloads which give you this version:

GNU ld (Arm GNU Toolchain 12.2.MPACBTI-Rel1 (Build arm-12-mpacbti.34)) 2.40.0.20230307

which is 2.40.

However, I still see all the warnings, as follows:

Warning: .drectve `-exclude-symbols:"_ZN77_$LT$alloc..raw_vec..RawVec$LT$T$C$A$GT$$u20$as$u20$core..ops..drop..Drop$GT$4drop17hb89ba7c74cb66113E" ' unrecognized
Warning: .drectve `-exclude-symbols:"_ZN5alloc7raw_vec19RawVec$LT$T$C$A$GT$7reserve21do_reserve_and_handle17heb762ccf93d8463bE" ' unrecognized
Warning: corrupt .drectve at end of def file

It's thus not correct to say that "GNU < 2.40" (see #112368 (comment)) will emit warnings, as 2.40.20230307 certainly still emits them.

I guess I'll have to wait for a new GNU Arm Tools release or build windows binaries somehow, neither of which are appealing. 😄

[workingjubilee]

But, on the other hand, from Rust's viewpoint, is it intentional that the "GNU" target produces a binary that is incompatible with "GNU" linker? I'm honestly not sure about the answer. I'm yet to figure out what the "GNU" in Rust's GNU toolchain/target actually means.

When you use a -gnu target, that is effectively meant to be, for most intents and purposes, "compatible with glibc". I have not really heard Rust team members discuss them in other terms. Obviously, there are some caveats here about dynamic loaders, but that isn't necessarily the same as the static linker.

[mati865]

It seems I can just ignore this warning (while the concern described in the LLVM's commit can be a real problem in future), but still I feel this is something that should be fixed if possible. I know, from LLVM's viewpoint, it's reasonable to assume the linker is lld and embed the directives that only lld can understand. But, on the other hand, from Rust's viewpoint, is it intentional that the "GNU" target produces a binary that is incompatible with "GNU" linker? I'm honestly not sure about the answer.

It's harmless warning. Previously binaries would contain unnecessary symbols and this newly introduced feature gets rid of them. Like already said here it's supported by GNU toolchain (since Binutils 2.40)

I'm yet to figure out what the "GNU" in Rust's GNU toolchain/target actually means.

Most importantly artifacts produced by Rust and GNU toolchain (GCC+Binutils+mingw-w64) are compatible with each other. Linking and the resulting binaries still work but a warning is shown when Binutils are too old to support feature used by LLVM. Surely this is not ideal (maybe there is a way to silence this warning from ld.bfd?) but I don't believe they are incompatible.

It's thus not correct to say that "GNU < 2.40" (see #112368 (comment)) will emit warnings, as 2.40.20230307 certainly still emits them.

2.40 does contain the required changes, either you have some environment issue or their version is not 2.40.
Using 2.40 Binutils from MSYS2:

$ rustc -vV
rustc 1.70.0 (90c541806 2023-05-31)
binary: rustc
commit-hash: 90c541806f23a127002de5b4038be731ba1458ca
commit-date: 2023-05-31
host: x86_64-pc-windows-gnu
release: 1.70.0
LLVM version: 16.0.2

$ cargo b --lib -r
   Compiling rust170_gnu_warning v0.1.0 (D:\tmp\rust170_gnu_warning)
    Finished release [optimized] target(s) in 0.51s

$ gcc -c main.c -o main.o

$ gcc -o testpkgrust170_gnu_warning.dll main.o -L./target/release -lrust170_gnu_warning -lws2_32 -ladvapi32 -luserenv -lbcrypt -lntdll

$ ld --version
GNU ld (GNU Binutils) 2.40
Copyright (C) 2023 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

$ gcc --version
gcc.exe (Rev7, Built by MSYS2 project) 13.1.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

When you use a -gnu target, that is effectively meant to be, for most intents and purposes, "compatible with glibc". I have not really heard people discuss them in other terms.

When speaking of windows-gnu target it's never about glibc but a GNU toolchain so GCC and Binutils (or compatible one like LLVM) coupled with MinGW.

[yutannihilation]

Thanks for the detailed explaination!

Surely this is not ideal (maybe there is a way to silence this warning from ld.bfd?) but I don't believe they are incompatible.

I now agree with you.

Since my problem is almost solved, I'm fine to close this issue. The only two reasons I keep this open are (1) I don't have enough knowledge to judge if this is a negligible issue (2) those who suffer from this warning might find this easier. Should I close this issue?

Regarding #112368 (comment), I also confirmed 2.40 contains the required change. So, it seems it's a problem on Arm's GCC toolchain's side, not on Rust's side.

[mati865]

It might be worth to keep it open for the compiler team to decide though.
While this warning is irrelevant it might scare more people.

[yutannihilation]

Sure, let's keep this open for a while.

[hydra]

2.40 does contain the required changes, either you have some environment issue or their version is not 2.40.
Using 2.40 Binutils from MSYS2:

Same here, ucrt64 environment, versions as follows.

$ rustc -vV
rustc 1.71.0-nightly (39c6804b9 2023-04-19)
binary: rustc
commit-hash: 39c6804b92aa202369e402525cee329556bc1db0
commit-date: 2023-04-19
host: x86_64-pc-windows-gnu
release: 1.71.0-nightly
LLVM version: 16.0.2

$ ld --version
GNU ld (GNU Binutils) 2.40

$ gcc --version
gcc.exe (Rev6, Built by MSYS2 project) 13.1.0

$ uname
MINGW64_NT-10.0-19045

However, I do cross-compilation, I'm compiling for various ST Arm MCUs, and it looks like the version of 'arm-none-eabi-ld' is < 2.4.0, so I'm probably seeing warnings from that version of 'ld'.

$ arm-none-eabi-gcc --version
arm-none-eabi-gcc.exe (Arm GNU Toolchain 12.2.Rel1 (Build arm-12.24)) 12.2.1 20221205
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ arm-none-eabi-ld --version
GNU ld (Arm GNU Toolchain 12.2.Rel1 (Build arm-12.24)) 2.39.0.20221210
Copyright (C) 2022 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

[mati865]

@hydra so those warnings appear when cross-compiling from windows-gnu to arm-none-eabi?

I think those directives shouldn't be used for targets other than Windows. Does it also happen when building from different host triple?

[hydra]

That said, I just tried with this version, the latest GNU Arm Tools release:

$ arm-none-eabi-ld --version
GNU ld (Arm GNU Toolchain 12.2.MPACBTI-Rel1 (Build arm-12-mpacbti.34)) 2.40.0.20230307

And still see errors, as below:

Warning: corrupt .drectve at end of def file
Warning: .drectve `-exclude-symbols:_ZN3std2rt10lang_start17h9c3ccb811358e938E ' unrecognized
Warning: corrupt .drectve at end of def file
Warning: .drectve `-exclude-symbols:"_ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17h355a3ebbaa224f8aE" ' unrecognized
Warning: .drectve `-exclude-symbols:_ZN4core3ops8function6FnOnce9call_once17haa9bcdad473de67cE ' unrecognized

It is 2.40.0.20230307, there are no other 'ld' instances in my path.

$ which ld
/ucrt64/bin/ld
$ which arm-none-eabi-ld
/d/Programs/GNUArmTools/active/bin/arm-none-eabi-ld
$ /d/Programs/GNUArmTools/active/bin/arm-none-eabi-ld --version
GNU ld (Arm GNU Toolchain 12.2.MPACBTI-Rel1 (Build arm-12-mpacbti.34)) 2.40.0.20230307

@hydra so those warnings appear when cross-compiling from windows-gnu to arm-none-eabi?

yes, as follows:

rustup:

$ rustup show
Default host: x86_64-pc-windows-gnu
rustup home:  D:\Users\Hydra\.rustup

installed targets for active toolchain
--------------------------------------

thumbv7em-none-eabihf
thumbv7m-none-eabi
x86_64-pc-windows-gnu


active toolchain
----------------

nightly-2023-06-17-x86_64-pc-windows-gnu (overridden by 'D:\Users\Hydra\Documents\dev\playground\test\rust-toolchain.toml')
rustc 1.72.0-nightly (6bba06146 2023-06-16)

I think those directives shouldn't be used for targets other than Windows. Does it also happen when building from different host triple?

Don't know, and I don't have any specific code to test that scenario. If you have example command lines/projects for me to compile let me know and I can try for you.

Suggestions?

[hydra]

As a further update to this, since the linker output now contains all the warnings, I just had to scroll back past 2800 (!!!) lines of warnings to get to the error message that caused by build to fail.

I suspect the size of build logs on CI servers will be filling up disks much quicker now too.

It's making things very unworkable here.

[hydra]

Is anyone working on this? Can we get it assigned to be worked on? Currently actual error messages get buried in the output, making development /extremely/ painful.

[mati865]

@hsn10 Rust doesn't call ld.bfd directly but instead relies on GCC (i686-w64-mingw32-gcc.exe or x86_64-w64-mingw32-gcc.exe), check if you have it in your PATH. When GCC is found, Rust will disable self-contained mode but you can also force it with -C link-self-contained=no.

[hsn10]

rustc needs what exe in path? that x86_64-w64 .. -gcc which comes bundled with rust?

because in my gcc 13 installation x86_64-w64-mingw32 is a directory with (ar.exe, ld.exe, as.exe, nm.exe, ld.bfd, etc), not single exe file.

[mati865]

$ which x86_64-w64-mingw32-gcc.exe
/d/msys64/mingw64/bin/x86_64-w64-mingw32-gcc.exe

[hsn10]

Its probably designed to work with this https://github.com/niXman/mingw-builds-binaries/releases mingw distribution. This is most passive distro and it will take a while until they upgrade gnu linker. Other distros are more active.

I am using this:

• https://nuwen.net/files/mingw/components-19.0.7z
• https://nuwen.net/mingw.html

[riking]

The last several comments are about a completely different issue, and binutils has released 2.42 (issue fixed in 2.40). I think this can be closed.

[mati865]

Rust still ships very old Binutils so this issue is not fixed.

[amyspark]

I've seen this reproduce with Binutils 2.42 and GCC 14.1.0 (I built the toolchain based on MSYS2's specs, with the host being x86_64-pc-windows-gnu).

[mati865]

Either there must have been some other toolchain in your PATH or Binutils have regressed. I haven't seen this problem with MSYS2 toolchains.

[Zxilly]

Can reproduce this error on https://github.com/ZNotify/cli/actions/runs/10145751720/job/28052213372

[mati865]

@Zxilly this warning doesn't cause build failures. In your case the linker has crashed, I'd suggest trying with different version.

[Zxilly]

@mati865 Thanks, but I didn't have any fixes in mind, so I simply removed the gnu target. This issue came up after the 1.70 release, but I can't reproduce it locally.

[mati865]

You probably have a different Binutils version locally and it doesn't crash.
Rust upgrade is not very relevant, Rust output has changed a bit in a way that causes this crash but a similar crash could also happen when you modify your code in a specific way.

[Zxilly]

For binutils I have 2.42 locally, and https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md shows the CI have the binutils 2.39. I'll continue to delve into this when I can.

[hsn10]

rust won't pick locally installed binutils automatically because auto detection code there can detect only one special distribution and that distribution ships outdated binutils.

niXman/mingw-builds-binaries#68

[mati865]

Rust will pick any linker from the PATH that is named <arch>-w64-mingw32-gcc so it's compatible with most, if not all distributions.