Inconsistent #[link(name = "")] resolution on Windows

[ralfbiedert]

Background

We have a Rust project (lets call it dylib) where we want to produce a shared library (.dll, .dylib, .so) for various platforms. As part of the build process we also want to verify the provided headers match the library.

To do that we have a separate project dylib_test, where we extract C doc tests into individual generated_nnn.c files, produce a generated.rs file that knows all C tests, and emits a #[test] for each, which we then want to run with cargo test.

The dylib project should only emit a cdylib, and the dylib_test should only link against that cdylib, to make sure the actual .dll, ... works.

Problem
When linking against dylib from the dylib_test crate, the #[link(name = "dylib")] attribute behaves inconsistently. On Mac and Linux the attribute works. On Windows, it fails with an

error: linking with `link.exe` failed: exit code: 1181
  |
  = note: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.21.27702\\bin\\HostX64\\x64\\link.exe" "/NOLOGO" "/NXCOMPAT" "/LIBPATH:C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.17sc3p6xub1nskk5.rcgu.o" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.1d4yl2ra8f02uq65.rcgu.o" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.2boci68ky9pbilmt.rcgu.o" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.2kg3da7ad2y6u5t4.rcgu.o" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.328elmwbbr69laus.rcgu.o" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.3pjpoaqleh1qku54.rcgu.o" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.48cxjx0647kyqroa.rcgu.o" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.4xn06m0s17xpy7q7.rcgu.o" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.bdw34kj2m6ietun.rcgu.o" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.dxf6w5mo3hfjrm.rcgu.o" "/OUT:D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.exe" "D:\\Development\\Source\\rust_issues\\target\\debug\\deps\\dylib_test-16b85ad62e2643c4.3tou2046e23p3ckg.rcgu.o" "/OPT:REF,NOICF" "/DEBUG" "/NATVIS:C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/LIBPATH:D:\\Development\\Source\\rust_issues\\target\\debug\\deps" "/LIBPATH:D:\\Development\\Source\\rust_issues\\target\\debug\\build\\dylib_test-df967a17b396e952\\out" "/LIBPATH:C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "dylib.lib" "test_harness.lib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libtest-5868cd1efdb1a14a.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libterm-4a1cd4f9337faede.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libgetopts-c94244661cc3c49c.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libunicode_width-5d83c260a3fd1c81.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd-69ffdb8026c42481.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libpanic_unwind-324700ef1bcafc8b.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libbacktrace-66fccb33cfc77b7d.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_demangle-429461051376b1a5.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libhashbrown-16c7dc58d869d64a.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_alloc-b5f6113de773ee4b.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libunwind-52ee0b65e04bcb17.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcfg_if-eee1f1d0cc78c8f9.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liblibc-f5bd27a471d11e8c.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liballoc-bb531cfdfdeadec5.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_core-48d94702399abeaa.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcore-edaeab1200806e65.rlib" "C:\\Users\\rb\\.rustup\\toolchains\\nightly-2019-07-08-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcompiler_builtins-5cb137db32dc726e.rlib" "kernel32.lib" "advapi32.lib" "ws2_32.lib" "userenv.lib" "msvcrt.lib"
  = note: LINK : fatal error LNK1181: cannot open input file 'dylib.lib'

I believe this is because Rust on Windows produces a dylib.dll and dylib.dll.lib (instead of a dylib.lib). When then resolving the library Rust apparently isn't aware that to resolve name="dylib" it should not only look for the (dylib.lib, dylib.dll) pair, but instead also for the (dylib.dll.lib, dylib.dll) pair.

Steps

1. Clone https://github.com/ralfbiedert/rust_issues/tree/dylib_issues
2. Run cargo test (on Windows)

Possible Solution(s)
When given a #[link(name = "dylib")], consider to look for dylib.dll.lib on Windows.

The current workaround is to specify #[link(name = "dylib.dll")], but from a cross-platform perspective that feels inconsistent.

Notes

Output of cargo version:

cargo 1.37.0-nightly (4c1fa54d1 2019-06-24)
rustc 1.38.0-nightly (6e310f2ab 2019-07-07)
x86_64-pc-windows-msvc

[alexcrichton]

Thanks for the report! I"ve transferred this issue to the rust-lang/rust repository since this is more about the output filenames of rustc than anything to do with Cargo

[retep998]

We can't change #[link(name = "dylib")] to look for dylib.dll.lib, due to the inherent limitations of relying on an external linker.

Producing dylib.dll.lib is intentional to not conflict with dylib.lib for static libraries.

I doubt there's anything we can do about this.

[ralfbiedert]

We can't change #[link(name = "dylib")] to look for dylib.dll.lib, due to the inherent limitations of relying on an external linker.

I'm not sure I follow. When invoking the linker for dylib_test, doesn't Rust know already that dylib is a [dependency] of crate-type = [ "cdylib" ] and that it just produced a dylib.dll.lib for it?

If during compilation of the dylib there can be logic to prevent a name conflict, can't there be the same logic when preparing the lib names during linking?

[retep998]

When you do #[link(name = "dylib")], rustc tells link.exe to link to dylib.lib. It cannot tell the linker to look for either dylib.lib or dylib.dll.lib, so it would have to tell the linker to link to only dylib.dll.lib which would be a breaking change. Currently cargo does not inform rustc of the cdylib and staticlib outputs from dependencies so rustc does not have that knowledge to make any decisions.

As for a workaround, you can use #[cfg_attr]:

#[cfg_attr(not(windows), link(name = "dylib"))]
#[cfg_attr(windows, link(name = "dylib.dll"))]

[ralfbiedert]

I just checked with gnu and msvc, the correct way seems to be:

#[cfg_attr(all(target_os = "windows", target_env = "msvc"), link(name = "dylib.dll"))]
#[cfg_attr(not(all(target_os = "windows", target_env = "msvc")), link(name = "dylib"))]
extern "C" { ... }

Although I wouldn't be surprised if other os / env combinations needs their own special handling.

[retep998]

Once #60260 lands, there will be windows-gnu targets that use the llvm linker instead of the mingw linker, and the llvm linker requires linking through an import library so you will need link(name = "dylib.dll") for those specific targets. Really I wish we'd just always emit import libraries on all windows targets for the sake of consistency.

[rossy62]

"Producing dylib.dll.lib is intentional to not conflict with dylib.lib for static libraries."

This makes sense in a linux output environment. But windows generates dylib.lib and dylib.exp dylib.dll for a DLL. If a windows users builds a DLL there cannot be a .lib of the same name in that directory. So the naming convention is not consistent with Windows IMHO.
When I implicitly link a DLL in my Native 'C' windows projects I use
#pragma comment(lib,"mylib.lib")
And the linker may look for mylib.lib at link time, but knows to load mylib.dll at runtime.
I can't link to RUST modules implicitly if the .exp,.lib,.dll are not the same name. I can if I rename them (this worked yesterday).
DLL's in Windows are a real PITA, and I'm pretty impressed I could get my C exe to load a RUST DLL, which is dependent on a native 'C' DLL at all. So KUDOS to all the RUST people that allowed this to work at all. It's really quite amazing. And it had to work for me to consider continuing to look into using RUST. So again.. AWESOME.
This is just a minor issue (filenames), but I'm thinking if it were fixed it might help adoption.