Linking fails with x86_64-pc-windows-gnu toolchain

[eiderdaus]

Mio defines NtCancelIoFileEx as follows:

#[link(name = "ntdll")]
extern "system" {
    /// See <https://processhacker.sourceforge.io/doc/ntioapi_8h.html#a0d4d550cad4d62d75b76961e25f6550c>
    ///
    /// This is an undocumented API and as such not part of <https://github.com/microsoft/win32metadata>
    /// from which `windows-sys` is generated, and also unlikely to be added, so
    /// we manually declare it here
    fn NtCancelIoFileEx(
        FileHandle: HANDLE,
        IoRequestToCancel: *mut IO_STATUS_BLOCK,
        IoStatusBlock: *mut IO_STATUS_BLOCK,
    ) -> NTSTATUS;
}

If I add fn main(){} and build with cargo build I get the following linker error:

error: linking with `x86_64-w64-mingw32-gcc` failed: exit code: 1
  |
  = note: "x86_64-w64-mingw32-gcc" "-fno-use-linker-plugin" "-Wl,--dynamicbase" "-Wl,--disable-auto-image-base" "-m64" "-Wl,--high-entropy-va" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\self-contained\\crt2.o" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsbegin.o" "C:\\Users\\eiderdaus\\AppData\\Local\\Temp\\rustcJSg3zJ\\symbols.o" "C:\\foo\\target\\debug\\deps\\foo-cb89f6e868929cf3.1bc41oyhwfcqyo6z.rcgu.o" "C:\\foo\\target\\debug\\deps\\foo-cb89f6e868929cf3.2e77gpka6erk4c1x.rcgu.o" "C:\\foo\\target\\debug\\deps\\foo-cb89f6e868929cf3.3wy12xbuznp9hor0.rcgu.o" "C:\\foo\\target\\debug\\deps\\foo-cb89f6e868929cf3.4f95o7n55leh2ke1.rcgu.o" "C:\\foo\\target\\debug\\deps\\foo-cb89f6e868929cf3.4rbije3ghxyb129a.rcgu.o" "C:\\foo\\target\\debug\\deps\\foo-cb89f6e868929cf3.53992k6tesbub9hu.rcgu.o" "-L" "C:\\foo\\target\\debug\\deps" "-L" "C:\\Users\\eiderdaus\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\windows_x86_64_gnu-0.42.0\\lib" "-L" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-lntdll" "-Wl,-Bstatic" "C:\\foo\\target\\debug\\deps\\libwindows_sys-87741728b4e67991.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd-583b0c1a8d918e48.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libpanic_unwind-32bbe0f97d4b5e74.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libobject-b4abf79acd3cfd5f.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libmemchr-01a39e8be25b0a33.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libaddr2line-4741964b98a78b52.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libgimli-ca9aa9ad07d84224.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_demangle-77da45e82831415d.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libstd_detect-6b504ee3232db397.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libhashbrown-f23e5590b13d2ba1.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libminiz_oxide-707623ebc84f4938.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libadler-6c1f623f129f8c4e.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_std_workspace_alloc-b83298f4db169fa7.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libunwind-24a266989a087673.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcfg_if-451066e352fc52df.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liblibc-f0b9b386ac2b1f89.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\liballoc-aecd3bcc2854e359.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\librustc_std_workspace_core-295042be5ab55e2d.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcore-427c90bb0249ec1c.rlib" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\libcompiler_builtins-efeda5f809566e63.rlib" "-Wl,-Bdynamic" "-lwindows" "-lkernel32" "-ladvapi32" "-luserenv" "-lkernel32" "-lws2_32" "-lbcrypt" "-lgcc_eh" "-l:libpthread.a" "-lmsvcrt" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-luser32" "-lkernel32" "-Wl,--nxcompat" "-nostartfiles" "-L" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib" "-L" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\self-contained" "-o" "C:\\foo\\target\\debug\\deps\\foo-cb89f6e868929cf3.exe" "-Wl,--gc-sections" "-no-pie" "-nodefaultlibs" "C:\\Users\\eiderdaus\\.rustup\\toolchains\\stable-x86_64-pc-windows-gnu\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib\\rsend.o"
  = note: ld: cannot find -lntdll


warning: `foo` (bin "foo") generated 1 warning
error: could not compile `foo` due to previous error; 1 warning emitted

Compiling the example from README.md fails with the same error (only with more dependencies in the linker command) and so does my Tokio-dependent application.

The reason is obviously #[link(name = "ntdll")]. There's no libntdll.a in my %USERPROFILE%\.rustup\toolchains\stable-x86_64-pc-windows-gnu\lib\rustlib\x86_64-pc-windows-gnu\lib\self-contained folder.

The windows-sys crate for example also provides some functions from ntdll.dll but those are annotated with link(name = "windows") and they provide their own libwindows.a. This works flawlessly for me. winapi-rs provides libwindows_ntdll.a in addition to many other libraries but it doesn't actually provide any functions from ntdll.dll (I think).

I am using Windows 10 and just reinstalled Rust freshly via Rustup, changing only the platform tuple to x86_64-pc-windows-gnu and leaving all other defaults alone. I am uncertain which version of Rust or mio introduced this issue. I only found out when I wanted to recompile a Tokio-dependent project which still worked a few months ago at least.

[Thomasdezeeuw]

Why are you not using x86_64-pc-windows-msvc?

[eiderdaus]

Why are you not using x86_64-pc-windows-msvc?

Do I really need to elaborate? Honestly your question baffles me. There are many reasons:

• It provides a much nicer "works out of the box" experience because it's way simpler to set up (literally asking Rustup to install x86_64-pc-windows-gnu was all I had to do, no MSVC or build tools needed).
• It doesn't burden my hard disk as much.
• If my code works with x86_64-pc-windows-gnu I'll be more likely to be able to cross-compile from Linux should I ever need to.
• I like open source:
  • It makes me independent from a single company and
  • open source software has other advantages (figuring those out is left as an exercise for the reader).
• I don't really see many advantages of the MSVC toolchain:
  • Apart from this exact issue above I never had any problems at all with the GNU toolchain.
  • Because debugging C++ using MSVC is such a nice experience one time I tried to setup debugging Rust with MSVC and miserably failed. The Rust extension for MSVC was barely working at the time and has not been updated since.
• At times I dislike MSFT.
• It's cheaper for companies and other entities that can't use MSVC for free. This is especially relevant to me.
  • In my opinion (and I'm not alone with that) Rust would be better off serving an MSFT-free toolchain by default on Windows.

[Thomasdezeeuw]

Why are you not using x86_64-pc-windows-msvc?

Do I really need to elaborate? Honestly your question baffles me.

Okay.

There are many reasons:
* It provides a much nicer "works out of the box" experience because it's way simpler to set up (literally asking Rustup to install x86_64-pc-windows-gnu was all I had to do, no MSVC or build tools needed).

* It doesn't burden my hard disk as much.

* If my code works with `x86_64-pc-windows-gnu` I'll be more likely to be able to cross-compile from Linux should I ever need to.

* I like open source:
  
  * It makes me independent from a single company and
  * open source software has other advantages (figuring those out is left as an exercise for the reader).

* I don't really see many advantages of the MSVC toolchain:
  
  * Apart from this exact issue above I never had any problems at all with the GNU toolchain.
  * Because debugging C++ using MSVC is such a nice experience one time I tried to setup debugging Rust with MSVC and miserably failed. The Rust extension for MSVC was barely working at the time and has not been updated since.

* At times I dislike MSFT.

But you're running Windows 10? Not judging or disagreeing mind you...

* It's cheaper for companies and other entities that can't use MSVC for free. This is especially relevant to me.
  
  * In my opinion ([and I'm not alone with that](https://internals.rust-lang.org/t/use-lld-by-default-on-windows-to-mitigate-microsoft-c-build-tools-licencing-issues/16606)) Rust would be better off serving an MSFT-free toolchain by default on Windows.

I was merely wondering if you a specific reason not use the default MSVC based target, and looking at the list it seems you do.

Currently we don't test with x86_64-pc-windows-gnu, which means we don't really support it, but we can always add support of course.

Does changing #[link(name = "ntdll")] to #[link(name = "windows")] work? Or windows_ntdll, which winapi seems to do?

Alternatively we can try adding it to windows-sys since it's now documented: https://learn.microsoft.com/en-us/windows/win32/devnotes/nt-cancel-io-file-ex (/cc @kennykerr).

[eiderdaus]

Okay.

Please forgive my unfriendlyness, I regretted my wording the moment I pressed the comment button.

But you're running Windows 10? Not judging or disagreeing mind you...

Well as you might imagine I have my reasons for that, too 😅

Does changing #[link(name = "ntdll")] to #[link(name = "windows")] work? Or windows_ntdll, which winapi seems to do?

Yes both do work. It's winapi_ntdll for winapi-rs, I have spelled it wrongly before. I just had to figure out how to patch dependent crates for testing. Is the [patch] section in Cargo.toml the way to go? Somehow it spits out the same linker error three times in a row now if I override mio with a path dependency that refers to an unchanged Mio repository.

Alternatively we can try adding it to windows-sys since it's now documented: https://learn.microsoft.com/en-us/windows/win32/devnotes/nt-cancel-io-file-ex (/cc @kennykerr).

That would be great. I think changing the link attribute without actually providing suitable linker import libraries makes this rather brittle as it depends on an implementation detail of windows-sys or winapi-rs.

[kennykerr]

Sure, I'll see about getting this function included in the Win32 metadata.

[kennykerr]

I opened microsoft/wdkmetadata#9 - if they can include the function then windows-rs can include the definition and generate new libs for all supported targets.

[kennykerr]

@Thomasdezeeuw In the meantime, you could try calling CancelIoEx and letting it call NtCancelIoFileEx on your behalf by copying the IO_STATUS_BLOCK value into the head of the OVERLAPPED. I think the first two fields of OVERLAPPED basically correspond to the IO_STATUS_BLOCK for the operation.

[Thomasdezeeuw]

@Thomasdezeeuw In the meantime, you could try calling CancelIoEx and letting it call NtCancelIoFileEx on your behalf by copying the IO_STATUS_BLOCK value into the head of the OVERLAPPED. I think the first two fields of OVERLAPPED basically correspond to the IO_STATUS_BLOCK for the operation.

I'm afraid I don't have a Windows machine available at the moment. Maybe someone from @tokio-rs/windows can give this a try?

[piscisaureus]

since it's now documented: https://learn.microsoft.com/en-us/windows/win32/devnotes/nt-cancel-io-file-ex

Don't read this documentation though! It is very much incorrect, reading it will make you dumber and your program crash.

The signature spelled out in the issue description is the right one.

[MegaBrutal]

This problem has massively affected me a month ago and when I found this issue, I left it as that, thinking that I'll get back to my project in the distant future, meanwhile I was also watching when mio gets a new release. Now I just randomly checked and it builds without problems for x86_64-pc-windows-gnu on Windows 10! Because there is no new version of mio, I assume it was fixed elsewhere. What happened?

[eiderdaus]

@kennykerr Though microsoft/wdkmetadata#9 was fixed in the meantime, I can not find a definition of NtCancelIoFileEx in the windows-sys crate. So I am wondering whether windows-sys just needs to be rebuilt based on the updated metadata? There seems to be no documentation on how to achieve this so I am unsure. Maybe open another issue in https://github.com/microsoft/windows-rs? ;-)

[kennykerr]

Yep, if you open an issue I'll have a reminder to update the metadata. 😉

[ChrisDenton]

What happened?

Rust now redistributes libntdll.a. Note though that this is technically an implementation detail and could change in the future. It's only included because std just so happens to be using it. To have native platform libraries available you're supposed to install a mingw compiler (if using gnu).

Of course, once the windows crates are updated this will no longer be an issue.

[kennykerr]

windows-targets 0.48.2 will include these imports.

[kennykerr]

microsoft/windows-rs#2573 updated the repo with the latest metadata. Please kick the tires and let me know if you experience any issues.

[Thomasdezeeuw]

Gave it a shot in #1688. Windows-sys contains a number of breaking changes, so the next version will likely be 0.49, which means a (slightly) painful update where we have to coordinate a little bit with the ecosystem. Other than that everything cargo checks out from macOS.

[eiderdaus]

I just tried an alternative workaround based on the newly introduced kind = "raw-dylib" link modifier but got this: error: Error calling dlltool 'dlltool.exe': program not found

I merely updated to Rust 1.71 and replaced the original #[link(name = "ntdll")] by #[link(name = "ntdll", kind = "raw-dylib")].

Can anyone confirm that this does not work before I open an issue against https://github.com/rust-lang/rust/?

[kennykerr]

This is a known issue: rust-lang/rust#103939

The windows-rs crates fully support raw-dylib via the windows_raw_dylib cfg option, but this is only an opt-in until raw-dylib support is ubiquitous and reliable, the latter is largely up to the target toolchains.

[kennykerr]

If all you need is NtCancelIoFileEx, one option would be to add a dependency on the upcoming semver-compatible windows-targets version 0.48.2 just for that function and otherwise stay with the current windows-sys version 0.48.0 until folks are ready to move to the next version of windows-sys that will include it.

windows_targets::link!("ntdll.dll" "system" fn NtCancelIoFileEx(filehandle: HANDLE, iorequesttocancel: *const IO_STATUS_BLOCK, iostatusblock: *mut IO_STATUS_BLOCK) -> NTSTATUS);

Let me know - happy to release an update to windows-targets.

[eiderdaus]

This is a known issue: rust-lang/rust#103939

Yeah I found this issue, too, in the meantime. Though I'd argue using stable-x86_64-pc-windows-gnu is not cross-compiling so the scope of the issue is larger than its title suggests. Probably raw-dylib only really works for -msvc targets at the moment and that hardly justifies its stabilization and official announcement on blog.rust-lang.org in my opinion...

Thanks for your offer regarding an update to windows-targets, but I'll be fine waiting a little longer. However on that note, when can we expect this next version of windows-sys? 😉

[kennykerr]

You can get raw-dylib working with GNU with some effort. I test raw-dylib support for windows-rs targeting both MSVC and GNU here:

https://github.com/microsoft/windows-rs/blob/master/.github/workflows/raw_dylib.yml

It relies on this workaround to deal with the dlltool problem:

https://github.com/microsoft/windows-rs/blob/master/.github/actions/fix-environment/action.yml

when can we expect this next version of windows-sys

I'm working furiously on stabilizing metadata, but until then the crate releases are on-demand only to avoid churn. Of course, if tokio needs an update and have tested against github I'm happy to publish an update upon request.

[Thomasdezeeuw]

If all you need is NtCancelIoFileEx, one option would be to add a dependency on the upcoming semver-compatible windows-targets version 0.48.2 just for that function and otherwise stay with the current windows-sys version 0.48.0 until folks are ready to move to the next version of windows-sys that will include it.

windows_targets::link!("ntdll.dll" "system" fn NtCancelIoFileEx(filehandle: HANDLE, iorequesttocancel: *const IO_STATUS_BLOCK, iostatusblock: *mut IO_STATUS_BLOCK) -> NTSTATUS);

Let me know - happy to release an update to windows-targets.

Adding NtCancelIoFileEx to a v0.48.x release would be great for Mio. For Tokio TLS we still need to support rust v1.46, so a MSRV bump would likely mean Mio v0.9.

[ghost]

NtCancelIoFileEx is an undocumented API. Why not abandon this ugly hacky approach and use IOCP or ioring-api instead?

[Thomasdezeeuw]

NtCancelIoFileEx is an undocumented API. Why not abandon this ugly hacky approach and use IOCP or ioring-api instead?

We already use IOCP and iouring is Linux only.

[ghost]

https://learn.microsoft.com/en-us/windows/win32/api/ioringapi/
ioring is not io_uring
@Thomasdezeeuw

[ghost]

Windows-rs has already provided ioringapi, which supports common I/O operations. I believe we no longer need to manually write bindings.