Fix cross compilation with clang-cl (v2) #230

Sporif · 2022-03-06T01:49:13Z

Supersedes #101.

This PR also fixes cross compilation of the C bindings and adds CI testing for 'Linux - Windows MSVC', but skips the 'C bindings intrinsics' test as that fails to build).

Successful build: https://github.com/Sporif/BLAKE3/actions/runs/1944217647

oconnor663 · 2022-03-09T22:17:56Z

.github/workflows/ci.yml

+        sudo ln -s /usr/bin/lld /usr/bin/lld-link
+        sudo apt-get install --quiet --no-install-recommends -y wine-stable winetricks
+        winetricks nocrashdialog
+        cargo install cargo-xwin --target x86_64-unknown-linux-gnu


Wow, thank you for figuring all this out. It's probably time for me to refactor all of these copy-pasted jobs into a Python script or something, so that additions like this aren't so painful in the future.

oconnor663 · 2022-03-09T22:27:04Z

build.rs

+        } else {
+            build.file("c/blake3_sse2_x86-64_windows_gnu.S");
+            build.file("c/blake3_sse41_x86-64_windows_gnu.S");
+            build.file("c/blake3_avx2_x86-64_windows_gnu.S");


@sneves are there any potential ABI mismatch issues we might need to worry about with compiling the Windows GNU .S files into a binary that's otherwise targeting the MSVC ABI? Like different callee-saved register conventions or whatever? Or is it just the assembly syntax that's different between the two?

Syntax only, yes.

ark0f · 2023-02-25T10:30:08Z

Any updates?

tbillington · 2023-11-07T11:20:49Z

Hey @oconnor663, just wanted to check if there was any chance of this PR being adopted/revisited?

I started applying the changes fresh to current master but the code has changed enough that I'm lost for how to translate the last of the original changes in this PR to the current api used in build.rs so I can test with the current version of blake3. Perhaps with some assistance I could get it over the line.

For context bevy recently started using blake3 for file hashing in the import pre-processing. We're unable to cross compile for now and investigating our options. Don't want you/project maintainers to feel pressured to attend to this though as there are backup alternatives :)

oconnor663 · 2023-11-07T15:58:08Z

Ah, apologies for dropping the ball. I'm going to put a note on my calendar to get to this PR this weekend.

Incidentally I've just recently been studying up on how ECS works, and I've been looking at what Bevy does for atomically reserving ranges of IDs. So I'm pumped to see BLAKE3 getting involved :)

oconnor663 · 2023-11-20T02:54:21Z

I'm trying to understand how this sort of cross-compilation works and how to test it. I'm getting some confusing results when I try an alternate approach with cross, where the master branch succeeds but this branch fails. Specifically following these instructions:

$ git clone https://github.com/cross-rs/cross
...
$ cd cross
$ git submodule update --init --remote
...
$ cargo build-docker-image x86_64-pc-windows-msvc-cross --tag local
... [suuuper long build]
$ cat > Cross.toml << EOF
[target.x86_64-pc-windows-msvc]
image = "ghcr.io/cross-rs/x86_64-pc-windows-msvc-cross:local"
EOF
$ cross test --target x86_64-pc-windows-msvc
...
test result: ok. 29 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.06s

If I rebase this PR and run the same tests, the last step fails:

$ cross test --target x86_64-pc-windows-msvc
...
  running: "cl.exe" "-nologo" "-MD" "-Z7" "-Brepro" "-W4" "-Fo/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_sse2_
x86-64_windows_gnu.o" "-c" "c/blake3_sse2_x86-64_windows_gnu.S"
  cargo:warning=cl : Command line warning D9024 : unrecognized source file type 'c/blake3_sse2_x86-64_windows_gnu.S', object file assumed

  cargo:warning=cl : Command line warning D9027 : source file 'c/blake3_sse2_x86-64_windows_gnu.S' ignored

  cl : Command line warning D9021 : no action performed
  exit status: 0
  running: "cl.exe" "-nologo" "-MD" "-Z7" "-Brepro" "-W4" "-Fo/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_sse41
_x86-64_windows_gnu.o" "-c" "c/blake3_sse41_x86-64_windows_gnu.S"
  cargo:warning=cl : Command line warning D9024 : unrecognized source file type 'c/blake3_sse41_x86-64_windows_gnu.S', object file assumed

  cargo:warning=cl : Command line warning D9027 : source file 'c/blake3_sse41_x86-64_windows_gnu.S' ignored

  cl : Command line warning D9021 : no action performed
  exit status: 0
  running: "cl.exe" "-nologo" "-MD" "-Z7" "-Brepro" "-W4" "-Fo/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_avx2_
x86-64_windows_gnu.o" "-c" "c/blake3_avx2_x86-64_windows_gnu.S"
  cargo:warning=cl : Command line warning D9024 : unrecognized source file type 'c/blake3_avx2_x86-64_windows_gnu.S', object file assumed

  cargo:warning=cl : Command line warning D9027 : source file 'c/blake3_avx2_x86-64_windows_gnu.S' ignored

  cl : Command line warning D9021 : no action performed
...
  running: "lib.exe" "-out:/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/libblake3_sse2_sse41_avx2_assembly.a" "-nologo" "/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_sse2_x86-64_windows_gnu.o" "/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_sse41_x86-64_windows_gnu.o" "/target/x86_64-pc-windows-msvc/debug/build/blake3-8bde2450d49dcdfd/out/c/blake3_avx2_x86-64_windows_gnu.o"
  \target\x86_64-pc-windows-msvc\debug\build\blake3-8bde2450d49dcdfd\out\c\blake3_sse2_x86-64_windows_gnu.o : fatal error LNK1107: invalid or corrupt file: cannot read at 0x4200
  exit status: 83

I'm guessing there's more than one way to cross-compile for the MSVC ABI, and whatever cross is doing there expects .asm files rather than .S files. Is that right? Is one of these approaches "more correct" than the other? Are there tweaks we could make to the build.rs changes here to make both approaches build?

shrimpwtf · 2023-12-28T17:32:41Z

Has this been fixed yet? Struggling to cross compile my binary because of this

oconnor663 · 2023-12-28T21:33:55Z

Not yet, I need help answering the questions above.

toothbrush7777777 · 2024-03-22T11:22:07Z

Hey @oconnor663

You may find this answer about differences between the MinGW and MSVC toolchains helpful.

Cross offers both toolchains. For the MSVC toolchain, Cross extracts headers and link libraries from the MSVC runtime packages, then runs the actual MSVC compiler under Wine emulation (both cl.exe and link.exe). Therefore, this PR would not work in this case since cl.exe wants the original MSVC assembly files.

You could get this PR to work using Cross, but you'd need to override the linker, C++ compiler and C compiler that Cross provides (the MSVC toolchain under Wine) to use instead the LLVM MSVC-compatible toolchain. Something like this should work:

[target.aarch64-unknown-linux-gnu]
pre-build = [
    # TODO: Install clang-cl, lld-link, etc.
    "export CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_LINKER=lld-link",
    # or maybe rust-lld would work?
    "export CC_x86_64_pc_windows_msvc=clang-cl",
    "export CXX_x86_64_pc_windows_msvc=clang-cl",
]

toothbrush7777777 · 2024-03-22T11:44:12Z

It should be possible to "fix" the build script to work by default with MSVC on Windows, MSVC on Wine using Cross, or clang-cl on any architecture (e.g. using cargo xwin) by modifying the use_msvc_asm function. Something like this should work:

fn use_msvc_asm() -> bool {
    const MSVC_NAMES: &[&str] = &["", "cl", "cl.exe"];
    let target_os = env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();
    let target_env = env::var("CARGO_CFG_TARGET_ENV").unwrap_or_default();
    let target_windows_msvc = target_os == "windows" && target_env == "msvc";
    let host_triple = env::var("HOST").unwrap_or_default();
    let target_triple = env::var("TARGET").unwrap_or_default();
    let cross_compiling = host_triple != target_triple;
    let cc = env::var("CC").unwrap_or_default().to_ascii_lowercase();
    if !target_windows_msvc {
        // We are not building for Windows with the MSVC toolchain.
        false
    } else if !cross_compiling && MSVC_NAMES.contains(cc) {
        // We are building on Windows with the MSVC toolchain (and not cross-compiling for another architecture or target).
        true
    } else {
        // We are cross-compiling to Windows with the MSVC toolchain.
        let target_arch = env::var("CARGO_CFG_TARGET_ARCH").unwrap_or_default();
        let target_vendor = env::var("CARGO_CFG_TARGET_VENDOR").unwrap_or_default();
        let cc = env::var(format!("CC_{target_arch}_{target_vendor}_windows_msvc")).unwrap_or_default().to_ascii_lowercase();
        // Check if we are using the MSVC compiler.
        MSVC_NAMES.contains(cc)
    }
}

Edit: This should support all the Windows targets now, including the UWP and new Windows 7 targets.

VitalyAnkh · 2024-09-24T10:42:41Z

Any updates? Does this PR need any assistance? If so, I'd be happy to help.

Fix cross compilation with clang-cl

f659ff7

Sporif force-pushed the fix-clang-cl-cross branch from b54ccfa to d5b8cf9 Compare March 6, 2022 14:46

Add Linux - Windows MSVC tests to Github CI

124a7d8

Sporif force-pushed the fix-clang-cl-cross branch from d5b8cf9 to 124a7d8 Compare March 7, 2022 08:05

oconnor663 reviewed Mar 9, 2022

View reviewed changes

tbillington mentioned this pull request Nov 7, 2023

WSL2 to Windows compilation fails in Bevy 0.12 bevyengine/bevy#10425

Open

This comment was marked as outdated.

Sign in to view

messense mentioned this pull request Aug 26, 2024

ml64.exe not found rust-cross/cargo-xwin#124

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix cross compilation with clang-cl (v2) #230

Fix cross compilation with clang-cl (v2) #230

Sporif commented Mar 6, 2022 •

edited

Loading

oconnor663 Mar 9, 2022

oconnor663 Mar 9, 2022

sneves Mar 10, 2022

ark0f commented Feb 25, 2023

tbillington commented Nov 7, 2023 •

edited

Loading

oconnor663 commented Nov 7, 2023

oconnor663 commented Nov 20, 2023

shrimpwtf commented Dec 28, 2023

oconnor663 commented Dec 28, 2023

toothbrush7777777 commented Mar 22, 2024 •

edited

Loading

toothbrush7777777 commented Mar 22, 2024 •

edited

Loading

This comment was marked as outdated.

VitalyAnkh commented Sep 24, 2024

Fix cross compilation with clang-cl (v2) #230

Are you sure you want to change the base?

Fix cross compilation with clang-cl (v2) #230

Conversation

Sporif commented Mar 6, 2022 • edited Loading

oconnor663 Mar 9, 2022

Choose a reason for hiding this comment

oconnor663 Mar 9, 2022

Choose a reason for hiding this comment

sneves Mar 10, 2022

Choose a reason for hiding this comment

ark0f commented Feb 25, 2023

tbillington commented Nov 7, 2023 • edited Loading

oconnor663 commented Nov 7, 2023

oconnor663 commented Nov 20, 2023

shrimpwtf commented Dec 28, 2023

oconnor663 commented Dec 28, 2023

toothbrush7777777 commented Mar 22, 2024 • edited Loading

toothbrush7777777 commented Mar 22, 2024 • edited Loading

This comment was marked as outdated.

VitalyAnkh commented Sep 24, 2024

Sporif commented Mar 6, 2022 •

edited

Loading

tbillington commented Nov 7, 2023 •

edited

Loading

toothbrush7777777 commented Mar 22, 2024 •

edited

Loading

toothbrush7777777 commented Mar 22, 2024 •

edited

Loading