-
Notifications
You must be signed in to change notification settings - Fork 655
Correct the Windows cross-compiling commands #13071
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Correct the Windows cross-compiling commands #13071
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13071
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit 4dc8bc2 with merge base d36e83a ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
in the test plan, can you add what steps did you take that failed before this PR and worked after |
@kimishpatel updated |
### Summary During the cross-compilation using Clang on Ubuntu for Windows, there are several commands that are executed on the host and they are failing. Add the guarding condition which checks for `CMAKE_CROSSCOMPILING` and picks the Linux alternative. ### Test plan <details> <summary>1. Create a docker image for cross compilation</summary> ``` ARG GROUP_ID=1000 ARG USER_ID=1000 ARG USER_NAME=docker-user FROM amd64/ubuntu:24.04 # https://bugs.launchpad.net/cloud-images/+bug/2005129 RUN userdel -r ubuntu ENV DEBIAN_FRONTEND=noninteractive ARG GROUP_ID ARG USER_ID ARG USER_NAME RUN apt-get -y update \ && apt-get install --no-install-recommends -y \ ccache \ curl \ lsb-release \ wget \ software-properties-common \ gnupg \ ca-certificates \ build-essential \ git \ make \ ninja-build \ patch \ && rm -rf /var/lib/apt/lists/* RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - \ && wget https://apt.llvm.org/llvm.sh \ && chmod +x llvm.sh \ && ./llvm.sh 19 all \ && apt update && apt install -y \ clang-19 \ lld-19 \ llvm-19 \ && rm -rf /var/lib/apt/lists/* RUN update-alternatives --install /usr/bin/clang-cl clang-cl /usr/bin/clang-cl-19 60 \ && update-alternatives --install /usr/bin/clang clang /usr/bin/clang-19 60 \ && update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-19 60 \ && update-alternatives --install /usr/bin/llvm-rc llvm-rc /usr/bin/llvm-rc-19 60 \ && update-alternatives --install /usr/bin/llvm-mt llvm-mt /usr/bin/llvm-mt-19 60 \ && update-alternatives --install /usr/bin/lld-link lld-link /usr/bin/lld-link-19 60 \ && update-alternatives --install /usr/bin/lldb lldb /usr/bin/lldb-19 60 \ && update-alternatives --install /usr/bin/llvm-lib llvm-lib /usr/bin/llvm-lib-19 60 \ && update-alternatives --install /usr/bin/llvm-ar llvm-ar /usr/bin/llvm-ar-19 60 \ && update-alternatives --install /usr/bin/llvm-ranlib llvm-ranlib /usr/bin/llvm-ranlib-19 60 \ && update-alternatives --install /usr/bin/llvm-nm llvm-nm /usr/bin/llvm-nm-19 60 \ && update-alternatives --install /usr/bin/llvm-objdump llvm-objdump /usr/bin/llvm-objdump-19 60 \ && update-alternatives --install /usr/bin/llvm-objcopy llvm-objcopy /usr/bin/llvm-objcopy-19 60 \ && update-alternatives --install /usr/bin/llvm-strip llvm-strip /usr/bin/llvm-strip-19 60 \ && update-alternatives --install /usr/bin/cc cc /usr/bin/clang-19 100 \ && update-alternatives --install /usr/bin/c++ c++ /usr/bin/clang++-19 100 \ && update-alternatives --install /usr/bin/ld ld /usr/bin/ld.lld-19 100 # ----------------------------------------------------------------------------- # user # ----------------------------------------------------------------------------- RUN groupadd --gid $GROUP_ID docker-user \ && useradd --uid $USER_ID --gid docker-user --create-home $USER_NAME USER $USER_NAME # ----------------------------------------------------------------------------- # python uv # ----------------------------------------------------------------------------- RUN curl -LsSf https://astral.sh/uv/0.7.17/install.sh | sh # ----------------------------------------------------------------------------- # rust, x86_64-pc-windows-msvc target # ----------------------------------------------------------------------------- RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --profile minimal -y # ----------------------------------------------------------------------------- # xwin # ----------------------------------------------------------------------------- RUN ~/.cargo/bin/cargo install xwin cargo-cache \ && ~/.cargo/bin/cargo-cache -a \ && ~/.cargo/bin/xwin --cache-dir /tmp/xwin-cache --accept-license --variant desktop --arch x86_64 --include-atl \ splat --preserve-ms-arch-notation --include-debug-libs --output ~/.xwin/x86_64 \ && rm -rf /tmp/xwin-cache ``` </details> <details> <summary>2. Create cmake dir. </summary> For some reason I had to disable AVX instructions to get it working, it is not the intention, the intention here is to use a window toolchain ``` cmake -G Ninja -B build-win \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_SYSTEM_NAME=Windows \ -DCMAKE_SYSTEM_VERSION=10.0 \ -DCMAKE_SYSTEM_PROCESSOR=AMD64 \ -DCMAKE_C_COMPILER=clang-cl \ -DCMAKE_CXX_COMPILER=clang-cl \ -DCMAKE_ASM_COMPILER=clang-cl \ -DCMAKE_RC_COMPILER=llvm-rc \ -DCMAKE_LINKER=lld-link \ -DCMAKE_C_COMPILER_TARGET=x86_64-pc-windows-msvc \ -DCMAKE_CXX_COMPILER_TARGET=x86_64-pc-windows-msvc \ -DCMAKE_ASM_COMPILER_TARGET=x86_64-pc-windows-msvc \ -DCMAKE_SYSROOT=/home/docker-user/.xwin/x86_64 \ -DCMAKE_FIND_ROOT_PATH=/home/docker-user/.xwin/x86_64 \ -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER \ -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY \ -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY \ -DCMAKE_CXX_FLAGS="/imsvc /home/docker-user/.xwin/x86_64/crt/include /imsvc /home/docker-user/.xwin/x86_64/sdk/include/ucrt /imsvc /home/docker-user/.xwin/x86_64/sdk/include/um /imsvc /home/docker-user/.xwin/x86_64/sdk/include/shared -Wno-unknown-argument" \ -DCMAKE_C_FLAGS="/imsvc /home/docker-user/.xwin/x86_64/crt/include /imsvc /home/docker-user/.xwin/x86_64/sdk/include/ucrt /imsvc /home/docker-user/.xwin/x86_64/sdk/include/um /imsvc /home/docker-user/.xwin/x86_64/sdk/include/shared -Wno-unknown-argument" \ -DCMAKE_EXE_LINKER_FLAGS="/libpath:/home/docker-user/.xwin/x86_64/crt/lib/x64 /libpath:/home/docker-user/.xwin/x86_64/sdk/lib/um/x64 /libpath:/home/docker-user/.xwin/x86_64/sdk/lib/ucrt/x64" \ -DCMAKE_SHARED_LINKER_FLAGS="/libpath:/home/docker-user/.xwin/x86_64/crt/lib/x64 /libpath:/home/docker-user/.xwin/x86_64/sdk/lib/um/x64 /libpath:/home/docker-user/.xwin/x86_64/sdk/lib/ucrt/x64" \ -DGFLAGS_INTTYPES_FORMAT=VC7 \ -DEXECUTORCH_BUILD_XNNPACK=ON \ -DXNNPACK_ENABLE_ASSEMBLY=OFF \ -DXNNPACK_BUILD_TESTS=OFF \ -DXNNPACK_BUILD_BENCHMARKS=OFF \ -DXNNPACK_ENABLE_AVX512F=OFF \ -DXNNPACK_ENABLE_AVX512SKX=OFF \ -DXNNPACK_ENABLE_AVX512VBMI=OFF \ -DXNNPACK_ENABLE_AVX512VNNI=OFF \ -DXNNPACK_ENABLE_AVX512VNNIGFNI=OFF \ -DXNNPACK_ENABLE_AVX512AMX=OFF \ -DXNNPACK_ENABLE_AVX512FP16=OFF \ --fresh ``` </details> <details> <summary>3. Build </summary> ``` cmake --build build-win --config Release --target xnnpack_schema ``` </details> The error will have ``` FAILED: schema/include/executorch/backends/xnnpack/serialization/schema_generated.h /mnt/executorch/build-win/schema/include/executorch/backends/xnnpack/serialization/schema_generated.h cd /mnt/executorch && /mnt/executorch/build-win/third-party/flatbuffers_external_project/bin/flatc --cpp --cpp-std c++11 --scoped-enums -o /mnt/executorch/build-win/schema/include/executorch/backends/xnnpack/serialization backends/xnnpack/serialization/runtime_schema.fbs && powershell -Command "Move-Item -Path /mnt/executorch/build-win/schema/include/executorch/backends/xnnpack/serialization/runtime_schema_generated.h -Destination /mnt/executorch/build-win/schema/include/executorch/backends/xnnpack/serialization/schema_generated.h" /bin/sh: 1: powershell: not found ninja: build stopped: subcommand failed. ```
Summary
During the cross-compilation using Clang on Ubuntu for Windows, there are several commands that are executed on the host and they are failing.
Add the guarding condition which checks for
CMAKE_CROSSCOMPILING
and picks the Linux alternative.Test plan
1. Create a docker image for cross compilation
2. Create cmake dir.
For some reason I had to disable AVX instructions to get it working, it is not the intention, the intention here is to use a window toolchain
3. Build
The error will have