Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create an abomination #99

Merged
merged 1 commit into from
May 9, 2023
Merged

Create an abomination #99

merged 1 commit into from
May 9, 2023

Conversation

bbqsrc
Copy link
Owner

@bbqsrc bbqsrc commented May 9, 2023

Fixes #92.

@TomieAi can you test this and let me know if it fixes your issue?

cargo install --git https://github.com/bbqsrc/cargo-ndk --branch feature/windows-r25-workaround

@bbqsrc bbqsrc force-pushed the feature/windows-r25-workaround branch 5 times, most recently from 671207f to 75cf74b Compare May 9, 2023 21:34
So there's a bug where NDK r25 doesn't handle command line args
properly to clang. Which is precisely what rustc calls, and breaks.

Highly paid teams at Google in their infinite wisdom implemented
argument handling with [reads notes] wizardry, wishful thinking,
and ... fucking batch scripts in 2023. `.cmd` files. For arg parsing! ON
WINDOWS! WHAT IN THE NAME OF ZOMBIE JESUS. Anyway.

So now cargo-ndk will use the actual Win32 function for parsing args
before handing them off to their cursed batch files. It took 2 hours to
workaround this issue. The bug in the NDK repo has been open for 3
months.

I should invoice Google at this point.

android/ndk#1856
@bbqsrc bbqsrc force-pushed the feature/windows-r25-workaround branch from 75cf74b to 35a315e Compare May 9, 2023 21:37
@bbqsrc bbqsrc merged commit 213c0d6 into main May 9, 2023
@bbqsrc
Copy link
Owner Author

bbqsrc commented May 9, 2023

Tested it on a number of major crates, it works.

@bbqsrc
Copy link
Owner Author

bbqsrc commented May 9, 2023

image

@rib
Copy link
Contributor

rib commented May 9, 2023

Huge thanks for taking a look at this issue, this was a big pain for me recently!

Unfortunately it looks like this breaks setting the 'platform' / api level because Clang itself parses the name of the executable to pull out the api level from the name, such that aarch64-linux-android29-clang would have an api level 29.

I guess the hard link names should be defined based on clang_suffix()

@rib
Copy link
Contributor

rib commented May 9, 2023

(maybe it works though since I suppose Command::().spawn() isn't like exec on Linux and so I suppose Clang should see the right name - my comment above was only based on reviewing the change, so I'm not entirely sure)

@bbqsrc
Copy link
Owner Author

bbqsrc commented May 10, 2023

Yeah it doesn't matter what the names are because it simply grabs the correct path and calls it via another environment variable.

Essentially it just maps the args and then calls whatever.cmd, which is still the original name. 🙂

@ScSofts
Copy link
Contributor

ScSofts commented May 10, 2023

My CARGO_HOME is set to F:\Cargo, and temp path is "C:\TEMP" haha

H:\rust\flashdrop>cargo ndk -t arm64-v8a build
[2023-05-10T12:41:32Z INFO  cargo_ndk::cli] Using NDK at path: D:\compiler\Android\ndk\25.0.8775105 (ANDROID_HOME)
[2023-05-10T12:41:32Z INFO  cargo_ndk::cli] NDK API level: 21
[2023-05-10T12:41:32Z INFO  cargo_ndk::cli] Building targets: arm64-v8a
[2023-05-10T12:41:32Z INFO  cargo_ndk::cli] Building arm64-v8a (aarch64-linux-android)
Cargo.rs:139   main: F:\Cargo\bin\cargo-ndk.exe f:"C:\\TEMP\\.tmpbaLHAs\\ar.exe"
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 17, kind: CrossesDevices, message: "
系统无法将文件移到不同的磁盘驱动器。" }', src\cargo.rs:140:81
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

@bbqsrc
Copy link
Owner Author

bbqsrc commented May 10, 2023

lol fucking Windows. I’ll fix today, thanks for reporting!

@ScSofts
Copy link
Contributor

ScSofts commented May 10, 2023

lol fucking Windows. I’ll fix today, thanks for reporting!

Thank you very much 👍

@ScSofts

This comment was marked as off-topic.

@TomieAi
Copy link

TomieAi commented May 10, 2023

I did

cargo install --git https://github.com/bbqsrc/cargo-ndk --branch feature/windows-r25-workaround

And test it

PS D:\rustpj\learn> cargo ndk -t armeabi-v7a -t arm64-v8a -o ./jniLibs build --release
[2023-05-10T17:40:17Z INFO  cargo_ndk::cli] Using NDK at path: D:\AndroidSDK\ndk\25.2.9519653 (NDK_HOME)
[2023-05-10T17:40:17Z INFO  cargo_ndk::cli] NDK API level: 21
[2023-05-10T17:40:17Z INFO  cargo_ndk::cli] Building targets: armeabi-v7a, arm64-v8a
[2023-05-10T17:40:17Z INFO  cargo_ndk::cli] Building armeabi-v7a (armv7-linux-androideabi)
   Compiling example v0.1.0 (D:\rustpj\learn)
    Finished release [optimized] target(s) in 0.09s
[2023-05-10T17:40:17Z INFO  cargo_ndk::cli] Building arm64-v8a (aarch64-linux-android)
   Compiling example v0.1.0 (D:\rustpj\learn)
    Finished release [optimized] target(s) in 0.09s
[2023-05-10T17:40:17Z INFO  cargo_ndk::cli] Copying libraries to ./jniLibs...
[2023-05-10T17:40:17Z ERROR cargo_ndk::cli] No .so files found in path "D:\\rustpj\\learn\\target\\armv7-linux-androideabi\\release"
[2023-05-10T17:40:17Z ERROR cargo_ndk::cli] Did you set the crate-type in Cargo.toml to include 'cdylib'?
[2023-05-10T17:40:17Z ERROR cargo_ndk::cli] For more info, see <https://doc.rust-lang.org/cargo/reference/cargo-targets.html#library>.
PS D:\rustpj\learn> 

my cargo.toml is same as the example folder on this repo.

but doing cargo ndk -t armeabi-v7a build only

[2023-05-10T17:43:07Z INFO  cargo_ndk::cli] Using NDK at path: D:\AndroidSDK\ndk\25.2.9519653 (NDK_HOME)
[2023-05-10T17:43:07Z INFO  cargo_ndk::cli] NDK API level: 21
[2023-05-10T17:43:07Z INFO  cargo_ndk::cli] Building targets: armeabi-v7a
[2023-05-10T17:43:07Z INFO  cargo_ndk::cli] Building armeabi-v7a (armv7-linux-androideabi)
   Compiling example v0.1.0 (D:\rustpj\learn)
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s

I dont see any .so output xD
hope soon it get fixed <3

@bbqsrc
Copy link
Owner Author

bbqsrc commented May 11, 2023

@TomieAi @rib try version 3.1.1 and let me know if it now works consistently for you.

@TomieAi
Copy link

TomieAi commented May 11, 2023

@TomieAi @rib try version 3.1.1 and let me know if it now works consistently for you.
omg you are the best xD it works <3 ty

PS D:\rustpj\learn> cargo ndk -t armeabi-v7a -t arm64-v8a -o ./jniLibs build --release
[2023-05-11T08:58:55Z INFO  cargo_ndk::cli] Using NDK at path: D:\AndroidSDK\ndk\25.2.9519653 (NDK_HOME)
[2023-05-11T08:58:55Z INFO  cargo_ndk::cli] NDK API level: 21
[2023-05-11T08:58:55Z INFO  cargo_ndk::cli] Building targets: armeabi-v7a, arm64-v8a
[2023-05-11T08:58:55Z INFO  cargo_ndk::cli] Building armeabi-v7a (armv7-linux-androideabi)
   Compiling example v0.1.0 (D:\rustpj\learn)
    Finished release [optimized] target(s) in 1.05s
[2023-05-11T08:58:57Z INFO  cargo_ndk::cli] Building arm64-v8a (aarch64-linux-android)
   Compiling example v0.1.0 (D:\rustpj\learn)
    Finished release [optimized] target(s) in 0.57s
[2023-05-11T08:58:57Z INFO  cargo_ndk::cli] Copying libraries to ./jniLibs...
[2023-05-11T08:58:57Z INFO  cargo_ndk::cli] D:\rustpj\learn\target\armv7-linux-androideabi\release\libexample.so -> ./jniLibs\armeabi-v7a\libexample.so
[2023-05-11T08:58:57Z INFO  cargo_ndk::cli] D:\rustpj\learn\target\aarch64-linux-android\release\libexample.so -> ./jniLibs\arm64-v8a\libexample.so
PS D:\rustpj\learn> 

got the .so files lets gooooooooooooooooooo xD


this might be out of the topic but. since we use android ndk to compile ... how do we pass cxx/c flags. where should i put it?

@bbqsrc
Copy link
Owner Author

bbqsrc commented May 11, 2023

The rules for doing that are the same as with cargo and rustc, cargo ndk still just passes through all the flags as it did before.

@rubenh2905
Copy link

rubenh2905 commented May 11, 2023

Love you, worked for me 🌹

@rib
Copy link
Contributor

rib commented May 11, 2023

@TomieAi @rib try version 3.1.1 and let me know if it now works consistently for you.

Unfortunately cargo ndk still isn't working for me on windows, with a project that does cross compile ok from Linux and also used to work with an older toolchain on windows.

The quoting issue is fixed, yay! (I was seeing an issue with the quoting of a linker script, similar to one of the other reports) but now I'm seeing an issue with the command line apparently being too long.

The following warnings were emitted during compilation:

warning: The command line is too long.
warning: Error: Errored with code 1

This is the command that it's upset about:

Click me
running: "C:\\Users\\Robert\\AppData\\Local\\Temp\\.tmpChFZH3\\cxx.exe" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\include\\common" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\include\\foundation" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\include\\pvd" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\include\\task" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source\\common\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/common/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source\\fastxml\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/fastxml/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/filebuf/include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/foundation" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/foundation/include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/foundation/unix" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/ccd" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/common" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/contact" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/convex" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/cooking" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/distance" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/gjk" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/hf" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/intersection" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/mesh" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/pcm" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/geomutils/src/sweep" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/include\\cooking" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/lowlevel\\api/include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/lowlevel\\common/include/collision" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/lowlevel\\common/include/pipeline" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/lowlevel\\common/include/utils" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/lowlevel/common/src/pipeline" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/lowlevel\\software/include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/lowlevel/software/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source\\lowlevelaabb\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/lowlevelaabb/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source\\lowleveldynamics\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/lowleveldynamics/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physx/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxcharacterkinematic/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxcooking/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxextensions/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxextensions/src/serialization" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxextensions/src/serialization/Binary" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxextensions/src/serialization\\File" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxextensions/src/serialization/Xml" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxextensions/src/tet" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source\\physxmetadata/core/include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxmetadata/core/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxmetadata/extensions\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxmetadata/extensions/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxvehicle/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxvehicle/src/physxmetadata\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/physxvehicle/src/physxmetadata/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source\\pvd\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/pvd/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source\\scenequery\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/scenequery/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source\\simulationcontroller\\include" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/simulationcontroller/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/task/src" "-I" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx\\pxshared\\include" "-Wall" "-Wextra" "--sysroot=C:\\Users\\Robert\\AppData\\Local\\Android\\Sdk\\ndk\\25.2.9519653\\toolchains/llvm/prebuilt\\windows-x86_64\\sysroot" "-std=c++14" "-w" "-DANDROID" "-DPX_PHYSX_STATIC_LIB" "-DDISABLE_CUDA_PHYSX" "-DPX_SUPPORT_PVD=1" "-DNDEBUG=1" "-o" "C:\\Users\\Robert\\src\\Embark\\ark\\target\\aarch64-linux-android\\release\\build\\physx-sys-fb5993a469625837\\out\\03f707125f972674-PsFastXml.o" "-c" "C:\\Users\\Robert\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.11.1\\physx/physx\\source/fastxml/src\\PsFastXml.cpp"

I have some vague recollection that by default windows programs have some really silly limit on the length of commands they can spawn, but I'm not sure off the top of my head what's needed to bump that limit.

@rib
Copy link
Contributor

rib commented May 11, 2023

It looks like that command is about 12k, which is > the 8k limit. I'm not quite sure how cargo / rustc overcome that limit though

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

NDK r25 does not handle quoted parameters to clang on Windows
5 participants