Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FR] Enable LLVM Polly #1096

Closed
a1batross opened this issue Oct 11, 2019 · 14 comments
Closed

[FR] Enable LLVM Polly #1096

a1batross opened this issue Oct 11, 2019 · 14 comments
Assignees
@stephenhines
Labels
enhancement

Comments

@a1batross
Copy link

a1batross commented Oct 11, 2019 •
edited
Loading

In our app, we don't write vectorized code by hand, instead we're relying on compiler autovectorization. In legacy NDK we used GCC's Graphite and now when we switched to latest NDK r20, I've noticed that Polly, LLVM's alternative to GCC Graphite, isn't enabled in NDK toolchains.

Simple way to check if it's enabled:

$ ./aarch64-linux-android21-clang -mllvm -polly -xc -c - -o test.o < /dev/null
clang (LLVM option parsing): Unknown command line argument '-polly'.  Try: 'clang (LLVM option parsing) -help'
clang (LLVM option parsing): Did you mean '-color'?
$ echo $?
1

When on Clang 8 from LLVM's APT repository it doesn't show an error:

$ clang-8 -target aarch64-linux-android21 -mllvm -polly -xc -c - -o test.o < /dev/null
$ echo $?
0

Our app is crossplatform, so testing Polly on Linux ARM machine is resulted performance boost around 10%. Is there any bugs on Android so it's not available?
@stephenhines stephenhines self-assigned this Oct 11, 2019
@stephenhines
Copy link
Collaborator

This is just something that we have not had time to evaluate or test in the context of Android. It is something we can look into for future releases, but I can't commit to a specific release yet.

@a1batross
Copy link
Author

@stephenhines thanks for answer!

Actually, I have my own research with Polly on Android as part of study of how I can use thirdparty Android NDK compiler, for example, Clang/LLVM from Ubuntu repositories, apt.llvm.org or self-built, but that set up to use NDK's sysroots.
Also, being a fan of hardfloat ABI, I also set compiler to use sysroot from older NDK with uncut hardfloat ABI support. This doesn't apply to the topic, but still was a nice try.

I did ran tests and benchmarks to ensure that this way of hack didn't generated worse code than NDK's toolchain.

So, here is results:

Toolchain Avg FPS
NDK Clang ARMv7 77 FPS
Host Clang softfp 77 FPS
Host Clang hardfp 75.8 FPS
Host Clang softfp Polly 85 FPS
Host Clang hardfp Polly 89 FPS
NDK Clang AArch64 100 FPS
Host Clang AArch64 Polly 102 FPS

We just ended up with renderer performance on AArch64, but for ARMv7 devices that half of our audience still uses it's given a nice performance boost.

@nekonomicon nekonomicon mentioned this issue Jan 31, 2020
Open
@stephenhines
Copy link
Collaborator

https://android-review.googlesource.com/c/toolchain/llvm_android/+/1261718 enables this for our toolchain. We'll be marking Polly support as experimental in an upcoming release of the NDK.

@DanAlbert DanAlbert added this to the r23 milestone Mar 19, 2020
@DanAlbert
Copy link
Member

I think that change missed the train for the r22 compiler, so triaging to r23. @stephenhines please correct me/move it if that's wrong.

@stephenhines stephenhines modified the milestones: r23, r22 Apr 14, 2020
@stephenhines
Copy link
Collaborator

This actually will make it for R22, since we need to take Yabin's update as well.

@DanAlbert
Copy link
Member

@stephenhines

$ prebuilts/clang/host/linux-x86/clang-r383902c/bin/clang++ -mllvm -polly foo.cpp -c
clang (LLVM option parsing): Unknown command line argument '-polly'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--color'?

Seems this wasn't actually enabled? @nickdesaulniers as well because he was interested in using this.

@stephenhines
Copy link
Collaborator

Yeah, I missed that the build configuration wasn't actually picked up. We need to explicitly build polly (actually "Polly") and then bundle the necessary libraries to make this work. Let's move this to r23 instead.

@stephenhines stephenhines modified the milestones: r22, r23 Jun 25, 2020
@DanAlbert
Copy link
Member

Bummer. https://android-review.googlesource.com/c/platform/ndk/+/1350665 fixes the changelog.

https://android-review.googlesource.com/c/platform/ndk/+/1350664 adds a test that we should submit when this gets added.

@stephenhines
Copy link
Collaborator

It does actually work, but Polly isn't built into the toolchain itself (and I am not 100% convinced whether we will want to do that just yet). You have to load the plugin for it to work, so the following command line works for me:

bin/clang-11 -Xclang -load -Xclang lib64/LLVMPolly.so -target aarch64-linux-android21 -mllvm -polly -xc -c - -o test.o < /dev/null

Note the both additional -load and the plugin location need to have a -Xclang argument in front to work properly. This method should work just fine in NDK r22, and we can look into including the plugin directly in Clang (removing the need for those extra arguments) for a future release.l

@DanAlbert
Copy link
Member

Ah, okay. I'll retriage so I can fix the docs and add the test then.

@DanAlbert DanAlbert modified the milestones: r23, r22 Jun 26, 2020
@DanAlbert
Copy link
Member

The windows prebuilts are missing the Polly library, so I'm not going to advertise this.

Also even with the library, needing to name LLVMPolly.so makes it annoying to write cross-platform build scripts, since you'd need to instead use LLVMPolly.dll for Windows. Getting it loaded by default for r23 is probably the right call.

@DanAlbert DanAlbert modified the milestones: r22, r23 Jun 26, 2020
@a1batross
Copy link
Author

Good to know it's gonna be available next revision.

Thank you <3

@DanAlbert
Copy link
Member

As long as you're not on windows you can use it with r22, fwiw. The flags listed above should work fine.

@a1batross
Copy link
Author

Yeah, I'm on Linux so I don't think it'll be a problem.

@DanAlbert DanAlbert removed this from the r23 milestone Jan 8, 2021
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 16, 2021
@grendello
Add support for NDK r23.
e0a607a 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
@grendello grendello mentioned this issue Aug 16, 2021
Merged
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 18, 2021
@grendello
Add support for NDK r23.
fe6c689 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 19, 2021
@grendello
Add support for NDK r23.
ffef7e9 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 19, 2021
@grendello
Add support for NDK r23.
4fa0b00 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 19, 2021
@grendello
Add support for NDK r23.
bef4bdd 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 23, 2021
@grendello
Add support for NDK r23.
b06ff72 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 23, 2021
@grendello
Add support for NDK r23.
a157ad1 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 23, 2021
@grendello
Add support for NDK r23.
d236eb1 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 26, 2021
@grendello
Add support for NDK r23.
38600a3 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 26, 2021
@grendello
Add support for NDK r23.
db273ce 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 26, 2021
@grendello
Add support for NDK r23.
b777cdc 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 30, 2021
@grendello
Add support for NDK r23.
ec083b4 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
grendello added a commit to grendello/xamarin-android that referenced this issue Aug 30, 2021
@grendello
Add support for NDK r23.
f44f2f4 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream changes:

* Includes Android 12 APIs.
* Updated LLVM to clang-r416183b, based on LLVM 12 development.
  * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
  * [Issue 1096]: Includes support for [Polly]. Enable by adding `-mllvm -polly`
    to your cflags.
  * [Issue 1230]: LLVM's libunwind is now used instead of libgcc for all
    architectures rather than just 32-bit Arm.
  * [Issue 1231]: LLVM's libclang_rt.builtins is now used instead of libgcc.
  * [Issue 1406]: Fixes crash with Neon intrinsic.
* Vulkan validation layer source and binaries are no longer shipped in the NDK.
  The latest are now posted directly to [GitHub](https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases).
* Vulkan tools source is also removed, specifically vulkan_wrapper.
  It should be downloaded upstream from [GitHub](https://github.com/KhronosGroup/Vulkan-Tools).
* The toolchain file (android.toolchain.cmake) is refactored to base on CMake's
  integrated Android support. This new toolchain file will be enabled by default
  for CMake 3.21 and newer. No user side change is expected. But if anything goes
  wrong, please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON` to
  restore the legacy behavior.
    * When using the new behavior (when using CMake 3.21+ and not explicitly
      selecting the legacy toolchain), **default build flags may change**. One
      of the primary goals was to reduce the behavior differences between our
      toolchain and CMake, and CMake's default flags do not always match the
      legacy toolchain file. Most notably, if using `CMAKE_BUILD_TYPE=Release`,
      your optimization type will likely be `-O3` instead of `-O2` or `-Oz`. See
      [Issue 1536] for more information.
* [Issue 929]: `find_library` now prefers shared libraries from the sysroot over
  static libraries.
* [Issue 1390]: ndk-build now warns when building a static executable with the
  wrong API level.
* [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather than using
  scan-build. clang-tidy performs all the same checks by default, and scan-build
  was no longer working. See the bug for more details, but no user-side changes
  should be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
jonpryor pushed a commit to dotnet/android that referenced this issue Aug 31, 2021
@grendello
[build] Add support for NDK r23. (#6073)
f361d99 
Context: https://github.com/android/ndk/wiki/Changelog-r23#changelog

The most important changes for Xamarin.Android:

NDK r23 removes GNU Binutils and so we need to switch to using our
bundled copy of them.

Upstream NDK r23changes:

  * Includes Android 12/API-31 APIs.

  * Updated LLVM to clang-r416183b, based on LLVM 12 development.
    * [Issue 1047]: Fixes crash when using ASan with the CFI unwinder.
    * [Issue 1096]: Includes support for [Polly].
      Enable by adding `-mllvm -polly` to your cflags.
    * [Issue 1230]: LLVM's `libunwind` is now used instead of `libgcc`
      for all architectures rather than just 32-bit ARM.
    * [Issue 1231]: LLVM's `libclang_rt.builtins` is now used instead
      of `libgcc`.
    * [Issue 1406]: Fixes crash with Neon intrinsic.

  * Vulkan validation layer source and binaries are no longer shipped
    in the NDK.  The latest are now posted directly to
    [KhronosGroup/Vulkan-ValidationLayers].

  * Vulkan tools source is also removed, specifically `vulkan_wrapper`.
    It should be downloaded upstream from [KhronosGroup/Vulkan-Tools].

  * Refactored the toolchain file `android.toolchain.cmake`, basing
    it on CMake's integrated Android support.  This new toolchain file
    will be enabled by default for CMake 3.21 and newer.
    No user side change is expected.  But if anything goes wrong,
    please file a bug and set `ANDROID_USE_LEGACY_TOOLCHAIN_FILE=ON`
    to restore the legacy behavior.
      * When using the new behavior (when using CMake 3.21+ and not
        explicitly selecting the legacy toolchain),
        **default build flags may change**.  One of the primary goals
        was to reduce the behavior differences between our toolchain
        and CMake, and CMake's default flags do not always match the
        legacy toolchain file.  Most notably, if using
        `CMAKE_BUILD_TYPE=Release`, your optimization type will likely
        be `-O3` instead of `-O2` or `-Oz`.
        See [Issue 1536] for more information.

  * [Issue 929]: `find_library` now prefers shared libraries from the
    sysroot over static libraries.

  * [Issue 1390]: `ndk-build` now warns when building a static
    executable with the wrong API level.

  * [Issue 1452]: `NDK_ANALYZE=1` now sets `APP_CLANG_TIDY=true` rather
    than using `scan-build`.  `clang-tidy` performs all the same checks
    by default, and `scan-build` was no longer working.
    See [Issue 1452] for more details; no user-side changes should
    be needed.

[Issue 929]: android/ndk#929
[Issue 1047]: android/ndk#1047
[Issue 1096]: android/ndk#1096
[Issue 1230]: android/ndk#1230
[Issue 1231]: android/ndk#1231
[Issue 1390]: android/ndk#1390
[Issue 1406]: android/ndk#1406
[Issue 1452]: android/ndk#1452
[Issue 1536]: android/ndk#1536
[Polly]: https://polly.llvm.org/
[KhronosGroup/Vulkan-ValidationLayers]: https://github.com/KhronosGroup/Vulkan-ValidationLayers/releases
[KhronosGroup/Vulkan-Tools]: https://github.com/KhronosGroup/Vulkan-Tools
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stephenhines stephenhines

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@DanAlbert @a1batross @stephenhines