Add CI jobs to cross-build arm64 macOS wheels #1362

mtreinish · 2021-09-28T18:53:55Z

Summary

This PR adds CI jobs to build qiskit-aer wheels on arm64 macOS wheels. Since no CI system provides native arm64
macOS environments these CI jobs are configured to cross compile arm64 binaries from an x86_64 macOS
environment. This has the limitation of not letting us execute the binaries but we can at least test building them in
PR CI and publish the built binaries at release time. The two jobs added here do just that, the first is run as part
of the Build workflow that runs on every proposed commit to test that we can at least compile on arm64 macOS
(for supported python versions). The second job is a release publishing job that will build and publish arm64
macOS wheels to PyPI whenever there is a release.

Details and comments

Fixes #1286
Fixes #1397

vvilpas · 2021-10-01T06:14:58Z

CMakeLists.txt

 if(CMAKE_HOST_SYSTEM_PROCESSOR STREQUAL "x86_64" OR CMAKE_HOST_SYSTEM_PROCESSOR STREQUAL "AMD64")
-	if(APPLE OR UNIX)
+    if(NOT CMAKE_OSX_ARCHITECTURES STREQUAL "arm64" AND (APPLE OR UNIX))


I think the problem is that CMAKE_OSX_ARCHITECTURES is not really set and that we are using CMAKE_HOST_SYSTEM_PROCESSOR instead of CMAKE_SYSTEM_PROCESSOR, so it incorrectly assumes is in x86_64.
I would try with:

if(CMAKE_SYSTEM_PROCESSOR STREQUAL "x86_64" OR CMAKE_SYSTEM_PROCESSOR STREQUAL "AMD64") if(APPLE OR UNIX)

CMakeLists.txt

Co-authored-by: Victor Villar <vvilpas@gmail.com>

This reverts commit 549d2d2.

This reverts commit 26efd30.

…ing used" This isn't needed I found the other location in a cmakefile triggering the compilation of the avx2 file so we don't need to print this anymore (it also wouldn't have shown what I was looking for). This reverts commit 9a6883c.

There was a second cmake file embedded in the source tree triggering the avx2 file compilation. This commit adds an if condition to that location as well to avoid trying to build avx2 on arm64.

jakelishman

Looks solid, and not too much of a change, which is nice!

jakelishman · 2022-01-18T18:58:54Z

qiskit/providers/aer/backends/wrappers/CMakeLists.txt

+    if (NOT CMAKE_OSX_ARCHITECTURES STREQUAL "arm64")
+      # We build SIMD filed separately, because they will be reached only if the
+      # machine running the code has SIMD support
+      set(SIMD_SOURCE_FILE "../../../../../src/simulators/statevector/qv_avx2.cpp")


../../../../../ lol. I know it's pre-existing, but is it worth quickly swapping it to ${PROJECT_SOURCE_DIR}?

Heh, sure I'll update this now to make this ready for merge and do that change as part of it

nonhermitian · 2022-01-18T20:20:11Z

The wheels work for me (well Py3.9 one did).

This commit finalizes the iterative process in earlier commits to prepare this PR for merging. It makes 3 primary changes, the first is to add the proper release wheel publish job. This is a dual of the job added in the previous commits that runs in the build ci workflow. That job but will run on tags and publish the built wheels to pypi. The build job that runs on every commit will still stay around to ensure we don't regress and break macOS arm64 builds since testing of it will be limited. The second change is a small update to the cmake files to clean them up slightly based on review comments. The last change here is adding a release note to document the addition of support for arm64 macOS.

jakelishman

🎉 🎉 🎉

jakelishman · 2022-01-18T20:57:51Z

.github/workflows/build.yml

+        run: python -m pip install -U cibuildwheel==2.1.2
+      - name: Build Wheels
+        env:
+          CIBW_SKIP: "cp310-* pp*"


This line will need updating in #1430 after merge, and similar in deploy.yml.

Yep, I'll update it here or in #1430 depending on which merges first

Actually on second thought it's not strictly needed as the cibuildwheel configuration already filters arm64 macOS wheels to >= 3.8 as those are the only python versions which officialy support arm64 macOS. So we don't strictly need to change anything here because it'll already skip 3.6 and 3.7

This PR adds CI jobs to build qiskit-aer wheels on arm64 macOS wheels. Since no CI system provides native arm64 macOS environments these CI jobs are configured to cross compile arm64 binaries from an x86_64 macOS environment. This has the limitation of not letting us execute the binaries but we can at least test building them in PR CI and publish the built binaries at release time. The two jobs added here do just that, the first is run as part of the Build workflow that runs on every proposed commit to test that we can at least compile on arm64 macOS (for supported python versions). The second job is a release publishing job that will build and publish arm64 macOS wheels to PyPI whenever there is a release. Fixes Qiskit#1286 Fixes Qiskit#1397

The supported platform list in the install documentation wasn't quite up to date. The supported platforms are also a bit more nuanced than that list made it appear. This commit expands the platform support section and documents the different levels of support for platforms and the details about each level. This should hopefully make it clearer to users which platforms are expected to work and what guarantees are made for each platform. This is related to Qiskit/qiskit#7553, Qiskit/qiskit#7549, and Qiskit/qiskit-aer#1362 which are expanding (or just changing the level of support) the supported platforms but each have a different set of tradeoffs that we should be documenting.

* Expand platform support section to install doc The supported platform list in the install documentation wasn't quite up to date. The supported platforms are also a bit more nuanced than that list made it appear. This commit expands the platform support section and documents the different levels of support for platforms and the details about each level. This should hopefully make it clearer to users which platforms are expected to work and what guarantees are made for each platform. This is related to Qiskit/qiskit#7553, Qiskit/qiskit#7549, and Qiskit/qiskit-aer#1362 which are expanding (or just changing the level of support) the supported platforms but each have a different set of tradeoffs that we should be documenting. * Apply suggestions from code review Co-authored-by: Jake Lishman <jake@binhbar.com> Co-authored-by: Jake Lishman <jake@binhbar.com>

This PR adds CI jobs to build qiskit-aer wheels on arm64 macOS wheels. Since no CI system provides native arm64 macOS environments these CI jobs are configured to cross compile arm64 binaries from an x86_64 macOS environment. This has the limitation of not letting us execute the binaries but we can at least test building them in PR CI and publish the built binaries at release time. The two jobs added here do just that, the first is run as part of the Build workflow that runs on every proposed commit to test that we can at least compile on arm64 macOS (for supported python versions). The second job is a release publishing job that will build and publish arm64 macOS wheels to PyPI whenever there is a release. Fixes #1286 Fixes #1397

This PR adds CI jobs to build qiskit-aer wheels on arm64 macOS wheels. Since no CI system provides native arm64 macOS environments these CI jobs are configured to cross compile arm64 binaries from an x86_64 macOS environment. This has the limitation of not letting us execute the binaries but we can at least test building them in PR CI and publish the built binaries at release time. The two jobs added here do just that, the first is run as part of the Build workflow that runs on every proposed commit to test that we can at least compile on arm64 macOS (for supported python versions). The second job is a release publishing job that will build and publish arm64 macOS wheels to PyPI whenever there is a release. Fixes Qiskit#1286 Fixes Qiskit#1397

…kage#1402) * Expand platform support section to install doc The supported platform list in the install documentation wasn't quite up to date. The supported platforms are also a bit more nuanced than that list made it appear. This commit expands the platform support section and documents the different levels of support for platforms and the details about each level. This should hopefully make it clearer to users which platforms are expected to work and what guarantees are made for each platform. This is related to Qiskit#7553, Qiskit#7549, and Qiskit/qiskit-aer#1362 which are expanding (or just changing the level of support) the supported platforms but each have a different set of tradeoffs that we should be documenting. * Apply suggestions from code review Co-authored-by: Jake Lishman <jake@binhbar.com> Co-authored-by: Jake Lishman <jake@binhbar.com>

…kage#1402) * Expand platform support section to install doc The supported platform list in the install documentation wasn't quite up to date. The supported platforms are also a bit more nuanced than that list made it appear. This commit expands the platform support section and documents the different levels of support for platforms and the details about each level. This should hopefully make it clearer to users which platforms are expected to work and what guarantees are made for each platform. This is related to Qiskit/qiskit#7553, Qiskit/qiskit#7549, and Qiskit/qiskit-aer#1362 which are expanding (or just changing the level of support) the supported platforms but each have a different set of tradeoffs that we should be documenting. * Apply suggestions from code review Co-authored-by: Jake Lishman <jake@binhbar.com> Co-authored-by: Jake Lishman <jake@binhbar.com>

mtreinish added the on hold Can not fix yet label Sep 28, 2021

mtreinish requested review from chriseclectic and vvilpas as code owners September 28, 2021 18:53

mtreinish mentioned this pull request Sep 28, 2021

ImportError for Qiskit Aer built from source for MacOS arm64 #1286

Closed

vvilpas reviewed Oct 1, 2021

View reviewed changes

CMakeLists.txt Outdated Show resolved Hide resolved

mtreinish and others added 10 commits December 7, 2021 10:27

Test wheels arm64 cross build

11ac8cd

Disable universal2 build and only build arm64 only

33de1be

Force conan to use armv8 to test

a226bc7

Try to avoid x86 paths when cross compiling

503d01b

Force armv8 if building for mac arm64 explicitly

492a2e8

Update CMakeLists.txt

11ac839

Co-authored-by: Victor Villar <vvilpas@gmail.com>

Use CMAKE_SYSTEM_PROCESSOR instead of HOST PROCESSOR

77d65f8

Fix typo

8cf7d65

Revert "Fix typo"

f55104c

This reverts commit 549d2d2.

Revert "Use CMAKE_SYSTEM_PROCESSOR instead of HOST PROCESSOR"

86fd342

This reverts commit 26efd30.

mtreinish force-pushed the vvilpas-vv_update_openmp branch from ecd70ba to 86fd342 Compare December 7, 2021 15:28

mtreinish changed the title ~~DNM: Test #1348 with cibuildwheel~~ [WIP] Add CI jobs to cross-build arm64 macOS wheels Dec 7, 2021

Merge branch 'main' into vvilpas-vv_update_openmp

92b1cef

JoelHBierman mentioned this pull request Dec 12, 2021

pip install fails on Mac M1 arm64 #1397

Closed

mtreinish added 7 commits January 18, 2022 10:36

Tweak logic in cmake files for avx2 enablement

7ab2eaf

Merge remote-tracking branch 'origin/main' into vvilpas-vv_update_openmp

67ce958

Try adding a space

d17b2d5

Add another arm64 avx2 skip

45bcf02

Add debug messages to try and figure out why avx2 is still being used

9a6883c

Only build avx2 file on macOS if not arm64

3752820

There was a second cmake file embedded in the source tree triggering the avx2 file compilation. This commit adds an if condition to that location as well to avoid trying to build avx2 on arm64.

jakelishman reviewed Jan 18, 2022

View reviewed changes

mtreinish changed the title ~~[WIP] Add CI jobs to cross-build arm64 macOS wheels~~ Add CI jobs to cross-build arm64 macOS wheels Jan 18, 2022

mtreinish added Changelog: New Feature Include in the Added section of the changelog stable-backport-potential The issue or PR might be minimal and/or import enough to backport to stable and removed on hold Can not fix yet labels Jan 18, 2022

jakelishman approved these changes Jan 18, 2022

View reviewed changes

jakelishman reviewed Jan 18, 2022

View reviewed changes

mtreinish merged commit f778f4b into Qiskit:main Jan 19, 2022

mtreinish deleted the vvilpas-vv_update_openmp branch January 19, 2022 19:51

mtreinish mentioned this pull request Jan 21, 2022

Expand platform support section to install doc Qiskit/qiskit-metapackage#1402

Merged

lazyoracle mentioned this pull request Feb 8, 2022

pip install qiskit does not work on M1-based macs Qiskit/qiskit-metapackage#1201

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CI jobs to cross-build arm64 macOS wheels #1362

Add CI jobs to cross-build arm64 macOS wheels #1362

mtreinish commented Sep 28, 2021 •

edited

Loading

vvilpas Oct 1, 2021

jakelishman left a comment

jakelishman Jan 18, 2022

mtreinish Jan 18, 2022

nonhermitian commented Jan 18, 2022

jakelishman left a comment

jakelishman Jan 18, 2022

mtreinish Jan 18, 2022

mtreinish Jan 18, 2022

Add CI jobs to cross-build arm64 macOS wheels #1362

Add CI jobs to cross-build arm64 macOS wheels #1362

Conversation

mtreinish commented Sep 28, 2021 • edited Loading

Summary

Details and comments

vvilpas Oct 1, 2021

Choose a reason for hiding this comment

jakelishman left a comment

Choose a reason for hiding this comment

jakelishman Jan 18, 2022

Choose a reason for hiding this comment

mtreinish Jan 18, 2022

Choose a reason for hiding this comment

nonhermitian commented Jan 18, 2022

jakelishman left a comment

Choose a reason for hiding this comment

jakelishman Jan 18, 2022

Choose a reason for hiding this comment

mtreinish Jan 18, 2022

Choose a reason for hiding this comment

mtreinish Jan 18, 2022

Choose a reason for hiding this comment

mtreinish commented Sep 28, 2021 •

edited

Loading