-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add cross-build wheel jobs for ppc64le and s390x #7549
Add cross-build wheel jobs for ppc64le and s390x #7549
Conversation
This commit adds a new CI job at release time to build precompiled binaries for s390x and ppc64le linux platforms. These platforms do not have precompiled wheels for any upstream dependencies (except for retworkx and tweedledum, which both publish binaries for both platforms). Since the only publically available CI service that has native s390x and ppc64le environments available is travis which doesn't provide sufficient quota to open source projects anymore to make it usable we have to rely on either cross compilation or emulation. Cross compiling with python wheels while possible is quite tricky to setup and configure in practice so emulation is used here. This is a much simpler path configure, especially because cibuildwheel has support for emulating non-x86 architectures via QEMU to build wheels. While QEMU emulation of other architectures is exceedingly slow, we have a 6 hour job time limit with github actions which should hopefully be sufficient to compile the binary and build wheels for all our supported python versions. This was previously attempted in Qiskit#5844 but we hit a timout issue reliably every time trying to build scipy from source. Compiling scipy is slow on a native system and it's an order of magnitude slower under qemu emulation. In Qiskit#5844 we exceeded the 6 hour job time limit just compiling scipy once for testing. To avoid this issue (since scipy doesn't publish s390x or ppc64le wheels) this commit skips the test runs on these jobs. This means we're solely building the wheels and not testing if they're valid. This also means installing scipy and other dependencies is still an exercise for the s390x and ppc64le users. But, if this works it will mean that at least for Qiskit there will not be a need to compile anything from source.
This commit should not be merged without being reverted first. It is changing the CI trigger to run on open pull requests and removing the twine upload step to trigger the wheel builds for testing and iteratively fixing issues. Once the builds work this will be reverted and the PR is ready for final review/merging.
Pull Request Test Coverage Report for Build 1728983295
💛 - Coveralls |
Testing shows this works well and we can build and publish ppc64le and s390x wheels on release. This reverts commit f06c0b6 and opens up the PR for review.
Ci worked, you can see the logs (and download the wheels) from here: https://github.com/Qiskit/qiskit-terra/actions/runs/1724756938 I'll spin up an s390x vm in a bit to test the wheel works |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In principle this looks good to me, but I don't really have the experience with virtualisation to meaningfully test the output wheels (nor the right software on my laptop, as far as I know!).
Do we want a release note to talk about the new wheels?
You could install QEMU locally for emulation like we do in CI to build these but I don't really recommend this as it's super slow. I normally test s390x via the LinuxONE community cloud which let you provision s390x VMs for open source work: https://developer.ibm.com/gettingstarted/ibm-linuxone/ (my account was stale so I'm waiting on a new one to be provisioned now). For ppc64le I don't have a good test mechanism short of knowing someone with a server.
Yeah a release note is a good call, I'll add one in a bit. |
In Qiskit#7102 we added support for Python 3.10 to Qiskit. However, numpy and scipy stopped publishing wheels for 32bit platforms in Python 3.10. This means that to test Python 3.10 32bit wheels we will have to compile numpy and scipy from source (which for scipy is prohibitively slow). In response this commit updates our cibuildwheel configuration to skip tests on 32bit Python 3.10 wheels since we can't reliably test the built wheels in CI and it would require users to build them from source. This is the same approach taken in PR Qiskit#7549 for ppc64le and s390x. For 32 bit platform users on python 3.10 this means they'll need to locally build numpy, scipy, and potentially other depencies from source to use Qiskit Terra with Python 3.10, but for the Qiskit components at least precompiled binaries will be available.
Approved, provided you're happy with your latest VM tests. It might not worth backporting this to 0.19.2, unless you reckon it'd be good to do a test release with the new platforms before the next minor. |
The supported platform list in the install documentation wasn't quite up to date. The supported platforms are also a bit more nuanced than that list made it appear. This commit expands the platform support section and documents the different levels of support for platforms and the details about each level. This should hopefully make it clearer to users which platforms are expected to work and what guarantees are made for each platform. This is related to Qiskit/qiskit#7553, Qiskit/qiskit#7549, and Qiskit/qiskit-aer#1362 which are expanding (or just changing the level of support) the supported platforms but each have a different set of tradeoffs that we should be documenting.
I've tested this on s390x (with the python 3.8 wheel) and it works fine (after compiling numpy and scipy from source):
|
Yeah, I think we can just hold this until 0.20.0 (this will also give us time to do a similar PR on aer). There's no pressing need here. It's not like either of these platforms have a lot of users. Looking at the CPU stats from PyPI over the last 30 days:
s390x clients are only 0.06% of our downloads and 0 ppc64le downloads. |
* Expand platform support section to install doc The supported platform list in the install documentation wasn't quite up to date. The supported platforms are also a bit more nuanced than that list made it appear. This commit expands the platform support section and documents the different levels of support for platforms and the details about each level. This should hopefully make it clearer to users which platforms are expected to work and what guarantees are made for each platform. This is related to Qiskit/qiskit#7553, Qiskit/qiskit#7549, and Qiskit/qiskit-aer#1362 which are expanding (or just changing the level of support) the supported platforms but each have a different set of tradeoffs that we should be documenting. * Apply suggestions from code review Co-authored-by: Jake Lishman <jake@binhbar.com> Co-authored-by: Jake Lishman <jake@binhbar.com>
* Drop support for 32bit py310 wheel builds In #7102 we added support for Python 3.10 to Qiskit. However, numpy and scipy stopped publishing wheels for 32bit platforms in Python 3.10. This means that to test Python 3.10 32bit wheels we will have to compile numpy and scipy from source (which for scipy is prohibitively slow). In response this commit updates our cibuildwheel configuration to skip tests on 32bit Python 3.10 wheels since we can't reliably test the built wheels in CI and it would require users to build them from source. This is the same approach taken in PR #7549 for ppc64le and s390x. For 32 bit platform users on python 3.10 this means they'll need to locally build numpy, scipy, and potentially other depencies from source to use Qiskit Terra with Python 3.10, but for the Qiskit components at least precompiled binaries will be available. * Update release note
I've tested this on ppc64le (with python 3.7, 3.8, and 3.9) and they worked fine.
|
* Drop support for 32bit py310 wheel builds In Qiskit#7102 we added support for Python 3.10 to Qiskit. However, numpy and scipy stopped publishing wheels for 32bit platforms in Python 3.10. This means that to test Python 3.10 32bit wheels we will have to compile numpy and scipy from source (which for scipy is prohibitively slow). In response this commit updates our cibuildwheel configuration to skip tests on 32bit Python 3.10 wheels since we can't reliably test the built wheels in CI and it would require users to build them from source. This is the same approach taken in PR Qiskit#7549 for ppc64le and s390x. For 32 bit platform users on python 3.10 this means they'll need to locally build numpy, scipy, and potentially other depencies from source to use Qiskit Terra with Python 3.10, but for the Qiskit components at least precompiled binaries will be available. * Update release note (cherry picked from commit 371a2bc)
* Drop support for 32bit py310 wheel builds In #7102 we added support for Python 3.10 to Qiskit. However, numpy and scipy stopped publishing wheels for 32bit platforms in Python 3.10. This means that to test Python 3.10 32bit wheels we will have to compile numpy and scipy from source (which for scipy is prohibitively slow). In response this commit updates our cibuildwheel configuration to skip tests on 32bit Python 3.10 wheels since we can't reliably test the built wheels in CI and it would require users to build them from source. This is the same approach taken in PR #7549 for ppc64le and s390x. For 32 bit platform users on python 3.10 this means they'll need to locally build numpy, scipy, and potentially other depencies from source to use Qiskit Terra with Python 3.10, but for the Qiskit components at least precompiled binaries will be available. * Update release note (cherry picked from commit 371a2bc)
* Add support for Python 3.10 (backport #7102) Merge notes: for the backport, we must ensure that Python 3.6 is still supported. * Add support for Python 3.10 Python 3.10.0 was released on 10-04-2021, this commit marks the support of Python 3.10 in qiskit-terra. It adds the supported python version in the package metadata and updates the CI configuration to run test jobs on Python 3.10 and build Python 3.10 wheels. * Fix typo * Update default envlist in tox.ini to include 3.10 * Bump cibuildwheel to the latest version This also takes the opportunity to deduplicate the cibuildwheel configuration using the pyproject.toml support in newer versions of cibuildwheel. The common options for all builds are put there and per build overrides (which are only for cross compiling arm wheels) are left as environment variables in the CI configuration. * Add missing cibuildwheel config to pyproject.toml * Ignore internal deprecation warning emitted by jupyter in ci * Fix black (cherry picked from commit 9a743fb) Co-authored-by: Jake Lishman <jake.lishman@ibm.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Skip musl builds of linux Co-authored-by: Matthew Treinish <mtreinish@kortar.org> * Update Windows wheel build VM The `vs2017-windows2016` is deprecated, and due to be removed in a few months; it is currently in the brownout period. `windows-latest` now contains the VS build tools we were using, so should be sufficient. * Drop support for 32bit py310 wheel builds (backport #7553) * Drop support for 32bit py310 wheel builds In #7102 we added support for Python 3.10 to Qiskit. However, numpy and scipy stopped publishing wheels for 32bit platforms in Python 3.10. This means that to test Python 3.10 32bit wheels we will have to compile numpy and scipy from source (which for scipy is prohibitively slow). In response this commit updates our cibuildwheel configuration to skip tests on 32bit Python 3.10 wheels since we can't reliably test the built wheels in CI and it would require users to build them from source. This is the same approach taken in PR #7549 for ppc64le and s390x. For 32 bit platform users on python 3.10 this means they'll need to locally build numpy, scipy, and potentially other depencies from source to use Qiskit Terra with Python 3.10, but for the Qiskit components at least precompiled binaries will be available. * Update release note (cherry picked from commit 371a2bc) * Use manylinux2014 on Python 3.10+ Numpy and Scipy are moving to drop manylinux2010 wheels on newer platforms, which gives us some cover to do the same. For the 0.19 series, though, we need to maintain the same manylinux compatibility we had before. To test, we need to ensure that we pull Numpy and Scipy in binary form only (forcing pip to install slightly older versions like 1.21 instead of 1.22), rather than attempting to build Numpy from source as part of our testing process. Co-authored-by: Matthew Treinish <mtreinish@kortar.org> Co-authored-by: Jake Lishman <jake.lishman@ibm.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
…kage#1402) * Expand platform support section to install doc The supported platform list in the install documentation wasn't quite up to date. The supported platforms are also a bit more nuanced than that list made it appear. This commit expands the platform support section and documents the different levels of support for platforms and the details about each level. This should hopefully make it clearer to users which platforms are expected to work and what guarantees are made for each platform. This is related to Qiskit#7553, Qiskit#7549, and Qiskit/qiskit-aer#1362 which are expanding (or just changing the level of support) the supported platforms but each have a different set of tradeoffs that we should be documenting. * Apply suggestions from code review Co-authored-by: Jake Lishman <jake@binhbar.com> Co-authored-by: Jake Lishman <jake@binhbar.com>
…kage#1402) * Expand platform support section to install doc The supported platform list in the install documentation wasn't quite up to date. The supported platforms are also a bit more nuanced than that list made it appear. This commit expands the platform support section and documents the different levels of support for platforms and the details about each level. This should hopefully make it clearer to users which platforms are expected to work and what guarantees are made for each platform. This is related to Qiskit#7553, Qiskit#7549, and Qiskit/qiskit-aer#1362 which are expanding (or just changing the level of support) the supported platforms but each have a different set of tradeoffs that we should be documenting. * Apply suggestions from code review Co-authored-by: Jake Lishman <jake@binhbar.com> Co-authored-by: Jake Lishman <jake@binhbar.com>
…kage#1402) * Expand platform support section to install doc The supported platform list in the install documentation wasn't quite up to date. The supported platforms are also a bit more nuanced than that list made it appear. This commit expands the platform support section and documents the different levels of support for platforms and the details about each level. This should hopefully make it clearer to users which platforms are expected to work and what guarantees are made for each platform. This is related to Qiskit/qiskit#7553, Qiskit/qiskit#7549, and Qiskit/qiskit-aer#1362 which are expanding (or just changing the level of support) the supported platforms but each have a different set of tradeoffs that we should be documenting. * Apply suggestions from code review Co-authored-by: Jake Lishman <jake@binhbar.com> Co-authored-by: Jake Lishman <jake@binhbar.com>
Summary
This commit adds a new CI job at release time to build precompiled
binaries for s390x and ppc64le linux platforms. These platforms do not
have precompiled wheels for any upstream dependencies (except for
retworkx and tweedledum, which both publish binaries for both platforms).
Since the only publically available CI service that has native s390x and
ppc64le environments available is travis which doesn't provide
sufficient quota to open source projects anymore to make it usable we
have to rely on either cross compilation or emulation. Cross compiling
with python wheels while possible is quite tricky to setup and configure
in practice so emulation is used here. This is a much simpler path
configure, especially because cibuildwheel has support for emulating
non-x86 architectures via QEMU to build wheels. While QEMU emulation of
other architectures is exceedingly slow, we have a 6 hour job time limit
with github actions which should hopefully be sufficient to compile the
binary and build wheels for all our supported python versions.
This was previously attempted in #5844 but we hit a timout issue
reliably every time trying to build scipy from source. Compiling scipy
is slow on a native system and it's an order of magnitude slower under
qemu emulation. In #5844 we exceeded the 6 hour job time limit just
compiling scipy once for testing. To avoid this issue (since scipy
doesn't publish s390x or ppc64le wheels) this commit skips the test
runs on these jobs. This means we're solely building the wheels and not
testing if they're valid. This also means installing scipy and other
dependencies is still an exercise for the s390x and ppc64le users. But,
if this works it will mean that at least for Qiskit there will not be a
need to compile anything from source.
Details and comments