Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Python ABI3 wheels instead of a wheel per Python version #1064

Merged
merged 30 commits into from
Apr 11, 2024

Conversation

IvanIsCoding
Copy link
Collaborator

@IvanIsCoding IvanIsCoding commented Jan 21, 2024

Closes #891

This PR switches rustworkx wheels to build against the Python stable ABI. For 0.15, we choose Python 3.8 as the minimum required version.

Most notably, we also remove our hard-coded splits for less common architectures. This simplifies our setup.

We also update cibuildwheel to 2.17.0, actions/download-artifact to v4 (which had breaking changes), and bumped the Python used in the tasks to 3.10.

@IvanIsCoding IvanIsCoding added this to the 0.15.0 milestone Jan 21, 2024
@coveralls
Copy link

coveralls commented Jan 21, 2024

Pull Request Test Coverage Report for Build 8649595792

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.03%) to 96.528%

Totals Coverage Status
Change from base Build 8639249844: 0.03%
Covered Lines: 17320
Relevant Lines: 17943

💛 - Coveralls

@IvanIsCoding IvanIsCoding changed the title [WIP] Build Python ABI3 wheels instead of a wheel per Python version Build Python ABI3 wheels instead of a wheel per Python version Jan 24, 2024
@IvanIsCoding
Copy link
Collaborator Author

IvanIsCoding commented Jan 24, 2024

I still need to remove the ppc64 musl build that also failed the 0.14 release and the test file... apart from that it should be fine

@IvanIsCoding
Copy link
Collaborator Author

@mtreinish this should be ready to review, all the wheels build in https://github.com/IvanIsCoding/rustworkx/actions/runs/8400723506

Copy link
Member

@mtreinish mtreinish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, and thanks for manually validating the builds. I just had 2 quick inline questions but not a blocker per say.

The other question I had was did you do any performance testing to see if there was a measurable impact from moving to abi3?

@@ -91,5 +92,6 @@ def readme():
"mpl": mpl_extras,
"graphviz": graphviz_extras,
"all": mpl_extras + graphviz_extras,
}
},
options=RUST_OPTS,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember having to set this flag, for building abi3 wheels what is requiring this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -29,14 +29,15 @@ jobs:
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.8'
python-version: '3.10'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it's good to bump this to 3.10 since 3.8 goes eol in october (which is a good reminder for us to start emitting a deprecation warning on 3.8 in 0.15.0). But I'm wondering if there was something that required us to move to 3.10 here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My reasoning was that macOS Arm only runs 3.10+ and I wanted to make it consistent

@IvanIsCoding
Copy link
Collaborator Author

LGTM, and thanks for manually validating the builds. I just had 2 quick inline questions but not a blocker per say.

The other question I had was did you do any performance testing to see if there was a measurable impact from moving to abi3?

I did not benchmark it, maybe we should run https://github.com/mtreinish/retworkx-comparison-benchmarks against the main branch vs this PR. Unfortunately the performance won't get better, but maybe it will not get that much worse?

@mtreinish
Copy link
Member

Let's go ahead and merge it now. We can benchmark it after it merges pretty easily. At least when we made this change with qiskit it wasn't really measurable and we can offset it potentially by leveraging PGO or something. But even if there is a small performance regression, the portability benefits are worth it.

@mtreinish mtreinish added the automerge Queue a approved PR for merging label Apr 11, 2024
@mergify mergify bot merged commit d5521e3 into Qiskit:main Apr 11, 2024
30 checks passed
@IvanIsCoding
Copy link
Collaborator Author

@BastianZim @wshanks FYI I don’t know how this affects https://github.com/conda-forge/rustworkx-feedstock but for 0.15.x we will be distributing less binaries. I hope it simplifies your work in conda too

@wshanks
Copy link

wshanks commented Apr 11, 2024

Thanks, @IvanIsCoding. Distributing less binaries doesn't help us directly because we build from source any way, but I hope we can make use of the fact that RustworkX is abi3 compatible in the future. Currently, changes to conda are needed because conda doesn't know where site-packages is outside of a Python version-specific path (like lib/python3.10/site-packages). There is a special noarch format that can do it but here we need something arch-specific but not Python specific. It has been discussed in conda-forge/conda-forge.github.io#1865. I also made a test with the Qiskit package in conda-forge/qiskit-terra-feedstock#41 where I found that conda seemed to hard-code the version range of the Python dependency as well when building a package with Python. I did find that abi3 builds of Qiskit worked for me across Python versions using a symlink to fake the site-packages location and forcing conda to ignore the Python version during install. Hopefully conda gets better support some day 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge Queue a approved PR for merging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate Python Stable ABI
4 participants