-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci(python): Add cibuildwheel setup for Python wheels #353
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #353 +/- ##
==========================================
- Coverage 88.08% 87.20% -0.89%
==========================================
Files 72 6 -66
Lines 11677 125 -11552
==========================================
- Hits 10286 109 -10177
+ Misses 1391 16 -1375 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
workflow looks good rest also looks sensible but my python is mid ^^
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good!
I added a whole bunch of comments, but many are also small nitpicks that are not crucial at all
|
||
# Build wheels weekly or when requested | ||
on: | ||
pull_request: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could also run in on every commit in main? (I agree it's certainly good to not let it run on every commit in every PR)
push:
branches:
- main
.github/workflows/python-wheels.yaml
Outdated
- {os: "windows-2019", label: "windows"} | ||
- {os: "macOS-11", label: "macOS"} | ||
# Uncomment when apache/arrow-nanoarrow is added to the arrow runner group | ||
# - {os: ["self-hosted", "arm"], label: "linux/arm64"} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BTW, we can build MacOS arm64 wheels here on github actions if we want (just not test them), that's what we do in other projects I am involved with
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In fact, that's already happening (because you defined CIBW_ARCHS_MACOS
to include both, I suppose)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could also try emulation on github actions here for linux arm64 (given it's a small package, that might be possible and not too slow). Then we wouldn't need any other self-hosted runner or different CI platform.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It took a second to figure it out, but it seems like the arm64 runner works!
- name: Install cibuildwheel | ||
run: python -m pip install cibuildwheel==2.15.0 | ||
|
||
- name: Build wheels | ||
run: | | ||
python -m cibuildwheel --output-dir wheelhouse python |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- name: Install cibuildwheel | |
run: python -m pip install cibuildwheel==2.15.0 | |
- name: Build wheels | |
run: | | |
python -m cibuildwheel --output-dir wheelhouse python | |
- name: Build wheels | |
uses: pypa/cibuildwheel@v2.16.2 |
I am not sure why the cibuildwheel README shows the manual way, but the docs recommend using the action (https://cibuildwheel.readthedocs.io/en/stable/setup/#github-actions)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had to revert to the previous version because of the arm self-hosted runner (for which the stock install-python action won't work).
python/pyproject.toml
Outdated
@@ -34,7 +34,7 @@ repository = "https://github.com/apache/arrow-nanoarrow" | |||
[build-system] | |||
requires = [ | |||
"setuptools >= 61.0.0", | |||
"setuptools-scm", | |||
"versioneer[toml]==0.29", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't needed if you use the copied miniver helper?
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
This PR adds a weekly/workflow dispatch job for building and testing Python wheels. This required a few housekeeping items:
Two outstanding issues that don't necessarily need to be solved in this PR: