Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build library and also Python wheels in CI #390

Draft
wants to merge 29 commits into
base: master
Choose a base branch
from

Conversation

zacharyburnett
Copy link
Contributor

@zacharyburnett zacharyburnett commented Nov 7, 2024

closes #389

attempt to use pypa/cibuildwheel pypa/build to build wheels for Linux, macOS, and Windows

EDIT: also run cmake and make install in another job to test the build

@jmr
Copy link
Member

jmr commented Nov 7, 2024

Thanks for trying this. I authorized and triggered the workflows. They all fail.

https://github.com/google/s2geometry/actions/runs/11724915532/job/32662569678?pr=390

Getting action download info
Error: Unable to resolve action `actions/checkout@v5`, unable to find version `v5`

Doesn't this also need to do something to install the dependencies like abseil-cpp?

@zacharyburnett
Copy link
Contributor Author

Thanks for trying this. I authorized and triggered the workflows. They all fail.

https://github.com/google/s2geometry/actions/runs/11724915532/job/32662569678?pr=390

Getting action download info
Error: Unable to resolve action `actions/checkout@v5`, unable to find version `v5`

oof, I always forget it only has v4 😅 I'll just pin it to a specific release

Doesn't this also need to do something to install the dependencies like abseil-cpp?

oh yes, I forgot about abseil

@jmr
Copy link
Member

jmr commented Nov 7, 2024

oof, I always forget it only has v4 😅 I'll just pin it to a specific release

Can you use v4 instead of v5? Or not specify a version? Why is a hash better?

I triggered it again for another batch of failures.

@zacharyburnett
Copy link
Contributor Author

zacharyburnett commented Nov 7, 2024

oof, I always forget it only has v4 😅 I'll just pin it to a specific release

Can you use v4 instead of v5? Or not specify a version? Why is a hash better?

it's better to use a commit hash for security, because if someone takes control of the upstream action repo they could theoretically retag an existing tag to point to malicious code

It's not a pressing issue though, so we could just change it to v4 here, you're right

@zacharyburnett
Copy link
Contributor Author

zacharyburnett commented Nov 7, 2024

thank you for being so patient as I iterate on this workflow! I'm attempting to make it easier to obtain wheels for s2geometry such that we could maybe use this downstream as a backend for our spherical_geometry package

@jmr
Copy link
Member

jmr commented Nov 7, 2024

I ran it again.

@zacharyburnett
Copy link
Contributor Author

zacharyburnett commented Nov 7, 2024

I'm able to run actions on my fork: https://github.com/zacharyburnett/s2geometry/actions

I'll try to get this running

.github/workflows/build.yml Outdated Show resolved Hide resolved
.github/workflows/build.yml Outdated Show resolved Hide resolved
@jmr
Copy link
Member

jmr commented Nov 8, 2024

Install the project...
-- Install configuration: ""
CMake Error at absl/base/cmake_install.cmake:46 (file):
  file cannot create directory: /usr/local/lib/pkgconfig.  Maybe need
  administrative privileges.
Call Stack (most recent call first):
  absl/cmake_install.cmake:47 (include)
  cmake_install.cmake:47 (include)

You probably need -DCMAKE_INSTALL_PREFIX=... for abseil-cpp.

I use -DCMAKE_PREFIX_PATH=/tmp/absl-install-17 -DCMAKE_INSTALL_PREFIX=/tmp/absl-install-17 -DABSL_ENABLE_INSTALL=ON -DBUILD_TESTING=off -DCMAKE_VERBOSE_MAKEFILE=on -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_CXX_STANDARD=17, but I'm not sure how many of these are really needed.

Then build s2geometry with -DCMAKE_PREFIX_PATH=/tmp/absl-install-17.

@zacharyburnett
Copy link
Contributor Author

Install the project...
-- Install configuration: ""
CMake Error at absl/base/cmake_install.cmake:46 (file):
  file cannot create directory: /usr/local/lib/pkgconfig.  Maybe need
  administrative privileges.
Call Stack (most recent call first):
  absl/cmake_install.cmake:47 (include)
  cmake_install.cmake:47 (include)

You probably need -DCMAKE_INSTALL_PREFIX=... for abseil-cpp.

I use -DCMAKE_PREFIX_PATH=/tmp/absl-install-17 -DCMAKE_INSTALL_PREFIX=/tmp/absl-install-17 -DABSL_ENABLE_INSTALL=ON -DBUILD_TESTING=off -DCMAKE_VERBOSE_MAKEFILE=on -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_CXX_STANDARD=17, but I'm not sure how many of these are really needed.

Then build s2geometry with -DCMAKE_PREFIX_PATH=/tmp/absl-install-17.

ah, thank you! I apologize, I'm not familiar with building C++ at all, so I appreciate your help

@zacharyburnett zacharyburnett changed the title build Python wheels in CI build library and also Python wheels in CI Nov 8, 2024
@zacharyburnett
Copy link
Contributor Author

zacharyburnett commented Nov 8, 2024

from testing on the fork, the cmake job seems to succeed building now, but the cibuildwheel still fails:
image

@jmr
Copy link
Member

jmr commented Nov 8, 2024

How do you make cibuildwheel show the cmake output? I can't see why it failed.

https://github.com/google/s2geometry/actions/runs/11747466338/job/32729571493?pr=390

@jmr
Copy link
Member

jmr commented Nov 8, 2024

@jmr
Copy link
Member

jmr commented Nov 8, 2024

nproc doesn't exist on macOS. sysctl -n hw.physicalcpu or logicalcpu is the equivalent.

@zacharyburnett zacharyburnett force-pushed the ci/build branch 7 times, most recently from 275c7e5 to ed5b5e7 Compare November 11, 2024 16:17
@zacharyburnett zacharyburnett force-pushed the ci/build branch 10 times, most recently from 09eae89 to edd019c Compare December 5, 2024 18:24
@zacharyburnett
Copy link
Contributor Author

so I've been QUITE dumb 😅

All this time I've been trying to recreate cibuildwheel's functionality because I didn't know how to include the built version of abseil-cpp in the container, BUT I neglected to realize that I could have just been installing it with yum and brew (see #390 (comment)). I'm testing with cibuilwheel now, with a before-all command to install the requisite libraries in the config

@zacharyburnett zacharyburnett force-pushed the ci/build branch 7 times, most recently from d88728b to ac53057 Compare December 5, 2024 19:20
@zacharyburnett
Copy link
Contributor Author

taking this back into draft until I can figure out the magic config for cibuildwheel that'll work

@jmr
Copy link
Member

jmr commented Dec 5, 2024

Ok, thanks for working so hard on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use cibuildwheel for Python CI
4 participants