Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to PEP 621, and add GitHub Actions configuration #118

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

freakboy3742
Copy link
Contributor

Stemming from #117, this PR attempts to restore CI configuration, independent of any other changes.

It upgrades the project configuration to PEP 621 format (i.e., pyproject.toml). A pyproject.toml file with a PEP517 configuration is required to avoid warnings (and, in the very near future installation errors); migrating the rest of the configuration was relatively trivial.

It adds a tox configuration to make it easier to run tests for multiple Python versions; and adds a GitHub Actions configuration to invoke that tox configuration across a matrix of Python versions.

Coverage reporting is done in the build, and a HTML report is uploaded as a build artefact.

@freakboy3742 freakboy3742 marked this pull request as ready for review September 10, 2024 03:18
@freakboy3742
Copy link
Contributor Author

@benfogle FYI - it's difficult to finalize this PR until GitHub Actions are actually running. I think this is everything that is needed, but I think actions won't actually run until there's at least a stub actions configuration in the main branch.

@freakboy3742
Copy link
Contributor Author

@benfogle I've triggered this CI configuration in my own repo; however, it's currently failing, with the same (or similar) errors to what I'm seeing locally.

You've mentioned that you're able to run the tests on Ubuntu 22.04 - however, I'm not sure I see how (at least, not without some extensive apt repo customization). The problems seen in the CI runs all stem from the fact that the 2 prebuilt environments (prebuilt_musl_arm_aarch64.tar.xz and prebuild_python3.9.0.tar.xz) have been built on (I think) Ubuntu 18.04. As a result, they reference libssl1.1.so and libffi6.so. These libraries are no longer available on Ubuntu 22.04 (OpenSSL 1.1.1 is unmaintained, and 22.04 ships with libffi7.so).

Ubuntu 18.04 is no longer available as a runtime environment on GitHub Actions, so that's not a good option for CI. However, it's not clear to me how the prebuilt environments were built (or even if that's something you'd want to be contributed as part of a PR, due to the potential security risk of binaries from third party).

Are you able to advise:

  1. How the prebuilt environments are built?
  2. How you'd like to proceed with updating those prebuilt environments for compatibility with Ubuntu 22.04 (or 24.04, given that is the current ubuntu-latest environment is in GitHub Actions).

@freakboy3742
Copy link
Contributor Author

@benfogle I've now tried running the test suite under Docker using Ubuntu 18.04, and I haven't had any more luck.

Most of these problems stem from the fact that the 18.04 ships with Python 3.6 - and more recent versions of pip and setuptools aren't backwards compatible that far. As a result, the numpy and bcrypt test_basic tests fail because the setup.py configurations of the specific packages that are referenced aren't compatible with recent versions of Python, and you end up in the general pre-PEP 517 hell of finding a combination of Python, pip, and setuptools that are mutually compatible.

I've tried rebuilding the prebuilt packages locally under 22.04, and didn't have any luck, either. My first attempt failed because the URL for zlib downloads has changed; on the second attempt, after an hour, the build failed at 90% trying to configure host-python-main-armhf, with an error complaining about a missing --with-build-python argument. From the look of it, that's because it's trying to build the main branch of Python with a set of configure arguments that haven't been sufficient since Python 3.10.

So - I'm not sure where to take things from here. It appears there's a lot of work needed to restore CI to a working state; however, I'm not sure of the best way to contribute that work. Any tips, suggestions or preferences would be gratefully accepted at this point.

@benfogle
Copy link
Owner

benfogle commented Oct 6, 2024

Thanks for this. You're 100% right that more needs to change to restore CI. I need to poke around at a few options and figure out what's best. It will probably involve converting the git lfs artifacts into docker images. I will likely base that work of off this branch. Thanks for being patient with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants