Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: migrate to pyproject.toml and hatch #371

Merged
merged 3 commits into from
Sep 21, 2023
Merged

chore: migrate to pyproject.toml and hatch #371

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9655922
chore: update pre-commit config
JP-Ellis Sep 13, 2023
5f65fa0
chore!: migrate to pyproject.toml and hatch
JP-Ellis Sep 12, 2023
6f82d7e
chore(ci): migrate cicd to hatch
JP-Ellis Sep 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 28 additions & 30 deletions .cirrus.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,51 +1,49 @@
BUILD_TEST_TASK_TEMPLATE: &BUILD_TEST_TASK_TEMPLATE
TEST_TEMPLATE: &TEST_TEMPLATE
arch_check_script:
- uname -am
test_script:
- python --version
- python -m pip install --upgrade pip
- python -m pip install -r requirements_dev.txt
- python -m flake8
- python -m pydocstyle pact
- python -m tox -e test
# - make examples
# TODO: Fix lints before enabling
- echo hatch run lint
# TODO: Implement the examples to work in hatch
- echo hatch run example
- hatch run test

linux_arm64_task:
linux_arm64_task:
env:
PATH: ${HOME}/.local/bin:${PATH}
matrix:
# - IMAGE: python:3.6-slim # This works locally, with cirrus run, but fails in CI
- IMAGE: python:3.7-slim
- IMAGE: python:3.8-slim
- IMAGE: python:3.9-slim
- IMAGE: python:3.10-slim
- IMAGE: "python:3.8-slim"
- IMAGE: "python:3.9-slim"
- IMAGE: "python:3.10-slim"
- IMAGE: "python:3.11-slim"
arm_container:
image: $IMAGE
install_script:
- apt update --yes && apt install --yes gcc make
<< : *BUILD_TEST_TASK_TEMPLATE

- apt update --yes
- apt install --yes gcc make
- python -m pip install --upgrade pip pipx
- pipx install hatch
<<: *TEST_TEMPLATE

macosx_arm64_task:
macos_instance:
image: ghcr.io/cirruslabs/macos-ventura-base:latest
env:
PATH: ${HOME}/.pyenv/shims:${PATH}
PATH: ${HOME}/.local/bin:${HOME}/.pyenv/shims:${PATH}
matrix:
- PYTHON: 3.6
- PYTHON: 3.7
- PYTHON: 3.8
- PYTHON: 3.9
- PYTHON: 3.10
- PYTHON: "3.8"
- PYTHON: "3.9"
- PYTHON: "3.10"
- PYTHON: "3.11"
install_script:
# Per the pyenv homebrew recommendations.
# https://github.com/pyenv/pyenv/wiki#suggested-build-environment
# - xcode-select --install # Unnecessary on Cirrus
- brew update
# - brew install openssl readline sqlite3 xz zlib
- brew update
- brew install pyenv
- pyenv install ${PYTHON}
- pyenv global ${PYTHON}
- pyenv rehash
## To install rosetta
# - softwareupdate --install-rosetta --agree-to-license
<< : *BUILD_TEST_TASK_TEMPLATE
- python -m pip install --upgrade pip pipx
- pyenv rehash
- pipx install hatch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we always want to run against the latest version of hatch, or do we want to pin this so it could be picked up via say a dependabot config (or similar) to ensure we can test version updates and ensure consistency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as a note, my first experience was that I needed to install hatch, which is where I came back to the CI scripts to see how we are installing it :)

pact-python on  feat/pyproject@origin:feat/pyproject via 🐳 desktop-linux via 🐍 v3.11.4 on ☁️  (eu-west-2) 
🕙15:02:03 ❯ make                                                

  clean      to clear build and distribution directories
  examples   to run the example end to end tests (consumer, fastapi, flask, messaging)
  consumer   to run the example consumer tests
  fastapi    to run the example FastApi provider tests
  flask      to run the example Flask provider tests
  messaging  to run the example messaging e2e tests
  package    to create a distribution package in /dist/
  release    to perform a release build, including deps, test, and package targets
  test       to run all tests


pact-python on  feat/pyproject@origin:feat/pyproject via 🐳 desktop-linux via 🐍 v3.11.4 on ☁️  (eu-west-2) 
🕙15:02:40 ❯ make clean
hatch clean
make: hatch: No such file or directory
make: *** [clean] Error 1

pact-python on  feat/pyproject@origin:feat/pyproject via 🐳 desktop-linux via 🐍 v3.11.4 on ☁️  (eu-west-2) 
🕙15:02:44 [🔴 USAGE] ❯ make package
hatch build
make: hatch: No such file or directory
make: *** [package] Error 1

pact-python on  feat/pyproject@origin:feat/pyproject via 🐳 desktop-linux via 🐍 v3.11.4 on ☁️  (eu-west-2) 
🕙15:02:51 [🔴 USAGE] ❯ make test   
hatch run all
make: hatch: No such file or directory
make: *** [test] Error 1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I think installing the latest version of hatch should generally not be an issue. Hatch is not an actually dependency of pact-python, and instead is just a CLI interface and a build backend. I would be very surprised if regular updates to hatch broke this.

Of course, should hatch come out with a completely overhaul in v2, then we might need to pin it then, but I'm happy to leave this as future problem.

In the same vein, I'm of two minds as to whether we should update system dependencies (apt and brew) as part of the workflow. Personally, I typically lean towards faster CI/CD builds and therefore use whatever packages are installed in the image, and then trust the CI/CD provider to regularly update these packages.

Lastly, would you like me to add a make command to install hatch? Or not really worth it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course, should hatch come out with a completely overhaul in v2, then we might need to pin it then, but I'm happy to leave this as future problem.

Yeah I'm okay with that, the CI will catch any changes, and alert, and we can action accordingly

In the same vein, I'm of two minds as to whether we should update system dependencies (apt and brew) as part of the workflow. Personally, I typically lean towards faster CI/CD builds and therefore use whatever packages are installed in the image, and then trust the CI/CD provider to regularly update these packages.

You get a more repeatable build by not pulling in new updates each version, I've always just updated as a matter of course, going from practises used in Docker files, but also find it incredibly annoying when building things locally that link dynamically, and the dep has been updated with a new named shared lib, causing my built programs to fail, until they are rebuilt. Also why I like statically compiled executables, that will always work, even if they are chonkier.

Lastly, would you like me to add a make command to install hatch? Or not really worth it?

could do, if we did I would favour it being installed with python, so it saves x platform package manager concerns

if we don't, just have it as a pre-req for development somewhere in the docs.

you could have a make doctor command, which checks if the hatch executable is available and alerts people on how to install it

- pyenv rehash
<<: *TEST_TEMPLATE
171 changes: 171 additions & 0 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
name: build

on:
push:
tags:
- v*
branches:
- master
pull_request:
branches:
- master

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true

env:
STABLE_PYTHON_VERSION: "3.11"
CIBW_BUILD_FRONTEND: build
CIBW_TEST_COMMAND: >
python -c
"from pact import EachLike;
assert EachLike(1).generate() == {'json_class': 'Pact::ArrayLike', 'contents': 1, 'min': 1}
"

jobs:
build-x86_64:
name: Build wheels on ${{ matrix.os }} (x86, 64-bit)
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
archs: x86_64
- os: macos-latest
archs: x86_64
- os: windows-latest
archs: AMD64

steps:
- uses: actions/checkout@v4
with:
# Fetch all tags
fetch-depth: 0

- name: Create wheels
uses: pypa/cibuildwheel@v2.15.0
env:
CIBW_ARCHS: ${{ matrix.archs }}

- name: Upload wheels
uses: actions/upload-artifact@v3
with:
name: wheels
path: ./wheelhouse/*.whl
if-no-files-found: error

build-x86:
name: Build wheels on ${{ matrix.os }} (x86, 32-bit)
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: windows-latest
archs: x86

steps:
- uses: actions/checkout@v4
with:
# Fetch all tags
fetch-depth: 0

- name: Create wheels
uses: pypa/cibuildwheel@v2.15.0
env:
CIBW_ARCHS: ${{ matrix.archs }}

- name: Upload wheels
uses: actions/upload-artifact@v3
with:
name: wheels
path: ./wheelhouse/*.whl
if-no-files-found: error

build-arm64:
name: Build wheels on ${{ matrix.os }} (arm64)
runs-on: ${{ matrix.os }}
# As this requires emulation, it's not worth running on PRs
if: >-
github.event_name == 'push' &&
startsWith(github.event.ref, 'refs/tags')
YOU54F marked this conversation as resolved.
Show resolved Hide resolved
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
archs: aarch64
build: "*manylinux*"
- os: macos-latest
archs: arm64
build: "*"

steps:
- uses: actions/checkout@v4
with:
# Fetch all tags
fetch-depth: 0

- name: Set up QEMU
if: matrix.os == 'ubuntu-latest'
uses: docker/setup-qemu-action@v3
with:
platforms: arm64

- name: Create wheels
uses: pypa/cibuildwheel@v2.15.0
env:
CIBW_ARCHS: ${{ matrix.archs }}
CIBW_BUILD: ${{ matrix.build }}

- name: Upload wheels
uses: actions/upload-artifact@v3
with:
name: wheels
path: ./wheelhouse/*.whl
if-no-files-found: error

check:
name: Check wheels
needs:
- build-x86_64
- build-x86
- build-arm64
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: ${{ env.STABLE_PYTHON_VERSION }}

- uses: actions/download-artifact@v3
with:
name: wheels
path: wheelhouse

- run: |
pipx run twine check --strict wheelhouse/*

publish:
name: Publish wheels
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
needs: [check]
runs-on: ubuntu-latest

steps:
- uses: actions/download-artifact@v3
with:
name: artifacts
path: wheelhouse

- name: Push build artifacts to PyPI
uses: pypa/gh-action-pypi-publish@v1.8.10
with:
skip_existing: true
user: ${{ secrets.PYPI_USERNAME }}
password: ${{ secrets.PYPI_PASSWORD }}
packages-dir: wheelhouse
54 changes: 0 additions & 54 deletions .github/workflows/build_and_test.yml
View file
Open in desktop

This file was deleted.

27 changes: 0 additions & 27 deletions .github/workflows/package_and_push_to_pypi.yml
View file
Open in desktop

This file was deleted.

61 changes: 61 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: test

on:
push:
branches:
- master
pull_request:
branches:
- master

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

env:
STABLE_PYTHON_VERSION: "3.11"

jobs:
run:
name: >-
Python ${{ matrix.python-version }}
on ${{ matrix.os }}

runs-on: ${{ matrix.os }}
continue-on-error: ${{ matrix.experimental }}

strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.8", "3.9", "3.10", "3.11"]
experimental: [false]
include:
- # Run tests against the next Python version, but no need for the full list of OSes.
os: ubuntu-latest
python-version: "3.12-dev"
experimental: true

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Install Hatch
run: pip install --upgrade hatch

- # TODO: Fix lints before enabling this
name: Lint
if: matrix.python-version == env.STABLE_PYTHON_VERSION && runner.os == 'Linux'
run: echo hatch run lint

- # TODO: Implement the examples to work in hatch
name: Examples
if: matrix.python-version == env.STABLE_PYTHON_VERSION && runner.os == 'Linux'
run: echo hatch run example

- name: Run tests and track code coverage
run: hatch run test
Loading