Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add cibuildwheel GHA #1283

Merged
merged 18 commits into from
Jun 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .ci/build_deps.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/usr/bin/bash

set -ex


download_and_build_netcdf() {
ocefpaf marked this conversation as resolved.
Show resolved Hide resolved
if [ ! -d "netcdf-c" ]; then
netcdf_url=https://github.com/Unidata/netcdf-c
netcdf_src=netcdf-c
netcdf_build=netcdf-build

git clone ${netcdf_url} -b v4.9.2 ${netcdf_src}

cmake ${netcdf_src} -B ${netcdf_build} \
-DENABLE_NETCDF4=on \
-DENABLE_HDF5=on \
-DENABLE_DAP=on \
-DENABLE_TESTS=off \
-DENABLE_PLUGIN_INSTALL=off \
-DBUILD_SHARED_LIBS=on \
-DCMAKE_BUILD_TYPE=Release

cmake --build ${netcdf_build} \
--target install
fi
}

download_and_build_netcdf
195 changes: 195 additions & 0 deletions .github/workflows/cibuildwheel.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
name: Wheels

on:
pull_request:
push:
tags:
- "v*"
release:
types:
- published

permissions:
contents: read

jobs:

build_sdist:
name: Build source distribution
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: actions/setup-python@v4
name: Install Python
with:
python-version: 3.x

- name: Install APT packages
if: contains(${{ matrix.os }}, 'ubuntu')
run: |
sudo apt update
sudo apt install libhdf5-dev libnetcdf-dev

- name: Build sdist
run: >
pip install build
&& python -m build --sdist . --outdir dist

- uses: actions/upload-artifact@v4
with:
name: pypi-artifacts
path: ${{ github.workspace }}/dist/*.tar.gz


build_bdist:
name: "Build ${{ matrix.os }} (${{ matrix.arch }}) wheels"
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-22.04
arch: x86_64
# - os: ubuntu-22.04
# arch: aarch64
- os: macos-14
arch: arm64
CIBW_ENVIRONMENT: MACOSX_DEPLOYMENT_TARGET=14.0
- os: macos-11
arch: x86_64
CIBW_ENVIRONMENT: MACOSX_DEPLOYMENT_TARGET=11.0

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: 'true'

- name: Build oldest and newest Python
shell: bash
# On PRs we run only oldest and newest Python versions to reduce CI load.
# Skips pypy and musllinux everywhere.
# We are buiding 38 and 312 for now.
# These needs to rotate every new Python release.
run: |
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
CIBW_SKIP="pp* cp36-* cp37-* *-musllinux* cp39-* cp310-* cp311-*"
else
CIBW_SKIP="pp* cp36-* cp37-* *-musllinux*"
fi
echo "CIBW_SKIP=$CIBW_SKIP" >> $GITHUB_ENV
echo "Setting CIBW_SKIP=$CIBW_SKIP"

- name: "Building ${{ matrix.os }} (${{ matrix.arch }}) wheels"
uses: pypa/cibuildwheel@v2.18.1
env:
CIBW_SKIP: ${{ env.CIBW_SKIP }}
CIBW_ARCHS: ${{ matrix.arch }}
CIBW_BUILD_FRONTEND: build
CIBW_MANYLINUX_X86_64_IMAGE: manylinux_2_28
CIBW_BEFORE_BUILD_LINUX: >
dnf install -y epel-release
&& dnf install -y hdf5-devel libcurl-devel
&& sh .ci/build_deps.sh
CIBW_ENVIRONMENT: ${{ matrix.CIBW_ENVIRONMENT }}
CIBW_BEFORE_BUILD_MACOS: brew install hdf5 netcdf
CIBW_TEST_REQUIRES: pytest cython packaging
CIBW_TEST_COMMAND: >
python -c "import netCDF4; print(f'netCDF4 v{netCDF4.__version__}')"
&& pytest -s -rxs -v {project}/test

- uses: actions/upload-artifact@v4
with:
name: pypi-artifacts-${{ matrix.os }}-${{ matrix.arch }}
path: ${{ github.workspace }}/wheelhouse/*.whl


build_wheels_windows:
name: Build wheels for ${{matrix.arch}} on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [windows-latest]
arch: [win_amd64]

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: 'true'

- uses: actions/setup-python@v4
name: Install Python
with:
python-version: 3.x

- name: Setup Micromamba Python ${{ matrix.python-version }}
uses: mamba-org/setup-micromamba@v1
with:
environment-name: build
init-shell: bash
create-args: >-
python=${{ matrix.python-version }} libnetcdf=4.9.2 --channel conda-forge

- name: Install cibuildwheel
run: |
python -m pip install --upgrade cibuildwheel delvewheel

- name: Build wheels for Windows (${{ matrix.arch }})
run: cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: "cp39-${{ matrix.arch }} cp310-${{ matrix.arch }} cp311-${{ matrix.arch }} cp312-${{ matrix.arch }}"
CIBW_ENVIRONMENT_WINDOWS: >
HDF5_DIR="C:\\Users\\runneradmin\\micromamba\\envs\\build\\Library"
netCDF4_DIR="C:\\Users\\runneradmin\\micromamba\\envs\\build\\Library"
PATH="C:\\Users\\runneradmin\\micromamba\\envs\\build\\Library\\bin;${PATH}"
CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: >
delvewheel show {wheel}
&& delvewheel repair -w {dest_dir} {wheel}
CIBW_TEST_REQUIRES: pytest cython packaging
CIBW_TEST_COMMAND: >
python -c "import netCDF4; print(f'netCDF4 v{netCDF4.__version__}')"
&& pytest -s -rxs -v {project}\\test

- uses: actions/upload-artifact@v4
with:
name: pypi-artifacts-${{ matrix.os }}-${{ matrix.arch }}
path: ${{ github.workspace }}/wheelhouse/*.whl


show-artifacts:
needs: [build_bdist, build_sdist, build_wheels_windows]
name: "Show artifacts"
runs-on: ubuntu-22.04
steps:
- uses: actions/download-artifact@v4
with:
pattern: pypi-artifacts*
path: ${{ github.workspace }}/dist
merge-multiple: true

- shell: bash
run: |
ls -lh ${{ github.workspace }}/dist


publish-artifacts-pypi:
needs: [build_bdist, build_sdist, build_wheels_windows]
name: "Publish to PyPI"
runs-on: ubuntu-22.04
# upload to PyPI for every tag starting with 'v'
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v')
steps:
- uses: actions/download-artifact@v3
with:
name: pypi-artifacts
path: ${{ github.workspace }}/dist

- uses: pypa/gh-action-pypi-publish@release/v1
with:
user: __token__
password: ${{ secrets.PYPI_PASSWORD }}
print_hash: true
1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ tests = [
"pytest",
]


[project.readme]
text = """\
netCDF version 4 has many features not found in earlier versions of the library,
Expand Down
4 changes: 0 additions & 4 deletions test/run_all.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,6 @@
m = __import__(os.path.splitext(f)[0])
testsuite.addTests(unittest.TestLoader().loadTestsFromModule(m))

# Run the test suite.
def test(verbosity=1):
runner = unittest.TextTestRunner(verbosity=verbosity)
runner.run(testsuite)
ocefpaf marked this conversation as resolved.
Show resolved Hide resolved

if __name__ == '__main__':
import numpy, cython
Expand Down
Loading