Skip to content

Run wheels workflow on PR and cibuildwheel github action #105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 6, 2023
Merged

Run wheels workflow on PR and cibuildwheel github action #105

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b6740ec
Add SUITESPARSE_FASTEST_BUILD option
alugowski Oct 5, 2023
67dcbdd
Build wheels on PR
alugowski Oct 5, 2023
9878080
Switch to cibuildwheel github action
alugowski Oct 6, 2023
703b7c6
Limit push wheels action run to only main branch
alugowski Oct 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 44 additions & 41 deletions .github/workflows/wheels.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ on:
# Enable Run Workflow button in GitHub UI
workflow_dispatch:

push:
branches: [ main ]

pull_request:

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
Expand Down Expand Up @@ -34,55 +39,47 @@ jobs:
build_wheels:
name: Wheels - ${{ matrix.cibw_archs }} - ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
# graphblas version to use if another one is not provided
default-graphblas-version: "8.2.0"

strategy:
fail-fast: false
matrix:
# Loosely based on scikit-learn's config:
# https://github.com/scikit-learn/scikit-learn/blob/main/.github/workflows/wheels.yml
include:
- os: windows-latest
python-version: "3.8"
platform_id: win_amd64
cibw_archs: "auto"
cibw_archs: "auto64"

# Linux 64 bit manylinux2014
# Linux x86
- os: ubuntu-latest
python-version: "3.8"
platform_id: manylinux_x86_64
manylinux_image: manylinux2014
cibw_archs: "native"
cibw_archs: "x86_64"
# skip musllinux
cibw_skip: "*musl*"

# Linux 64 bit manylinux2014 for aarch64
# Linux aarch64
# Separate runner because this requires emulation (only x86 runners are available) and is very slow.
- os: ubuntu-latest
python-version: "3.8"
platform_id: manylinux_x86_64
manylinux_image: manylinux2014
cibw_archs: "aarch64"
# numpy wheels not available for aarch64 PyPy or musllinux
cibw_skip: "pp* *musl*"

# macOS x86
- os: macos-latest
python-version: "3.8"
platform_id: macosx_x86_64
cibw_archs: "x86_64"

# Use x86 macOS runner to build ARM.
# macOS Apple Silicon cross-compiled on x86 macOS runner.
# GitHub does not offer Apple Silicon yet (only for self-hosted).
# See https://github.com/github/roadmap/issues/528
- os: macos-latest
python-version: "3.8"
platform_id: macosx_x86_64
cibw_archs: "arm64"
# Skip macOS ARM tests on Intel runner.
cibw_test_skip: "*-macosx_arm64"

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

# aarch64 Linux builds are cross-compiled on x86 runners using emulation
# see https://cibuildwheel.readthedocs.io/en/stable/faq/#emulation
- name: Setup QEMU (for aarch64)
Expand All @@ -98,6 +95,20 @@ jobs:
run: |
echo "SUITESPARSE_FAST_BUILD=1" >> $GITHUB_ENV

- name: Setup for testing
if: github.event_name == 'push' || github.event_name == 'pull_request'
# Ask suitesparse.sh to compile in the fastest way possible and provide a GB version to build
run: |
echo "SUITESPARSE_FASTEST_BUILD=1" >> $GITHUB_ENV
echo "GB_VERSION_REF=refs/tags/${{ env.default-graphblas-version }}.0" >> $GITHUB_ENV
shell: bash

- name: Setup GraphBLAS version from git tag
if: ${{ startsWith(github.ref, 'refs/tags/') }}
# If this is a tagged ref, like a release, then use the tag for the graphblas version
run: echo "GB_VERSION_REF=${{ github.ref }}" >> $GITHUB_ENV
shell: bash

- name: Install tools (macOS)
if: contains(matrix.os, 'macos')
# Install coreutils which includes `nproc` used by `make -j` in suitesparse.sh
Expand All @@ -112,18 +123,20 @@ jobs:
sh add_arm_to_libomp_dylib.sh
fi

- name: Build Wheels
- uses: pypa/cibuildwheel@v2.16
with:
output-dir: wheelhouse
env:
# very verbose
CIBW_BUILD_VERBOSITY: 3

# Build SuiteSparse
CIBW_BEFORE_ALL: bash suitesparse.sh ${{ github.ref }}
CIBW_BEFORE_ALL: bash suitesparse.sh ${{ env.GB_VERSION_REF }}

# Install FFI dev library, needed for Python 3.12
CIBW_BEFORE_BUILD_LINUX: yum install -y libffi-devel

CIBW_ENVIRONMENT_LINUX: SUITESPARSE_FAST_BUILD=${{ env.SUITESPARSE_FAST_BUILD }}
CIBW_ENVIRONMENT_PASS_LINUX: SUITESPARSE_FAST_BUILD SUITESPARSE_FASTEST_BUILD

# CMAKE_GNUtoMS=ON asks suitesparse.sh to build libraries in MSVC style on Windows.
CIBW_ENVIRONMENT_WINDOWS: CMAKE_GNUtoMS=ON GRAPHBLAS_PREFIX="C:/GraphBLAS"
Expand All @@ -138,11 +151,8 @@ jobs:
# Architectures to build specified in matrix
CIBW_ARCHS: ${{ matrix.cibw_archs }}

# No 32-bit builds
# no musllinux
# no PyPy aarch64 (only due to build speed, numpy does not ship aarch64 pypy wheels)
# as of writing numpy does not support pypy 3.10
CIBW_SKIP: "*-win32 *_i686 *musl* pp*aarch64 pp310*"
CIBW_SKIP: "${{ matrix.cibw_skip }} pp310*"

# Use delvewheel on Windows.
# This copies graphblas.dll into the wheel. "repair" in cibuildwheel parlance includes copying any shared
Expand All @@ -159,14 +169,7 @@ jobs:
# run tests
CIBW_TEST_COMMAND: "pytest --pyargs suitesparse_graphblas -s -k test_print_jit_config && pytest -v --pyargs suitesparse_graphblas"

# GitHub Actions macOS Intel runner cannot run ARM tests. Uncomment to silence warning.
CIBW_TEST_SKIP: "*-macosx_arm64"

run: |
python -m pip install --upgrade pip
python -m pip install cibuildwheel
python -m cibuildwheel --output-dir wheelhouse .
shell: bash
CIBW_TEST_SKIP: ${{ matrix.cibw_test_skip }}

- uses: actions/upload-artifact@v3
id: uploadAttempt1
Expand All @@ -189,8 +192,8 @@ jobs:
name: Upload to PyPI
needs: [build_wheels, build_sdist]
runs-on: ubuntu-latest
if: github.repository == 'GraphBLAS/python-suitesparse-graphblas'
# if: github.event_name == 'release' && github.event.action == 'published'
# only upload releases to PyPI
if: github.repository == 'GraphBLAS/python-suitesparse-graphblas' && github.event_name == 'release' && github.event.action == 'published'

steps:
- uses: actions/setup-python@v4
Expand All @@ -212,4 +215,4 @@ jobs:

# Test PyPI:
# password: ${{ secrets.TEST_PYPI_API_TOKEN }}
# repository_url: https://test.pypi.org/legacy/
# repository-url: https://test.pypi.org/legacy/
23 changes: 21 additions & 2 deletions suitesparse.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,27 @@ if [ -n "${SUITESPARSE_FAST_BUILD}" ]; then
echo "#define GxB_NO_UINT8 1" >> ../Source/GB_control.h
fi

# Disable all Source/Generated2 kernels. For workflow development only.
#cmake_params+=(-DCMAKE_CUDA_DEV=1)
if [ -n "${SUITESPARSE_FASTEST_BUILD}" ]; then
echo "suitesparse.sh: Fastest build requested."
# Fastest build possible. For use in development and automated tests that do not depend on performance.

echo "#define GxB_NO_BOOL 1" >> ../Source/GB_control.h
echo "#define GxB_NO_FP32 1" >> ../Source/GB_control.h
echo "#define GxB_NO_FP64 1" >> ../Source/GB_control.h
echo "#define GxB_NO_FC32 1" >> ../Source/GB_control.h
echo "#define GxB_NO_FC64 1" >> ../Source/GB_control.h
echo "#define GxB_NO_INT16 1" >> ../Source/GB_control.h
echo "#define GxB_NO_INT32 1" >> ../Source/GB_control.h
echo "#define GxB_NO_INT64 1" >> ../Source/GB_control.h
echo "#define GxB_NO_INT8 1" >> ../Source/GB_control.h
echo "#define GxB_NO_UINT16 1" >> ../Source/GB_control.h
echo "#define GxB_NO_UINT32 1" >> ../Source/GB_control.h
echo "#define GxB_NO_UINT64 1" >> ../Source/GB_control.h
echo "#define GxB_NO_UINT8 1" >> ../Source/GB_control.h

# Disable all Source/Generated2 kernels. For workflow development only.
cmake_params+=(-DCMAKE_CUDA_DEV=1)
fi

# Use `-DJITINIT=2` so that the JIT functionality is available, but disabled by default.
# Level 2, "run", means that pre-JIT kernels may be used, which does not require a compiler at runtime.
Expand Down