Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(build): use uv for cibuildwheel #3695

Merged
merged 18 commits into from
Jun 11, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 7 additions & 16 deletions .github/workflows/build_wheel.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,30 +74,27 @@ jobs:
with:
# https://github.com/pypa/setuptools_scm/issues/480
fetch-depth: 0
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
if: runner.os != 'Linux'
njzjz marked this conversation as resolved.
Show resolved Hide resolved
- uses: docker/setup-qemu-action@v3
name: Setup QEMU
if: matrix.platform_id == 'manylinux_aarch64' && matrix.os == 'ubuntu-latest'
# detect version in advance. See #3168
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.11'
cache: 'pip'
if: matrix.dp_pkg_name == 'deepmd-kit-cu11'
- run: |
python -m pip install setuptools_scm
python -c "from setuptools_scm import get_version;print('SETUPTOOLS_SCM_PRETEND_VERSION='+get_version())" >> $GITHUB_ENV
echo "SETUPTOOLS_SCM_PRETEND_VERSION=$(pipx run uv tool run --from setuptools_scm python -m setuptools_scm)" >> $GITHUB_ENV
rm -rf .git
if: matrix.dp_pkg_name == 'deepmd-kit-cu11'
- name: Build wheels
uses: pypa/cibuildwheel@v2.18
uses: pypa/cibuildwheel@v2.19
env:
CIBW_BUILD_VERBOSITY: 1
CIBW_ARCHS: all
CIBW_BUILD: cp${{ matrix.python }}-${{ matrix.platform_id }}
DP_VARIANT: ${{ matrix.dp_variant }}
CUDA_VERSION: ${{ matrix.cuda_version }}
DP_PKG_NAME: ${{ matrix.dp_pkg_name }}
CIBW_BUILD_FRONTEND: 'build[uv]'
- uses: actions/upload-artifact@v4
with:
name: cibw-cp${{ matrix.python }}-${{ matrix.platform_id }}-cu${{ matrix.cuda_version }}-${{ strategy.job-index }}
Expand All @@ -109,14 +106,8 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-python@v5
name: Install Python
with:
python-version: '3.11'
cache: 'pip'
- run: python -m pip install build
- name: Build sdist
run: python -m build --sdist
run: pipx run uv tool run --with build[uv] --from build python -m build --installer uv --sdist

- uses: actions/upload-artifact@v4
with:
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,4 @@ build_cc_tests
build_c_tests
build_c/
libdeepmd_c/
.uv/
7 changes: 7 additions & 0 deletions backend/find_tensorflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,11 +156,18 @@ def get_tf_requirement(tf_version: str = "") -> dict:
"tensorflow; platform_machine=='aarch64' or (platform_machine=='arm64' and platform_system == 'Darwin')",
# https://github.com/tensorflow/tensorflow/issues/61830
"tensorflow-cpu!=2.15.*; platform_system=='Windows'",
# TODO: build(wheel): unpin h5py on aarch64
# Revert after https://github.com/h5py/h5py/issues/2408 is fixed;
# or set UV_PREFER_BINARY when https://github.com/astral-sh/uv/issues/1794 is resolved.
# 3.6.0 is the first version to have aarch64 wheels.
"h5py>=3.6.0,<3.11.0; platform_system=='Linux' and platform_machine=='aarch64'",
*extra_requires,
],
"gpu": [
"tensorflow",
"tensorflow-metal; platform_machine=='arm64' and platform_system == 'Darwin'",
# See above.
"h5py>=3.6.0,<3.11.0; platform_system=='Linux' and platform_machine=='aarch64'",
wanghan-iapcm marked this conversation as resolved.
Show resolved Hide resolved
*extra_requires,
],
**extra_select,
Expand Down
6 changes: 6 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,12 @@ before-all = [
"""rpm --import https://repo.almalinux.org/almalinux/RPM-GPG-KEY-AlmaLinux""",
"""{ if [ "$(uname -m)" = "x86_64" ] ; then yum config-manager --add-repo http://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo && yum install -y cuda-nvcc-${CUDA_VERSION/./-} cuda-cudart-devel-${CUDA_VERSION/./-}; fi }""",
"yum install -y mpich-devel",
# uv is not available in the old manylinux image
"""{ if [ "$(uname -m)" = "x86_64" ] ; then pipx install uv; fi }""",
]
before-build = [
# old build doesn't support uv
"""{ if [ "$(uname -m)" = "x86_64" ] ; then uv pip install --system -U build; fi }""",
]

[tool.cibuildwheel.windows]
Expand Down
Loading