-
Notifications
You must be signed in to change notification settings - Fork 995
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: New workflow to build wheels #2866
Merged
Merged
Changes from 3 commits
Commits
Show all changes
61 commits
Select commit
Hold shift + click to select a range
c245e20
Add new build workflow
kevjumba b0256b9
Fix
kevjumba 609557a
Fix
kevjumba 090c7af
fix
kevjumba 6ed11be
Fix
kevjumba 6d5d9a0
Fix
kevjumba c3a55e5
Fix
kevjumba 5110ef9
Fix
kevjumba d5a8f36
Fix
kevjumba 6be258d
Fix
kevjumba 4125536
Fix
kevjumba e829bbd
Fix
kevjumba 7fdda76
Fix
kevjumba 21d07d7
Fix
kevjumba 7a37d94
Fix
kevjumba 1a7033e
Fix
kevjumba 3ad9ce2
Fix
kevjumba 2730f96
Fix
kevjumba 1bb556f
Fix
kevjumba 8028bf9
Fix
kevjumba fbada35
Fix
kevjumba f16d464
Fix
kevjumba 58c5dda
Fix
kevjumba 485444f
Fix
kevjumba c55bcb2
Fix
kevjumba 95eb809
Fix
kevjumba 0e586a0
Fix
kevjumba 79745f2
Fix
kevjumba 1352bfc
Fix
kevjumba 120d6d0
Fix
kevjumba f7ad4de
Fix
kevjumba ab70a0d
Fix
kevjumba 4acef81
Fix
kevjumba e917726
Fix
kevjumba 46ecf7c
Fix
kevjumba 1ed9e3b
Fix
kevjumba 299a6d7
Fix
kevjumba 1b8d91f
Fix
kevjumba a36f302
Fix
kevjumba cea432f
Fix
kevjumba b32f4f3
Fix
kevjumba 1542796
Fix
kevjumba 036b02a
Fix
kevjumba 68286be
Fix
kevjumba 627001a
Fix
kevjumba f636b53
Fix
kevjumba 1dd685f
Fix
kevjumba b3f5995
Fix
kevjumba 470c6a6
Fix
kevjumba 20953f2
Fix
kevjumba 831359f
Fix
kevjumba 0a6c621
Fix
kevjumba eb5aa24
Fix
kevjumba 5000966
Fix
kevjumba e8dd17c
Fix
kevjumba 8a8f994
Finally working version
kevjumba 0a6fffc
Fix
kevjumba 1d9feab
Fix
kevjumba 978ce22
Fix
kevjumba e132df5
Fix
kevjumba 1c00b99
Fix
kevjumba File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,184 @@ | ||
name: build_wheels | ||
|
||
on: workflow_dispatch | ||
|
||
jobs: | ||
get-version: | ||
runs-on: ubuntu-latest | ||
outputs: | ||
release_version: ${{ steps.get_release_version.outputs.release_version }} | ||
version_without_prefix: ${{ steps.get_release_version_without_prefix.outputs.version_without_prefix }} | ||
highest_semver_tag: ${{ steps.get_highest_semver.outputs.highest_semver_tag }} | ||
steps: | ||
- uses: actions/checkout@v2 | ||
- name: Get release version | ||
id: get_release_version | ||
run: echo ::set-output name=release_version::${GITHUB_REF#refs/*/} | ||
- name: Get release version without prefix | ||
id: get_release_version_without_prefix | ||
env: | ||
RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }} | ||
run: | | ||
echo ::set-output name=version_without_prefix::${RELEASE_VERSION:1} | ||
- name: Get highest semver | ||
id: get_highest_semver | ||
env: | ||
RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }} | ||
run: | | ||
source infra/scripts/setup-common-functions.sh | ||
SEMVER_REGEX='^v[0-9]+\.[0-9]+\.[0-9]+(-([0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*))?$' | ||
if echo "${RELEASE_VERSION}" | grep -P "$SEMVER_REGEX" &>/dev/null ; then | ||
echo ::set-output name=highest_semver_tag::$(get_tag_release -m) | ||
fi | ||
- name: Check output | ||
env: | ||
RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }} | ||
VERSION_WITHOUT_PREFIX: ${{ steps.get_release_version_without_prefix.outputs.version_without_prefix }} | ||
HIGHEST_SEMVER_TAG: ${{ steps.get_highest_semver.outputs.highest_semver_tag }} | ||
run: | | ||
echo $RELEASE_VERSION | ||
echo $VERSION_WITHOUT_PREFIX | ||
echo $HIGHEST_SEMVER_TAG | ||
|
||
build-python-sdk: | ||
name: Build wheels on ${{ matrix.os }} | ||
runs-on: ${{ matrix.os }} | ||
strategy: | ||
matrix: | ||
os: [ ubuntu-latest, macos-10.15 ] | ||
steps: | ||
- uses: actions/checkout@v2 | ||
- name: Setup Node | ||
uses: actions/setup-node@v2 | ||
with: | ||
node-version: '17.x' | ||
registry-url: 'https://registry.npmjs.org' | ||
- name: Build UI | ||
run: make build-ui | ||
- name: Build wheels | ||
uses: pypa/cibuildwheel@v2.4.0 | ||
env: | ||
CIBW_BUILD: "cp3*_x86_64" | ||
CIBW_SKIP: "cp36-* *-musllinux_x86_64 cp310-macosx_x86_64" | ||
CIBW_ARCHS: "native" | ||
CIBW_ENVIRONMENT: > | ||
COMPILE_GO=True PATH=$PATH:/usr/local/go/bin | ||
CIBW_BEFORE_ALL_LINUX: | | ||
curl -o go.tar.gz https://dl.google.com/go/go1.18.2.linux-amd64.tar.gz | ||
tar -C /usr/local -xzf go.tar.gz | ||
go version | ||
CIBW_BEFORE_ALL_MACOS: | | ||
curl -o python.pkg https://www.python.org/ftp/python/3.9.12/python-3.9.12-macosx10.9.pkg | ||
sudo installer -pkg python.pkg -target / | ||
CIBW_BEFORE_BUILD: | | ||
make install-protoc-dependencies | ||
make install-go-proto-dependencies | ||
make install-go-ci-dependencies | ||
|
||
- uses: actions/upload-artifact@v2 | ||
with: | ||
name: wheels | ||
path: ./wheelhouse/*.whl | ||
|
||
|
||
build-python-sdk-macos-py310: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why don't we move this into There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done |
||
runs-on: macos-10.15 | ||
steps: | ||
- uses: actions/checkout@v2 | ||
- name: Setup Python | ||
id: setup-python | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: "3.10" | ||
architecture: x64 | ||
- name: Setup Node | ||
uses: actions/setup-node@v2 | ||
with: | ||
node-version: '17.x' | ||
registry-url: 'https://registry.npmjs.org' | ||
- name: Build and install dependencies | ||
run: | | ||
pip install -U pip setuptools wheel twine | ||
make install-protoc-dependencies | ||
make install-go-proto-dependencies | ||
make install-go-ci-dependencies | ||
make build-ui | ||
- name: Build | ||
run: | | ||
python3 setup.py sdist bdist_wheel | ||
|
||
- uses: actions/upload-artifact@v2 | ||
with: | ||
name: wheels | ||
path: dist/* | ||
|
||
verify-python-wheel: | ||
runs-on: ${{ matrix.os }} | ||
needs: [build-python-sdk, build-python-sdk-macos-py310] | ||
strategy: | ||
matrix: | ||
os: [ ubuntu-latest, macos-10.15 ] | ||
python-version: [ "3.7", "3.8", "3.9", "3.10"] | ||
from-source: [ True, False ] | ||
env: | ||
# this script is for testing servers | ||
# it starts server with timeout and checks whether process killed by timeout (started healthy) or died by itself | ||
TEST_SCRIPT: | | ||
timeout 10s $@ & pid=$! | ||
wait $pid | ||
ret=$? | ||
if [[ $ret -ne 124 ]] | ||
then | ||
exit $ret | ||
else | ||
echo "Succeeded!" | ||
fi | ||
steps: | ||
- name: Setup Python | ||
id: setup-python | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
architecture: x64 | ||
- uses: actions/setup-go@v3 | ||
with: | ||
go-version: '>=1.17.0' | ||
- uses: actions/download-artifact@v2 | ||
with: | ||
name: wheels | ||
path: dist | ||
- name: Install wheel | ||
if: ${{ !matrix.from-source }} | ||
# try to install all wheels; only the current platform wheel should be actually installed | ||
run: | | ||
cd dist/ | ||
pip install wheel | ||
for f in *.whl; do pip install $f || true; done | ||
- name: Install go from sdist | ||
if: ${{ matrix.from-source }} | ||
env: | ||
COMPILE_GO: "True" | ||
run: | | ||
pip install 'grpcio-tools==1.44.0' 'pybindgen==0.22.0' | ||
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26.0 | ||
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1.0 | ||
pip install dist/*tar.gz | ||
- name: Install OS X dependencies | ||
if: matrix.os == 'macos-10.15' | ||
run: brew install coreutils | ||
- name: Smoke test | ||
run: | | ||
feast init test_repo | ||
cd test_repo/ | ||
feast apply | ||
echo "$TEST_SCRIPT" > run-and-wait.sh | ||
bash run-and-wait.sh feast serve | ||
bash run-and-wait.sh feast ui | ||
# We disable this test for the Python 3.10 binary since it does not include Go. | ||
- name: Smoke test with go | ||
if: matrix.python-version != '3.10' || matrix.os == 'ubuntu-latest' | ||
run: | | ||
pip install cffi | ||
printf "\ngo_feature_retrieval: True" >> feature_store.yaml | ||
echo "$TEST_SCRIPT" > run-and-wait.sh | ||
bash run-and-wait.sh feast serve |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit,
build-python-wheel
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.