Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Port CI to GitHub Actions #780

Merged
merged 39 commits into from
Sep 21, 2020
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
8f96b8f
ci: Run lints on release branches
jan-auer Sep 18, 2020
462f094
ci: Add default-feature tests to GHA
jan-auer Sep 18, 2020
94a585d
ci: Fix workflow matrix config
jan-auer Sep 18, 2020
2ef460c
ci: Run correct test command
jan-auer Sep 18, 2020
312845f
ci: Fix cache key for test job
jan-auer Sep 18, 2020
bf2f6a7
ci: Start redis and test all features
jan-auer Sep 18, 2020
9a86f37
ci: Split test jobs
jan-auer Sep 18, 2020
0079cf4
ci: Expose Redis ports
jan-auer Sep 18, 2020
910b144
ci: Clean out old jobs
jan-auer Sep 18, 2020
a6e7914
build: Disable debug information in debug builds
jan-auer Sep 18, 2020
df73c86
ci: Port integration tests to GHA
jan-auer Sep 18, 2020
73f8bc2
ci: Simplify integration tests
jan-auer Sep 18, 2020
a779cc3
ci: Port python library tests to GHA
jan-auer Sep 18, 2020
9ad447f
ci: Test Zeus uploads
jan-auer Sep 18, 2020
92401eb
ci: Fix actions syntax
jan-auer Sep 18, 2020
0b71181
ci: Simplify npx commands
jan-auer Sep 18, 2020
1d76e86
ci: Split up workflows
jan-auer Sep 18, 2020
e835278
ci: Add builds for Linux, macOS and Windows
jan-auer Sep 18, 2020
5a2e998
ci: Fix actions syntax
jan-auer Sep 18, 2020
e5d8bfd
ci: Fix build jobs
jan-auer Sep 18, 2020
db42ff8
ci: Fix build and debug bundling
jan-auer Sep 18, 2020
b3625d8
ci: Clean up old builds
jan-auer Sep 18, 2020
1375bb0
ci: Lint and style-check python code
jan-auer Sep 18, 2020
2af0fde
ci: Fix windows release build uploads
jan-auer Sep 18, 2020
ea6c7f5
ci: Fix lint python dependencies
jan-auer Sep 18, 2020
37b4aa3
fix: Actual python formatting
jan-auer Sep 18, 2020
bcc6519
ci: Set kafka partition count to 1
jan-auer Sep 18, 2020
f2b3cc0
ci: Port library builds to GHA
jan-auer Sep 18, 2020
7ca2734
ci: Split build workflow
jan-auer Sep 18, 2020
5c29000
ci: Simplify docker builds
jan-auer Sep 18, 2020
f9ccb9a
ci: Build for macOS 10.15
jan-auer Sep 18, 2020
5805067
ci: Install wheel for wheel builds
jan-auer Sep 18, 2020
3e35d67
ci: Do not build manylinux with -it
jan-auer Sep 18, 2020
da94db2
ci: Try expression syntax
jan-auer Sep 18, 2020
0e1bc96
ci: Disable release builds on PRs
jan-auer Sep 18, 2020
f026f29
fix(craft): Remove Travis CI context for library releases
jan-auer Sep 21, 2020
7b97e4b
meta: Changelog
jan-auer Sep 21, 2020
0859f0d
build: Bump macOS wheel version
jan-auer Sep 21, 2020
bf2375f
build: Bump minimum craft requirement
jan-auer Sep 21, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 0 additions & 70 deletions .appveyor.yml

This file was deleted.

116 changes: 116 additions & 0 deletions .github/workflows/build_binary.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
name: Binary Release Build

on:
push:
branches:
- release/**

jobs:
linux:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use a matrix here, but tbh what you have is probably more clear despite some duplication

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I had a matrix at first, but ended up choosing this for the cleaner config. As soon as craft supports GitHub artifacts, the biggest chunk of duplication (zeus uploads) will be gone and this will be neat and clean. Would propose to revisit once we can move off Zeus.

name: Linux
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
with:
submodules: recursive

- name: Build in Docker
run: scripts/docker-build-linux.sh
env:
BUILD_ARCH: x86_64
RELAY_FEATURES: ssl

- name: Bundle Debug File
run: zip -r relay-debug.zip target/x86_64-unknown-linux-gnu/release/relay.debug

- uses: actions/setup-node@v1

- name: Upload to Zeus
env:
ZEUS_API_TOKEN: ${{ secrets.ZEUS_API_TOKEN }}
ZEUS_HOOK_BASE: ${{ secrets.ZEUS_HOOK_BASE }}
run: |
npm install -D @zeus-ci/cli
npx zeus job update -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}
npx zeus upload -b ${{ github.run_id }} -j ${{ github.job }} -t "application/octet-stream" -n relay-Linux-x86_64 target/x86_64-unknown-linux-gnu/release/relay
npx zeus upload -b ${{ github.run_id }} -j ${{ github.job }} -t "application/zip" -n relay-Linux-x86_64-debug.zip relay-debug.zip
npx zeus job update --status=passed -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}

macos:
name: macOS
runs-on: macos-10.15

steps:
- uses: actions/checkout@v2
with:
submodules: recursive

- uses: actions-rs/toolchain@v1
id: toolchain
with:
toolchain: stable
profile: minimal
override: true

- name: Run Cargo Build
uses: actions-rs/cargo@v1
with:
command: build
args: --manifest-path=relay/Cargo.toml --release --features ssl

- name: Bundle dSYM
run: zip -r relay-dsym.zip target/release/relay.dSYM

- uses: actions/setup-node@v1

- name: Upload to Zeus
env:
ZEUS_API_TOKEN: ${{ secrets.ZEUS_API_TOKEN }}
ZEUS_HOOK_BASE: ${{ secrets.ZEUS_HOOK_BASE }}
run: |
npm install -D @zeus-ci/cli
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of curiosity, what was wrong with install -g and running the binary directly?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I can try that. Originally, I used just npx, and then noticed that it is reinstalling on every invocation. On Azure Pipelines, one cannot run global installations, but that may work on GitHub actions.

npx zeus job update -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}
npx zeus upload -b ${{ github.run_id }} -j ${{ github.job }} -t "application/octet-stream" -n relay-Darwin-x86_64 target/release/relay
npx zeus upload -b ${{ github.run_id }} -j ${{ github.job }} -t "application/zip" -n relay-Darwin-x86_64-dsym.zip relay-dsym.zip
npx zeus job update --status=passed -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}

windows:
name: Windows
runs-on: windows-2019

steps:
- uses: actions/checkout@v2
with:
submodules: recursive

- uses: actions-rs/toolchain@v1
id: toolchain
with:
toolchain: stable
profile: minimal
override: true

- name: Run Cargo Build
uses: actions-rs/cargo@v1
with:
command: build
args: --manifest-path=relay/Cargo.toml --release --features ssl

- name: Bundle PDB
run: |
Install-Module 7Zip4PowerShell -Force -Verbose
7z a .\relay-pdb.zip .\target\release\relay.pdb

- uses: actions/setup-node@v1

- name: Upload to Zeus
env:
ZEUS_API_TOKEN: ${{ secrets.ZEUS_API_TOKEN }}
ZEUS_HOOK_BASE: ${{ secrets.ZEUS_HOOK_BASE }}
run: |
npm install -D @zeus-ci/cli
npx zeus job update -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}
npx zeus upload -b ${{ github.run_id }} -j ${{ github.job }} -t "application/octet-stream" -n relay-Windows-x86_64.exe target/release/relay.exe
npx zeus upload -b ${{ github.run_id }} -j ${{ github.job }} -t "application/zip" -n relay-Windows-x86_64-pdb.zip relay-pdb.zip
npx zeus job update --status=passed -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}
105 changes: 105 additions & 0 deletions .github/workflows/build_library.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
name: Library Release Build

on:
push:
branches:
- release-library/**

jobs:
linux:
strategy:
fail-fast: false
matrix:
build-arch: [i686, x86_64]

name: Python Linux ${{ matrix.build-arch }}
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
with:
submodules: recursive

- name: Build in Docker
run: scripts/docker-manylinux.sh
env:
BUILD_ARCH: ${{ matrix.build-arch }}

- uses: actions/setup-node@v1

- name: Upload to Zeus
env:
ZEUS_API_TOKEN: ${{ secrets.ZEUS_API_TOKEN }}
ZEUS_HOOK_BASE: ${{ secrets.ZEUS_HOOK_BASE }}
run: |
npm install -D @zeus-ci/cli
npx zeus job update -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}
npx zeus upload -b ${{ github.run_id }} -j ${{ github.job }} -t "application/zip+wheel" py/dist/*
npx zeus job update --status=passed -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}

macos:
name: Python macOS
runs-on: macos-10.15

steps:
- uses: actions/checkout@v2
with:
submodules: recursive

- uses: actions-rs/toolchain@v1
id: toolchain
with:
toolchain: stable
profile: minimal
override: true

- uses: actions/setup-python@v2
with:
python-version: 2.7

- name: Build Wheel
run: |
pip install wheel
python setup.py bdist_wheel
working-directory: py

- uses: actions/setup-node@v1

- name: Upload to Zeus
env:
ZEUS_API_TOKEN: ${{ secrets.ZEUS_API_TOKEN }}
ZEUS_HOOK_BASE: ${{ secrets.ZEUS_HOOK_BASE }}
run: |
npm install -D @zeus-ci/cli
npx zeus job update -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}
npx zeus upload -b ${{ github.run_id }} -j ${{ github.job }} -t "application/zip+wheel" py/dist/*
npx zeus job update --status=passed -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}

sdist:
name: Python sdist
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
with:
submodules: recursive

- uses: actions/setup-python@v2
with:
python-version: 2.7

- name: Build sdist
run: python setup.py sdist --format=zip
working-directory: py

- uses: actions/setup-node@v1

- name: Upload to Zeus
env:
ZEUS_API_TOKEN: ${{ secrets.ZEUS_API_TOKEN }}
ZEUS_HOOK_BASE: ${{ secrets.ZEUS_HOOK_BASE }}
run: |
npm install -D @zeus-ci/cli
npx zeus job update -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}
npx zeus upload -b ${{ github.run_id }} -j ${{ github.job }} -t "application/zip+wheel" py/dist/*
npx zeus job update --status=passed -b ${{ github.run_id }} -j ${{ github.job }} -r ${{ github.sha }}
Loading