Skip to content

Commit

Permalink
ci: Split out worflows
Browse files Browse the repository at this point in the history
This change splits out our GitHub workflows to run conditionally,
depending on which files change in a pull request.

* `build`: builds the proxy in release mode wh
* `check-each`: checks that each Cargo.toml checks cleanly on its own.
  This used to be a serial task, but has been converted to a matrix job.
* `deps`: Only runs on dependency changes to check cargo-deny
* `integration`: Runs integration tests on code & dependency changes.
* `lint`: Runs clippy, fmt, and doc on all source changes. Not run on
  dependency changes.
* `test`: Runs unit tests on code and dependency changes

This prevents doing needless work on dependency changes (as these are
extremely common) and makes it easier to rerun narrower workflows, if
necessary. The best part of this, though, is that we've made
`check-each` a matrix job so that its tests can be parallelized.

We'll update the github settings to make none of these checks strictly
required.

Signed-off-by: Oliver Gould <ver@buoyant.io>
  • Loading branch information
olix0r committed Feb 11, 2022
1 parent 0530c00 commit a4197af
Show file tree
Hide file tree
Showing 8 changed files with 217 additions and 118 deletions.
30 changes: 30 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Builds the proxy as if it were a release.
name: build

permissions:
contents: read

on:
pull_request:
paths:
# We don't run this workflow on dependency changes. It's mainly intended to determine how long
# a release build takes given Linkerd changes. We don't really need to run this on every
# dependabot change, though.
- "**/*.rs"
- .github/workflows/build.yml

env:
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
RUST_BACKTRACE: short
RUSTUP_MAX_RETRIES: 10

jobs:
release:
timeout-minutes: 20
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
- env:
CARGO_RELEASE: "1"
run: make build
54 changes: 54 additions & 0 deletions .github/workflows/check-each.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Check each crate independently. Helps to catch dependency issues.
name: check-each

permissions:
contents: read

on:
pull_request:
paths:
- Cargo.lock
- "**/*.rs"
- "**/Cargo.toml"
- .github/workflows/check-each.yml

env:
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
RUST_BACKTRACE: short
RUSTUP_MAX_RETRIES: 10

jobs:
enumerate:
timeout-minutes: 3
runs-on: ubuntu-latest
container:
image: docker://rust:1.56.1-buster
env:
DEBIAN_FRONTEND: noninteractive
steps:
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
- run: apt update && apt install -y jq
- run: cargo fetch
- name: list crates
id: list-crates
run: |
echo "::set-output name=crates::$(cargo metadata --frozen --format-version=1 \
| jq -cr "[.packages[] | select(.manifest_path | startswith(\"$PWD\")) | .name | select(. != \"linkerd-meshtls-boring\")]")"
outputs:
crates: ${{ steps.list-crates.outputs.crates }}`

check:
needs: enumerate
timeout-minutes: 20
runs-on: ubuntu-latest
container:
image: docker://rust:1.56.1-buster
strategy:
matrix:
crate: ${{ fromJson(needs.enumerate.outputs.crates) }}
steps:
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
- run: cargo fetch
- run: cargo check -p ${{ matrix.crate }} --frozen --all-targets

15 changes: 9 additions & 6 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
@@ -1,21 +1,24 @@
name: Coverage
name: coverage

permissions:
contents: read

# Run daily at 11AM UTC (3AM PST).
on:
pull_request:
paths:
- .github/workflows/coverage.yml
schedule:
# cron: minute hour day month day-of-week
# Run daily at 11AM UTC (3AM PST).
- cron: '0 11 * * *'

env:
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
CARGO_TARPAULIN_VERSION: 0.18.5
DEBIAN_FRONTEND: noninteractive
RUST_BACKTRACE: short
RUSTUP_MAX_RETRIES: 10

permissions:
contents: read

jobs:
test:
name: codecov
Expand Down
41 changes: 41 additions & 0 deletions .github/workflows/deps.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Audits dependencies with cargo-deny
name: deps

permissions:
contents: read

on:
pull_request:
paths:
- Cargo.lock
- .github/workflows/deps.yml

env:
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
RUST_BACKTRACE: short
RUSTUP_MAX_RETRIES: 10

jobs:
# Check for security advisories.
#
# Failures are not fatal, since issues are opened in the linkerd2 repo via rustsecbot.
advisories:
timeout-minutes: 10
runs-on: ubuntu-latest
continue-on-error: true
steps:
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
- uses: EmbarkStudios/cargo-deny-action@4340bbf5bc9e7034fae7c4857e9ab87cab35c905
with:
command: check advisories

# Audit licenses, unreleased crates, and unexpected duplicate versions.
bans:
timeout-minutes: 10
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
- uses: EmbarkStudios/cargo-deny-action@4340bbf5bc9e7034fae7c4857e9ab87cab35c905
with:
command: check bans licenses sources
55 changes: 15 additions & 40 deletions .github/workflows/slow.yml → .github/workflows/integration.yml
Original file line number Diff line number Diff line change
@@ -1,53 +1,28 @@
# Each job typically runs for more than 5 minutes.
name: slow
# Run integration tests
name: integration

permissions:
contents: read

on:
pull_request: {}
pull_request:
paths:
- Cargo.lock
- "**/*.rs"
- "**/*.toml"
- .github/workflows/integration.yml

env:
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
RUST_BACKTRACE: short
RUSTUP_MAX_RETRIES: 10

permissions:
contents: read

# Run only the app-level tests. These may take longer to compile (usually due to very large stack
# types) and have the potential to be flakey as they depend on opening sockets and may have timing
# sensitivity.
jobs:

# Iterate through all (non-fuzzer) sub-crates to ensure each compiles independently.
check-each-crate:
timeout-minutes: 20
runs-on: ubuntu-latest
container:
image: docker://rust:1.56.1-buster
steps:
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
- run: |
for toml in $(find . -mindepth 2 \
-not -path '*/fuzz/*' \
-not -path './linkerd/meshtls/boring/*' \
-name Cargo.toml \
| sort -r)
do
d="${toml%/*}"
echo "# $d"
(cd $d ; cargo check --all-targets)
done
check-release:
timeout-minutes: 20
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
- env:
CARGO_RELEASE: "1"
run: make build

# Run only the app-level tests. These may take longer to compile (usually due to very large stack
# types) and have the potential to be flakey as they depend on opening sockets and may have timing
# sensitivity.
test-integration:
test:
timeout-minutes: 20
runs-on: ubuntu-latest
container:
Expand Down
52 changes: 52 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Each job should typically run in under 5 minutes.
name: lint

permissions:
contents: read

on:
pull_request:
paths:
- "**/*.rs"
- .github/workflows/lint.yml

env:
CARGO_INCREMENTAL: 0
CARGO_NET_RETRY: 10
RUST_BACKTRACE: short
RUSTUP_MAX_RETRIES: 10

jobs:
clippy:
timeout-minutes: 10
runs-on: ubuntu-latest
container:
image: docker://rust:1.56.1-buster
steps:
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
- run: rustup component add clippy
- run: cargo clippy --all --exclude=linkerd-meshtls-boring

fmt:
timeout-minutes: 10
runs-on: ubuntu-latest
container:
image: docker://rust:1.56.1-buster
steps:
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
- run: rustup component add rustfmt
- run: make check-fmt

docs:
timeout-minutes: 10
runs-on: ubuntu-latest
container:
image: docker://rust:1.56.1-buster
steps:
- uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579
- run: |
cargo doc --all --no-deps \
--exclude=linkerd-meshtls \
--exclude=linkerd-meshtls-boring \
--exclude=linkerd-meshtls-rustls
5 changes: 2 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Release
name: release

on:
push:
Expand Down Expand Up @@ -68,7 +68,6 @@ jobs:

release:
needs: [package]
name: GitHub Release
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
Expand All @@ -91,7 +90,7 @@ jobs:
- name: display structure of downloaded files
run: ls -R artifacts

- name: release
- name: publish
uses: softprops/action-gh-release@1e07f4398721186383de40550babbdf2b84acfc5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down
Loading

0 comments on commit a4197af

Please sign in to comment.