Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: switch scheduled jobs to nix built kernel #1280

Merged
merged 1 commit into from
Feb 3, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/actions/restore-kernel-cache/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: 'Restore kernel cache'
description: 'Restore a previously built kernel cache, failing if not available'
inputs:
git-repo:
required: true
type: string
branch:
required: true
type: string

runs:
using: "composite"
steps:
- name: Get hash from repo/branch
run: echo "SCHED_EXT_KERNEL_COMMIT=$(git ls-remote ${{ inputs.git-repo }} heads/${{ inputs.branch }} | awk '{print $1}')" >> $GITHUB_ENV
shell: bash

- name: Load cache
uses: actions/cache/restore@v4
with:
fail-on-cache-miss: true
key: kernel-build-${{ env.SCHED_EXT_KERNEL_COMMIT }}-7
path: |
linux/arch/x86/boot/bzImage
linux/usr/include
linux/**/*.h
91 changes: 8 additions & 83 deletions .github/workflows/bpf-next-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,74 +6,12 @@ on:
- cron: "0 */6 * * *"

jobs:
build-kernel-nix:
build-kernel:
uses: ./.github/workflows/build-kernel.yml
with:
git-repo: "https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git"
branch: for-next

build-kernel:
runs-on: ubuntu-24.04
steps:
# prevent cache permission errors
- run: sudo chown root /usr/bin/tar && sudo chmod u+s /usr/bin/tar
# redundancy to exit fast
- run: echo 'debconf debconf/frontend select Noninteractive' | sudo debconf-set-selections
- run: sudo apt-get update
- run: sudo apt-get install -y git --no-install-recommends
# get latest head commit of sched_ext for-next
- run: echo "SCHED_EXT_KERNEL_COMMIT=$(git ls-remote https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git heads/for-next | awk '{print $1}')" >> $GITHUB_ENV

- uses: actions/checkout@v4

# use cached kernel if available, create after job if not
- name: Cache Kernel
id: cache-kernel
uses: actions/cache@v4
with:
path: |
linux/arch/x86/boot/bzImage
linux/usr/include
linux/**/*.h
key: kernel-build-${{ env.SCHED_EXT_KERNEL_COMMIT }}-bpf-1

- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }}
uses: ./.github/actions/install-deps-action

# cache virtiofsd (goes away w/ 24.04)
- name: Cache virtiofsd
id: cache-virtiofsd
uses: actions/cache@v4
with:
path: |
/usr/lib/virtiofsd
key: virtiofsd-binary
- if: ${{ steps.cache-virtiofsd.outputs.cache-hit != 'true' && steps.cache-kernel.outputs.cache-hit != 'true' }}
run: cargo install virtiofsd && sudo cp -a ~/.cargo/bin/virtiofsd /usr/lib/


- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }}
name: Clone Kernel
# Get the latest sched-ext enabled kernel directly from the korg
# for-next branch
run: git clone --single-branch -b for-next --depth 1 https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git linux

# guard rail because we are caching
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }}
run: cd linux && git checkout ${{ env.SCHED_EXT_KERNEL_COMMIT }}

- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }}
# Print the latest commit of the checked out sched-ext kernel
run: cd linux && git log -1 --pretty=format:"%h %ad %s" --date=short

- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }}
# Build a minimal kernel (with sched-ext enabled) using virtme-ng
run: cd linux && vng -v --build --config ../.github/workflows/sched-ext.config

- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }}
# Generate kernel headers
run: cd linux && make headers

integration-test:
runs-on: ubuntu-24.04
needs: build-kernel
Expand All @@ -82,9 +20,15 @@ jobs:
scheduler: [ scx_bpfland, scx_lavd, scx_layered, scx_rlfifo, scx_rustland, scx_rusty ]
fail-fast: false
steps:
- uses: actions/checkout@v4

- uses: ./.github/actions/restore-kernel-cache
with:
git-repo: "https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git"
branch: for-next

# prevent cache permission errors
- run: sudo chown root /usr/bin/tar && sudo chmod u+s /usr/bin/tar
- uses: actions/checkout@v4
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.scheduler }}
Expand All @@ -101,25 +45,6 @@ jobs:
- if: ${{ steps.cache-virtiofsd.outputs.cache-hit != 'true' }}
run: cargo install virtiofsd && sudo cp -a ~/.cargo/bin/virtiofsd /usr/lib/

# get latest head commit of sched_ext for-next
- run: echo "SCHED_EXT_KERNEL_COMMIT=$(git ls-remote https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git heads/for-next | awk '{print $1}')" >> $GITHUB_ENV

# use cached kernel if available, create after job if not
- name: Cache Kernel
id: cache-kernel
uses: actions/cache@v4
with:
path: |
linux/arch/x86/boot/bzImage
linux/usr/include
linux/**/*.h
key: kernel-build-${{ env.SCHED_EXT_KERNEL_COMMIT }}-bpf-1

# need to re-run job when kernel head changes between build and test running.
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }}
name: exit if cache stale
run: exit -1

# veristat
- run: wget https://github.com/libbpf/veristat/releases/download/v0.3.2/veristat-v0.3.2-amd64.tar.gz
- run: tar -xvf veristat-v0.3.2-amd64.tar.gz && sudo cp veristat /usr/bin/
Expand Down
Loading