Skip to content

Commit

Permalink
Merge #6197: ci: always build guix, save artifacts
Browse files Browse the repository at this point in the history
770651a set hosts in guix-check (pasta)
580bbe6 feat: improve guix building; run always, save artifacts (pasta)
101a315 refactor: simplify caching setup, add a restore key to actually cache besides 1 run (pasta)
1b139e4 feat: automatically run guix-build on all tags pushed (pasta)

Pull request description:

  ## Issue being fixed or feature implemented
  Previously, we only ran guix on 1 machine for all hosts; this slowed it down a lot. Let's move to GitHub action runners, but run them all separately. Then upload the artifacts.

  In the future there is significant caching I can add that should help a lot more. But currently, takes about 1 hour

  ## What was done?

  ## How Has This Been Tested?
  see: https://github.com/PastaPastaPasta/dash/actions/runs/10345024600

  ## Breaking Changes
  None

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    utACK 770651a

Tree-SHA512: 639b95c3b6a26f205ed00c138a9189f915cfc36a815516035e59ceda82675414b1bd31a361b33449b5e4c58a7655f3a7d616b362c23f7fa75e72b1284be06b9e
  • Loading branch information
PastaPastaPasta committed Aug 12, 2024
2 parents bc73522 + 770651a commit 41a67dc
Showing 1 changed file with 67 additions and 31 deletions.
98 changes: 67 additions & 31 deletions .github/workflows/guix-build.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
name: Guix Build

permissions:
packages: write

on:
pull_request:
types: [ labeled ]
workflow_dispatch:
pull_request_target:
push:

jobs:
build:
runs-on: [ "self-hosted", "linux", "x64", "ubuntu-core" ]
if: contains(github.event.pull_request.labels.*.name, 'guix-build')
timeout-minutes: 480
build-image:
runs-on: ubuntu-latest
outputs:
image-tag: ${{ steps.prepare.outputs.image-tag }}
repo-name: ${{ steps.prepare.outputs.repo-name }}
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -22,37 +25,72 @@ jobs:
uses: docker/setup-buildx-action@v3

- name: Commit variables
id: dockerfile
id: prepare
run: |
echo "hash=$(sha256sum ./dash/contrib/containers/guix/Dockerfile | cut -d ' ' -f1)" >> $GITHUB_OUTPUT
echo "host_user_id=$(id -u)" >> $GITHUB_OUTPUT
echo "host_group_id=$(id -g)" >> $GITHUB_OUTPUT
BRANCH_NAME=$(echo "${GITHUB_REF##*/}" | tr '[:upper:]' '[:lower:]')
REPO_NAME=$(echo "${{ github.repository }}" | tr '[:upper:]' '[:lower:]')
echo "::set-output name=image-tag::${BRANCH_NAME}"
echo "::set-output name=repo-name::${REPO_NAME}"
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build Docker image
uses: docker/build-push-action@v5
uses: docker/build-push-action@v6
with:
context: ${{ github.workspace }}/dash
build-args: |
USER_ID=${{ steps.dockerfile.outputs.host_user_id }}
GROUP_ID=${{ steps.dockerfile.outputs.host_group_id }}
USER_ID=${{ steps.prepare.outputs.host_user_id }}
GROUP_ID=${{ steps.prepare.outputs.host_group_id }}
build-contexts: |
docker_root=${{ github.workspace }}/dash/contrib/containers/guix
file: ./dash/contrib/containers/guix/Dockerfile
load: true
tags: guix_ubuntu:latest
cache-from: type=gha
cache-to: type=gha,mode=max
push: true
tags: |
ghcr.io/${{ steps.prepare.outputs.repo-name }}/dashcore-guix-builder:${{ steps.prepare.outputs.image-tag }}
ghcr.io/${{ steps.prepare.outputs.repo-name }}/dashcore-guix-builder:latest
cache-from: type=registry,ref=ghcr.io/${{ steps.prepare.outputs.repo-name }}/dashcore-guix-builder:latest
cache-to: type=inline,mode=max

build:
needs: build-image
# runs-on: [ "self-hosted", "linux", "x64", "ubuntu-core" ]
runs-on: ubuntu-latest
# if: ${{ contains(github.event.pull_request.labels.*.name, 'guix-build') }}
strategy:
matrix:
build_target: [x86_64-linux-gnu, arm-linux-gnueabihf, aarch64-linux-gnu, riscv64-linux-gnu, x86_64-w64-mingw32, x86_64-apple-darwin, arm64-apple-darwin]

- name: Restore Guix cache and depends
timeout-minutes: 480
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
path: dash
fetch-depth: 0

- name: Cache Guix and depends
id: guix-cache-restore
uses: actions/cache/restore@v3
uses: actions/cache@v3
with:
path: |
${{ github.workspace }}/.cache
${{ github.workspace }}/dash/depends/built
${{ github.workspace }}/dash/depends/sources
${{ github.workspace }}/dash/depends/work
key: ${{ runner.os }}-guix
/gnu/store
key: ${{ runner.os }}-guix-${{ matrix.build_target }}-${{ github.sha }}
restore-keys: |
${{ runner.os }}-guix-${{ matrix.build_target }}
${{ runner.os }}-guix-
- name: Create .cache folder if missing
if: steps.guix-cache-restore.outputs.cache-hit != 'true'
Expand All @@ -67,8 +105,8 @@ jobs:
-v ${{ github.workspace }}/dash:/src/dash \
-v ${{ github.workspace }}/.cache:/home/ubuntu/.cache \
-w /src/dash \
guix_ubuntu:latest && \
docker exec guix-daemon bash -c '/usr/local/bin/guix-start'
ghcr.io/${{ needs.build-image.outputs.repo-name }}/dashcore-guix-builder:${{ needs.build-image.outputs.image-tag }} && \
docker exec guix-daemon bash -c 'HOSTS=${{ matrix.build_target }} /usr/local/bin/guix-start'
- name: Ensure build passes
run: |
Expand All @@ -77,17 +115,15 @@ jobs:
exit 1
fi
- name: Save Guix cache and depends
id: guix-cache-save
uses: actions/cache/save@v3
- name: Compute SHA256 checksums
continue-on-error: true # It will complain on depending on only some hosts
run: |
HOSTS=${{ matrix.build_target }} ./dash/contrib/containers/guix/scripts/guix-check ${{ github.workspace }}/dash
- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: guix-artifacts-${{ matrix.build_target }}
path: |
${{ github.workspace }}/.cache
${{ github.workspace }}/dash/depends/built
${{ github.workspace }}/dash/depends/sources
${{ github.workspace }}/dash/depends/work
key: ${{ steps.guix-cache-restore.outputs.cache-primary-key }}
${{ github.workspace }}/dash/guix-build*/output/${{ matrix.build_target }}/
- name: Compute SHA256 checksums
run: |
./dash/contrib/containers/guix/scripts/guix-check ${{ github.workspace }}/dash

0 comments on commit 41a67dc

Please sign in to comment.