Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ghaf nix build GitHub actions #282

Closed
wants to merge 3 commits into from
Closed

Ghaf nix build GitHub actions #282

wants to merge 3 commits into from

Conversation

henrirosten
Copy link
Collaborator

@henrirosten henrirosten commented Sep 26, 2023 •
edited
Loading

Run nix build in Github Actions:

  • Uses cross-compile targets, but also allows emulated builds.
  • Builds are run on GitHub-hosted runners.
  • Makes use of Cachix to push the build artifacts for later re-use. Using Cachix cache isn't strictly required, but might be useful for many reasons e.g.: to speed-up the PR builds, to allow faster local builds, and to offload the bursts of concurrent builds away from the main cache (vedenemo). See: https://app.cachix.org/cache/ghaf-dev.
  • All builds are run concurrently, see the build matrix configuration for more details. From the current targets, 'lenovo-x1-carbon-gen11-debug' seems to take longest. Execution time obviously depends on how much needs to be re-built, currently the best case for the whole build matrix to finish is around 40 minutes. We start-off with limited amount of targets, to limit the github action minutes used by this new workflow.
  • Some (not fully developed) ideas to improve the execution time:
    • Would it be possible to have this CI build output (subsets) that would not change unless the relevant inputs changed, meaning all build results would be read from the cache if the target had been built earlier with the same inputs. Currently, the main build results (store-disk.squashf, nixos.img) are re-build every time a new commit is pushed, even if the commit didn't introduce any changes to the inputs - reference.

For more information, see: https://ssrc.atlassian.net/wiki/spaces/SP/pages/837648757/Running+nix+build+in+github+actions.

@henrirosten
Run nix build in Github Actions
f2580bb 
Signed-off-by: Henri Rosten <henri.rosten@unikie.com>
@henrirosten
Run the build with cachix watch-exec
9c92170 
Runs the build with cachix watch-exec to start pushing nix store
paths to the cache in the background already during the build, so
other build jobs could re-use them while still building.

Signed-off-by: Henri Rosten <henri.rosten@unikie.com>
@mikatammi
Copy link
Contributor

I already want to give 👍 at this point

@henrirosten
Copy link
Collaborator Author

henrirosten commented Sep 26, 2023 •
edited
Loading

Commit 795c3ee limits the initial number of build targets. The reason is, we want to trial this new workflow and monitor how much of the tiiuae organization github-hosted runner minutes this new workflow starts to consume. If needed, and if the quota allows, we can add more targets later. At the same time, we should also look for ways to improve the overall build times. See: #282 (comment).

Intentionally not squashing the commits.

@henrirosten henrirosten marked this pull request as ready for review September 26, 2023 14:44
@henrirosten henrirosten marked this pull request as draft September 27, 2023 09:50
@henrirosten
Limit the number of initial build targets
7e063ad 
- Start-off with just two build targets.
- Rename the cachix authentication token.
- Replace cachix cache substituter with the 'official' non-trial
  version.
- Remove debug steps: printing nix.conf and timing the build
  commands.
- Remove 'workflow_dispatch' event from the workflow triggers, to
  *not* allow manually re-triggering the action.

Signed-off-by: Henri Rosten <henri.rosten@unikie.com>
@henrirosten henrirosten mentioned this pull request Oct 2, 2023
Merged
@henrirosten
Copy link
Collaborator Author

Closing this PR in favor of: #293

@henrirosten henrirosten closed this Oct 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brianmcgillion brianmcgillion brianmcgillion approved these changes

@mikatammi mikatammi Awaiting requested review from mikatammi

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@henrirosten @mikatammi @brianmcgillion