This guide explains how to convert GitHub Actions workflow files from YAML to CUE, check those workflows are valid, and then use CUE's tooling layer to regenerate YAML.
This allows you to switch to CUE as a source of truth for GitHub Actions workflows and perform client-side validation, without GitHub needing to know you're managing your workflows with CUE.
| ❗ WARNING ❗ |
|---|
This guide requires that you use cue version v0.11.0-alpha.2 or later. The process described below won't work with earlier versions. Check the version of your cue command by running cue version, and upgrade it if needed. |
- You have a set of GitHub Actions workflow files.
- The examples shown in this guide use the state of the first commit of CUE's github-actions-example repository, but you don't need to use that repository in any way.
- You have
CUE installed
locally -- this allows you to run
cuecommands- You must have version
v0.11.0-alpha.2or later installed.
- You must have version
- You have a GitHub account -- this allows you to use the CUE Central Registry.
- You have a Central Registry account -- this allows you to fetch a schema to validate your GitHub Actions workflows.
- You have
gitinstalled.
Change directory into the root of the repository that contains your GitHub Actions workflow files, and ensure you start this process with a clean git state, with no modified files. For example:
💻 terminal
cd github-actions-example # our example repository
git status # should report "working tree clean"Initialise a CUE module named after the organisation and repository you're working with. For example:
💻 terminal
cue mod init github.com/cue-examples/github-actions-exampleUse cue to import your YAML workflow files:
💻 terminal
cue import ./.github/workflows/ --with-context -p github -f -l workflows: -l 'strings.TrimSuffix(path.Base(filename),path.Ext(filename))'Check that a CUE file has been created for each YAML workflow in the
.github/workflows directory. For example:
💻 terminal
ls .github/workflows/Your output should look similar to this, with matching pairs of YAML and CUE files:
workflow1.cue
workflow1.yml
workflow2.cue
workflow2.yml
Observe that each workflow has been imported into the workflows struct, at a
location derived from its original file name:
💻 terminal
head -5 .github/workflows/*.cueThe output should reflect your workflows. In our example:
==> .github/workflows/workflow1.cue <==
package github
workflows: workflow1: {
on: [
"push",
==> .github/workflows/workflow2.cue <==
package github
workflows: workflow2: {
on: [
"push",
Create a directory called github to hold your CUE-based GitHub Actions
workflow files. For example:
💻 terminal
mkdir -p internal/ci/githubYou may change the hierarchy and naming of github's parent directories to
suit your repository layout. If you do so, you will need to adapt some commands
and CUE code as you follow this guide.
Move the newly-created CUE files into their dedicated directory. For example:
💻 terminal
mv ./.github/workflows/*.cue internal/ci/githubRun this command, and follow the instructions it displays:
💻 terminal
cue loginThis will allow you to fetch modules from the Central Registry.
💻 terminal
cue mod get github.com/cue-tmp/jsonschema-pub/exp1/githubactions@v0.3.0This command specifies a precise version of the GitHub Actions module in order to make sure that this process is reproducible.
The GitHub Actions module looks like it has a temporary location because it was created as part of the CUE project's work to figure out how and where to store third-party schemas. Whilst it will eventually live at a more permanent and appropriate location (which will then be reflected in this guide), this version of the module won't disappear from its "temporary" location - so it's safe to use!
We need to tell CUE to apply the schema to each workflow.
To do this we'll create a file at internal/ci/github/workflows.cue in our
example.
However, if the workflow imports that you performed earlier already created a
file with that same path and name, then simply select a different CUE filename
that doesn't already exist. Place the file in the internal/ci/github/
directory.
💾 internal/ci/github/workflows.cue
package github
import "github.com/cue-tmp/jsonschema-pub/exp1/githubactions"
// Each member of the workflows struct must be a valid #Workflow.
workflows: [_]: githubactions.#Workflow💻 terminal
cue vet ./internal/ci/githubIf this command fails and produces any output, then CUE believes that at least one of your workflows isn't valid. It's very likely that CUE is correct (and has found a problem) even if GitHub Actions manages to process your workflow files successfully -- because GitHub Actions can be lax and overly permissive in enforcing its own schema rules! You'll need to resolve this before continuing, by updating your workflows inside your new CUE files. If you're having difficulty fixing them, please come and ask for help in the friendly CUE Slack workspace or Discord server!
Create a CUE file at internal/ci/github/ci_tool.cue, containing the following workflow command.
Adapt the element commented with TODO:
💾 internal/ci/github/ci_tool.cue
package github
import (
"path"
"encoding/yaml"
"tool/file"
)
_goos: string @tag(os,var=os)
// Regenerate all workflow files
command: regenerate: {
workflow_files: {
// TODO: update _toolFile to reflect the directory hierarchy containing this file.
let _toolFile = "internal/ci/github/ci_tool.cue"
let _workflowDir = path.FromSlash(".github/workflows", path.Unix)
let _donotedit = "Code generated by \(_toolFile); DO NOT EDIT."
clean: {
glob: file.Glob & {
glob: path.Join([_workflowDir, "*.yml"], _goos)
files: [...string]
}
for _, _filename in glob.files {
"Delete \(_filename)": file.RemoveAll & {path: _filename}
}
}
create: {
for _workflowName, _workflow in workflows
let _filename = _workflowName + ".yml" {
"Generate \(_filename)": file.Create & {
$after: [for v in clean {v}]
filename: path.Join([_workflowDir, _filename], _goos)
contents: "# \(_donotedit)\n\n\(yaml.Marshal(_workflow))"
}
}
}
}
}Make the modification indicated by the TODO comment.
This workflow command will export each CUE-based workflow back into its required YAML file, on demand.
With the modified ci_tool.cue file in place, check that the regenerate
workflow command is available from a shell sitting at the root of the
repository. For example:
💻 terminal
cd $(git rev-parse --show-toplevel) # make sure we're sitting at the repository root
cue help cmd regenerate ./internal/ci/github # the "./" prefix is requiredYour output must begin with the following:
Regenerate all workflow files
Usage:
cue cmd regenerate [flags]
| ❗ WARNING ❗ |
|---|
If you don't see the usage explanation for the regenerate workflow command (or if you receive an error message) then either your workflow command isn't set up as CUE requires, or you're running a CUE version older than v0.11.0-alpha.2. If you've upgraded to at least that version but the usage explanation still isn't being displayed then: (1) double check the contents of the ci_tool.cue file and the modifications you made to it; (2) make sure its location in the repository is precisely as given in this guide; (3) ensure the filename is exactly ci_tool.cue; (4) check that the internal/ci/github/workflows.cue file has the same contents as shown above; (5) run cue vet ./internal/ci/github and check that your workflows actually validate successfully - in other words: were they truly valid before you even started this process? Lastly, make sure you've followed all the steps in this guide, and that you invoked the cue help command from the repository's root directory. If you get really stuck, please come and join the CUE community and ask for some help! |
Run the regenerate workflow command to produce YAML workflow files from CUE. For
example:
💻 terminal
cue cmd regenerate ./internal/ci/github # the "./" prefix is requiredCheck that each YAML workflow file has a single change from the original:
💻 terminal
git diff .github/workflows/Your output should look similar to the following example:
diff --git a/.github/workflows/workflow1.yml b/.github/workflows/workflow1.yml
--- a/.github/workflows/workflow1.yml
+++ b/.github/workflows/workflow1.yml
@@ -1,3 +1,5 @@
+# Code generated by internal/ci/github/ci_tool.cue; DO NOT EDIT.
+
"on":
- push
- pull_request
diff --git a/.github/workflows/workflow2.yml b/.github/workflows/workflow2.yml
--- a/.github/workflows/workflow2.yml
+++ b/.github/workflows/workflow2.yml
@@ -1,3 +1,5 @@
+# Code generated by internal/ci/github/ci_tool.cue; DO NOT EDIT.
+
"on":
- push
- pull_requestThe only change in each YAML file is the addition of a header that warns the reader not to edit the file directly.
Add your files to git. For example:
💻 terminal
git add .github/workflows/ internal/ci/github/ cue.mod/module.cueMake sure to include your slightly modified YAML workflow files in
.github/workflows/ along with all the new files in internal/ci/github/ and
your cue.mod/module.cue file.
Commit your files to git, with an appropriate commit message:
💻 terminal
git commit -m "ci: create CUE sources for GHA workflows"Well done - your GitHub Actions workflow files have been imported into CUE!
They can now be managed using CUE, leading to safer and more predictable changes. The use of a schema to check your workflows means that you will catch and fix many types of mistake earlier than before, without waiting for the slow "git add/commit/push; check if CI fails" cycle.
From now on, each time you make a change to a CUE workflow file, immediately regenerate the YAML files required by GitHub Actions, and commit your changes to all the CUE and YAML files. For example:
💻 terminal
cue cmd regenerate ./internal/ci/github/ # the "./" prefix is required
git add .github/workflows/ internal/ci/github/
git commit -m "ci: added new release workflow" # example message