Scaffold for buildkite CI. Fixes #4801 #4814
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nholland94
reviewed
Apr 30, 2020
.buildkite/src/Lib/Map.dhall
Outdated
| -- Maps are lists of special "entry" records. This is made explicit here | ||
| let Entry = { Type = \(value: Type) -> { mapKey : Text, mapValue : value } } in | ||
| { Entry = Entry, Type = \(value : Type) -> List (Entry.Type value) } | ||
|
|
There was a problem hiding this comment.
Prelude provides this functionality in it's map related exports. See Prelude.Map.Type and Prelude.Map.Entry.
| Pipeline.Config::{ | ||
| spec = ./Spec.dhall, | ||
| steps = [ | ||
| Command.Config::{ command = [ "echo \"hello\"" ], label = "Test Echo", key = "hello", target = <Large | Small>.Small } |
There was a problem hiding this comment.
You should make an alias for this size union so that you don't have to always use this syntax.
| -- Add any new jobs here | ||
| [ | ||
| ./jobs/Sample/Spec.dhall, | ||
| ./jobs/Sample2/Spec.dhall |
There was a problem hiding this comment.
Should we auto-gen this dhall file?
There was a problem hiding this comment.
Yes, but I'll add a TODO for that and we can do it later
4 tasks
|
@yourbuddyconner ✅ ? @mrmr1993 ✅ ? |
nholland94
approved these changes
Apr 30, 2020
mrmr1993
approved these changes
Apr 30, 2020
bkase
added
the
ready-to-merge
|
Preview: Built with commit 9b6ce4c |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Initial scaffolding for BuildKite CI system
Summary
For now, builds are generated manually, but it is very easy to trigger them automatically on PRs.
There is no infrastructure for running build agents yet -- so to iterate on builds you can run an agent locally. I have included an
agent.nixwith instructions on how to run the agent on your own computer.Currently, the agents do not shell out to docker containers so make sure you have installed the environment necessary to run your pipeline before running your agent.
It works
Included are two samples with separate "dirtyWhen" filters. When the branch includes changes from something in
src/libor a file containingtransitionin it the pipeline executes as follows:And if your branch does not include such changed files:
Implementation
Pipelines are defined in Dhall. To support monorepo triaging, there is one top-level pipeline that dynamically pushes individual "Job"s to buildkite when a dirtyWhen trigger occurs.
BuildKite talks to
.buildkite/pipeline.dhallwhich loads./.buildkite/src/Monorepo.dhall. Jobs are defined in./.buildkite/src/jobs/and two sample jobs are provided as a reference.Inline comments in the Dhall code can provide more background.
Note that detecting dirty files is handled through the
scripts/generate_diff.shscript.Epic: #4762