Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Remove magic strings from Source types #57

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
4cfa108
Add Source data structures
pmengelbert Dec 11, 2023
5a22b1c
Add `GetRef` and `SetRef` methods to `Source`
pmengelbert Dec 11, 2023
91022b6
Switch on embedded union variant
pmengelbert Dec 11, 2023
c804300
Implement switching logic on `Source` variants
pmengelbert Dec 11, 2023
f0036cf
Remove `KeepGitDir` field from `Source`.
pmengelbert Dec 11, 2023
a51da27
Rename fields on some Source variants
pmengelbert Dec 12, 2023
945ff2f
Implement source.processArgs
pmengelbert Dec 15, 2023
d0f6817
Add `SourceLocal` (local Source variant)
pmengelbert Dec 15, 2023
f31c8d0
Add validation functions
pmengelbert Dec 15, 2023
851331c
Allow `Cmd` subfield on docker image sources only
pmengelbert Dec 15, 2023
320f146
Remove dead code
pmengelbert Dec 15, 2023
e649cad
Remove JSON schema extension
pmengelbert Dec 15, 2023
30aa0b8
Match some tests fixtures to new spec format
pmengelbert Dec 18, 2023
5a723c7
Retool `SourceBuild`
pmengelbert Dec 18, 2023
19aefe6
Fix more yaml fixtures
pmengelbert Dec 18, 2023
b3570a2
All tests but one are passing
pmengelbert Dec 18, 2023
b33b3af
Redesign `SourceBuild` struct
pmengelbert Jan 8, 2024
7b6a875
Save progress
pmengelbert Jan 8, 2024
b84de6f
save
pmengelbert Jan 9, 2024
1612df8
Fix local source
pmengelbert Jan 9, 2024
fa813ed
Run `go generate`
pmengelbert Jan 9, 2024
74db6e1
Update comment for autogenerated docs
pmengelbert Jan 9, 2024
ac0bffb
Update patching test to reflect new spec format
pmengelbert Jan 9, 2024
1076923
Fix `frontend` test fixture
pmengelbert Jan 9, 2024
f068072
Fix runc test fixture
pmengelbert Jan 9, 2024
991772e
Fix example spec files and accompanying docs
pmengelbert Jan 9, 2024
8b795d1
Remove unused empty file
pmengelbert Jan 10, 2024
007524c
Clean up validation function
pmengelbert Jan 10, 2024
44084e4
Remove dead code
pmengelbert Jan 10, 2024
9554d2b
Remove local source type
pmengelbert Jan 10, 2024
4519995
Remove unused function
pmengelbert Jan 10, 2024
e40bf10
Run `go generate`
pmengelbert Jan 10, 2024
8380747
Use strings for DockerFile/Inline on build sources
pmengelbert Jan 10, 2024
a3ddd8a
Add error to default case
pmengelbert Jan 10, 2024
140a9a9
Allow `Build` Source type to use any Source as the context
pmengelbert Jan 10, 2024
1a864c1
Address linter errors
pmengelbert Jan 10, 2024
d76f447
Remove `Path` field from `SourceContext`
pmengelbert Jan 12, 2024
c2a8230
Make yaml specs conform to Git source type layout
pmengelbert Jan 12, 2024
b3a2642
Remove unused default cases from switch statements
pmengelbert Jan 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/examples/go-md2man-1.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# syntax=ghcr.io/azure/dalec/frontend:latest
name: go-md2man
version: 2.0.3
packager: Dalec Example
Expand All @@ -8,7 +9,9 @@ website: https://github.com/cpuguy83/go-md2man

sources:
src:
ref: https://github.com/cpuguy83/go-md2man.git#v2.0.3
git:
url: https://github.com/cpuguy83/go-md2man.git
commit: "v2.0.3"

dependencies:
build:
Expand All @@ -30,4 +33,4 @@ artifacts:

image:
entrypoint: go-md2man
cmd: --help
cmd: --help
33 changes: 19 additions & 14 deletions docs/examples/go-md2man-2.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,22 +9,27 @@ website: https://github.com/cpuguy83/go-md2man

sources:
src:
ref: https://github.com/cpuguy83/go-md2man.git#v2.0.3
git:
url: https://github.com/cpuguy83/go-md2man.git
commit: "v2.0.3"
gomods: # This is required when the build environment does not allow network access. This downloads all the go modules.
ref: docker-image://mcr.microsoft.com/oss/go/microsoft/golang:1.21
path: /build/gomodcache # This is the path we will be extracing after running the command below.
cmd:
dir: /build/src
mounts:
# Mount the "src" source, specified above, so our command has access to it.
- dest: /build/src
spec:
ref: source://src
steps:
- command: go mod download
env:
# This variable controls where the go modules are downloaded to.
GOMODCACHE: /build/gomodcache
image:
ref: mcr.microsoft.com/oss/go/microsoft/golang:1.21
cmd:
dir: /build/src
mounts:
# Mount a source (inline, under `spec`), so our command has access to it.
- dest: /build/src
spec:
git:
url: https://github.com/cpuguy83/go-md2man.git
commit: "v2.0.3"
steps:
- command: go mod download
env:
# This variable controls where the go modules are downloaded to.
GOMODCACHE: /build/gomodcache

dependencies:
build:
Expand Down
44 changes: 26 additions & 18 deletions docs/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,9 @@ website: https://github.com/cpuguy83/go-md2man

sources:
src:
ref: https://github.com/cpuguy83/go-md2man.git#v2.0.3
git:
url: https://github.com/cpuguy83/go-md2man.git
commit: "v2.0.3"

dependencies:
build:
Expand Down Expand Up @@ -166,8 +168,8 @@ image:
In the `sources` section there is a single source called `src` that references
the github repo at tag v2.0.3. The name `src` is arbitrary, however this is
where the source will be checked out to in the build phase. You can add
multiple sources, in the build phase they will be checked out to the name you
give them.
multiple sources, and in the build phase they will be checked out to the name
you give them.

One thing to note, in many build systems you will not have access to the
internet while building the package, and indeed that is the case with the
Expand All @@ -182,14 +184,15 @@ everything needed to build the package (aside from dependencies on other
packages). Source packages can be published to a package repository and then
another system can download the source package and build it.

In the case case of the above example, we need to include the go modules in the
In the case of the above example, we need to include the go modules in the
list of sources. We'll accomplish this by add a source which will run `go mod
download` in a docker image with the `src` source mounted and then extract the
go modules from the resulting filesystem.

*Note*: See the full example from [examples/go-md2man.yml](examples/go-md2man-2.yml)

```yaml
# syntax=ghcr.io/azure/dalec/frontend:latest
name: go-md2man
version: 2.0.3
packager: Dalec Example
Expand All @@ -200,22 +203,27 @@ website: https://github.com/cpuguy83/go-md2man

sources:
src:
ref: https://github.com/cpuguy83/go-md2man.git#v2.0.3
git:
url: https://github.com/cpuguy83/go-md2man.git
commit: "v2.0.3"
gomods: # This is required when the build environment does not allow network access. This downloads all the go modules.
ref: docker-image://mcr.microsoft.com/oss/go/microsoft/golang:1.21
path: /build/gomodcache # This is the path we will be extracing after running the command below.
cmd:
dir: /build/src
mounts:
# Mount the "src" source, specified above, so our command has access to it.
- dest: /build/src
spec:
ref: source://src
steps:
- command: go mod download
env:
# This variable controls where the go modules are downloaded to.
GOMODCACHE: /build/gomodcache
image:
ref: mcr.microsoft.com/oss/go/microsoft/golang:1.21
cmd:
dir: /build/src
mounts:
# Mount a source (inline, under `spec`), so our command has access to it.
- dest: /build/src
spec:
git:
url: https://github.com/cpuguy83/go-md2man.git
commit: "v2.0.3"
steps:
- command: go mod download
env:
# This variable controls where the go modules are downloaded to.
GOMODCACHE: /build/gomodcache

dependencies:
build:
Expand Down
156 changes: 104 additions & 52 deletions docs/spec.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -63,35 +63,6 @@
"type": "object",
"description": "Artifacts describes all the artifacts to include in the package."
},
"BuildSpec": {
"properties": {
"target": {
"type": "string",
"description": "Target specifies the build target to use.\nIf unset, the default target is determined by the frontend implementation (e.g. the dockerfile frontend uses the last build stage as the default)."
},
"args": {
"additionalProperties": {
"type": "string"
},
"type": "object",
"description": "Args are the build args to pass to the build."
},
"file": {
"type": "string",
"description": "File is the path to the build file in the build context\nIf not set the default is assumed by buildkit to be `Dockerfile` at the root of the context.\nThis is exclusive with [Inline]"
},
"inline": {
"type": "string",
"description": "Inline is an inline build spec to use.\nThis can be used to specify a dockerfile instead of using one in the build context\nThis is exclusive with [File]",
"examples": [
"FROM busybox\nRUN echo hello world"
]
}
},
"additionalProperties": false,
"type": "object",
"description": "BuildSpec is used to generate source from a build."
},
"BuildStep": {
"properties": {
"command": {
Expand Down Expand Up @@ -211,7 +182,7 @@
"type": "object",
"description": "CheckOutput is used to specify the exepcted output of a check, such as stdout/stderr or a file."
},
"CmdSpec": {
"Command": {
"properties": {
"dir": {
"type": "string",
Expand Down Expand Up @@ -251,7 +222,7 @@
"required": [
"steps"
],
"description": "CmdSpec is used to execute a command to generate a source from a docker image."
"description": "Command is used to execute a command to generate a source from a docker image."
},
"FileCheckOutput": {
"properties": {
Expand Down Expand Up @@ -437,17 +408,21 @@
},
"Source": {
"properties": {
"ref": {
"type": "string",
"pattern": "^((context|docker-image|git|http|https|source)://.+)|((context|build)://)$",
"description": "Ref is a unique identifier for the source.\nExample: \"docker-image://busybox:latest\", \"https://github.com/moby/buildkit.git#master\", \"context://\n\nWhen a source is specified as part of a [CmdSpec], it may also be used to reference a top-level source.",
"examples": [
"docker-image://busybox:latest",
"https://github.com/moby/buildkit.git#master",
"build://",
"context://",
"context://some/path/in/build/context"
]
"image": {
"$ref": "#/$defs/SourceDockerImage",
"description": "This is an embedded union representing all of the possible source types.\nExactly one must be non-nil, with all other cases being errors.\n\n=== Begin Source Variants ==="
},
"git": {
"$ref": "#/$defs/SourceGit"
},
"https": {
"$ref": "#/$defs/SourceHTTPS"
},
"context": {
"$ref": "#/$defs/SourceContext"
},
"build": {
"$ref": "#/$defs/SourceBuild"
},
"path": {
"type": "string",
Expand All @@ -466,26 +441,103 @@
},
"type": "array",
"description": "Excludes is a list of paths underneath `Path` to exclude, everything else is included"
}
},
"additionalProperties": false,
"type": "object",
"description": "Source defines a source to be used in the build."
},
"SourceBuild": {
"properties": {
"source": {
"$ref": "#/$defs/Source",
"description": "A source specification to use as the context for the Dockerfile build"
},
"keep_git_dir": {
"type": "boolean",
"description": "KeepGitDir is used to keep the .git directory after fetching the source for git references."
"dockerfile": {
"type": "string",
"description": "DockerFile is the path to the build file in the build context\nIf not set the default is assumed by buildkit to be `Dockerfile` at the root of the context.\nThis is exclusive with [Inline]"
},
"cmd": {
"$ref": "#/$defs/CmdSpec",
"description": "Cmd is used to generate the source from a command.\nThis can be used when Ref is \"docker-image://\"\nIf ref is \"cmd://\", this is required."
"inline": {
"type": "string",
"description": "Inline is an inline build spec to use.\nThis can be used to specify a dockerfile instead of using one in the build context\nThis is exclusive with [File]",
"examples": [
"FROM busybox\nRUN echo hello world"
]
},
"build": {
"$ref": "#/$defs/BuildSpec",
"description": "Build is used to generate source from a build.\nThis is used when [Ref]` is \"build://\"\nThe context for the build is assumed too be specified in after `build://` in the ref, e.g. `build://https://github.com/moby/buildkit.git#master`\nWhen nothing is specified after `build://`, the context is assumed to be the current build context."
"target": {
"type": "string",
"description": "Target specifies the build target to use.\nIf unset, the default target is determined by the frontend implementation\n(e.g. the dockerfile frontend uses the last build stage as the default)."
},
"args": {
"additionalProperties": {
"type": "string"
},
"type": "object",
"description": "Args are the build args to pass to the build."
}
},
"additionalProperties": false,
"type": "object",
"description": "SourceBuild is used to generate source from a DockerFile build, either inline or from a local file."
},
"SourceContext": {
"properties": {
"name": {
"type": "string",
"description": "Name is the name of the build context. By default, it is the magic name\n`context`, recognized by Docker as the default context."
}
},
"additionalProperties": false,
"type": "object",
"description": "SourceContext is used to generate a source from a build context."
},
"SourceDockerImage": {
"properties": {
"ref": {
"type": "string"
},
"cmd": {
"$ref": "#/$defs/Command"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"ref"
]
},
"SourceGit": {
"properties": {
"url": {
"type": "string"
},
"commit": {
"type": "string"
},
"keepGitDir": {
"type": "boolean"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"url",
"commit",
"keepGitDir"
]
},
"SourceHTTPS": {
"properties": {
"url": {
"type": "string"
}
},
"additionalProperties": false,
"type": "object",
"required": [
"url"
],
"description": "Source defines a source to be used in the build."
"description": "No longer supports `.git` URLs as git repos."
},
"SourceMount": {
"properties": {
Expand Down
1 change: 0 additions & 1 deletion docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,6 @@ tests:
-
dest: /target/mount/path
spec:
ref: build://
build:
inline: |
FROM busybox
Expand Down
Loading
Loading