Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add workflow version #2476

Merged
merged 31 commits into from
Nov 6, 2023
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
a98a808
WIP: add version to workflow configs
6543 Jun 6, 2023
401851a
Merge remote-tracking branch 'upstream/main' into pipeline-config_ver…
qwerty287 Sep 19, 2023
b165813
Merge branch 'main' into pipeline-config_version
qwerty287 Sep 19, 2023
84f72ad
Update pipeline/frontend/yaml/error.go
qwerty287 Sep 19, 2023
01ff70b
fix tests
qwerty287 Sep 19, 2023
8e9379c
format
qwerty287 Sep 19, 2023
fbce04f
update schema
qwerty287 Sep 19, 2023
7e2bfde
update example yamls
qwerty287 Sep 20, 2023
2608aea
Merge branch 'main' into pipeline-config_version
qwerty287 Sep 20, 2023
4469518
Merge branch 'main' into pipeline-config_version
qwerty287 Sep 23, 2023
3179dfc
Merge branch 'main' into pipeline-config_version
qwerty287 Oct 4, 2023
e784ac5
Merge branch 'main' into pipeline-config_version
qwerty287 Oct 9, 2023
cd5449a
Make version optional (default 1)
qwerty287 Oct 9, 2023
a923fc5
Update pipeline/schema/schema.json
qwerty287 Oct 9, 2023
5b7bee7
Merge branch 'main' into pipeline-config_version
qwerty287 Oct 13, 2023
93819c1
Merge branch 'main' into pipeline-config_version
qwerty287 Oct 15, 2023
7125107
Merge branch 'main' into pipeline-config_version
qwerty287 Oct 23, 2023
917e88a
Merge branch 'main' into pipeline-config_version
qwerty287 Oct 25, 2023
53d90b4
Merge branch 'main' into pipeline-config_version
qwerty287 Oct 27, 2023
8780114
Merge branch 'main' into pipeline-config_version
qwerty287 Nov 1, 2023
6891032
Merge branch 'main' into pipeline-config_version
qwerty287 Nov 2, 2023
e212b8b
Merge branch 'main' into pipeline-config_version
6543 Nov 2, 2023
1ac5c0e
move default version to constant
qwerty287 Nov 2, 2023
a5d46b6
Merge branch 'main' into pipeline-config_version
qwerty287 Nov 2, 2023
caef4a9
Update parse.go
qwerty287 Nov 2, 2023
aad400f
fix year
qwerty287 Nov 2, 2023
532b052
Merge branch 'main' into pipeline-config_version
qwerty287 Nov 3, 2023
245c9ea
Merge branch 'main' into pipeline-config_version
qwerty287 Nov 4, 2023
57cb25d
fix missing import
qwerty287 Nov 4, 2023
adc5aad
Merge branch 'main' into pipeline-config_version
qwerty287 Nov 4, 2023
c7b0947
Merge branch 'main' into pipeline-config_version
6543 Nov 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .woodpecker/binaries.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

depends_on:
- test
- web
Expand Down
2 changes: 2 additions & 0 deletions .woodpecker/docker.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

when:
- event: [pull_request, tag]
- event: push
Expand Down
2 changes: 2 additions & 0 deletions .woodpecker/docs.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

qwerty287 marked this conversation as resolved.
Show resolved Hide resolved
when:
- event: pull_request
- event: push
Expand Down
2 changes: 2 additions & 0 deletions .woodpecker/release-helper.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

steps:
release-helper:
image: woodpeckerci/plugin-ready-release-go:0.6.1
Expand Down
2 changes: 2 additions & 0 deletions .woodpecker/securityscan.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

when:
- event: [pull_request, cron]
- event: push
Expand Down
2 changes: 2 additions & 0 deletions .woodpecker/test.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

when:
- event: [pull_request, tag]
- event: push
Expand Down
2 changes: 2 additions & 0 deletions .woodpecker/web.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

when:
- event: [pull_request, tag]
- event: push
Expand Down
1 change: 1 addition & 0 deletions contrib/woodpecker-test-repo/.woodpecker/demo.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
version: 1
steps:
demo:
image: 'alpine'
Expand Down
1 change: 1 addition & 0 deletions contrib/woodpecker-test-repo/.woodpecker/test.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
version: 1
steps:
test_1:
image: 'alpine'
Expand Down
2 changes: 2 additions & 0 deletions pipeline/frontend/yaml/error.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ package yaml

import "errors"

var ErrUnsuportedVersion = errors.New("unsuported pipeline config version detected")

// PipelineParseError is an error that occurs when the pipeline parsing fails.
type PipelineParseError struct {
Err error
Expand Down
5 changes: 4 additions & 1 deletion pipeline/frontend/yaml/linter/linter_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import (
func TestLint(t *testing.T) {
testdatas := []struct{ Title, Data string }{{
Title: "map", Data: `
version: 1
steps:
build:
image: docker
Expand All @@ -44,6 +45,7 @@ services:
`,
}, {
Title: "list", Data: `
version: 1
steps:
- name: build
image: docker
Expand All @@ -62,6 +64,7 @@ steps:
`,
}, {
Title: "merge maps", Data: `
version: 1
variables:
step_template: &base-step
image: golang:1.19
Expand Down Expand Up @@ -151,7 +154,7 @@ func TestLintErrors(t *testing.T) {
}

for _, test := range testdata {
conf, err := yaml.ParseString(test.from)
conf, err := yaml.ParseString("version: 1\n" + test.from)
if err != nil {
t.Fatalf("Cannot unmarshal yaml %q. Error: %s", test.from, err)
}
Expand Down
28 changes: 26 additions & 2 deletions pipeline/frontend/yaml/parse.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,20 @@ import (

// ParseBytes parses the configuration from bytes b.
func ParseBytes(b []byte) (*types.Workflow, error) {
yamlVersion, err := checkVersion(b)
if err != nil {
return nil, &PipelineParseError{Err: err}
}

out := new(types.Workflow)
err := xyaml.Unmarshal(b, out)
err = xyaml.Unmarshal(b, out)
if err != nil {
return nil, err
return nil, &PipelineParseError{Err: err}
}

// make sure detected version is set
out.Version = yamlVersion

return out, nil
}

Expand All @@ -37,3 +45,19 @@ func ParseString(s string) (*types.Workflow, error) {
[]byte(s),
)
}

func checkVersion(b []byte) (int, error) {
ver := struct {
Version int `yaml:"version"`
qwerty287 marked this conversation as resolved.
Show resolved Hide resolved
}{}
_ = xyaml.Unmarshal(b, &ver)
if ver.Version == 0 {
// default: version 1
return 1, nil
qwerty287 marked this conversation as resolved.
Show resolved Hide resolved
}

if ver.Version != Version {
return 0, ErrUnsuportedVersion
}
return ver.Version, nil
}
19 changes: 19 additions & 0 deletions pipeline/frontend/yaml/parse_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,16 @@ func TestParse(t *testing.T) {
g.Assert(out.Steps.ContainerList[1].When.Constraints[0].Event.Include).Equal([]string{"success"})
})

g.It("Should unmarshal with default version", func() {
out, err := ParseString(sampleYamlDefaultVersion)
if err != nil {
g.Fail(err)
}
g.Assert(len(out.Steps.ContainerList)).Equal(1)
g.Assert(out.Steps.ContainerList[0].Name).Equal("notify_success")
g.Assert(out.Steps.ContainerList[0].Image).Equal("xyz")
})

matchConfig, err := ParseString(sampleYaml)
if err != nil {
g.Fail(err)
Expand Down Expand Up @@ -138,6 +148,7 @@ func TestParse(t *testing.T) {
}

var sampleYaml = `
version: 1
image: hello-world
when:
- event:
Expand Down Expand Up @@ -189,7 +200,14 @@ runs_on:
- failure
`

var sampleYamlDefaultVersion = `
steps:
- name: notify_success
image: xyz
`

var simpleYamlAnchors = `
version: 1
vars:
image: &image plugins/slack
steps:
Expand All @@ -198,6 +216,7 @@ steps:
`

var sampleVarYaml = `
version: 1
_slack: &SLACK
image: plugins/slack
steps:
Expand Down
1 change: 1 addition & 0 deletions pipeline/frontend/yaml/types/workflow.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ type (
DependsOn []string `yaml:"depends_on,omitempty"`
RunsOn []string `yaml:"runs_on,omitempty"`
SkipClone bool `yaml:"skip_clone"`
Version int `yaml:"version"`

// Undocumented
Cache base.StringOrSlice `yaml:"cache,omitempty"`
Expand Down
18 changes: 18 additions & 0 deletions pipeline/frontend/yaml/version.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
// Copyright 2022 Woodpecker Authors
qwerty287 marked this conversation as resolved.
Show resolved Hide resolved
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package yaml

// Version of this package and it's subpackages
const Version = 1
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-branches-array.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

branches: [main, pages]

steps:
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-branches-exclude-include.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

branches:
include: main
exclude: [develop, feature/*]
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-branches.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

branches: main

steps:
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-broken.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

branches: main

matri:
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-clone-skip.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

steps:
test:
image: alpine
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-clone.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

clone:
git:
image: plugins/git:next
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-labels.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

labels:
location: europe
weather: sun
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-matrix.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

steps:
test:
image: golang:${GO_VERSION}
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-merge-map-and-sequence.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

variables:
step_template: &base-step
image: golang:1.19
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-multi.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

steps:
deploy:
image: golang
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-pipeline-when.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

when:
- branch: [main, deploy]
event: push
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-platform.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

platform: linux/amd64

steps:
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-plugin.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

steps:
build:
image: golang
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-run-on.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

steps:
build:
image: golang
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-service.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

steps:
build:
image: golang
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-step.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

steps:
image:
image: golang
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-when.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

steps:
when-branch:
image: alpine
Expand Down
2 changes: 2 additions & 0 deletions pipeline/schema/.woodpecker/test-workspace.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
version: 1

workspace:
base: /go
path: src/github.com/octocat/hello-world
Expand Down
5 changes: 3 additions & 2 deletions pipeline/schema/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"branches": { "$ref": "#/definitions/branches" },
"when": { "$ref": "#/definitions/pipeline_when" },
"steps": { "$ref": "#/definitions/step_list" },
"pipeline": { "$ref": "#/definitions/step_list", "description": "depricated, use steps" },
"pipeline": { "$ref": "#/definitions/step_list", "description": "deprecated, use steps" },
"services": { "$ref": "#/definitions/services" },
"workspace": { "$ref": "#/definitions/workspace" },
"matrix": { "$ref": "#/definitions/matrix" },
Expand All @@ -34,7 +34,8 @@
"type": "array",
"minLength": 1,
"items": { "type": "string" }
}
},
"version": { "type": "number", "default": 1 }
},
"definitions": {
"clone": {
Expand Down
Loading