Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CSE-13] extended error handling for workflows #16

Merged
merged 23 commits into from
Mar 15, 2019
Merged

[CSE-13] extended error handling for workflows #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
de94f1d
wip
decarboxy Mar 11, 2019
0d483a1
mixed case imports cause all sorts of problems, switch to lowercase
decarboxy Mar 11, 2019
62bfd2e
fixing build issu
decarboxy Mar 11, 2019
2a4c83c
fixing error deserialization
decarboxy Mar 11, 2019
3ea6e7b
fixing error deserialization
decarboxy Mar 11, 2019
f162e32
unmatched string logic
decarboxy Mar 11, 2019
28da250
make workflows fail on error trigger
decarboxy Mar 11, 2019
3657786
properly evaluate workflow failures
decarboxy Mar 11, 2019
b4b17c4
dev version bump
decarboxy Mar 11, 2019
b361ea0
serialize errors and warnings into wf crd
decarboxy Mar 12, 2019
28d97a8
ugh go types
decarboxy Mar 12, 2019
193ffcf
rewriting error handling to support file sources
decarboxy Mar 13, 2019
059df92
temporarily commenting out a test
decarboxy Mar 13, 2019
c23f096
fixing warning handler
decarboxy Mar 13, 2019
bf6763b
fixing error handling
decarboxy Mar 13, 2019
39dca2f
cleaning up types
decarboxy Mar 14, 2019
41e7b10
updating codegen
decarboxy Mar 14, 2019
799b533
fixing version
decarboxy Mar 14, 2019
1b46346
updating codegen
decarboxy Mar 14, 2019
4cd1489
add podname and stage name to error result
decarboxy Mar 14, 2019
7b4c254
version 2.5.0->2.4.0
decarboxy Mar 14, 2019
0ce2ccd
ErrorCondition->ExceptionCondition
decarboxy Mar 15, 2019
1878655
codegen
decarboxy Mar 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .argo-ci/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ spec:
- name: revision
value: master
- name: repo
value: https://github.com/CyrusBiotechnology/argo.git
value: https://github.com/cyrusbiotechnology/argo.git
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you hit a bug related to casing?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. go's codegen tool insists on generating code with lowercase imports, and if you have a mix of lowercase and mixed case imports in your source base, your code wont compile with an error about import case collisions. it's "very cool"


templates:
- name: argo-ci
Expand Down Expand Up @@ -39,31 +39,31 @@ spec:
- name: cmd
artifacts:
- name: code
path: /go/src/github.com/CyrusBiotechnology/argo
path: /go/src/github.com/cyrusbiotechnology/argo
git:
repo: "{{workflow.parameters.repo}}"
revision: "{{workflow.parameters.revision}}"
container:
image: argoproj/argo-ci-builder:latest
command: [sh, -c]
args: ["{{inputs.parameters.cmd}}"]
workingDir: /go/src/github.com/CyrusBiotechnology/argo
workingDir: /go/src/github.com/cyrusbiotechnology/argo

- name: ci-dind
inputs:
parameters:
- name: cmd
artifacts:
- name: code
path: /go/src/github.com/CyrusBiotechnology/argo
path: /go/src/github.com/cyrusbiotechnology/argo
git:
repo: "{{workflow.parameters.repo}}"
revision: "{{workflow.parameters.revision}}"
container:
image: argoproj/argo-ci-builder:latest
command: [sh, -c]
args: ["until docker ps; do sleep 3; done && {{inputs.parameters.cmd}}"]
workingDir: /go/src/github.com/CyrusBiotechnology/argo
workingDir: /go/src/github.com/cyrusbiotechnology/argo
env:
- name: DOCKER_HOST
value: 127.0.0.1
Expand Down
2 changes: 1 addition & 1 deletion Branding Assets.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Argo Branding Assets
## Logo
![Argo Logo](https://github.com/CyrusBiotechnology/argo/blob/master/argo-logo600.png "Argo Logo")
![Argo Logo](https://github.com/cyrusbiotechnology/argo/blob/master/argo-logo600.png "Argo Logo")
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ Go to https://groups.google.com/forum/#!forum/argoproj

### Quickstart
```
$ go get github.com/CyrusBiotechnology/argo
$ cd $(go env GOPATH)/src/github.com/CyrusBiotechnology/argo
$ go get github.com/cyrusbiotechnology/argo
$ cd $(go env GOPATH)/src/github.com/cyrusbiotechnology/argo
$ dep ensure -vendor-only
$ make
```
Expand Down
8 changes: 4 additions & 4 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ def runUtilityCommand(buildCommand) {
// Run an arbitrary command inside the docker builder image
sh "docker run --rm " +
"-v ${pwd()}/dist/pkg:/root/go/pkg " +
"-v ${pwd()}:/root/go/src/github.com/CyrusBiotechnology/argo " +
"-w /root/go/src/github.com/CyrusBiotechnology/argo argo-builder ${buildCommand}"
"-v ${pwd()}:/root/go/src/github.com/cyrusbiotechnology/argo " +
"-w /root/go/src/github.com/cyrusbiotechnology/argo argo-builder ${buildCommand}"
}

pipeline {
Expand Down Expand Up @@ -85,8 +85,8 @@ pipeline {
stage('push CLI to artifactory') {
steps {
withCredentials([usernamePassword(credentialsId: 'Artifactory', usernameVariable: 'ARTI_NAME', passwordVariable: 'ARTI_PASS')]) {
runUtilityCommand("curl -u ${ARTI_NAME}:${ARTI_PASS} -T /root/go/src/github.com/CyrusBiotechnology/argo/dist/argo-darwin-amd64 https://cyrusbio.jfrog.io/cyrusbio/argo-cli/argo-mac-${VERSION}")
runUtilityCommand("curl -u ${ARTI_NAME}:${ARTI_PASS} -T /root/go/src/github.com/CyrusBiotechnology/argo/dist/argo-linux-amd64 https://cyrusbio.jfrog.io/cyrusbio/argo-cli/argo-linux-${VERSION}")
runUtilityCommand("curl -u ${ARTI_NAME}:${ARTI_PASS} -T /root/go/src/github.com/cyrusbiotechnology/argo/dist/argo-darwin-amd64 https://cyrusbio.jfrog.io/cyrusbio/argo-cli/argo-mac-${VERSION}")
runUtilityCommand("curl -u ${ARTI_NAME}:${ARTI_PASS} -T /root/go/src/github.com/cyrusbiotechnology/argo/dist/argo-linux-amd64 https://cyrusbio.jfrog.io/cyrusbio/argo-cli/argo-linux-${VERSION}")
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
PACKAGE=github.com/CyrusBiotechnology/argo
PACKAGE=github.com/cyrusbiotechnology/argo
CURRENT_DIR=$(shell pwd)
DIST_DIR=${CURRENT_DIR}/dist
ARGO_CLI_NAME=argo
Expand Down
6 changes: 3 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ Argo is an open source container-native workflow engine for getting work done on
* Argo with Kubernetes puts a cloud-scale supercomputer at your fingertips.

## Documentation
* [Get started here](https://github.com/CyrusBiotechnology/argo/blob/master/demo.md)
* [How to write Argo workflow specs](https://github.com/CyrusBiotechnology/argo/blob/master/examples/README.md)
* [How to configure your artifact repository](https://github.com/CyrusBiotechnology/argo/blob/master/ARTIFACT_REPO.md)
* [Get started here](https://github.com/cyrusbiotechnology/argo/blob/master/demo.md)
* [How to write Argo workflow specs](https://github.com/cyrusbiotechnology/argo/blob/master/examples/README.md)
* [How to configure your artifact repository](https://github.com/cyrusbiotechnology/argo/blob/master/ARTIFACT_REPO.md)

## Features
* DAG or Steps based declaration of workflows
Expand Down
2 changes: 1 addition & 1 deletion VERSION
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.3.0
2.4.0
60 changes: 59 additions & 1 deletion api/openapi-spec/swagger.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"swagger": "2.0",
"info": {
"title": "Argo",
"version": "v2.2.1"
"version": "v2.4.0"
},
"paths": {},
"definitions": {
Expand Down Expand Up @@ -229,6 +229,52 @@
}
}
},
"io.argoproj.workflow.v1alpha1.ExceptionCondition": {
"description": "ExceptionCondition is a container for defining an error or warning rule",
"required": [
"name"
],
"properties": {
"message": {
"type": "string"
},
"name": {
"type": "string"
},
"patternMatched": {
"type": "string"
},
"patternUnmatched": {
"type": "string"
},
"source": {
"type": "string"
}
}
},
"io.argoproj.workflow.v1alpha1.ExceptionResult": {
"description": "ExceptionResult contains the results on an extended error or warning condition evaluation",
"required": [
"name",
"message",
"podName",
"stepName"
],
"properties": {
"message": {
"type": "string"
},
"name": {
"type": "string"
},
"podName": {
"type": "string"
},
"stepName": {
"type": "string"
}
}
},
"io.argoproj.workflow.v1alpha1.GCSArtifact": {
"description": "GCSArtifact is the location of a GCS artifact",
"required": [
Expand Down Expand Up @@ -821,6 +867,12 @@
"description": "DAG template subtype which runs a DAG",
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.DAGTemplate"
},
"errors": {
"type": "array",
"items": {
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ExceptionCondition"
}
},
"inputs": {
"description": "Inputs describe what inputs parameters and artifacts are supplied to this template",
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.Inputs"
Expand Down Expand Up @@ -888,6 +940,12 @@
"items": {
"$ref": "#/definitions/io.k8s.api.core.v1.Toleration"
}
},
"warnings": {
"type": "array",
"items": {
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.ExceptionCondition"
}
}
}
},
Expand Down
6 changes: 3 additions & 3 deletions cmd/argo/commands/common.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ import (
"strconv"
"strings"

wfv1 "github.com/CyrusBiotechnology/argo/pkg/apis/workflow/v1alpha1"
wfclientset "github.com/CyrusBiotechnology/argo/pkg/client/clientset/versioned"
"github.com/CyrusBiotechnology/argo/pkg/client/clientset/versioned/typed/workflow/v1alpha1"
wfv1 "github.com/cyrusbiotechnology/argo/pkg/apis/workflow/v1alpha1"
wfclientset "github.com/cyrusbiotechnology/argo/pkg/client/clientset/versioned"
"github.com/cyrusbiotechnology/argo/pkg/client/clientset/versioned/typed/workflow/v1alpha1"
"github.com/spf13/cobra"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/commands/delete.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
"github.com/spf13/cobra"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

"github.com/CyrusBiotechnology/argo/workflow/common"
"github.com/cyrusbiotechnology/argo/workflow/common"
)

var (
Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/commands/get.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import (
"github.com/spf13/cobra"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

wfv1 "github.com/CyrusBiotechnology/argo/pkg/apis/workflow/v1alpha1"
wfv1 "github.com/cyrusbiotechnology/argo/pkg/apis/workflow/v1alpha1"
)

const onExitSuffix = "onExit"
Expand Down
4 changes: 2 additions & 2 deletions cmd/argo/commands/lint.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ import (
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"

cmdutil "github.com/CyrusBiotechnology/argo/util/cmd"
"github.com/CyrusBiotechnology/argo/workflow/validate"
cmdutil "github.com/cyrusbiotechnology/argo/util/cmd"
"github.com/cyrusbiotechnology/argo/workflow/validate"
)

func NewLintCommand() *cobra.Command {
Expand Down
8 changes: 4 additions & 4 deletions cmd/argo/commands/list.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ import (
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/selection"

wfv1 "github.com/CyrusBiotechnology/argo/pkg/apis/workflow/v1alpha1"
"github.com/CyrusBiotechnology/argo/pkg/client/clientset/versioned/typed/workflow/v1alpha1"
"github.com/CyrusBiotechnology/argo/workflow/common"
"github.com/CyrusBiotechnology/argo/workflow/util"
wfv1 "github.com/cyrusbiotechnology/argo/pkg/apis/workflow/v1alpha1"
"github.com/cyrusbiotechnology/argo/pkg/client/clientset/versioned/typed/workflow/v1alpha1"
"github.com/cyrusbiotechnology/argo/workflow/common"
"github.com/cyrusbiotechnology/argo/workflow/util"
)

type listFlags struct {
Expand Down
6 changes: 3 additions & 3 deletions cmd/argo/commands/logs.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ import (
"sync"
"time"

"github.com/CyrusBiotechnology/argo/pkg/apis/workflow/v1alpha1"
wfclientset "github.com/CyrusBiotechnology/argo/pkg/client/clientset/versioned"
wfinformers "github.com/CyrusBiotechnology/argo/pkg/client/informers/externalversions"
"github.com/argoproj/pkg/errors"
"github.com/cyrusbiotechnology/argo/pkg/apis/workflow/v1alpha1"
wfclientset "github.com/cyrusbiotechnology/argo/pkg/client/clientset/versioned"
wfinformers "github.com/cyrusbiotechnology/argo/pkg/client/informers/externalversions"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"k8s.io/api/core/v1"
Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/commands/resubmit.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ package commands
import (
"os"

"github.com/CyrusBiotechnology/argo/workflow/util"
"github.com/argoproj/pkg/errors"
"github.com/cyrusbiotechnology/argo/workflow/util"
"github.com/spf13/cobra"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/commands/resume.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"log"
"os"

"github.com/CyrusBiotechnology/argo/workflow/util"
"github.com/cyrusbiotechnology/argo/workflow/util"
"github.com/spf13/cobra"
)

Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/commands/retry.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"github.com/spf13/cobra"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

"github.com/CyrusBiotechnology/argo/workflow/util"
"github.com/cyrusbiotechnology/argo/workflow/util"
)

func NewRetryCommand() *cobra.Command {
Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/commands/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package commands
import (
"os"

"github.com/CyrusBiotechnology/argo/util/cmd"
"github.com/cyrusbiotechnology/argo/util/cmd"
"github.com/spf13/cobra"
"k8s.io/client-go/tools/clientcmd"
)
Expand Down
8 changes: 4 additions & 4 deletions cmd/argo/commands/submit.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ import (
"github.com/argoproj/pkg/json"
"github.com/spf13/cobra"

wfv1 "github.com/CyrusBiotechnology/argo/pkg/apis/workflow/v1alpha1"
cmdutil "github.com/CyrusBiotechnology/argo/util/cmd"
"github.com/CyrusBiotechnology/argo/workflow/common"
"github.com/CyrusBiotechnology/argo/workflow/util"
wfv1 "github.com/cyrusbiotechnology/argo/pkg/apis/workflow/v1alpha1"
cmdutil "github.com/cyrusbiotechnology/argo/util/cmd"
"github.com/cyrusbiotechnology/argo/workflow/common"
"github.com/cyrusbiotechnology/argo/workflow/util"
)

// cliSubmitOpts holds submition options specific to CLI submission (e.g. controlling output)
Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/commands/suspend.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"log"
"os"

"github.com/CyrusBiotechnology/argo/workflow/util"
"github.com/cyrusbiotechnology/argo/workflow/util"
"github.com/spf13/cobra"
)

Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/commands/terminate.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"github.com/argoproj/pkg/errors"
"github.com/spf13/cobra"

"github.com/CyrusBiotechnology/argo/workflow/util"
"github.com/cyrusbiotechnology/argo/workflow/util"
)

func NewTerminateCommand() *cobra.Command {
Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/commands/wait.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ import (
"os"
"sync"

wfv1 "github.com/CyrusBiotechnology/argo/pkg/apis/workflow/v1alpha1"
"github.com/argoproj/pkg/errors"
wfv1 "github.com/cyrusbiotechnology/argo/pkg/apis/workflow/v1alpha1"
"github.com/spf13/cobra"
apierr "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/commands/watch.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/fields"

wfv1 "github.com/CyrusBiotechnology/argo/pkg/apis/workflow/v1alpha1"
wfv1 "github.com/cyrusbiotechnology/argo/pkg/apis/workflow/v1alpha1"
)

func NewWatchCommand() *cobra.Command {
Expand Down
2 changes: 1 addition & 1 deletion cmd/argo/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"fmt"
"os"

"github.com/CyrusBiotechnology/argo/cmd/argo/commands"
"github.com/cyrusbiotechnology/argo/cmd/argo/commands"
// load the gcp plugin (required to authenticate against GKE clusters).
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
// load the oidc plugin (required to authenticate with OpenID Connect).
Expand Down
2 changes: 1 addition & 1 deletion cmd/argoexec/commands/resource.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package commands
import (
"os"

"github.com/CyrusBiotechnology/argo/workflow/common"
"github.com/cyrusbiotechnology/argo/workflow/common"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
Expand Down
12 changes: 6 additions & 6 deletions cmd/argoexec/commands/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ import (
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/tools/clientcmd"

"github.com/CyrusBiotechnology/argo"
"github.com/CyrusBiotechnology/argo/util/cmd"
"github.com/CyrusBiotechnology/argo/workflow/common"
"github.com/CyrusBiotechnology/argo/workflow/executor"
"github.com/CyrusBiotechnology/argo/workflow/executor/docker"
"github.com/CyrusBiotechnology/argo/workflow/executor/kubelet"
"github.com/cyrusbiotechnology/argo"
"github.com/cyrusbiotechnology/argo/util/cmd"
"github.com/cyrusbiotechnology/argo/workflow/common"
"github.com/cyrusbiotechnology/argo/workflow/executor"
"github.com/cyrusbiotechnology/argo/workflow/executor/docker"
"github.com/cyrusbiotechnology/argo/workflow/executor/kubelet"
)

const (
Expand Down
Loading