fix: Mark non-fulfilled taskSetNodes error when agent pod failed. Fixes #12703

[jswxstw]

Fixes #12703

Motivation

1. Node status is non-fulfilled when agent pod failed.
2. argo-cluster-role can not get secrets argo-workflows-agent-ca-certificates

Modifications

1. Mark non-fulfilled taskSetNodes error when agent pod failed.
2. add secrets argo-workflows-agent-ca-certificates get permission to argo-cluster-role

Verification

UT

Delete hello-executor-plugin serviceaccount to simulate agent pod failure scenario.

1. single step example

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-
spec:
  entrypoint: main
  templates:
    - name: main
      plugin:
        hello: { }

# argo get hello-q8h7f
Name:                hello-q8h7f
Namespace:           argo
ServiceAccount:      unset (will run with the default ServiceAccount)
Status:              Error
Message:             create agent pod failed with reason:"serviceaccounts "hello-executor-plugin" not found"
Conditions:
 PodRunning          False
 Completed           True
Created:             Fri Mar 01 15:55:18 +0800 (53 seconds ago)
Started:             Fri Mar 01 15:55:18 +0800 (53 seconds ago)
Finished:            Fri Mar 01 15:55:19 +0800 (52 seconds ago)
Duration:            1 second
Progress:            0/1

STEP            TEMPLATE  PODNAME  DURATION  MESSAGE
 ⚠ hello-q8h7f  main                         create agent pod failed with reason:"serviceaccounts "hello-executor-plugin" not found"

2. parallel steps example

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: hello-
spec:
  entrypoint: main
  templates:
    - name: main
      steps:
      - - name: plugin
          template: hello-plugin
        - name: container
          template: hello-container
    - name: hello-plugin
      plugin:
        hello: {}
    - name: hello-container
      container:
        image: docker/whalesay:latest
        command: [sh, -c]
        args: ["echo hello"]

# argo get hello-dtkm9
Name:                hello-dtkm9
Namespace:           argo
ServiceAccount:      unset (will run with the default ServiceAccount)
Status:              Failed
Message:             child 'hello-dtkm9-3577871718' failed
Conditions:
 PodRunning          False
 Completed           True
Created:             Fri Mar 01 15:55:35 +0800 (28 seconds ago)
Started:             Fri Mar 01 15:55:35 +0800 (28 seconds ago)
Finished:            Fri Mar 01 15:55:45 +0800 (18 seconds ago)
Duration:            10 seconds
Progress:            1/2
ResourcesDuration:   0s*(1 cpu),4s*(100Mi memory)

STEP              TEMPLATE         PODNAME                                 DURATION  MESSAGE
 ✖ hello-dtkm9    main                                                               child 'hello-dtkm9-3577871718' failed
 └─┬─✔ container  hello-container  hello-dtkm9-hello-container-1541734788  7s
   └─⚠ plugin     hello-plugin                                                       create agent pod failed with reason:"serviceaccounts "hello-executor-plugin" not found"

[juliev0]

Using node-level errors seems good as far as I can tell. Want to confirm - when a standard (non-Agent) Pod fails to be created, I assume that the Node results in NodeError, and thus this would be the same as that?

[jswxstw]

You are right, if creating non-agent pod with non-transient error or pod executing with error, the node will be marked as NodeError.