Retries use an incorrect parameter set

[davidboren]

Is this a BUG REPORT or FEATURE REQUEST?:
BUG REPORT

What happened:
When tasks within a dag share a template, on retry these tasks will use an incorrect parameter set (Possibly belonging to the first task registered with said template)

What you expected to happen:
A retry should use the same parameter set as the first attempt

How to reproduce it (as minimally and precisely as possible):
The following dag workflow has a first task, A, and 3 subsequent tasks, B, C, and D, which each have A as their only dependency. The retries are set on the template (shared between all tasks) with a limit of 3. The tasks are simple python scripts that are parameterized via an env variable that takes in TASK_NAME as an input parameter, and the script raises an exception when the TASK_NAME is C or D. The first try of C and D uses the correct parameterization and fails, but the retries of both tasks, though they show the correct parameterization in the ui, are actually parameterized with A, causing them to pass on the second try rather than fail again, as they ought to.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-retry-example
spec:
  entrypoint: dag
  templates:
  - name: dag
    dag:
      tasks:
      - name: A
        template: raise-exception
        arguments:
          parameters:
          - name: task-name
            value: A
      - name: B
        dependencies: [A]
        template: raise-exception
        arguments:
          parameters:
          - name: task-name
            value: B
      - name: C
        dependencies: [A]
        template: raise-exception
        arguments:
          parameters:
          - name: task-name
            value: C
      - name: D
        dependencies: [A]
        template: raise-exception
        arguments:
          parameters:
          - name: task-name
            value: D

  - name: raise-exception
    inputs:
      parameters:
      - name: task-name
    retryStrategy:
      limit: 3
    script:
      image: python:alpine3.6
      command: [python]
      env:
      - name: TASK_NAME
        value: '{{inputs.parameters.task-name}}'
      source: |
        import os
        print(f"Task Name: {os.environ['TASK_NAME']}")
        if os.environ["TASK_NAME"] in ("C", "D"):
            raise Exception("Bad juju")

Anything else we need to know?:

Environment:

• Argo version:

argo: v2.4.2
  BuildDate: 2019-10-21T18:39:52Z
  GitCommit: 675c66267f0c916de0f233d8101aa0646acb46d4
  GitTreeState: clean
  GitTag: v2.4.2
  GoVersion: go1.11.5
  Compiler: gc
  Platform: darwin/amd64

• Kubernetes version :

Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.11", GitCommit:"637c7e288581ee40ab4ca210618a89a555b6e7e9", GitTreeState:"clean", BuildDate:"2018-11-26T14:25:46Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Other debugging information (if applicable):

• workflow result:

Name:                dag-retry-example97d8g
Namespace:           argo-events
ServiceAccount:      default
Status:              Succeeded
Created:             Wed Oct 30 22:05:36 -0700 (15 minutes ago)
Started:             Wed Oct 30 22:05:36 -0700 (15 minutes ago)
Finished:            Wed Oct 30 22:05:52 -0700 (15 minutes ago)
Duration:            16 seconds

STEP                             PODNAME                            DURATION  MESSAGE
 ✔ dag-retry-example97d8g (dag)
 ├-✔ A(0) (raise-exception)      dag-retry-example97d8g-853219092   8s
 ├-✔ C (raise-exception)
 | ├-✖ C(0) (raise-exception)    dag-retry-example97d8g-2378465938  3s        failed with exit code 1
 | └-✔ C(1) (raise-exception)    dag-retry-example97d8g-700556943   3s
 ├-✔ D (raise-exception)
 | ├-✖ D(0) (raise-exception)    dag-retry-example97d8g-1308248661  3s        failed with exit code 1
 | └-✔ D(1) (raise-exception)    dag-retry-example97d8g-1107064328  2s
 └-✔ B(0) (raise-exception)      dag-retry-example97d8g-2510019383  3s

[simster7]

Possibly related to #1713

[simster7]

Hello @davidboren, v2.4.3 was just released with a fix for this issue, please take a look! Closing