fix ValidateStepResultsVariables to validate stepResults only #8264

jkhelil · 2024-09-16T14:24:31Z

Changes

There is a mixture of validating step result and task result in both ValidateStepResultsVariables
ValidateStepResultsVariables should use step results set of name to only validate StepResults. TaskResults are validated via the function ValidateTaskResultsVariables

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
Has Tests included if any functionality added or changed
pre-commit Passed
Follows the commit message standard
Meets the Tekton contributor standards (including functionality, content, code)
Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

NONE

jkhelil · 2024-09-16T14:30:31Z

/kind bug

tekton-robot · 2024-09-16T14:36:07Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-16T14:37:10Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-16T14:37:24Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-17T08:19:39Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-17T08:21:48Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-17T08:23:33Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-17T08:56:12Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-17T09:00:37Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-17T11:34:28Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-17T11:35:05Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-17T11:36:24Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-09-17T12:04:27Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

jkhelil · 2024-09-23T08:07:34Z

/test pull-tekton-pipeline-beta-integration-tests

jkhelil · 2024-09-23T08:57:39Z

/test pull-tekton-pipeline-beta-integration-tests

afrittoli · 2024-09-23T09:43:52Z

/test pull-tekton-pipeline-go-coverage-df

tekton-robot · 2024-09-23T09:43:55Z

@afrittoli: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test pull-tekton-pipeline-alpha-integration-tests
/test pull-tekton-pipeline-beta-integration-tests
/test pull-tekton-pipeline-build-tests
/test pull-tekton-pipeline-integration-tests
/test pull-tekton-pipeline-unit-tests

The following commands are available to trigger optional jobs:

/test pull-tekton-pipeline-go-coverage

Use /test all to run all jobs.

In response to this:

/test pull-tekton-pipeline-go-coverage-df

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot · 2024-09-23T09:56:29Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

savitaashture · 2024-09-24T14:23:21Z

@vdemeester I can see it working with the reproducer scenario

➜  tekton tkn tr describe test-ndn9d
Name:              test-ndn9d
Namespace:         default
Task Ref:          collect-data
Service Account:   default
Timeout:           1h0m0s
Labels:
 app.kubernetes.io/managed-by=tekton-pipelines
 tekton.dev/task=collect-data
Annotations:
 pipeline.tekton.dev/release=f1fc5c5

🌡️  Status

STARTED         DURATION    STATUS
3 minutes ago   21s         Succeeded

📝 Results

 NAME           VALUE
 ∙ resultsDir   tom

🦶 Steps

 NAME             STATUS
 ∙ collect-data   Completed
 ∙ reduce         Completed

k get pods
Containers:
  step-collect-data:
    Container ID:  containerd://971efd7afdea2403d40c149e73ee1c534004e9ead4263cf5ae127bc54d1cc083
    Image:         quay.io/konflux-ci/release-service-utils:e633d51cd41d73e4b3310face21bb980af7a662f
    Image ID:      quay.io/konflux-ci/release-service-utils@sha256:13624fbf75efd88c3220aa19b1423786f4cacfd9abab9042e934ff6114addae0
    Port:          <none>
    Host Port:     <none>
    Command:
      /tekton/bin/entrypoint
    Args:
      -wait_file
      /tekton/downward/ready
      -wait_file_content
      -post_file
      /tekton/run/0/out
      -termination_path
      /tekton/termination
      -step_metadata_dir
      /tekton/run/0/status
      -step_results
      singleComponentMode
      -results
      resultsDir
      -entrypoint
      /tekton/scripts/script-0-8v22c
      --
    State:          Terminated
      Reason:       Completed
      Message:      [{"key":"resultsDir","value":"tom","type":1},{"key":"singleComponentMode","value":"scott","type":4},{"key":"StartedAt","value":"2024-09-18T11:31:45.793Z","type":3}]
      Exit Code:    0
      Started:      Wed, 18 Sep 2024 13:31:43 +0200
      Finished:     Wed, 18 Sep 2024 13:31:45 +0200
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /tekton/artifacts from tekton-internal-artifacts (rw)
      /tekton/bin from tekton-internal-bin (ro)
      /tekton/creds from tekton-creds-init-home-0 (rw)
      /tekton/downward from tekton-internal-downward (ro)
      /tekton/home from tekton-internal-home (rw)
      /tekton/results from tekton-internal-results (rw)
      /tekton/run/0 from tekton-internal-run-0 (rw)
      /tekton/run/1 from tekton-internal-run-1 (ro)
      /tekton/scripts from tekton-internal-scripts (ro)
      /tekton/steps from tekton-internal-steps (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-rdjnz (ro)
      /workspace from tekton-internal-workspace (rw)
  step-reduce:
    Container ID:  containerd://0d791fafaa4f65428d3170626c3a53e63873e908aa431cc53ab7832ac70122f3
    Image:         quay.io/konflux-ci/release-service-utils:e633d51cd41d73e4b3310face21bb980af7a662f
    Image ID:      quay.io/konflux-ci/release-service-utils@sha256:13624fbf75efd88c3220aa19b1423786f4cacfd9abab9042e934ff6114addae0
    Port:          <none>
    Host Port:     <none>
    Command:
      /tekton/bin/entrypoint
    Args:
      -wait_file
      /tekton/run/0/out
      -post_file
      /tekton/run/1/out
      -termination_path
      /tekton/termination
      -step_metadata_dir
      /tekton/run/1/status
      -results
      resultsDir
      -entrypoint
      /tekton/scripts/script-1-cf47m
      --
    State:          Terminated
      Reason:       Completed
      Message:      [{"key":"resultsDir","value":"tom","type":1},{"key":"StartedAt","value":"2024-09-18T11:31:45.915Z","type":3}]
      Exit Code:    0
      Started:      Wed, 18 Sep 2024 13:31:43 +0200
      Finished:     Wed, 18 Sep 2024 13:31:47 +0200

@jkhelil It would be great if this is covered as part of e2e if possible to add

tekton-robot · 2024-10-28T12:07:52Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tekton-robot · 2024-10-28T12:19:37Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

tekton-robot · 2024-10-28T12:19:59Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	98.6%	-0.0

afrittoli

Thank for catching and fixing this issue.
I think there is an issue with the current fix though - see my comments inline.
/hold

afrittoli · 2024-10-28T12:52:25Z

pkg/apis/pipeline/v1alpha1/stepaction_validation_test.go

-	}, {
-		name: "step script refers to nonexistent result",
-		fields: fields{
-			Image: "my-image",
-			Script: `
-			#!/usr/bin/env bash
-			date | tee $(results.non-exist.path)`,
-			Results: []v1.StepResult{{Name: "a-result"}},
-		},
-		expectedError: apis.FieldError{
-			Message: `non-existent variable in "\n\t\t\t#!/usr/bin/env bash\n\t\t\tdate | tee $(results.non-exist.path)"`,
-			Paths:   []string{"script"},
-		},


The fact that you have to remove this test here highlights another problem in the current codebase.
You want to fix an issue in how we validate an inline step in a Task definition, and this has a side effect on the validation of a StepAction, which should not happen.

In the task-stepaction-results.yaml example that you provided, an inline step references to a Task result, which is valid and should work, and doesn't work today, which is the problem that needs to be fixed.
This stems from the fact that ValidateStepResultsVariables does not have enough context to validate Task level references, which are validated instead by ValidateTaskResultsVariables as you mentioned.

The change you proposed, however, modifies code used for validation of StepActions, which is not ok, as exemplified by the test you removed, which should continue to pass. The author of a StepAction has no knowledge about the Tasks that may use it, and should not reference any variable outside of the known context.

A correct fix for the issue should include, on top of the fix already done, a change in the stepaction_validation code, which, instead of using ValidateStepResultsVariables from the task validation, should define its own ValidateStepActionResultsVariables, which should look like what ValidateStepResultsVariables looks today.

Unit tests should not be removed and we should make sure we have a unit tests that validates ValidateTaskResultsVariables (which we most likely already have).

@afrittoli I think you're right to add ValidateStepActionResultsVariables that validates step results only because it has only the step results context and hence i removed the snippet of tests there (we dont have the task result context when using stepAction.Validate)
I am also fixing ValidateStepResultsVariables to add task results in the function args so we can validate the step script against both step and task result because we have them in context as this method is now only called in task validation context

@afrittoli Can you have a look to my comment pleae

tekton-robot · 2024-10-29T09:16:29Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	97.1%	-1.6
pkg/apis/pipeline/v1beta1/task_validation.go	98.6%	98.3%	-0.2

tekton-robot · 2024-10-29T09:19:33Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	97.1%	-1.6
pkg/apis/pipeline/v1beta1/task_validation.go	98.6%	98.3%	-0.2

…nd fix ValidateStepResultsVariables to include task results validation

tekton-robot · 2024-10-29T10:38:39Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	97.1%	-1.6
pkg/apis/pipeline/v1beta1/task_validation.go	98.6%	98.3%	-0.2

tekton-robot · 2024-10-29T10:41:25Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/task_validation.go	98.6%	97.1%	-1.6
pkg/apis/pipeline/v1beta1/task_validation.go	98.6%	98.3%	-0.2

tekton-robot added release-note-none Denotes a PR that doesnt merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Sep 16, 2024

tekton-robot requested review from abayer and jerop September 16, 2024 14:24

jkhelil force-pushed the fix_validateTaskResultsVariables branch from a7cd2bb to 8e0fa48 Compare September 16, 2024 14:25

tekton-robot added the kind/bug Categorizes issue or PR as related to a bug. label Sep 16, 2024

jkhelil force-pushed the fix_validateTaskResultsVariables branch from 8e0fa48 to 0c6cd04 Compare September 17, 2024 08:06

tekton-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Sep 17, 2024

jkhelil force-pushed the fix_validateTaskResultsVariables branch from 0c6cd04 to 18dee1a Compare September 17, 2024 08:08

jkhelil changed the title ~~fix ValidateStepResultsVariables and validateTaskResultsVariables~~ fix ValidateStepResultsVariables to validate stepResults only Sep 17, 2024

jkhelil force-pushed the fix_validateTaskResultsVariables branch from 18dee1a to 7c5e691 Compare September 17, 2024 08:45

tekton-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 17, 2024

jkhelil force-pushed the fix_validateTaskResultsVariables branch 2 times, most recently from e078cdd to 98e4c6f Compare September 17, 2024 11:23

jkhelil force-pushed the fix_validateTaskResultsVariables branch from 98e4c6f to f1fc5c5 Compare September 17, 2024 11:53

afrittoli added this to the Pipeline v0.64 milestone Sep 23, 2024

tekton-robot modified the milestones: Pipeline v0.64, Pipeline v0.65 Sep 25, 2024

vdemeester approved these changes Sep 27, 2024

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 27, 2024

jkhelil force-pushed the fix_validateTaskResultsVariables branch from f1fc5c5 to 5a87c4a Compare October 28, 2024 12:07

tekton-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 28, 2024

afrittoli reviewed Oct 28, 2024

View reviewed changes

tekton-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 28, 2024

vdemeester modified the milestones: Pipeline v0.65, Pipeline v0.66 Oct 28, 2024

jkhelil force-pushed the fix_validateTaskResultsVariables branch from 5a87c4a to 49fb44c Compare October 29, 2024 09:02

tekton-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 29, 2024

Add ValidateStepActionResultsVariables to validate stepResults only a…

1a958c3

…nd fix ValidateStepResultsVariables to include task results validation

jkhelil force-pushed the fix_validateTaskResultsVariables branch from 49fb44c to 1a958c3 Compare October 29, 2024 10:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix ValidateStepResultsVariables to validate stepResults only #8264

fix ValidateStepResultsVariables to validate stepResults only #8264

jkhelil commented Sep 16, 2024 •

edited by afrittoli

Loading

jkhelil commented Sep 16, 2024

tekton-robot commented Sep 16, 2024

tekton-robot commented Sep 16, 2024

tekton-robot commented Sep 16, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

jkhelil commented Sep 23, 2024

jkhelil commented Sep 23, 2024

afrittoli commented Sep 23, 2024

tekton-robot commented Sep 23, 2024

tekton-robot commented Sep 23, 2024

savitaashture commented Sep 24, 2024

tekton-robot commented Oct 28, 2024

tekton-robot commented Oct 28, 2024

tekton-robot commented Oct 28, 2024

afrittoli left a comment

afrittoli Oct 28, 2024

jkhelil Oct 29, 2024

jkhelil Nov 7, 2024

tekton-robot commented Oct 29, 2024

tekton-robot commented Oct 29, 2024

tekton-robot commented Oct 29, 2024

tekton-robot commented Oct 29, 2024

fix ValidateStepResultsVariables to validate stepResults only #8264

Are you sure you want to change the base?

fix ValidateStepResultsVariables to validate stepResults only #8264

Conversation

jkhelil commented Sep 16, 2024 • edited by afrittoli Loading

Changes

Submitter Checklist

Release Notes

jkhelil commented Sep 16, 2024

tekton-robot commented Sep 16, 2024

tekton-robot commented Sep 16, 2024

tekton-robot commented Sep 16, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

tekton-robot commented Sep 17, 2024

jkhelil commented Sep 23, 2024

jkhelil commented Sep 23, 2024

afrittoli commented Sep 23, 2024

tekton-robot commented Sep 23, 2024

tekton-robot commented Sep 23, 2024

savitaashture commented Sep 24, 2024

tekton-robot commented Oct 28, 2024

tekton-robot commented Oct 28, 2024

tekton-robot commented Oct 28, 2024

afrittoli left a comment

Choose a reason for hiding this comment

afrittoli Oct 28, 2024

Choose a reason for hiding this comment

jkhelil Oct 29, 2024

Choose a reason for hiding this comment

jkhelil Nov 7, 2024

Choose a reason for hiding this comment

tekton-robot commented Oct 29, 2024

tekton-robot commented Oct 29, 2024

tekton-robot commented Oct 29, 2024

tekton-robot commented Oct 29, 2024

jkhelil commented Sep 16, 2024 •

edited by afrittoli

Loading