[TEP074] Remove Storage, Git and Generic PipelineResources #6150

JeromeJu · 2023-02-12T19:02:04Z

This is going to be the last PR for PipelineResources removal.

Changes

This commit removes the Git, Storage and Generic PipelineResources:

removes the Storage resources support at pkg/apis/resource/v1alpha1/storage API
removes the Git resources support at pkg/apis/resource/v1alpha1/git API
removes the generic logics for input and output resources
removes the pkg/artifacts API
their corresponding unit test cases, examples and integration tests
renames:
- ResolvedTaskResources Struct renamed to ResolvedTask
- pkg/reconciler/taskrun/validate_resources.go to pkg/reconciler/taskrun/validate_taskrun.go
- pkg/reconciler/taskrun/resources/taskresourceresolution.go to pkg/reconciler/taskrun/resources/taskresolution.go

Reasons why removing Git, Storage Resources andpkg/artifacts together:

Why bundle Git and Storage resources removal?
The Git and Storage PipelineResources both uses artifact storage, which made removing Storage PipelineResources along to be intervened by changes on Git Resources that are inappropriate to separate. See the decision made after attempting to remove Storage Resources along for more detail.
Why bundle removal of Storage resources with the artifacts package?
The artifacts package has also dependencies on storage resources for the artifacts storage setup, which made it hard to separate the removal of storage resources first. See the decision after the attempt to remove artifacts package first.

The coverage loss is expected since the pipelineResources is removed.

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Has Docs included if any changes are user facing
Has Tests included if any functionality added or changed
Follows the commit message standard
Meets the Tekton contributor standards (including
functionality, content, code)
Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

action required: please migrate off of `git` and `storage` `pipelineresources` as they are removed, please refer to the doc at https://github.com/tektoncd/pipeline/blob/main/docs/pipelineresources.md#replacing-an-git-resource  https://github.com/tektoncd/pipeline/blob/main/docs/pipelineresources.md#replacing-an-storage-resource

installing this release on a cluster with tasks and pipelines that use pipeline resources may break them

pipelineresources are removed in this release

tekton-robot · 2023-02-12T19:02:07Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

tekton-robot · 2023-02-12T19:07:49Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/resource/v1alpha1/pipelineresource_validation.go	88.0%	75.0%	-13.0
pkg/reconciler/pipelinerun/pipelinerun.go	87.2%	86.9%	-0.3
pkg/reconciler/pipelinerun/resources/input_output_steps.go	96.9%	80.0%	-16.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.9%	94.6%	-0.3
pkg/reconciler/taskrun/resources/input_resources.go	90.9%	8.6%	-82.3
pkg/reconciler/taskrun/resources/output_resource.go	89.1%	75.9%	-13.3
pkg/reconciler/taskrun/resources/taskresourceresolution.go	100.0%	94.7%	-5.3
pkg/reconciler/taskrun/taskrun.go	84.5%	83.8%	-0.7
pkg/reconciler/taskrun/validate_resources.go	98.5%	97.8%	-0.7
test/controller.go	29.1%	27.8%	-1.3

tekton-robot · 2023-02-12T19:09:49Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/resource/v1alpha1/pipelineresource_validation.go	88.0%	75.0%	-13.0
pkg/reconciler/pipelinerun/pipelinerun.go	87.2%	86.9%	-0.3
pkg/reconciler/pipelinerun/resources/input_output_steps.go	96.9%	80.0%	-16.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.9%	94.6%	-0.3
pkg/reconciler/taskrun/resources/input_resources.go	90.9%	8.6%	-82.3
pkg/reconciler/taskrun/resources/output_resource.go	89.1%	75.9%	-13.3
pkg/reconciler/taskrun/resources/taskresourceresolution.go	100.0%	94.7%	-5.3
pkg/reconciler/taskrun/taskrun.go	84.5%	83.8%	-0.7
pkg/reconciler/taskrun/validate_resources.go	98.5%	97.8%	-0.7
test/controller.go	29.1%	27.8%	-1.3

tekton-robot · 2023-02-13T13:35:55Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/resource/v1alpha1/pipelineresource_validation.go	88.0%	75.0%	-13.0
pkg/reconciler/pipelinerun/pipelinerun.go	87.2%	86.9%	-0.3
pkg/reconciler/pipelinerun/resources/input_output_steps.go	96.9%	80.0%	-16.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.9%	94.6%	-0.3
pkg/reconciler/taskrun/resources/output_resource.go	89.1%	72.7%	-16.4
pkg/reconciler/taskrun/resources/taskresourceresolution.go	100.0%	94.7%	-5.3
pkg/reconciler/taskrun/taskrun.go	84.5%	84.1%	-0.4
pkg/reconciler/taskrun/validate_resources.go	98.5%	98.2%	-0.4
test/controller.go	29.1%	27.8%	-1.3

tekton-robot · 2023-02-13T13:42:39Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/resource/v1alpha1/pipelineresource_validation.go	88.0%	75.0%	-13.0
pkg/reconciler/pipelinerun/pipelinerun.go	87.2%	86.9%	-0.3
pkg/reconciler/pipelinerun/resources/input_output_steps.go	96.9%	80.0%	-16.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.9%	94.6%	-0.3
pkg/reconciler/taskrun/resources/output_resource.go	89.1%	72.7%	-16.4
pkg/reconciler/taskrun/resources/taskresourceresolution.go	100.0%	94.7%	-5.3
pkg/reconciler/taskrun/taskrun.go	84.5%	84.1%	-0.4
pkg/reconciler/taskrun/validate_resources.go	98.5%	98.2%	-0.4
test/controller.go	29.1%	27.8%	-1.3

tekton-robot · 2023-02-13T16:13:21Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/resource/v1alpha1/pipelineresource_validation.go	88.0%	72.7%	-15.3
pkg/reconciler/pipelinerun/pipelinerun.go	87.2%	86.9%	-0.3
pkg/reconciler/pipelinerun/resources/input_output_steps.go	96.9%	80.0%	-16.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.9%	94.7%	-0.3
pkg/reconciler/taskrun/resources/output_resource.go	89.1%	72.7%	-16.4
pkg/reconciler/taskrun/resources/taskresourceresolution.go	100.0%	89.5%	-10.5
pkg/reconciler/taskrun/taskrun.go	84.5%	84.1%	-0.4
pkg/reconciler/taskrun/validate_resources.go	98.5%	98.2%	-0.4
test/controller.go	29.1%	27.8%	-1.3

tekton-robot · 2023-02-13T16:17:55Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/resource/v1alpha1/pipelineresource_validation.go	88.0%	72.7%	-15.3
pkg/reconciler/pipelinerun/pipelinerun.go	87.2%	86.9%	-0.3
pkg/reconciler/pipelinerun/resources/input_output_steps.go	96.9%	80.0%	-16.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.9%	94.7%	-0.3
pkg/reconciler/taskrun/resources/output_resource.go	89.1%	72.7%	-16.4
pkg/reconciler/taskrun/resources/taskresourceresolution.go	100.0%	94.7%	-5.3
pkg/reconciler/taskrun/taskrun.go	84.5%	84.1%	-0.4
pkg/reconciler/taskrun/validate_resources.go	98.5%	98.2%	-0.4
test/controller.go	29.1%	27.8%	-1.3

tekton-robot · 2023-02-13T16:18:04Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/resource/v1alpha1/pipelineresource_validation.go	88.0%	72.7%	-15.3
pkg/reconciler/pipelinerun/pipelinerun.go	87.2%	86.9%	-0.3
pkg/reconciler/pipelinerun/resources/input_output_steps.go	96.9%	80.0%	-16.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.9%	94.7%	-0.3
pkg/reconciler/taskrun/resources/output_resource.go	89.1%	72.7%	-16.4
pkg/reconciler/taskrun/resources/taskresourceresolution.go	100.0%	94.7%	-5.3
pkg/reconciler/taskrun/taskrun.go	84.5%	84.1%	-0.4
pkg/reconciler/taskrun/validate_resources.go	98.5%	98.2%	-0.4
test/controller.go	29.1%	27.8%	-1.3

chitrangpatel

Thanks for doing this!! This is a lot of removal. It must have been extremely tedious.

pkg/apis/pipeline/v1beta1/task_validation_test.go

pkg/apis/resource/v1alpha1/pipelineresource_validation_test.go

docs/variables.md

pkg/pipelinerunmetrics/metrics_test.go

pkg/reconciler/pipelinerun/resources/input_output_steps.go

tekton-robot · 2023-03-07T17:59:27Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1beta1/pipeline_conversion.go	94.8%	96.1%	1.3
pkg/apis/pipeline/v1beta1/pipeline_types.go	96.5%	96.3%	-0.2
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.3%	99.0%	-0.2
pkg/apis/pipeline/v1beta1/pipelinerun_conversion.go	95.7%	96.8%	1.1
pkg/apis/pipeline/v1beta1/resource_types.go	86.0%	80.0%	-6.0
pkg/apis/pipeline/v1beta1/task_conversion.go	94.4%	97.4%	2.9
pkg/apis/pipeline/v1beta1/task_validation.go	97.1%	96.9%	-0.1
pkg/apis/pipeline/v1beta1/taskrun_conversion.go	92.3%	93.1%	0.8
pkg/apis/pipeline/v1beta1/taskrun_validation.go	97.8%	97.7%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	87.4%	88.3%	0.9
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	94.8%	96.0%	1.1
pkg/reconciler/taskrun/controller.go	95.2%	95.0%	-0.2
pkg/reconciler/taskrun/resources/apply.go	99.3%	99.2%	-0.1
pkg/reconciler/taskrun/taskrun.go	84.8%	84.9%	0.1
pkg/reconciler/taskrun/validate_taskrun.go	Do not exist	97.8%
test/controller.go	29.1%	28.9%	-0.2

JeromeJu · 2023-03-07T19:10:54Z

/retest

dibyom · 2023-03-07T20:25:26Z

/lgtm

CRD pipelineresources has been removed in tektoncd#6150. We should also remove it from delete_pipeline_resources. Signed-off-by: Yongxuan Zhang yongxuanzhang@google.com

CRD pipelineresources has been removed in #6150. We should also remove it from delete_pipeline_resources. Signed-off-by: Yongxuan Zhang yongxuanzhang@google.com

wlynch · 2023-03-23T22:51:37Z

Heads up, I think this may be a similar problematic change as #6206 where we're removing both the functionality and API client types in the same change.

Can we consider reverting API fields back to tombstones at least for the v1beta1 client? 🙏

lbernick · 2023-03-27T14:19:23Z

Heads up, I think this may be a similar problematic change as #6206 where we're removing both the functionality and API client types in the same change.

Can we consider reverting API fields back to tombstones at least for the v1beta1 client? 🙏

There may have been some discussion around this already but I'm having trouble finding it (#6150 (comment) is the closest thing I'm seeing). I think this is reasonable; happy to open a PR

Config-artifact-pvc was introduce in https://github.com/tektoncd/pipeline/pull/866/files which was deleted in tektoncd#6150 These two resources are assumed to be used and needed by the storage pipeline resources which are gone now. These must be no longer needed and safe to clean up. Signed-off-by: Priti Desai <pdesai@us.ibm.com>

Config-artifact-pvc was introduce in https://github.com/tektoncd/pipeline/pull/866/files which was deleted in #6150 These two resources are assumed to be used and needed by the storage pipeline resources which are gone now. These must be no longer needed and safe to clean up. Signed-off-by: Priti Desai <pdesai@us.ibm.com>

Robocat is stuck on Pipelines `v20230307-771fb665f2` due to an error in the nightly deploy TaskRun. See for example from the most recent run: ``` 2024-02-15T03:01:00.091452354Z + kubectl apply --kubeconfig /workspace/targetCluster/kubeconfig -k pipeline/overlays/robocat 2024-02-15T03:01:02.736006696Z error: no matches for Id ~G_v1_ConfigMap|tekton-pipelines|config-artifact-bucket; failed to find unique target for patch ~G_v1_ConfigMap|config-artifact-bucket ``` The referenced ConfigMap was removed from Pipelines in tektoncd/pipeline#6150 which was merged on March 7th. Remove the file from the overlay as it's no longer used.

The link to example files was broken, as those files were removed in: tektoncd/pipeline#6150 Update the link to match the example that is now being used: https://github.com/tektoncd/dashboard/blob/main/docs/tutorial

The link to example files was broken, as those files were removed in: tektoncd/pipeline#6150 Update the link to match the example that is now being used: https://github.com/tektoncd/dashboard/blob/main/docs/tutorial Also update the example images

tekton-robot requested review from dibyom and dlorenc February 12, 2023 19:02

JeromeJu changed the title ~~Rm storage git resources~~ [TEP074] Remove Storage and Git PipelineResources Feb 13, 2023

tekton-robot added release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. and removed release-note-none Denotes a PR that doesnt merit a release note. labels Feb 13, 2023

JeromeJu marked this pull request as ready for review February 13, 2023 13:36

tekton-robot requested a review from imjasonh February 13, 2023 13:36

tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 13, 2023

JeromeJu mentioned this pull request Feb 13, 2023

[TEP074] Remove Storage pipelineResources #6014

Closed

7 tasks

JeromeJu force-pushed the rm-storage-git-resources branch 2 times, most recently from a3e8f86 to 4d876da Compare February 13, 2023 16:10

JeromeJu mentioned this pull request Feb 13, 2023

[TEP074] Remove Git PipelineResources #6135

Closed

7 tasks

This was referenced Feb 13, 2023

Handle PipelineResources when swapping v1 to the storage version #5967

Closed

[TEP074] Remove Image pipelineResources #6002

Merged

chitrangpatel approved these changes Feb 13, 2023

View reviewed changes

lbernick reviewed Feb 13, 2023

View reviewed changes

JeromeJu force-pushed the rm-storage-git-resources branch from 4d876da to 17627b1 Compare February 13, 2023 20:56

tekton-robot assigned dibyom Mar 7, 2023

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Mar 7, 2023

tekton-robot merged commit 81876e6 into tektoncd:main Mar 7, 2023

Yongxuanzhang mentioned this pull request Mar 15, 2023

remove pipelineresources from delete_pipeline_resources #6357

Merged

7 tasks

lbernick mentioned this pull request Mar 21, 2023

[TEP-0074] Remove git-init code #6417

Merged

4 tasks

EmmaMunley mentioned this pull request Mar 22, 2023

TEP-0118: Update TaskRun Validation for Matrix Include Params #6418

Merged

4 tasks

This was referenced Mar 24, 2023

Remove PipelineResource support tektoncd/chains#734

Closed

Breaking API change: v0.46 omits data for existing API objects (PipelineResources) #6430

Closed

lbernick mentioned this pull request Mar 30, 2023

Revert removal of PipelineResources related fields #6436

Merged

4 tasks

This was referenced Apr 13, 2023

remove e2e pipeline resource tests tektoncd/cli#1987

Merged

skip pipelineresource tests tektoncd/plumbing#1392

Merged

kabhiibm mentioned this pull request Apr 21, 2023

Remove resource test in cli tektoncd/plumbing#1398

Merged

2 tasks

lbernick mentioned this pull request Jun 7, 2023

[release-v0.44.x] Fix spammy logs #6780

Merged

pritidesai mentioned this pull request Jul 5, 2023

cleaning up config-artifact-* #6902

Merged

7 tasks

AlanGreene mentioned this pull request Feb 15, 2024

Remove unused file causing Pipelines nightly deploy to fail tektoncd/plumbing#1819

Merged

2 tasks

briangleeson mentioned this pull request Jul 18, 2024

Update obsolete link in kind walk-through tektoncd/dashboard#3507

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TEP074] Remove Storage, Git and Generic PipelineResources #6150

[TEP074] Remove Storage, Git and Generic PipelineResources #6150

JeromeJu commented Feb 12, 2023 •

edited

Loading

tekton-robot commented Feb 12, 2023

tekton-robot commented Feb 12, 2023

tekton-robot commented Feb 12, 2023

tekton-robot commented Feb 13, 2023

tekton-robot commented Feb 13, 2023

tekton-robot commented Feb 13, 2023

tekton-robot commented Feb 13, 2023

tekton-robot commented Feb 13, 2023

chitrangpatel left a comment

tekton-robot commented Mar 7, 2023

JeromeJu commented Mar 7, 2023

dibyom commented Mar 7, 2023

wlynch commented Mar 23, 2023

lbernick commented Mar 27, 2023

[TEP074] Remove Storage, Git and Generic PipelineResources #6150

[TEP074] Remove Storage, Git and Generic PipelineResources #6150

Conversation

JeromeJu commented Feb 12, 2023 • edited Loading

Changes

Submitter Checklist

Release Notes

tekton-robot commented Feb 12, 2023

tekton-robot commented Feb 12, 2023

tekton-robot commented Feb 12, 2023

tekton-robot commented Feb 13, 2023

tekton-robot commented Feb 13, 2023

tekton-robot commented Feb 13, 2023

tekton-robot commented Feb 13, 2023

tekton-robot commented Feb 13, 2023

chitrangpatel left a comment

Choose a reason for hiding this comment

tekton-robot commented Mar 7, 2023

JeromeJu commented Mar 7, 2023

dibyom commented Mar 7, 2023

wlynch commented Mar 23, 2023

lbernick commented Mar 27, 2023

JeromeJu commented Feb 12, 2023 •

edited

Loading