Watcher: Pipeline resource cleanup

[wlynch]

Feature request

After a TaskRun/PipelineRun is completed and successfully uploaded, we should delete the etcd based data to free up resources on the user's cluster.

Requirements:

• etcd based data should not be deleted until the Task/PipelineRun is complete and we've confirmed that the data has been successfully uploaded to the API.
• There should be a configurable grace period for how long we wait before we deleting so that other integrations relying on the CRD API directly don't break.
• Cleanup behavior / grace period should be flag/configmap gated

Use case

Task/PipelineRun clean up has been a much requested feature, but is at odds with results storage since once the CRD object is gone from etcd, the results are gone with it.

Making this a feature of the Results watcher means that we can ensure that this clean up happens after results have been stored durably.

See:

• Introduce runHistoryLimit pipeline#2332
• Adding support to delete completed TaskRun or PipelineRun automatically pipeline#1486
• Introduce runHistoryLimit pipeline#2332
• About deleting completed TaskRun or PipelineRun automatically pipeline#1429
• Using CronJobs to automatically clean up completed Runs experimental#479
• PipelineRun cleanup and cancelation pipeline#2452

[tekton-robot]

@XinruZhang: GitHub didn't allow me to assign the following users: me.

Note that only tektoncd members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign me

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[XinruZhang]

/assign xinruzhang