Implement Pending PipelineRun status (TEP-0015) #3522

jbarrick-mesosphere · 2020-11-13T21:26:09Z

Changes

Implements the Pending PipelineRun status as described in TEP-0015. (based on #3223)

/kind feature

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

Includes tests (if functionality changed/added)
Includes docs (if user facing)
Commit messages follow commit message best practices
Release notes block has been filled in or deleted (only if no user facing changes)

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

Added PipelineRunPending setting to PipelineRun Spec Status to allow creating PipelineRuns in a Pending state.

tekton-robot · 2020-11-13T21:26:25Z

Hi @jbarrick-mesosphere. Thanks for your PR.

I'm waiting for a tektoncd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bobcatfish

Thanks @jbarrick-mesosphere !!

What do you think about maybe including an end to end test as well? https://github.com/tektoncd/pipeline/tree/master/test#end-to-end-tests

that would give you some flexibility to do something like: create a pipelinerun with a quick timeout, but pending, make sure it doesn't time out, then remove pending and verify it succeeds or times out or something (i.e. will make sure more pieces work together)

pkg/reconciler/pipelinerun/pipelinerun_test.go

bobcatfish · 2020-11-16T18:48:10Z

btw this is a relatively small PR so what im suggesting might be overkill but probably the fastest way to get this merged might be to break it up a little, e.g. if you had a PR with just the types and the validation, that could probably get merged a lot faster than this entire PR

jbarrick-mesosphere · 2020-11-16T22:17:50Z

btw this is a relatively small PR so what im suggesting might be overkill but probably the fastest way to get this merged might be to break it up a little, e.g. if you had a PR with just the types and the validation, that could probably get merged a lot faster than this entire PR

Thanks, if this makes it significantly easier for you as a reviewer I'm happy to do this, but I'm in no rush to get the API changes merged in without the implementation.

pritidesai · 2020-11-17T00:49:38Z

/ok-to-test

Logistics 😄 - please squash the commits

tekton-robot · 2020-11-17T00:52:26Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	86.1%	85.6%	-0.5
pkg/apis/pipeline/v1beta1/pipelinerun_types.go	75.4%	75.8%	0.4

FogDong · 2020-11-17T03:05:06Z

Thanks! I'll close my original PR after this is merged😸

imjasonh · 2020-11-17T03:26:49Z

docs/pipelineruns.md

+ status: "PipelineRunPending"
+```
+
+To start the PipelineRun, delete the `.spec.status` field.


I'd say "unset the .spec.status field, or update its value cancel it"

imjasonh · 2020-11-17T03:27:03Z

internal/builder/v1beta1/pipeline.go

@@ -114,11 +114,16 @@ func PipelineDescription(desc string) PipelineSpecOp {
 }
 }

-// PipelineRunCancelled sets the status to cancel to the TaskRunSpec.
+// PipelineRunCancelled sets the status to cancel to the PielineRunSpec.


typo here: PipelineRunSpec

pkg/apis/pipeline/v1beta1/pipelinerun_validation_test.go

imjasonh · 2020-11-17T03:29:50Z

I notice that the TEP says it's a non-goal to support pending TaskRuns, "as that would requiring pausing the Kubernetes pods themselves" -- I think that's actually a bit misstated -- pending TaskRuns could just not create Pods, the same way that pending PipelineRuns just don't create TaskRuns. I think it's worth reconsidering implementing pending TaskRuns for consistency, or at least coming up with a better rationale for not doing so.

bobcatfish · 2020-11-17T16:21:59Z

Thanks, if this makes it significantly easier for you as a reviewer I'm happy to do this, but I'm in no rush to get the API changes merged in without the implementation.

I think I'm just over-applying this principle and this PR is pretty easy to review as it is 👍

jbarrick-mesosphere · 2020-11-17T20:39:29Z

I notice that the TEP says it's a non-goal to support pending TaskRuns, "as that would requiring pausing the Kubernetes pods themselves" -- I think that's actually a bit misstated -- pending TaskRuns could just not create Pods, the same way that pending PipelineRuns just don't create TaskRuns. I think it's worth reconsidering implementing pending TaskRuns for consistency, or at least coming up with a better rationale for not doing so.

Nice catch, yeah this is left over from when the proposal was to actually pause a running pipeline.

jbarrick-mesosphere · 2020-11-17T21:48:23Z

Thanks @jbarrick-mesosphere !!

What do you think about maybe including an end to end test as well? https://github.com/tektoncd/pipeline/tree/master/test#end-to-end-tests

Added an e2e test. 👍

tekton-robot · 2020-11-17T21:50:03Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	86.1%	85.6%	-0.5
pkg/apis/pipeline/v1beta1/pipelinerun_types.go	75.4%	75.8%	0.4
pkg/reconciler/pipelinerun/pipelinerun.go	84.4%	84.8%	0.4

tekton-robot · 2020-11-17T21:53:30Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	86.1%	85.6%	-0.5
pkg/apis/pipeline/v1beta1/pipelinerun_types.go	75.4%	75.8%	0.4
pkg/reconciler/pipelinerun/pipelinerun.go	84.4%	84.8%	0.4

tekton-robot · 2020-11-17T22:15:07Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	86.1%	85.6%	-0.5
pkg/apis/pipeline/v1beta1/pipelinerun_types.go	75.4%	75.8%	0.4
pkg/reconciler/pipelinerun/pipelinerun.go	84.4%	84.8%	0.4

Allows creation of Pending PipelineRuns which are not started until the pending status is cleared. Co-authored-by: Tianxin Dong <dongtianxin.dongtx@bytedance.com>

tekton-robot · 2020-11-18T16:48:48Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
internal/builder/v1beta1/pipeline.go	86.1%	85.6%	-0.5
pkg/apis/pipeline/v1beta1/pipelinerun_types.go	75.4%	75.8%	0.4
pkg/reconciler/pipelinerun/pipelinerun.go	84.4%	84.8%	0.4

imjasonh · 2020-11-18T18:57:18Z

Nice catch, yeah this is left over from when the proposal was to actually pause a running pipeline.

I see. Is the intention to support creating pending TaskRuns as well then? Either way, can you update the TEP with this?

jbarrick-mesosphere · 2020-11-18T19:32:18Z

I see. Is the intention to support creating pending TaskRuns as well then?

I don't currently have a use-case for this.

Either way, can you update the TEP with this?

I can update the TEP, but that is outside of the scope of this PR.

imjasonh · 2020-11-23T14:51:11Z

I don't currently have a use-case for this.

The initial TEP proposal PR says it will cover TaskRuns: tektoncd/community#203

Another recent TEP, tektoncd/community#228, assumes pending TaskRuns to limit concurrency during PipelineRun execution.

jbarrick-mesosphere · 2020-11-23T19:03:17Z

The initial TEP proposal PR says it will cover TaskRuns: tektoncd/community#203

I wrote the TEP, it doesn't say that.

Another recent TEP, tektoncd/community#228, assumes pending TaskRuns to limit concurrency during PipelineRun execution.

That would be a discussion to have on that TEP's PR.

imjasonh · 2020-11-23T20:03:50Z

I wrote the TEP, it doesn't say that.

Hey, sorry, I think I spoke imprecisely. I just meant the PR's title as merged was "Add a pending setting to Tekton PipelineRun and TaskRuns", so it seemed like we had intended to support it at some point. It's fine that the TEP's scope eventually narrowed -- better that than the opposite! -- and that the actually-merged TEP didn't cover pending TaskRuns at all.

Your previous comment had been about lacking a use case, and I found the pipeline-concurrency TEP as a use case which was depending on pending TaskRuns, seemingly assuming they'd be added as part of your TEP. You're absolutely right that the pipeline-concurrency TEP is the place to discuss the addition of pending TaskRuns as a requirement for concurrency-limited pipelines.

(To be clear, none of this discussion should block this PR, which otherwise looks great to me.)

If you felt so inclined to extend this work in another PR to include pending TaskRuns, being as you have recent experience implementing such a change, I'm sure the authors of the pipeline-concurrency TEP would appreciate it. If that's outside your needs and you'd prefer the author of that TEP or the community at large to support pending TaskRuns, that's fine too. Thanks for your contributions of all kinds throughout this process. :)

jbarrick-mesosphere · 2020-11-30T20:40:47Z

This should be ready for review.

FogDong · 2021-01-04T12:35:56Z

Do we have any plans to move forward this PR? 🤔

vincent-pli · 2021-01-07T00:28:41Z

Any process of the PR, here we have a case and expect to leverage this feature:

There is a specific task named terminator and when arrive the task, the pipeline should be pause, and before cancel need to do some cleanup work...
@imjasonh @bobcatfish

imjasonh

Sorry I didn't realize this PR was still pending (pun intended), it looks great, and thanks for adding tests!

/lgtm

ghost · 2021-01-26T18:12:48Z

pkg/apis/pipeline/v1beta1/pipelinerun_types.go

@@ -202,6 +207,8 @@ const (
 // PipelineRunSpecStatusCancelled indicates that the user wants to cancel the task,
 // if not already cancelled or terminated
 PipelineRunSpecStatusCancelled = "PipelineRunCancelled"
+
+ PipelineRunSpecStatusPending = "PipelineRunPending"


nit: I think it's worth adding a comment here similar to PipelineRunSpecStatusCancelled describing its purpose. Something like:

// PipelineRunSpecStatusPending indicates that the user wants to postpone starting a PipelineRun // until some condition is met

ghost · 2021-01-27T14:32:03Z

Given how long this PR has been open and that I only have a nit about a comment, I'm happy to add that comment in a follow-up PR.

/approve

tekton-robot · 2021-01-27T14:32:09Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbwsg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [sbwsg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 13, 2020

tekton-robot requested review from dlorenc and imjasonh November 13, 2020 21:26

tekton-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Nov 13, 2020

jbarrick-mesosphere changed the title ~~Pause~~ Implement Pending PipelineRun status (TEP-0015) Nov 13, 2020

jbarrick-mesosphere mentioned this pull request Nov 16, 2020

TEP-0015 - Add a pending setting to Tekton PipelineRun and TaskRuns tektoncd/community#203

Merged

bobcatfish reviewed Nov 16, 2020

View reviewed changes

pkg/reconciler/pipelinerun/pipelinerun_test.go Show resolved Hide resolved

tekton-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 17, 2020

imjasonh reviewed Nov 17, 2020

View reviewed changes

jbarrick-mesosphere force-pushed the pause branch from 5579f40 to 5986e9b Compare November 18, 2020 16:45

Feat: Add pending condition in pipeline run.

d5e6f66

Allows creation of Pending PipelineRuns which are not started until the pending status is cleared. Co-authored-by: Tianxin Dong <dongtianxin.dongtx@bytedance.com>

jbarrick-mesosphere force-pushed the pause branch from 5986e9b to d5e6f66 Compare November 18, 2020 16:46

jbarrick-mesosphere requested review from bobcatfish and imjasonh November 30, 2020 20:40

imjasonh reviewed Jan 26, 2021

View reviewed changes

tekton-robot assigned imjasonh Jan 26, 2021

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 26, 2021

ghost reviewed Jan 26, 2021

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 27, 2021

tekton-robot merged commit fbbcf71 into tektoncd:master Jan 27, 2021

ghost mentioned this pull request Jan 27, 2021

Add comment to PipelineRunSpecStatusPending #3723

Merged

3 tasks

AlanGreene mentioned this pull request Feb 17, 2021

Ensure PipelineRuns created in pending state are handled correctly tektoncd/dashboard#1944

Closed

bobcatfish mentioned this pull request Jul 12, 2021

When does a timeout counter actually start? #4078

Open

bobcatfish mentioned this pull request Aug 17, 2021

TEP-0060: Propose remote resolution experimental controller tektoncd/community#493

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Pending PipelineRun status (TEP-0015) #3522

Implement Pending PipelineRun status (TEP-0015) #3522

jbarrick-mesosphere commented Nov 13, 2020 •

edited

Loading

tekton-robot commented Nov 13, 2020

bobcatfish left a comment

bobcatfish commented Nov 16, 2020

jbarrick-mesosphere commented Nov 16, 2020

pritidesai commented Nov 17, 2020

tekton-robot commented Nov 17, 2020

FogDong commented Nov 17, 2020

imjasonh Nov 17, 2020

imjasonh Nov 17, 2020

imjasonh commented Nov 17, 2020

bobcatfish commented Nov 17, 2020

jbarrick-mesosphere commented Nov 17, 2020

jbarrick-mesosphere commented Nov 17, 2020

tekton-robot commented Nov 17, 2020

tekton-robot commented Nov 17, 2020

tekton-robot commented Nov 17, 2020

tekton-robot commented Nov 18, 2020

imjasonh commented Nov 18, 2020

jbarrick-mesosphere commented Nov 18, 2020

imjasonh commented Nov 23, 2020

jbarrick-mesosphere commented Nov 23, 2020

imjasonh commented Nov 23, 2020

jbarrick-mesosphere commented Nov 30, 2020

FogDong commented Jan 4, 2021

vincent-pli commented Jan 7, 2021

imjasonh left a comment

ghost Jan 26, 2021

ghost commented Jan 27, 2021

tekton-robot commented Jan 27, 2021

Implement Pending PipelineRun status (TEP-0015) #3522

Implement Pending PipelineRun status (TEP-0015) #3522

Conversation

jbarrick-mesosphere commented Nov 13, 2020 • edited Loading

Changes

Submitter Checklist

Reviewer Notes

Release Notes

tekton-robot commented Nov 13, 2020

bobcatfish left a comment

Choose a reason for hiding this comment

bobcatfish commented Nov 16, 2020

jbarrick-mesosphere commented Nov 16, 2020

pritidesai commented Nov 17, 2020

tekton-robot commented Nov 17, 2020

FogDong commented Nov 17, 2020

imjasonh Nov 17, 2020

Choose a reason for hiding this comment

imjasonh Nov 17, 2020

Choose a reason for hiding this comment

imjasonh commented Nov 17, 2020

bobcatfish commented Nov 17, 2020

jbarrick-mesosphere commented Nov 17, 2020

jbarrick-mesosphere commented Nov 17, 2020

tekton-robot commented Nov 17, 2020

tekton-robot commented Nov 17, 2020

tekton-robot commented Nov 17, 2020

tekton-robot commented Nov 18, 2020

imjasonh commented Nov 18, 2020

jbarrick-mesosphere commented Nov 18, 2020

imjasonh commented Nov 23, 2020

jbarrick-mesosphere commented Nov 23, 2020

imjasonh commented Nov 23, 2020

jbarrick-mesosphere commented Nov 30, 2020

FogDong commented Jan 4, 2021

vincent-pli commented Jan 7, 2021

imjasonh left a comment

Choose a reason for hiding this comment

ghost Jan 26, 2021

Choose a reason for hiding this comment

ghost commented Jan 27, 2021

tekton-robot commented Jan 27, 2021

jbarrick-mesosphere commented Nov 13, 2020 •

edited

Loading