Towards v1 API

[vdemeester]

This issue is here to track work towards the v1 API of tektoncd/pipeline. It is loosely based on the initial doc around this, and will be update as we go.

Motivation

In order to declare Pipeline stable and ready to use in production
to our users and customer, we need to give them some
guarantees. Although v1beta1 has some guarantees, most users are
waiting for a v1 API set that they can rely on for the long term.

If we can, in addition, come with a set of rules that would help us
decide when a feature request should be considered as required or not
for a v1 API, this would be nice 🙃.

Identified work

• API versions and new features Support Kubernetes style feature-gates for new API fields #3459 @vdemeester @skaegi @afrittoli
• Better credential UX Improve UX of getting credentials into Tasks #2343 @sbwsg
  • Consider projected volumes Consider better use of ProjectedVolumes #3551 (in relation to credential UX @sbwsg) — timebox work on investigation
• Additionnal workspace support (@sbwsg tracking issue to be created on workspaces)
• PipelineResource future Pipeline Resources Redesign #1673 @jerop @sbwsg @vdemeester
  • Tekton PipelineResources Beta Roadmap
• Resources Limit/Request & LimitRange support Improve UX of Step Resource Requests #2986 @vdemeester
• Failure strategies "when" expressions do not match user expectations #3345 [proposal] Capturing step exit codes in Tasks #2800 @jerop (if there is anything else to add)
• Custom tasks Implement Custom Tasks #3133 Integrate custom tasks into Pipelines #3463 @imjasonh @GregDritschler
• Migration path from v1beta1 to v1 @vdemeester
  • Documentation
  • Conversion webhook, …

Nice to have

• Redirect output streams Print all CRD as YAML on test failure #182 @chhsia0
• Pipeline concurrency limit provide a pipeline concurrency limit #1305
• Custom Scheduler Investigate to create a Custom Scheduler to schedule TaskRun pods #3052 - Timebox work on investigation
• Tekton OCI bundle enabled by default @vdemeester
• Proxy Support Proxy support for Pipeline *and* tasks #3090 facilitate mount of certs for HTTPS Proxy #2288 TEP-0033 Add tep for proxy support community#232 @piyush-garg
• PipelineRun and TaskRun Status PipelineRun and TaskRun Status #3792

To triage

• Provide a separate PipelineRun-volumeClaimTemplate, that can be mapped to multiple workspaces Allow volume from volumeClaimTemplate to be used in more than one workspace #3440 (in relation to workspaces @sbwsg)
• Task parallelism and Regional clusters - supported? Task parallelism and Regional clusters - supported? #3563

Use stories (to cover)

We should come with a bunch of user stories (from actual users) to highlight the requirements.

• A standard go project pipeline (lint, build, test)
• A standard Java project pipeline (lint, package, test, publis, …) — maven and gradle versions
• A source-to-image project pipeline (build image, test, deployment)
• A source-to-image in knative project pipeline (build image, deployment, test, deployment)
• Tekton pipeline project pipeline (lint, build, test, e2e test against a cluster, …)
• A canary deployment pipeline
• A “matrix” build pipeline (build, test, … with some different env’ variables — using CustomResource)
• A Kubernetes pipeline execution engine for Kubeflow pipeline to leverage ML pipeline use cases (Python DSL to Tekton pipelineRuns, Data Transformation/processing, test, deploy and monitor k8s model training CR, model examination)
• Add yours here

See also here 😅

/area epic
/area roadmap
/kind feature
/assign

/cc @tektoncd/core-maintainers

[jlpettersson]

I would like to see #3440 in v1 - and deprecate the current place were volumeClaimTemplate is implemented. That would be a more intuitive API - easier to understand and easier to work with. I really regret the current design.

I also think #3551 would be nice to do / try out before v1.

Replace the Affinity Assistant with a Custom Scheduler #3052 would also be nice - even though not strictly an API-thing, but before 1.0. Personally I would prefer the current constraints since I, as default expect 1) that declared parallel tasks can execute in parallel - with PVC, 2) that pipelineruns can not deadlock depending on how I use PVCs

[bobcatfish]

I think that making some changes to the constraints the affinity assistant imposes before v1 would make a lot of sense - for example I really don't like it that (unless im understanding incorrectly) trying to use 2 PVCs with 1 Task right now will not be allowed (unless you disable the affinity assistant). Even if we keep that, I think it shouldn't be the default.

[jlpettersson]

@bobcatfish I opened #3563 for discussion

[bobcatfish]

@vdemeester maybe we can triage #3163 and potentially have it as part of our v1 roadmap?

[vdemeester]

@vdemeester maybe we can triage #3163 and potentially have it as part of our v1 roadmap?

Except in " Migration path from v1beta1 to v1 " I don't see how it affects the v1 API 🤔.

[bobcatfish]

I think it's part of the migration path @vdemeester - when we add v1, we'll have another API type to worry about having end to end tests for

[bobcatfish]

@vdemeester possible item to triage: have we discussed at all revisiting the structure of the status sections of TaskRun and PipelineRun? I don't think these were designed as carefully as the spec sections and there is some odd stuff in there. Or we can say it's good enough and go to v1 - but I think if we wanted to change it at all, we'd want to try to do it pre-v1

[vdemeester]

@vdemeester possible item to triage: have we discussed at all revisiting the structure of the status sections of TaskRun and PipelineRun? I don't think these were designed as carefully as the spec sections and there is some odd stuff in there. Or we can say it's good enough and go to v1 - but I think if we wanted to change it at all, we'd want to try to do it pre-v1

We can definitely discuss that :)

[bobcatfish]

Another item to add to our list (from our governing board meeting):

• Fixing the security issues identified at https://security.lfx.linuxfoundation.org/#/a092M00001G3xhxQAB/foundation-details

[jerop]

@vdemeester we missed you in the API meeting on Monday where we discussed adding #3792 to the V1 work here, what do you think?

[vdemeester]

@vdemeester we missed you in the API meeting on Monday where we discussed adding #3792 to the V1 work here, what do you think?

We should definitely add it to the list of things to discuss and work on yes 😉

[ghost]

We might want to add the "phasing out HOME / workingDir" things as part of v1 work too:

#2013
#1836

[bobcatfish]

Another item to add to our list - I think @sbwsg 's comment touches on this - but we have a bunch of feature flags we can probably remove (i.e. set the behavior they guard as the default): https://github.com/tektoncd/pipeline/blob/master/docs/install.md#customizing-the-pipelines-controller-behavior (maybe these are just the creds related flags?)

[kobaji]

The following comment is written in another issue #4048 because I thought this should be published as a new issue.

Hello @vdemeester and all thank you very much for publishing this issue.
Regarding this issue, I would like to know something.

Could you please tell me how I could reach the exact months or quarter of the Tekton pipeline's v1 API release?

I have checked the following documents but I could not have found the information.

- https://github.com/tektoncd/pipeline/blob/main/roadmap.md
- https://github.com/tektoncd/community/blob/main/roadmap.md

That information must be helpful for our team would contribute to Tekton.
Thank you.

[bobcatfish]

another thing to think about adding ... or removing that is XD: volumes + volume mounts (do these provide anything we need in addition to workspaces?) #2058

[bobcatfish]

@kobaji I don't think we have any firm dates but are hoping for early 2022 (lemme know if you have different thoughts @vdemeester @afrittoli )

[vdemeester]

another thing to think about adding ... or removing that is XD: volumes + volume mounts (do these provide anything we need in addition to workspaces?) #2058

Right, so on this, I am going to open an issue that is slightly related, on volume, workspace, configmap and secrets.

@kobaji I don't think we have any firm dates but are hoping for early 2022 (lemme know if you have different thoughts @vdemeester @afrittoli )

Yeah that's more or less what I have in mind. And again, this is for a v1 API, not necessarly a 1.0 release of tekton (that could happen before, or after).

[vdemeester]

#4055 ^^

[kobaji]

@kobaji I don't think we have any firm dates but are hoping for early 2022 (lemme know if you have different thoughts @vdemeester @afrittoli )

Yeah that's more or less what I have in mind. And again, this is for a v1 API, not necessarly a 1.0 release of tekton (that could happen before, or after).

Thank you @bobcatfish , @vdemeester for your kindness.
I will keep having a look regarding Tekton release.
Thank you again for your kind help.

[gabemontero]

Hey folks - I seem to recall a while back in API WG discussion around possibly deprecating ClusterTasks (with possibly bundles as a replacement).

Is my memory totally faulty here? I'm not seeing any reference to it here, nor am I know finding it in the design doc bookmarks I still have, nor seeing any open/closed issues with ClusterTask in the title, so perhaps my memory is faulty :-/

If not though, is that deprecation still being considered, or has it been dismissed?

thanks !

[vdemeester]

Is my memory totally faulty here? I'm not seeing any reference to it here, nor am I know finding it in the design doc bookmarks I still have, nor seeing any open/closed issues with ClusterTask in the title, so perhaps my memory is faulty :-/

If not though, is that deprecation still being considered, or has it been dismissed?

Your memory is not faulty, and indeed even though we discussed it, we never created an issue around it (and we probably should indeed 😛 )

[bobcatfish]

Another one i think we should add to our v1 list is to tackle #3497 - decide if we're okay with the current functionality going forward or not.

[jerop]

consider if we need to tackle #4399 before v1

[ghost]

Created #4476 for discussion around CluserTask

[vdemeester]

@jerop how do we want to maintain this compared/with https://github.com/orgs/tektoncd/projects/10 and the V1 TEP ?

[lbernick]

I'd prefer using the project board since it makes it easier to add/remove issues, track their status, and add commentary about each one

[vdemeester]

@lbernick I tend to agree 😇

[jerop]

@jerop how do we want to maintain this compared/with https://github.com/orgs/tektoncd/projects/10 and the V1 TEP ?

@vdemeester @lbernick agree with both of you that the project board is great for tracking items, and I'd add that the V1 TEP is useful for reaching alignment on what items are in scope before they are added to the board -- thank you @vdemeester for scoping the items in this issue, it's been so useful and strongly influenced both the TEP and project board 🙏

[StevenACoffman]

BTW, I just thought to mention here that the various Tekton Roadmaps are all still listed as "2021 Roadmaps" such as https://github.com/tektoncd/pipeline/blob/main/roadmap.md and https://github.com/tektoncd/community/blob/main/roadmap.md#2021-roadmap (there are more).

[lbernick]

/close

using project board for tracking

[tekton-robot]

@lbernick: Closing this issue.

In response to this:

/close

using project board for tracking

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[vdemeester]

BTW, I just thought to mention here that the various Tekton Roadmaps are all still listed as "2021 Roadmaps" such as https://github.com/tektoncd/pipeline/blob/main/roadmap.md and https://github.com/tektoncd/community/blob/main/roadmap.md#2021-roadmap (there are more).

Yeah we need to refine those.

[jerop]

@StevenACoffman @vdemeester moved those roadmap items into a project view so that it's live: https://github.com/orgs/tektoncd/projects/16/views/15