Towards v1 API #3548

vdemeester · 2020-11-20T10:57:38Z

This issue is here to track work towards the v1 API of tektoncd/pipeline. It is loosely based on the initial doc around this, and will be update as we go.

Motivation

In order to declare Pipeline stable and ready to use in production
to our users and customer, we need to give them some
guarantees. Although v1beta1 has some guarantees, most users are
waiting for a v1 API set that they can rely on for the long term.

If we can, in addition, come with a set of rules that would help us
decide when a feature request should be considered as required or not
for a v1 API, this would be nice 🙃.

Identified work

API versions and new features Support Kubernetes style feature-gates for new API fields #3459 @vdemeester @skaegi @afrittoli
Better credential UX Improve UX of getting credentials into Tasks #2343 @sbwsg
- Consider projected volumes Consider better use of ProjectedVolumes #3551 (in relation to credential UX @sbwsg) — timebox work on investigation
Additionnal workspace support (@sbwsg tracking issue to be created on workspaces)
PipelineResource future Pipeline Resources Redesign #1673 @jerop @sbwsg @vdemeester
- Tekton PipelineResources Beta Roadmap
Resources Limit/Request & LimitRange support Improve UX of Step Resource Requests #2986 @vdemeester
Failure strategies "when" expressions do not match user expectations #3345 [proposal] Capturing step exit codes in Tasks #2800 @jerop (if there is anything else to add)
Custom tasks Implement Custom Tasks #3133 Integrate custom tasks into Pipelines #3463 @imjasonh @GregDritschler
Migration path from v1beta1 to v1 @vdemeester
- Documentation
- Conversion webhook, …

Nice to have

Redirect output streams Print all CRD as YAML on test failure #182 @chhsia0
Pipeline concurrency limit provide a pipeline concurrency limit #1305
Custom Scheduler Investigate to create a Custom Scheduler to schedule TaskRun pods #3052 - Timebox work on investigation
Tekton OCI bundle enabled by default @vdemeester
Proxy Support Proxy support for Pipeline *and* tasks #3090 facilitate mount of certs for HTTPS Proxy #2288 TEP-0033 Add tep for proxy support community#232 @piyush-garg
PipelineRun and TaskRun Status PipelineRun and TaskRun Status #3792

To triage

Provide a separate PipelineRun-volumeClaimTemplate, that can be mapped to multiple workspaces Allow volume from volumeClaimTemplate to be used in more than one workspace #3440 (in relation to workspaces @sbwsg)
Task parallelism and Regional clusters - supported? Task parallelism and Regional clusters - supported? #3563

Use stories (to cover)

We should come with a bunch of user stories (from actual users) to highlight the requirements.

A standard go project pipeline (lint, build, test)
A standard Java project pipeline (lint, package, test, publis, …) — maven and gradle versions
A source-to-image project pipeline (build image, test, deployment)
A source-to-image in knative project pipeline (build image, deployment, test, deployment)
Tekton pipeline project pipeline (lint, build, test, e2e test against a cluster, …)
A canary deployment pipeline
A “matrix” build pipeline (build, test, … with some different env’ variables — using CustomResource)
A Kubernetes pipeline execution engine for Kubeflow pipeline to leverage ML pipeline use cases (Python DSL to Tekton pipelineRuns, Data Transformation/processing, test, deploy and monitor k8s model training CR, model examination)
Add yours here

See also here 😅

/area epic
/area roadmap
/kind feature
/assign

/cc @tektoncd/core-maintainers

The text was updated successfully, but these errors were encountered:

jlpettersson · 2020-11-20T23:38:38Z

I would like to see #3440 in v1 - and deprecate the current place were volumeClaimTemplate is implemented. That would be a more intuitive API - easier to understand and easier to work with. I really regret the current design.

I also think #3551 would be nice to do / try out before v1.

Replace the Affinity Assistant with a Custom Scheduler #3052 would also be nice - even though not strictly an API-thing, but before 1.0. Personally I would prefer the current constraints since I, as default expect 1) that declared parallel tasks can execute in parallel - with PVC, 2) that pipelineruns can not deadlock depending on how I use PVCs

bobcatfish · 2020-11-24T15:07:45Z

I think that making some changes to the constraints the affinity assistant imposes before v1 would make a lot of sense - for example I really don't like it that (unless im understanding incorrectly) trying to use 2 PVCs with 1 Task right now will not be allowed (unless you disable the affinity assistant). Even if we keep that, I think it shouldn't be the default.

jlpettersson · 2020-11-25T23:46:19Z

@bobcatfish I opened #3563 for discussion

bobcatfish · 2020-12-01T16:47:50Z

@vdemeester maybe we can triage #3163 and potentially have it as part of our v1 roadmap?

vdemeester · 2020-12-01T17:01:40Z

@vdemeester maybe we can triage #3163 and potentially have it as part of our v1 roadmap?

Except in " Migration path from v1beta1 to v1 " I don't see how it affects the v1 API 🤔.

bobcatfish · 2020-12-01T21:20:56Z

I think it's part of the migration path @vdemeester - when we add v1, we'll have another API type to worry about having end to end tests for

bobcatfish · 2021-01-12T21:25:59Z

@vdemeester possible item to triage: have we discussed at all revisiting the structure of the status sections of TaskRun and PipelineRun? I don't think these were designed as carefully as the spec sections and there is some odd stuff in there. Or we can say it's good enough and go to v1 - but I think if we wanted to change it at all, we'd want to try to do it pre-v1

vdemeester · 2021-01-13T09:14:02Z

@vdemeester possible item to triage: have we discussed at all revisiting the structure of the status sections of TaskRun and PipelineRun? I don't think these were designed as carefully as the spec sections and there is some odd stuff in there. Or we can say it's good enough and go to v1 - but I think if we wanted to change it at all, we'd want to try to do it pre-v1

We can definitely discuss that :)

bobcatfish · 2021-02-04T15:56:13Z

Another item to add to our list (from our governing board meeting):

Fixing the security issues identified at https://security.lfx.linuxfoundation.org/#/a092M00001G3xhxQAB/foundation-details

jerop · 2021-02-25T17:23:29Z

@vdemeester we missed you in the API meeting on Monday where we discussed adding #3792 to the V1 work here, what do you think?

vdemeester · 2021-03-01T11:14:54Z

@vdemeester we missed you in the API meeting on Monday where we discussed adding #3792 to the V1 work here, what do you think?

We should definitely add it to the list of things to discuss and work on yes 😉

ghost · 2021-03-09T14:48:51Z

We might want to add the "phasing out HOME / workingDir" things as part of v1 work too:

#2013
#1836

bobcatfish · 2021-03-09T18:15:10Z

Another item to add to our list - I think @sbwsg 's comment touches on this - but we have a bunch of feature flags we can probably remove (i.e. set the behavior they guard as the default): https://github.com/tektoncd/pipeline/blob/master/docs/install.md#customizing-the-pipelines-controller-behavior (maybe these are just the creds related flags?)

kobaji · 2021-06-16T02:21:54Z

The following comment is written in another issue #4048 because I thought this should be published as a new issue.

Hello @vdemeester and all thank you very much for publishing this issue.
Regarding this issue, I would like to know something.

Could you please tell me how I could reach the exact months or quarter of the Tekton pipeline's v1 API release?

I have checked the following documents but I could not have found the information.

- https://github.com/tektoncd/pipeline/blob/main/roadmap.md
- https://github.com/tektoncd/community/blob/main/roadmap.md

That information must be helpful for our team would contribute to Tekton.
Thank you.

bobcatfish · 2021-06-22T17:55:34Z

another thing to think about adding ... or removing that is XD: volumes + volume mounts (do these provide anything we need in addition to workspaces?) #2058

bobcatfish · 2021-06-22T17:56:04Z

@kobaji I don't think we have any firm dates but are hoping for early 2022 (lemme know if you have different thoughts @vdemeester @afrittoli )

vdemeester · 2021-06-23T07:53:15Z

another thing to think about adding ... or removing that is XD: volumes + volume mounts (do these provide anything we need in addition to workspaces?) #2058

Right, so on this, I am going to open an issue that is slightly related, on volume, workspace, configmap and secrets.

@kobaji I don't think we have any firm dates but are hoping for early 2022 (lemme know if you have different thoughts @vdemeester @afrittoli )

Yeah that's more or less what I have in mind. And again, this is for a v1 API, not necessarly a 1.0 release of tekton (that could happen before, or after).

vdemeester · 2021-06-23T08:23:19Z

#4055 ^^

kobaji · 2021-07-01T23:49:35Z

@kobaji I don't think we have any firm dates but are hoping for early 2022 (lemme know if you have different thoughts @vdemeester @afrittoli )

Yeah that's more or less what I have in mind. And again, this is for a v1 API, not necessarly a 1.0 release of tekton (that could happen before, or after).

Thank you @bobcatfish , @vdemeester for your kindness.
I will keep having a look regarding Tekton release.
Thank you again for your kind help.

This change is a clarification and update to the migration and upgrade plan for skipping strategies. The implementable proposal is to change the scope of `when` expressions to guard the `Task`. This is a backwards-incompatible change. To make the transition smooth, we plan to: - provide a feature flag, `scope-when-expressions-to-task`, which: - will default to `scope-when-expressions-to-task` : "false" to guard a `Task` and its dependent `Tasks` - can be set to `scope-when-expressions-to-task` : "true" to guard a `Task` only - after 9 months, per the [Tekton API compatibility policy](https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md#alpha-beta-and-ga), we'll flip the feature flag and default to ` scope-when-expressions-to-task` : "true" [February 2022] - in the next release, we'll remove the feature flag and `when` expressions will be scoped to guard a `Task` only going forward [March 2022] - when we do [v1 release](tektoncd/pipeline#3548) (projected for early 2022), we will have `when` expressions guarding a `Task` only both in _beta_ and _v1_ We also plan to over-communicate during the migration in Slack, email and working group meetings.

gabemontero · 2021-10-20T14:52:03Z

Hey folks - I seem to recall a while back in API WG discussion around possibly deprecating ClusterTasks (with possibly bundles as a replacement).

Is my memory totally faulty here? I'm not seeing any reference to it here, nor am I know finding it in the design doc bookmarks I still have, nor seeing any open/closed issues with ClusterTask in the title, so perhaps my memory is faulty :-/

If not though, is that deprecation still being considered, or has it been dismissed?

thanks !

vdemeester · 2021-10-20T15:12:39Z

Is my memory totally faulty here? I'm not seeing any reference to it here, nor am I know finding it in the design doc bookmarks I still have, nor seeing any open/closed issues with ClusterTask in the title, so perhaps my memory is faulty :-/

If not though, is that deprecation still being considered, or has it been dismissed?

Your memory is not faulty, and indeed even though we discussed it, we never created an issue around it (and we probably should indeed 😛 )

bobcatfish · 2021-11-09T21:56:25Z

Another one i think we should add to our v1 list is to tackle #3497 - decide if we're okay with the current functionality going forward or not.

jerop · 2021-11-29T21:05:37Z

consider if we need to tackle #4399 before v1

ghost · 2022-01-12T22:06:01Z

Created #4476 for discussion around CluserTask

vdemeester · 2022-04-06T01:29:46Z

@jerop how do we want to maintain this compared/with https://github.com/orgs/tektoncd/projects/10 and the V1 TEP ?

lbernick · 2022-04-06T14:45:12Z

I'd prefer using the project board since it makes it easier to add/remove issues, track their status, and add commentary about each one

vdemeester · 2022-04-06T17:25:24Z

@lbernick I tend to agree 😇

jerop · 2022-04-06T21:14:19Z

@jerop how do we want to maintain this compared/with https://github.com/orgs/tektoncd/projects/10 and the V1 TEP ?

@vdemeester @lbernick agree with both of you that the project board is great for tracking items, and I'd add that the V1 TEP is useful for reaching alignment on what items are in scope before they are added to the board -- thank you @vdemeester for scoping the items in this issue, it's been so useful and strongly influenced both the TEP and project board 🙏

StevenACoffman · 2022-04-27T17:06:19Z

BTW, I just thought to mention here that the various Tekton Roadmaps are all still listed as "2021 Roadmaps" such as https://github.com/tektoncd/pipeline/blob/main/roadmap.md and https://github.com/tektoncd/community/blob/main/roadmap.md#2021-roadmap (there are more).

lbernick · 2022-05-31T21:13:48Z

/close

using project board for tracking

tekton-robot · 2022-05-31T21:13:51Z

@lbernick: Closing this issue.

In response to this:

/close

using project board for tracking

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

vdemeester · 2022-06-01T07:05:22Z

BTW, I just thought to mention here that the various Tekton Roadmaps are all still listed as "2021 Roadmaps" such as https://github.com/tektoncd/pipeline/blob/main/roadmap.md and https://github.com/tektoncd/community/blob/main/roadmap.md#2021-roadmap (there are more).

Yeah we need to refine those.

jerop · 2022-08-23T17:22:09Z

@StevenACoffman @vdemeester moved those roadmap items into a project view so that it's live: https://github.com/orgs/tektoncd/projects/16/views/15

tekton-robot assigned vdemeester Nov 20, 2020

tekton-robot added area/epic Issues that should be considered as Epics (aka multiple sub-tasks, …) area/roadmap Issues that are part of the project (or organization) roadmap (usually an epic) kind/feature Categorizes issue or PR as related to a new feature. labels Nov 20, 2020

vdemeester added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Nov 20, 2020

vdemeester mentioned this issue Jan 4, 2021

Could you please tell me when Tekton is planned to GA?? tektoncd/community#297

Closed

jerop mentioned this issue Feb 25, 2021

PipelineRun and TaskRun Status #3792

Closed

jerop mentioned this issue Jun 1, 2021

TEP-0059: Skipping Strategies tektoncd/community#407

Merged

kobaji mentioned this issue Jun 18, 2021

Which month or quarter is the Tekton pipeline planned to v1 API release? #4048

Closed

vdemeester mentioned this issue Jun 23, 2021

Workspace, configmap, secret and typing #4055

Open

jerop mentioned this issue Jul 20, 2021

TEP-0059: Skipping Strategies - Migration Plan tektoncd/community#481

Merged

jerop mentioned this issue Oct 5, 2021

Consider better use of ProjectedVolumes #3551

Open

lbernick mentioned this issue Nov 29, 2021

[Problem Statement] TEP-0096: Pipelines V1 Release tektoncd/community#567

Merged

tekton-robot closed this as completed May 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Towards v1 API #3548

Towards v1 API #3548

vdemeester commented Nov 20, 2020 •

edited by bobcatfish

Loading

jlpettersson commented Nov 20, 2020 •

edited

Loading

bobcatfish commented Nov 24, 2020

jlpettersson commented Nov 25, 2020

bobcatfish commented Dec 1, 2020

vdemeester commented Dec 1, 2020

bobcatfish commented Dec 1, 2020

bobcatfish commented Jan 12, 2021

vdemeester commented Jan 13, 2021

bobcatfish commented Feb 4, 2021

jerop commented Feb 25, 2021 •

edited

Loading

vdemeester commented Mar 1, 2021

ghost commented Mar 9, 2021

bobcatfish commented Mar 9, 2021

kobaji commented Jun 16, 2021 •

edited

Loading

bobcatfish commented Jun 22, 2021

bobcatfish commented Jun 22, 2021

vdemeester commented Jun 23, 2021

vdemeester commented Jun 23, 2021

kobaji commented Jul 1, 2021

gabemontero commented Oct 20, 2021

vdemeester commented Oct 20, 2021

bobcatfish commented Nov 9, 2021

jerop commented Nov 29, 2021

ghost commented Jan 12, 2022

vdemeester commented Apr 6, 2022

lbernick commented Apr 6, 2022

vdemeester commented Apr 6, 2022

jerop commented Apr 6, 2022 •

edited

Loading

StevenACoffman commented Apr 27, 2022

lbernick commented May 31, 2022

tekton-robot commented May 31, 2022

vdemeester commented Jun 1, 2022

jerop commented Aug 23, 2022

Towards v1 API #3548

Towards v1 API #3548

Comments

vdemeester commented Nov 20, 2020 • edited by bobcatfish Loading

Motivation

Identified work

Nice to have

To triage

Use stories (to cover)

jlpettersson commented Nov 20, 2020 • edited Loading

bobcatfish commented Nov 24, 2020

jlpettersson commented Nov 25, 2020

bobcatfish commented Dec 1, 2020

vdemeester commented Dec 1, 2020

bobcatfish commented Dec 1, 2020

bobcatfish commented Jan 12, 2021

vdemeester commented Jan 13, 2021

bobcatfish commented Feb 4, 2021

jerop commented Feb 25, 2021 • edited Loading

vdemeester commented Mar 1, 2021

ghost commented Mar 9, 2021

bobcatfish commented Mar 9, 2021

kobaji commented Jun 16, 2021 • edited Loading

bobcatfish commented Jun 22, 2021

bobcatfish commented Jun 22, 2021

vdemeester commented Jun 23, 2021

vdemeester commented Jun 23, 2021

kobaji commented Jul 1, 2021

gabemontero commented Oct 20, 2021

vdemeester commented Oct 20, 2021

bobcatfish commented Nov 9, 2021

jerop commented Nov 29, 2021

ghost commented Jan 12, 2022

vdemeester commented Apr 6, 2022

lbernick commented Apr 6, 2022

vdemeester commented Apr 6, 2022

jerop commented Apr 6, 2022 • edited Loading

StevenACoffman commented Apr 27, 2022

lbernick commented May 31, 2022

tekton-robot commented May 31, 2022

vdemeester commented Jun 1, 2022

jerop commented Aug 23, 2022

vdemeester commented Nov 20, 2020 •

edited by bobcatfish

Loading

jlpettersson commented Nov 20, 2020 •

edited

Loading

jerop commented Feb 25, 2021 •

edited

Loading

kobaji commented Jun 16, 2021 •

edited

Loading

jerop commented Apr 6, 2022 •

edited

Loading