sig-api-machinery: add "Graduate CustomResourceDefinitions to GA" #990
Conversation
k8s-ci-robot
added
cncf-cla: yes
k8s-ci-robot
added
the
size/L
jpbetz
force-pushed
the
extensibility-ga-kep
branch
5 times, most recently
from
c3a91bc to
69a5269
Compare
|
@lavalamp @deads2k, @liggitt @sttts @mbohlool @roycaihw Could we get a first past review? We'd like to get this in for the 1.15 KEP deadline on 4/30 since the Conversion webhooks to beta part of this targets 1.15. Note that we can continue to iterate on the other sections of this afterward since we're targeting 1.16 for CRDs to GA. |
jpbetz
force-pushed
the
extensibility-ga-kep
branch
from
69a5269 to
987c3e9
Compare
|
|
||
| See [umbrella issue](https://github.com/kubernetes/kubernetes/issues/58682) for status. | ||
|
|
||
| ## Post-GA tasks |
There was a problem hiding this comment.
"Potential Post-GA tasks"
Some of them below are far from consensus or good enough understanding. We won't decide anything about those in this KEP.
There was a problem hiding this comment.
Maybe we should not try to list them comprehensibly at all or give detail below. A link to an umbrella github issue is enough IMO.
There was a problem hiding this comment.
Sounds good. I've replaced this section with a link to the umbrella issue where readers can find an up-to-date post GA task list.
There was a problem hiding this comment.
Re-added this per @lavalamp's request. Added a note that the list is a snapshot from when KEP was written and to see the umbrella issue for current status.
There was a problem hiding this comment.
Yeah, the umbrella issue is not a good historical record. The purpose of this section is to inform people that we didn't forget about these things, we just decided they don't block v1 GA--you might want to make that a bit clearer in the text in a followup.
jpbetz
force-pushed
the
extensibility-ga-kep
branch
from
987c3e9 to
6c53cc1
Compare
jpbetz
force-pushed
the
extensibility-ga-kep
branch
from
6c53cc1 to
802b920
Compare
|
|
||
| See the [umbrella issue](https://github.com/kubernetes/kubernetes/issues/58682) for details on Post-GA tasks. | ||
|
|
||
| ### Arbitrary subresources as Post-GA task |
There was a problem hiding this comment.
as nice as this is as a sketch, I would remove it from this KEP. This topic will move into its own KEP as soon as somebody seriously works on it. We have decided to do this post-GA though. So no need here yet.
There was a problem hiding this comment.
Sounds good. We wanted to think through the basics to make sure CRD API v1 would be compatible with adding arbitrary subresources later. But I agree that it doesn't need to be here in the KEP. We can discuss the details with those interested separately.
There was a problem hiding this comment.
We could start a gdoc at some point to work towards a 1.16 KEP for that.
jpbetz
force-pushed
the
extensibility-ga-kep
branch
from
802b920 to
20096f0
Compare
| Currently custom resource subresources are v1beta1 and provide support for the | ||
| /status and /scale subresources. We plan to graduate this to GA as part of the | ||
| CRD to GA graduation. See [Arbitrary subresources as Post-GA | ||
| task](#arbitrary-subresources-as-post-ga-task) details on post GA tasks. |
There was a problem hiding this comment.
Maybe just call out arbitrary subresources as a non-goal / post-v1 task?
There was a problem hiding this comment.
Sounds good. I've re-added the inline list of post GA tasks back to this document including this one.
|
|
||
| TODO: complete this list | ||
|
|
||
| * Ensure v1beta1 and v1 CRDs are round trippable |
There was a problem hiding this comment.
This doesn't sound like an e2e test?
There was a problem hiding this comment.
Updated to "Custom resources should be readable and writable at all available versions (test for round trip-ability)"
jpbetz
force-pushed
the
extensibility-ga-kep
branch
from
20096f0 to
e6553ed
Compare
|
Thanks for the review @lavalamp. If it's okay, we'd like to merge this and finish the remaining TODOs via separate PRs, and then get 'implementable' approval after that. I've opened #1004 to track CRD webhook conversion items that need to go into 1.15 separate from this KEP. So now this KEP should be targeting 1.16 only and doesn't need to be rushed. |
|
@jpbetz: You must be a member of the kubernetes/kubernetes-milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your and have them propose you as an additional delegate for this responsibility. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
|
||
| ### CRD v1 schemas are restricted to a subset of the OpenAPI specification | ||
|
|
||
| See [Vanilla OpenAPI Subset Design](https://docs.google.com/document/d/1pcGlbmw-2Y0JJs9hsYnSBXamgG9TfWtHY6eh80zSTd8) |
| OpenAPI subset definition (next point). An update of the [old Pruning for | ||
| CustomResources KEP](https://github.com/kubernetes/enhancements/pull/709) and the implementation | ||
| ([pruning PR](https://github.com/kubernetes/kubernetes/pull/64558), [defaulting | ||
| PR](https://github.com/kubernetes/kubernetes/pull/63604)), are follow-ups as soon as unblocked. |
|
|
||
| * See “Required for GA” issues tracked via the [CRD Project Board](https://github.com/orgs/kubernetes/projects/28). | ||
|
|
||
| For additional details on already completed features, see the [Umbrella Issue](https://github.com/kubernetes/kubernetes/issues/58682). |
There was a problem hiding this comment.
We need to spell these out in a followup, at least a list with bullet points; issues are mutable and therefore not a good historical reference. This document needs to collect what needed to be done at the time it was written.
|
I'm OK with merging and building on this. Please fix the links @sttts pointed out in a followup :) The TODOs need to TODONEs before this can be implementable, I think. /lgtm |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jpbetz, lavalamp The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
| * Support proto encoding for custom resources (https://github.com/kubernetes/kubernetes/issues/63677) | ||
| * labelSelectorPath should be allowed not be under .status (https://github.com/kubernetes/kubernetes/issues/66688) | ||
| * Support arbitrary non-standard subresources for CRDs (https://github.com/kubernetes/kubernetes/issues/72637) | ||
| * OpenAPI v3 is published for all resources, including custom resources |
There was a problem hiding this comment.
We have found some client issues dealing with unexpected openapi conditions during our alpha stage. I'm concerned about promoting to GA without seeing this be functional because we're already talking about stripping back the set of supported features and I don't want to make GA objects invalid.
There was a problem hiding this comment.
This is talking about publishing openapi v3, which we don't do for any resources today, including in-tree ones.
The openapi subset KEP is intended to ensure we limit ourselves to what can be published for openapi v3, but exposing openapi/v3 endpoints is a distinct effort
We'd like to graduate CRDs sub-features to beta in 1.15 and then graduate CRDs to GA in a following release, ideally 1.16.
The purpose of this KEP is to establish consensus for the remaining essential improvements needed to move CRDs to GA. In particular, we'd like to agree on the features and tests required.
A couple sections, particularly the test and scale goals sections, are not complete. Feedback on what we should include in those sections would be very helpful.