preserveUnknownFields=false doesn't work with crdVersions=v1 #476

jonnylangefeld · 2020-08-22T01:31:50Z

crdVersions=v1 which is the new default, doesn't work in conjunction with preserveUnknownFields=false. The spec.preserveUnknownFields simply doesn't appear in the crd. There is an integration test for v1beta1, but not for v1.

I'm running the following command in a kubebuilder repo:

controller-gen "crd:preserveUnknownFields=false,crdVersions=v1" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases

I'm using defaults in my schema and if applying to a 1.16.13-gke.1 cluster, I get:

The CustomResourceDefinition "foo.example.com" is invalid: spec.preserveUnknownFields: Invalid value: true: must be false in order to use defaults in the schema

If I add spec.preserveUnknownFields manually to the crd it works

My workaround in the makefile right now is

yq w -i config/crd/bases/cluster.paas.getcruise.com_clusters.yaml spec.preserveUnknownFields false

The text was updated successfully, but these errors were encountered:

pugangxa · 2020-09-03T13:34:38Z

I'm hitting the same issue, any ideas?

npiganeau · 2020-09-12T17:23:44Z

I had the same issue here. It was an issue about upgrading from v1beta1 to v1 CRD.
The spec.preserveUnknownFields was true in the cluster (probably because that was the default in v1beta1).
When I upgraded to v1, spec.preserveUnknownFields was not set in the manifest, but because it was in the cluster, I had the spec.preserveUnknownFields: Invalid value: true: must be false in order to use defaults in the schema error.

The solution was simply to manually add spec.preserveUnknownFields: false in the generated CRD manifest and apply it once.
From then, it worked again when applying controller-gen generated manifests (without spec.preserveUnknownFields set).

fejta-bot · 2020-12-11T18:08:47Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot · 2021-01-10T18:54:17Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

Update tools to newer versions. Note that despite preserveUnknownFields, this is not included in the generated CRDs due to [1]. Importing our local kustomize hack to include it here does not seem worthwhile since it is mostly an upgrade issue. [1] - kubernetes-sigs/controller-tools#476

bhiravabhatla · 2021-02-09T12:11:42Z

Any updates on this?

fejta-bot · 2021-03-11T12:17:05Z

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

k8s-ci-robot · 2021-03-11T12:17:12Z

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot · 2021-03-28T00:06:30Z

@elanv: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen
/remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

By generating v1 instead of v1beta1, kubectl explain now works. preserveUnknownFields is needed for explain to work when upgrading from CRD v1beta1 to v1 but controller-gen has a bug [1] that causes it not be included in the generated CRD yaml. Use kustomize to add it the CRDs. Keep the webhooks at their prior version until we decide to or must update them. The new controller-gen defaults to newer version. [1] - kubernetes-sigs/controller-tools#476

kevindelgado · 2021-07-22T17:13:04Z

/reopen

k8s-ci-robot · 2021-07-22T17:13:08Z

@kevindelgado: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kevindelgado · 2021-07-22T17:13:19Z

/assign

k8s-triage-robot · 2021-12-28T02:22:17Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue or PR with /reopen
Mark this issue or PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot · 2021-12-28T02:22:28Z

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue or PR with /reopen

Mark this issue or PR as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

By generating v1 instead of v1beta1, kubectl explain now works. preserveUnknownFields is needed for explain to work when upgrading from CRD v1beta1 to v1 but controller-gen has a bug [1] that causes it not be included in the generated CRD yaml. Use kustomize to add it the CRDs. Keep the webhooks at their prior version until we decide to or must update them. The new controller-gen defaults to newer version. [1] - kubernetes-sigs/controller-tools#476

derekperkins · 2022-11-07T03:46:51Z

Still hitting this problem, though I was able to manually edit the CRD to set it to false

Update tools to newer versions. Note that despite preserveUnknownFields, this is not included in the generated CRDs due to [1]. Importing our local kustomize hack to include it here does not seem worthwhile since it is mostly an upgrade issue. [1] - kubernetes-sigs/controller-tools#476

By generating v1 instead of v1beta1, kubectl explain now works. preserveUnknownFields is needed for explain to work when upgrading from CRD v1beta1 to v1 but controller-gen has a bug [1] that causes it not be included in the generated CRD yaml. Use kustomize to add it the CRDs. Keep the webhooks at their prior version until we decide to or must update them. The new controller-gen defaults to newer version. [1] - kubernetes-sigs/controller-tools#476

Explicitly set `preserveUnknownFields: false` to enable strict CRD validation and prohibit unknown fields in test environment first. Conversion of CRD from v1beta1 to v1 (see #2991) resulted in `preserveUnknownFields: true`: ``` $ kubectl --context=example get crd platformcredentialssets.zalando.org -o yaml | grep preserveUnknownFields preserveUnknownFields: true message: 'spec.preserveUnknownFields: Invalid value: true: must be false' ``` and requires explicit false value to reset. See also kubernetes-sigs/controller-tools#476 (comment) Signed-off-by: Alexander Yastrebov <alexander.yastrebov@zalando.de>

AlexanderYastrebov · 2024-04-09T16:44:56Z

Now it is trivial to implement like:

$ git --no-pager diff -- pkg/crd/gen.go
diff --git a/pkg/crd/gen.go b/pkg/crd/gen.go
index 9eb4126b..cb6dd964 100644
--- a/pkg/crd/gen.go
+++ b/pkg/crd/gen.go
@@ -100,6 +100,12 @@ func transformRemoveCRDStatus(obj map[string]interface{}) error {
        return nil
 }
 
+// transformPreserveUnknownFields ensures we spec.preserveUnknownFields=false.
+func transformPreserveUnknownFields(obj map[string]interface{}) error {
+       obj["spec"].(map[interface{}]interface{})["preserveUnknownFields"] = false
+       return nil
+}
+
 func (g Generator) Generate(ctx *genall.GenerationContext) error {
        parser := &Parser{
                Collector: ctx.Collector,
@@ -171,7 +177,7 @@ func (g Generator) Generate(ctx *genall.GenerationContext) error {
                        } else {
                                fileName = fmt.Sprintf("%s_%s.%s.yaml", crdRaw.Spec.Group, crdRaw.Spec.Names.Plural, crdVersions[i])
                        }
-                       if err := ctx.WriteYAML(fileName, headerText, []interface{}{crd}, genall.WithTransform(transformRemoveCRDStatus), genall.WithTransform(genall.TransformRemoveCreationTimestamp)); err != nil {
+                       if err := ctx.WriteYAML(fileName, headerText, []interface{}{crd}, genall.WithTransform(transformRemoveCRDStatus), genall.WithTransform(genall.TransformRemoveCreationTimestamp), genall.WithTransform(transformPreserveUnknownFields)); err != nil {
                                return err
                        }
                }

but unfortunately crd:preserveUnknownFields marker support was removed by #607

sbueringer · 2024-04-09T18:09:40Z

/remove-lifecycle rotten
/reopen

@AlexanderYastrebov Feel free to open a PR (please with a corresponding test)

k8s-ci-robot · 2024-04-09T18:09:44Z

@sbueringer: Reopened this issue.

In response to this:

/remove-lifecycle rotten
/reopen

@AlexanderYastrebov Feel free to open a PR (please with a corresponding test)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Reinstate crd:preserveUnknownFields option removed by kubernetes-sigs#607 to allow specifying the value of deprecated spec.preserveUnknownFields CRD field. This is useful for updating CRDs that were automatically converted from v1beta1 version which had true as a default value and resulted in: ``` $ kubectl get crd foo.bar -o yaml | grep preserveUnknownFields preserveUnknownFields: true message: 'spec.preserveUnknownFields: Invalid value: true: must be false' ``` Fixes kubernetes-sigs#476

Reinstate crd:preserveUnknownFields option removed by kubernetes-sigs#607 to allow specifying the value of deprecated spec.preserveUnknownFields CRD field. This is useful for updating CRDs that were automatically converted from v1beta1 version which had true as a default value and resulted in: ``` $ kubectl get crd foo.bar -o yaml | grep preserveUnknownFields preserveUnknownFields: true message: 'spec.preserveUnknownFields: Invalid value: true: must be false' ``` For kubernetes-sigs#476

AlexanderYastrebov · 2024-04-10T13:25:28Z

Feel free to open a PR (please with a corresponding test)

Ok, created #912

Reinstate crd:preserveUnknownFields option removed by kubernetes-sigs#607 to allow specifying the value of deprecated spec.preserveUnknownFields CRD field. This is useful for updating CRDs that were automatically converted from v1beta1 version which had true as a default value and resulted in: ``` $ kubectl get crd foo.bar -o yaml | grep preserveUnknownFields preserveUnknownFields: true message: 'spec.preserveUnknownFields: Invalid value: true: must be false' ``` For kubernetes-sigs#476

* crd: allow specifying spec.preserveUnknownFields Reinstate crd:preserveUnknownFields option removed by #607 to allow specifying the value of deprecated spec.preserveUnknownFields CRD field. This is useful for updating CRDs that were automatically converted from v1beta1 version which had true as a default value and resulted in: ``` $ kubectl get crd foo.bar -o yaml | grep preserveUnknownFields preserveUnknownFields: true message: 'spec.preserveUnknownFields: Invalid value: true: must be false' ``` For #476 * crd: rename marker to DeprecatedV1beta1CompatibilityPreserveUnknownFields

k8s-triage-robot · 2024-07-09T13:40:46Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

AlexanderYastrebov · 2024-07-09T14:03:08Z

@sbueringer @alvaroaleman I guess we can close this as done?

sbueringer · 2024-07-10T04:28:54Z

Yes we can, thx!

/close

k8s-ci-robot · 2024-07-10T04:28:58Z

@sbueringer: Closing this issue.

In response to this:

Yes we can, thx!

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

This was referenced Sep 3, 2020

Support Kubernetes 1.18 (Update controller-gen and crds accordingly) kserve/kserve#1077

Closed

✨ Set default value for corev1.Protocol type if not already specified #480

Merged

pugangxa mentioned this issue Oct 10, 2020

Support Kubernetes 1.18 kserve/kserve#1128

Merged

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 11, 2020

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jan 10, 2021

bryanv mentioned this issue Jan 29, 2021

Generate V1 CRDs vmware-archive/vm-operator-api#56

Merged

bhiravabhatla mentioned this issue Mar 2, 2021

Bump deprecated apiversions open-telemetry/opentelemetry-operator#188

Closed

k8s-ci-robot closed this as completed Mar 11, 2021

johscheuer mentioned this issue Mar 16, 2021

Remove workaround for apiexetensions FoundationDB/fdb-kubernetes-operator#551

Closed

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Mar 28, 2021

aiyengar2 mentioned this issue Apr 21, 2021

Fix CRD chart regression and fix CRDs for upgrades rancher/charts#1131

Merged

metacosm mentioned this issue May 7, 2021

Generated crd has a validation error fabric8io/kubernetes-client#3082

Closed

k8s-ci-robot reopened this Jul 22, 2021

rainest mentioned this issue Dec 2, 2021

Add KongIngress types for generated CRDs Kong/kubernetes-ingress-controller#1971

Merged

1 task

k8s-ci-robot closed this as completed Dec 28, 2021

kimwnasptd mentioned this issue Mar 31, 2022

Support K8s 1.22 in notebook controller kubeflow/kubeflow#6374

Merged

jala-dx mentioned this issue Jun 30, 2022

Add Rook add-on version 1.6.11 replicatedhq/kURL#3067

Merged

AlexanderYastrebov mentioned this issue Apr 9, 2024

PlatformCredentialsSet: configure spec.preserveUnknownFields=false zalando-incubator/kubernetes-on-aws#7284

Merged

k8s-ci-robot reopened this Apr 9, 2024

k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Apr 9, 2024

AlexanderYastrebov mentioned this issue Apr 10, 2024

✨ crd: allow specifying spec.preserveUnknownFields #912

Merged

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 9, 2024

k8s-ci-robot closed this as completed Jul 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

preserveUnknownFields=false doesn't work with crdVersions=v1 #476

preserveUnknownFields=false doesn't work with crdVersions=v1 #476

jonnylangefeld commented Aug 22, 2020

pugangxa commented Sep 3, 2020

npiganeau commented Sep 12, 2020

fejta-bot commented Dec 11, 2020

fejta-bot commented Jan 10, 2021

bhiravabhatla commented Feb 9, 2021

fejta-bot commented Mar 11, 2021

k8s-ci-robot commented Mar 11, 2021

k8s-ci-robot commented Mar 28, 2021

kevindelgado commented Jul 22, 2021

k8s-ci-robot commented Jul 22, 2021

kevindelgado commented Jul 22, 2021

k8s-triage-robot commented Dec 28, 2021

k8s-ci-robot commented Dec 28, 2021

derekperkins commented Nov 7, 2022 •

edited

Loading

AlexanderYastrebov commented Apr 9, 2024

sbueringer commented Apr 9, 2024

k8s-ci-robot commented Apr 9, 2024

AlexanderYastrebov commented Apr 10, 2024

k8s-triage-robot commented Jul 9, 2024

AlexanderYastrebov commented Jul 9, 2024

sbueringer commented Jul 10, 2024

k8s-ci-robot commented Jul 10, 2024

preserveUnknownFields=false doesn't work with crdVersions=v1 #476

preserveUnknownFields=false doesn't work with crdVersions=v1 #476

Comments

jonnylangefeld commented Aug 22, 2020

pugangxa commented Sep 3, 2020

npiganeau commented Sep 12, 2020

fejta-bot commented Dec 11, 2020

fejta-bot commented Jan 10, 2021

bhiravabhatla commented Feb 9, 2021

fejta-bot commented Mar 11, 2021

k8s-ci-robot commented Mar 11, 2021

k8s-ci-robot commented Mar 28, 2021

kevindelgado commented Jul 22, 2021

k8s-ci-robot commented Jul 22, 2021

kevindelgado commented Jul 22, 2021

k8s-triage-robot commented Dec 28, 2021

k8s-ci-robot commented Dec 28, 2021

derekperkins commented Nov 7, 2022 • edited Loading

AlexanderYastrebov commented Apr 9, 2024

sbueringer commented Apr 9, 2024

k8s-ci-robot commented Apr 9, 2024

AlexanderYastrebov commented Apr 10, 2024

k8s-triage-robot commented Jul 9, 2024

AlexanderYastrebov commented Jul 9, 2024

sbueringer commented Jul 10, 2024

k8s-ci-robot commented Jul 10, 2024

derekperkins commented Nov 7, 2022 •

edited

Loading