-
Notifications
You must be signed in to change notification settings - Fork 502
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix CEL tests for v1.28+ #3325
Fix CEL tests for v1.28+ #3325
Conversation
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: robscott The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
// Starting in k8s v1.28, CEL error messages stopped adding spec and status prefixes to path names | ||
wantLAdjusted := strings.ReplaceAll(wantL, "spec.", "") | ||
wantLAdjusted = strings.ReplaceAll(wantLAdjusted, "status.", "") | ||
|
||
// Enum validation messages changed in k8s v1.28: | ||
// Before: must be one of ['Exact', 'PathPrefix', 'RegularExpression'] | ||
// After: supported values: "Exact", "PathPrefix", "RegularExpression" | ||
if strings.Contains(wantLAdjusted, "must be one of") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jpbetz do you know of any other changes on the horizon for CEL error messages? Any better ideas for how we can make tests like this both support multiple versions and ensure that we're returning useful error messages to users? As long as this set of workarounds remains small, I think this is ~OK, but would hate to turn this into a massive function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not aware of any on the horizon.. we're trying to apply a light touch to these things as we do understand Hyrum's law is in full effect. But we'll keep a close eye out.
cc @TristonianJones @cici37 for visibility
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those errors don't look like they're from CEL itself, but perhaps would be from some layer up the stack within K8s maybe?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the quick responses! Definitely get that this is Hyrum's law in practice, just wondering if there's anywhere we can easily track potential upstream changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I looked more closely.
I don't believe the enum error is from CEL specifically. Although it is from openapi so still under api-machinery. This might be more durably matched by checking the error type, which consistently be FieldValueNotSupported
. We do sometimes change error strings, but the error types are fixed.
The removal of status and spec prefixes on path names is surprising, and does not look correct. @cici37 would you look more closely at this issue?
e34b3f8
to
34d6243
Compare
34d6243
to
1277007
Compare
/retest |
Thanks a lot @robscott for taking this on! /lgtm |
/cherrypick release-1.1 |
@mlavacca: #3325 failed to apply on top of branch "release-1.1":
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
* Fix CEL tests for v1.28+ (#3325) * feat: validate CRDs on specific k8s versions (#3316) Signed-off-by: Mattia Lavacca <lavacca.mattia@gmail.com> * finalize verify->crds-validation migration (#3330) Signed-off-by: Mattia Lavacca <lavacca.mattia@gmail.com> * Fixing kind v1.30 image ref (#3329) --------- Signed-off-by: Mattia Lavacca <lavacca.mattia@gmail.com> Co-authored-by: Rob Scott <robertjscott@google.com>
What type of PR is this?
/kind cleanup
/kind failing-test
What this PR does / why we need it:
As @mlavacca discovered in #3316, our CEL tests were failing in v1.28+. Fortunately it was just a variety of changes to how error messages are presented to users. I've also tested these changes against v1.27 and v1.29.
Does this PR introduce a user-facing change?:
/cc @gauravkghildiyal @jpbetz