chore: moving k8s-native-validation feature to beta #3476

JaydipGabani · 2024-08-07T20:57:50Z

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #

Special notes for your reviewer:

JaydipGabani · 2024-08-07T21:00:40Z

website/docs/validating-admission-policy.md

-### Option 2: Install with Gatekeeper deployment
-Edit the applicable deployments (`controller-manager` and `audit`), and update the following commandline flags:
- Set `--experimental-enable-k8s-native-validation=true`
-


Since we are defaulting to true I removed this doc section. However, we should add experimental-enable-k8s-native-validation is beta and is turned on by default somewhere in the docs.

@ritazh @maxsmythe @sozercan any suggestion on place on the doc for this^^?

Added a note above in the same file, here - https://github.com/open-policy-agent/gatekeeper/pull/3476/files#diff-6f51397980ac58fff52f433210aa369e522c495bdd049d01ae538eaa3b9118b1R43

you should mention the flag is beta, default to true, set to false to opt-out

codecov-commenter · 2024-08-07T21:12:33Z

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.

Project coverage is 48.07%. Comparing base (3350319) to head (a533a7a).
Report is 119 commits behind head on master.

Files	Patch %	Lines
cmd/gator/verify/verify.go	0.00%	2 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (3350319) and HEAD (a533a7a). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (3350319) HEAD (a533a7a)

unittests 2 1

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3476      +/-   ##
==========================================
- Coverage   54.49%   48.07%   -6.42%     
==========================================
  Files         134      219      +85     
  Lines       12329    15165    +2836     
==========================================
+ Hits         6719     7291     +572     
- Misses       5116     7059    +1943     
- Partials      494      815     +321

Flag	Coverage Δ
unittests	`48.07% <33.33%> (-6.42%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

sozercan · 2024-08-07T23:06:58Z

cmd/gator/verify/verify.go

@@ -47,8 +47,8 @@ func init() {
 		`print extended test output`)
 	Cmd.Flags().BoolVarP(&includeTrace, "trace", "t", false,
 		`include a trace for the underlying constraint framework evaluation`)
-	Cmd.Flags().BoolVarP(&flagEnableK8sCel, "experimental-enable-k8s-native-validation", "", false,
-		`PROTOTYPE (not stable): enable the validating admission policy driver`)
+	Cmd.Flags().BoolVarP(&flagEnableK8sCel, "experimental-enable-k8s-native-validation", "", true,


we can't take any action now, but we should not call flags experimental in the name again

Should we not? "experimental" sends the message "this may break/go away, do not rely on it", I see no undesireable user impact here. If anything, this means users who were using an experiment are put on notice that its behavior is changing and they should double-check that wont be a problem for them.

Are there impacts other than backwards compatibility? Because, again, backwards compatibility is explicitly not a concern for experimental/alpha things.

Also, given this is no longer experimental, we should change the flag name.

Changing or removing flags will cause the pod to crash during in place upgrade. That's not a good experience even if the feature is in alpha. A feature flag can be used in the future even after the feature is stable to allow users to opt-out. Therefore, starting a flag with "experiment" is odd if it can be used in the future after the feature is stable.

My point is that the flag cannot be used in the future, and that is fine. That is why we used "experimental", we should feel free to change the flag name.

One option can be to leave the old flag but render it useless. This would avoid crashing. The flag should still go away eventually.

However, I would still like to understand how/why a Helm chart cannot handle a binary changing its flags.

Concerns about UX due to breaking changes are specifically excluded for experimental/alpha features.

This is only true if we plan to never allow users to disable a new feature that is in development. If a new feature can be disabled AFTER it is stable, then it should NOT change or have "experimental" in the flag while it is alpha/beta. Do we not want users to ever disable this feature AFTER it is stable? If so, then we just need to deprecate the flag and return warnings for few releases before removing the flag completely. But if we do plan to allow users to disable a feature, then IMO it should NOT start with "experimental".

I refer you to the K8s API definitions for alpha/beta. They have a policy for flags that say alpha flags can be removed immediately:

https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecating-a-flag-or-cli

I don't think it's reasonable to hold ourselves to a higher standard than K8s itself.

I also return to the meaning of "experimental"/"alpha" which is:

"The developers are still working on this feature. They make no claim to have any idea as to its final behavior or how it will be configured. Breaking changes are expected"

This isn't about whether we want to continue with a feature gate or not, it's that, at the time of implementation, we were not sure whether/how the feature should work or be gated and wanted to warn the user so that they knew breaking changes are expected.

Whether we want to include "experimental" in a flag name in the future or not... I don't really care much either way. So long as users are put on notice that the API/flag is unstable, the core goal is accomplished.

As to the core question of whether its okay to rename a flag that is in alpha, the answer simply has to be yes. For any reason. We could decide a different name more fully communicates the flag's significance, the underlying behavior the flag controls could change, we could simply not like the name.

Beta is when backwards compatibility becomes a concern, and that is why we want to be careful before graduating a feature to beta, and that is why the flag name needs to change now, because we will not be able to change it in the future -- because we will be held to the requirements of being backwards compatible.

I agree if we need to rename a flag before beta it makes sense. i just want to avoid having to rename when we go to beta because we started with experimental.

I think both of our concerns can be addressed with:

return a warning for experimental and deprecating flags

try not to name flags with experimental to avoid having to change it later

If everyone agrees, I suggest renaming this flag to enable-k8s-native-validation since we are moving to beta.

Makes sense to me.

sozercan · 2024-08-07T23:08:36Z

website/docs/validating-admission-policy.md

@@ -40,6 +40,10 @@ In summary, these are potential options when running Gatekeeper:

 Find out more about different [enforcement points](enforcement-points.md)

+:::note
+CEL validation is in beta and `--experimental-enable-k8s-native-validation` is turned on by default.


do we want to say something like "CEL policy validation through Gatekeeper and VAP generation is in ..."

sozercan

minor suggestion, otherwise LGTM. Thanks!

ritazh · 2024-08-07T23:48:41Z

cmd/build/helmify/static/README.md

@@ -166,7 +166,7 @@ information._
 | mutatingWebhookURL                                         | Custom URL for Kubernetes API server to use to reach the mutating webhook pod. If not set, the default of connecting via the kubernetes service endpoint is used.                                                                                                                              | `null`                                                                                                                                                                |
 | emitAdmissionEvents                                        | Emit K8s events in configurable namespace for admission violations (alpha feature)                                                                                                                                                                                                             | `false`                                                                                                                                                               |
 | emitAuditEvents                                            | Emit K8s events in configurable namespace for audit violations (alpha feature)                                                                                                                                                                                                                 | `false`                                                                                                                                                               |
-| enableK8sNativeValidation                                  | Enable the K8s Native Validating driver to allow constraint templates to use rules written in VAP-style CEL (alpha feature)                                                                                                                                                                                                                 | `false`                                                                                                                                                               |
+| enableK8sNativeValidation                                  | Enable the K8s Native Validating driver to allow constraint templates to use rules written in VAP-style CEL (beta feature)                                                                                                                                                                                                                 | `false`                                                                                                                                                               |


default should be true

website/docs/validating-admission-policy.md

…it on by default Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

… on by default Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

ritazh · 2024-08-08T16:21:37Z

website/docs/validating-admission-policy.md

@@ -40,6 +40,10 @@ In summary, these are potential options when running Gatekeeper:

 Find out more about different [enforcement points](enforcement-points.md)

+:::note
+CEL validation in constraint templates through Gatekeeper is in beta and `--experimental-enable-k8s-native-validation` is turned on by default. Set --experimental-enable-k8s-native-validation=false` to disable evaluating CEL in constraint templates.


https://github.com/open-policy-agent/gatekeeper/pull/3476/files#diff-6f51397980ac58fff52f433210aa369e522c495bdd049d01ae538eaa3b9118b1R6-R8 should be updated too for 3.17 and this text can be added there instead of here.

Feature State: Gatekeeper version v3.17 (beta)
❗ This feature is beta, subject to change (feedback is welcome!). It is enabled by default. Set --experimental-enable-k8s-native-validation=false` to disable evaluating CEL in constraint templates.

ValidatingAdmissionPolicy is different than CEL validation in Gatekeeper, wouldn't updating in the section you mentioned be missleading? It might seem like we moved VAP/VAPB generation to beta. We probably need to distinguish between generation and CEL validation. How about the below?

VAP management through Gatekeeper:
Feature State: Gatekeeper version v3.16 (alpha)
❗ This feature is alpha, subject to change (feedback is welcome!). It is enabled by default.

CEL validation in Gatekeeper:
Feature State: Gatekeeper version v3.17 (beta)
❗ This feature is beta, subject to change (feedback is welcome!). It is enabled by default. Set --experimental-enable-k8s-native-validation=false` to disable evaluating CEL in constraint templates.

CEL validation in Gatekeeper:
Feature State: Gatekeeper version v3.17 (beta)
❗ This feature is beta, subject to change (feedback is welcome!). It is enabled by default. Set --experimental-enable-k8s-native-validation=false` to disable evaluating CEL in constraint templates.

VAP management through Gatekeeper:
Feature State: Gatekeeper version v3.16 (alpha)
❗ This feature is alpha, subject to change (feedback is welcome!). It is disabled by default unless explicitly enabled via feature flag and/or via constraint template.

Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

ritazh

LGTM

…ve-validation Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

maxsmythe

LGTM

ritazh

LGTM

JaydipGabani requested a review from a team as a code owner August 7, 2024 20:57

JaydipGabani changed the title ~~chore: moving experimental-enable-k8s-native-validation to beta and turning …~~ chore: moving experimental-enable-k8s-native-validation to beta Aug 7, 2024

JaydipGabani commented Aug 7, 2024

View reviewed changes

sozercan reviewed Aug 7, 2024

View reviewed changes

sozercan approved these changes Aug 7, 2024

View reviewed changes

ritazh reviewed Aug 7, 2024

View reviewed changes

website/docs/validating-admission-policy.md Show resolved Hide resolved

JaydipGabani added 5 commits August 8, 2024 02:07

moving experimental-enable-k8s-native-validation to beta and turning …

3c0e5fd

…it on by default Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

updating makefile to remove enableK8sNativeValidation as it is turned…

b62e900

… on by default Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

adding note for moving CEL validation to beta in docs

9597954

Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

addressing doc updates

ed5ea6c

Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

fixing rebase

6f0cb45

Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

JaydipGabani force-pushed the cel-validation-beta branch from 8e83a51 to 6f0cb45 Compare August 8, 2024 02:10

fixing rebase

7bf8f8d

Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

JaydipGabani requested a review from ritazh August 8, 2024 02:12

JaydipGabani force-pushed the cel-validation-beta branch from 7d67ac1 to 7bf8f8d Compare August 8, 2024 02:15

ritazh reviewed Aug 8, 2024

View reviewed changes

JaydipGabani requested a review from ritazh August 8, 2024 18:52

updating docs

7868066

Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

JaydipGabani mentioned this pull request Aug 9, 2024

chore: Removing setting alpha flags for vap/vapb generation unless explicitly set through helm #3481

Merged

ritazh approved these changes Aug 9, 2024

View reviewed changes

JaydipGabani requested a review from maxsmythe August 9, 2024 00:43

ritazh changed the title ~~chore: moving experimental-enable-k8s-native-validation to beta~~ chore: moving enable-k8s-native-validation to beta Aug 9, 2024

ritazh changed the title ~~chore: moving enable-k8s-native-validation to beta~~ chore: moving k8s-native-validation feature to beta Aug 9, 2024

renaming experimental-enable-k8s-native-validation -> enable-k8s-nati…

a533a7a

…ve-validation Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>

maxsmythe approved these changes Aug 9, 2024

View reviewed changes

ritazh approved these changes Aug 9, 2024

View reviewed changes

Merge branch 'master' into cel-validation-beta

77a685b

sozercan merged commit c5ee94e into open-policy-agent:master Aug 9, 2024
8 checks passed

JaydipGabani mentioned this pull request Aug 9, 2024

Log alpha status for existing alpha features/flags when in use #3485

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: moving k8s-native-validation feature to beta #3476

chore: moving k8s-native-validation feature to beta #3476

JaydipGabani commented Aug 7, 2024 •

edited

Loading

JaydipGabani Aug 7, 2024

JaydipGabani Aug 7, 2024

ritazh Aug 7, 2024

codecov-commenter commented Aug 7, 2024 •

edited

Loading

sozercan Aug 7, 2024 •

edited

Loading

maxsmythe Aug 8, 2024

maxsmythe Aug 8, 2024

ritazh Aug 8, 2024

maxsmythe Aug 8, 2024

maxsmythe Aug 8, 2024

ritazh Aug 9, 2024 •

edited

Loading

maxsmythe Aug 9, 2024

ritazh Aug 9, 2024 •

edited

Loading

maxsmythe Aug 9, 2024

sozercan Aug 7, 2024

sozercan left a comment

ritazh Aug 7, 2024

ritazh Aug 8, 2024 •

edited

Loading

JaydipGabani Aug 8, 2024

ritazh Aug 8, 2024

ritazh left a comment

maxsmythe left a comment

ritazh left a comment

chore: moving k8s-native-validation feature to beta #3476

chore: moving k8s-native-validation feature to beta #3476

Conversation

JaydipGabani commented Aug 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Aug 7, 2024 • edited Loading

Codecov Report

sozercan Aug 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ritazh Aug 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ritazh Aug 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sozercan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ritazh Aug 8, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ritazh left a comment

Choose a reason for hiding this comment

maxsmythe left a comment

Choose a reason for hiding this comment

ritazh left a comment

Choose a reason for hiding this comment

JaydipGabani commented Aug 7, 2024 •

edited

Loading

codecov-commenter commented Aug 7, 2024 •

edited

Loading

sozercan Aug 7, 2024 •

edited

Loading

ritazh Aug 9, 2024 •

edited

Loading

ritazh Aug 9, 2024 •

edited

Loading

ritazh Aug 8, 2024 •

edited

Loading