Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server Side Field Validation #2

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
e29d84f
Strict schema validation for POST and PUT
kevindelgado Aug 18, 2021
2591d7d
Add benchmarking for strict validation
kevindelgado Aug 18, 2021
f424ec6
add benchmarking to integration tests
kevindelgado Sep 14, 2021
9923290
fieldValidation returns enum, error
kevindelgado Sep 15, 2021
3cb499e
WIP start addressing PR feedback
kevindelgado Sep 21, 2021
526e0cd
Address feedback, improve integration/bench tests
kevindelgado Sep 29, 2021
0564cca
Strict Validation for CR jsonpatch
kevindelgado Aug 26, 2021
0d8d87e
determine strict serialization in the patch handler
kevindelgado Aug 30, 2021
5b82235
First pass at integration test
kevindelgado Sep 1, 2021
488bdf1
cleanup test, move to apiserver_test.go
kevindelgado Sep 2, 2021
853193f
wip, stash logging
kevindelgado Sep 9, 2021
3229e2a
fix patch crd test params
kevindelgado Oct 6, 2021
4194448
benchmark working, test is broken
kevindelgado Oct 6, 2021
e5bb865
wip debugging
kevindelgado Oct 6, 2021
8df8d69
fixed it by getting rid of preserve unknown fieds in the test
kevindelgado Oct 6, 2021
81f8e01
structural pruning now returns the pruned fields
kevindelgado Oct 6, 2021
9a2e946
clean up debug logging in customresource_handler.go
kevindelgado Oct 6, 2021
256d442
benchmark remaining merge-patch possibilities
kevindelgado Oct 6, 2021
b3e58bd
wip, stash logging
kevindelgado Sep 9, 2021
32d4bc4
wip more pritning
kevindelgado Sep 16, 2021
802ec6d
wip save pre debug cleanup
kevindelgado Sep 20, 2021
9da68d7
WIP factor deleteFromKeys, stash pre help chat
kevindelgado Sep 20, 2021
e19c591
IT WORKS
kevindelgado Sep 21, 2021
d45a665
wip, hacky test working pre cleanup
kevindelgado Sep 21, 2021
42ccfc9
WIP cleanup, move test out of apply
kevindelgado Sep 22, 2021
3a5136e
fix SMP test
kevindelgado Sep 23, 2021
a09a237
cleanup converter debug printing
kevindelgado Sep 23, 2021
e6d2249
WIP stash
kevindelgado Sep 28, 2021
904316f
cleanup debug logging
kevindelgado Oct 7, 2021
9275598
consistent test naming
kevindelgado Oct 11, 2021
f06b582
fix fieldValidation for SMP after unifying
kevindelgado Oct 11, 2021
62ead39
Add SMP benchmark test
kevindelgado Oct 12, 2021
712623e
remove converter debug logging
kevindelgado Oct 12, 2021
58fffc8
Refactor SMP tests into separate file
kevindelgado Oct 12, 2021
772b5c6
using helper breaks patch CRD test for some reason
kevindelgado Oct 12, 2021
2685b57
refactor patchCRD to properly use helper
kevindelgado Oct 12, 2021
9bb72d5
refactor fieldValidation post test
kevindelgado Oct 12, 2021
a2f0549
rename BenchmarkFieldValidationPostPut
kevindelgado Oct 12, 2021
d607258
fix panics, add commentary
kevindelgado Oct 12, 2021
d5fec06
default to strict decoder, use in create handler
kevindelgado Oct 13, 2021
c077c26
default strict decoder in update handler
kevindelgado Oct 13, 2021
68fa22c
bisect 1
kevindelgado Oct 13, 2021
acc474a
bisect 2
kevindelgado Oct 13, 2021
7a42d3a
bisect 3
kevindelgado Oct 13, 2021
5d75f7a
revert removal of strict serializer because it breaks patch CRD
kevindelgado Oct 13, 2021
15e6eb5
bisect 1
kevindelgado Oct 13, 2021
892b2c0
bisect 2
kevindelgado Oct 13, 2021
17327a5
wip, patch crd kinda working
kevindelgado Oct 14, 2021
8c67e1e
wip, checkpoint patch crd cleanup
kevindelgado Oct 14, 2021
98e0901
cleanup crd-patch
kevindelgado Oct 14, 2021
cc03962
start smp cleanup
kevindelgado Oct 14, 2021
35399e4
move fieldValidationDirective enum to runtime pkg
kevindelgado Oct 14, 2021
73eb90a
remove StrictSerializer from scope
kevindelgado Oct 14, 2021
5ff723b
refactor SMP
kevindelgado Oct 14, 2021
8339937
Add warning for POST/PUT
kevindelgado Oct 14, 2021
67ddac6
Add warning for SMP Patch
kevindelgado Oct 14, 2021
2e084cb
Add warn for PatchCRD
kevindelgado Oct 14, 2021
9e24323
cleanup comment
kevindelgado Oct 14, 2021
d57bcc9
Implement {Create,Update,Patch}Options for field validation
kevindelgado Oct 19, 2021
fb910cd
Add feature gate for FieldValidation
kevindelgado Oct 19, 2021
8bc736a
Add JSONPatch tests
kevindelgado Oct 20, 2021
d445e14
wip: debug smp
kevindelgado Oct 20, 2021
7731220
fix unstructured converter
kevindelgado Oct 21, 2021
adcd819
SMP converter, remove debug printing and add commentary
kevindelgado Oct 22, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pkg/features/kube_features.go
Original file line number Diff line number Diff line change
Expand Up @@ -904,6 +904,7 @@ var defaultKubernetesFeatureGates = map[featuregate.Feature]featuregate.FeatureS
genericfeatures.WarningHeaders: {Default: true, PreRelease: featuregate.GA, LockToDefault: true}, // remove in 1.24
genericfeatures.OpenAPIEnums: {Default: false, PreRelease: featuregate.Alpha},
genericfeatures.CustomResourceValidationExpressions: {Default: false, PreRelease: featuregate.Alpha},
genericfeatures.FieldValidation: {Default: false, PreRelease: featuregate.Alpha},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not completely sure about the name yet, what about ... StrictValidation, or both StrictFieldsValidation?

// features that enable backwards compatibility but are scheduled to be removed
// ...
HPAScaleToZero: {Default: false, PreRelease: featuregate.Alpha},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -841,12 +841,13 @@ func (r *crdHandler) getOrCreateServingInfoFor(uid types.UID, name string) (*crd

// CRDs explicitly do not support protobuf, but some objects returned by the API server do
negotiatedSerializer := unstructuredNegotiatedSerializer{
typer: typer,
creator: creator,
converter: safeConverter,
structuralSchemas: structuralSchemas,
structuralSchemaGK: kind.GroupKind(),
preserveUnknownFields: crd.Spec.PreserveUnknownFields,
typer: typer,
creator: creator,
converter: safeConverter,
structuralSchemas: structuralSchemas,
structuralSchemaGK: kind.GroupKind(),
preserveUnknownFields: crd.Spec.PreserveUnknownFields,
persistStrictDecodingErrors: true,
kevindelgado marked this conversation as resolved.
Show resolved Hide resolved
}
var standardSerializers []runtime.SerializerInfo
for _, s := range negotiatedSerializer.SupportedMediaTypes() {
Expand Down Expand Up @@ -1032,9 +1033,10 @@ type unstructuredNegotiatedSerializer struct {
creator runtime.ObjectCreater
converter runtime.ObjectConvertor

structuralSchemas map[string]*structuralschema.Structural // by version
structuralSchemaGK schema.GroupKind
preserveUnknownFields bool
structuralSchemas map[string]*structuralschema.Structural // by version
structuralSchemaGK schema.GroupKind
preserveUnknownFields bool
persistStrictDecodingErrors bool
}

func (s unstructuredNegotiatedSerializer) SupportedMediaTypes() []runtime.SerializerInfo {
Expand Down Expand Up @@ -1077,7 +1079,7 @@ func (s unstructuredNegotiatedSerializer) EncoderForVersion(encoder runtime.Enco
}

func (s unstructuredNegotiatedSerializer) DecoderToVersion(decoder runtime.Decoder, gv runtime.GroupVersioner) runtime.Decoder {
d := schemaCoercingDecoder{delegate: decoder, validator: unstructuredSchemaCoercer{structuralSchemas: s.structuralSchemas, structuralSchemaGK: s.structuralSchemaGK, preserveUnknownFields: s.preserveUnknownFields}}
d := schemaCoercingDecoder{delegate: decoder, validator: unstructuredSchemaCoercer{structuralSchemas: s.structuralSchemas, structuralSchemaGK: s.structuralSchemaGK, preserveUnknownFields: s.preserveUnknownFields, persistStrictDecodingErrors: s.persistStrictDecodingErrors}}
return versioning.NewCodec(nil, d, runtime.UnsafeObjectConvertor(Scheme), Scheme, Scheme, unstructuredDefaulter{
delegate: Scheme,
structuralSchemas: s.structuralSchemas,
Expand Down Expand Up @@ -1237,6 +1239,9 @@ func (d schemaCoercingDecoder) Decode(data []byte, defaults *schema.GroupVersion
}
if u, ok := obj.(*unstructured.Unstructured); ok {
if err := d.validator.apply(u); err != nil {
if runtime.IsStrictDecodingError(err) {
return obj, gvk, err
}
return nil, gvk, err
Comment on lines +1243 to 1245

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've never seen this type of return signature where one doesn't check for err != nil but obj != nil and I can see how it'd be confusing for people.

}
}
Expand Down Expand Up @@ -1296,9 +1301,10 @@ type unstructuredSchemaCoercer struct {
dropInvalidMetadata bool
repairGeneration bool

structuralSchemas map[string]*structuralschema.Structural
structuralSchemaGK schema.GroupKind
preserveUnknownFields bool
structuralSchemas map[string]*structuralschema.Structural
structuralSchemaGK schema.GroupKind
preserveUnknownFields bool
persistStrictDecodingErrors bool
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"persist" reads strangely... maybe returnUnknownFieldPaths?

}

func (v *unstructuredSchemaCoercer) apply(u *unstructured.Unstructured) error {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this signature is private, would it be clearer to make the list of unknown fields be returned as a separate return value?

func (v *unstructuredSchemaCoercer) apply(u *unstructured.Unstructured) (unknownFieldPaths []string, err error)

That would let the caller determine how to use the list of unknown field paths, would preserve order, and would distinguish between validation errors and unknown fields

Expand All @@ -1321,10 +1327,12 @@ func (v *unstructuredSchemaCoercer) apply(u *unstructured.Unstructured) error {
if err != nil {
return err
}

pruned := map[string]bool{}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could possibly have used an empty struct here, struct{} rather than bool, though I don't know if it matters.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also prefer var pruned map[string]... which avoids an unused map allocation

if gv.Group == v.structuralSchemaGK.Group && kind == v.structuralSchemaGK.Kind {
if !v.preserveUnknownFields {
// TODO: switch over pruning and coercing at the root to schemaobjectmeta.Coerce too
structuralpruning.Prune(u.Object, v.structuralSchemas[gv.Version], false)
// TODO: switch over pruning and coercing at the root to schemaobjectmeta.Coerce too
pruned = structuralpruning.Prune(u.Object, v.structuralSchemas[gv.Version], false)
structuraldefaulting.PruneNonNullableNullsWithoutDefaults(u.Object, v.structuralSchemas[gv.Version])
}
Comment on lines +1335 to 1337

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if we're not pruning, we can't fail on unknown fields?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that seems correct... if we're preserving fields not mentioned in the schema, we're effectively saying all field names are permissible

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, this mostly exists for compatibility reason, doesn't mean people necessarily wanted arbitrary fields? I think it's useful even as a warning for example.

if err := schemaobjectmeta.Coerce(nil, u.Object, v.structuralSchemas[gv.Version], false, v.dropInvalidMetadata); err != nil {
Expand All @@ -1348,6 +1356,17 @@ func (v *unstructuredSchemaCoercer) apply(u *unstructured.Unstructured) error {
return err
}
}
// collect all the strict decoding errors and return them
if len(pruned) > 0 && v.persistStrictDecodingErrors {
allStrictErrs := make([]error, len(pruned))
i := 0
for unknownField, _ := range pruned {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iterating over a map makes the order non-deterministic, which is not ideal

also see suggestion above about making this low-level function just return the list of unknown field paths and letting the caller turn that into warnings/messages

allStrictErrs[i] = fmt.Errorf("unknown field: %s", unknownField)
i++
}
Comment on lines +1361 to +1366

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
allStrictErrs := make([]error, len(pruned))
i := 0
for unknownField, _ := range pruned {
allStrictErrs[i] = fmt.Errorf("unknown field: %s", unknownField)
i++
}
allStrictErrs := make([]error, 0, len(pruned))
for unknownField := range pruned {
allStrictErrs = append(allStrictErrs, fmt.Errorf("unknown field: %s", unknownField))
}

err := runtime.NewStrictDecodingError(allStrictErrs)
return err
}

return nil
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ import (
// Prune removes object fields in obj which are not specified in s. It skips TypeMeta and ObjectMeta fields
// if XEmbeddedResource is set to true, or for the root if isResourceRoot=true, i.e. it does not
// prune unknown metadata fields.
func Prune(obj interface{}, s *structuralschema.Structural, isResourceRoot bool) {
// It returns the set of fields that it prunes.
func Prune(obj interface{}, s *structuralschema.Structural, isResourceRoot bool) map[string]bool {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this have to be a map, or can it be a list? What are the string? The type is not explaining a lot about what it is.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see suggestion above about making this return a list of prunedFieldPaths []string

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

if isResourceRoot {
if s == nil {
s = &structuralschema.Structural{}
Expand All @@ -34,7 +35,7 @@ func Prune(obj interface{}, s *structuralschema.Structural, isResourceRoot bool)
s = &clone
}
}
prune(obj, s)
return prune(obj, s)
}

var metaFields = map[string]bool{
Expand All @@ -43,51 +44,71 @@ var metaFields = map[string]bool{
"metadata": true,
}

func prune(x interface{}, s *structuralschema.Structural) {
func prune(x interface{}, s *structuralschema.Structural) map[string]bool {
if s != nil && s.XPreserveUnknownFields {
skipPrune(x, s)
return
return skipPrune(x, s)
}

pruning := map[string]bool{}
switch x := x.(type) {
case map[string]interface{}:
if s == nil {
for k := range x {
if !metaFields[k] {
pruning[k] = true
}
delete(x, k)
}
return
return pruning
}
for k, v := range x {
if s.XEmbeddedResource && metaFields[k] {
continue
}
prop, ok := s.Properties[k]
if ok {
prune(v, &prop)
pruned := prune(v, &prop)
for k, b := range pruned {
pruning[k] = b
}
} else if s.AdditionalProperties != nil {
prune(v, s.AdditionalProperties.Structural)
pruned := prune(v, s.AdditionalProperties.Structural)
for k, b := range pruned {
pruning[k] = b
}
} else {
if !metaFields[k] {
pruning[k] = true
}
delete(x, k)
}
}
case []interface{}:
if s == nil {
for _, v := range x {
prune(v, nil)
pruned := prune(v, nil)
for k, b := range pruned {
pruning[k] = b
}
}
return
return pruning
}
for _, v := range x {
prune(v, s.Items)
pruned := prune(v, s.Items)
for k, b := range pruned {
pruning[k] = b
}
}
default:
// scalars, do nothing
}
return pruning
}

func skipPrune(x interface{}, s *structuralschema.Structural) {
func skipPrune(x interface{}, s *structuralschema.Structural) map[string]bool {
pruning := map[string]bool{}
if s == nil {
return
return pruning
}

switch x := x.(type) {
Expand All @@ -97,16 +118,26 @@ func skipPrune(x interface{}, s *structuralschema.Structural) {
continue
}
if prop, ok := s.Properties[k]; ok {
prune(v, &prop)
pruned := prune(v, &prop)
for k, b := range pruned {
pruning[k] = b
}
} else if s.AdditionalProperties != nil {
prune(v, s.AdditionalProperties.Structural)
pruned := prune(v, s.AdditionalProperties.Structural)
for k, b := range pruned {
pruning[k] = b
}
}
}
case []interface{}:
for _, v := range x {
skipPrune(v, s.Items)
skipPruned := skipPrune(v, s.Items)
for k, b := range skipPruned {
pruning[k] = b
}
}
default:
// scalars, do nothing
}
return pruning
}
30 changes: 30 additions & 0 deletions staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -544,6 +544,16 @@ type CreateOptions struct {
// as defined by https://golang.org/pkg/unicode/#IsPrint.
// +optional
FieldManager string `json:"fieldManager,omitempty" protobuf:"bytes,3,name=fieldManager"`

// fieldValidation determines how the server should respond to
kevindelgado marked this conversation as resolved.
Show resolved Hide resolved
// unknown/duplicate fields.
// TODO: Do we still need a protobuf tag if protobuf is not supported?
kevindelgado marked this conversation as resolved.
Show resolved Hide resolved
// Valid values are:
// - Ignore: ignore's unknown/duplicate fields
// - Strict: fail the request on unknown/duplicate fields
// - Warn: respond with a warning, but successfully serve the request.
Comment on lines +552 to +554

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So have we decided that warn is to expensive to be the default (and that Ignore is unnecessary?)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest ordering these in increasing severity:

Ignore
Warn
Strict

And document what the default is on servers that support the fieldValidation option (I expected we could default to Warn)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reordered the values. Waiting on another round of feedback before defaulting to warn to confirm that is what we want to do

// +optional
FieldValidation string `json:"fieldValidation,omitempty"`
}

// +k8s:conversion-gen:explicit-from=net/url.Values
Expand Down Expand Up @@ -577,6 +587,16 @@ type PatchOptions struct {
// types (JsonPatch, MergePatch, StrategicMergePatch).
// +optional
FieldManager string `json:"fieldManager,omitempty" protobuf:"bytes,3,name=fieldManager"`

// fieldValidation determines how the server should respond to
// unknown/duplicate fields.
// TODO: Do we still need a protobuf tag if protobuf is not supported?
// Valid values are:
// - Ignore: ignore's unknown/duplicate fields
kevindelgado marked this conversation as resolved.
Show resolved Hide resolved
// - Strict: fail the request on unknown/duplicate fields
// - Warn: respond with a warning, but successfully serve the request.
// +optional
FieldValidation string `json:"fieldValidation,omitempty"`
}

// ApplyOptions may be provided when applying an API object.
Expand Down Expand Up @@ -632,6 +652,16 @@ type UpdateOptions struct {
// as defined by https://golang.org/pkg/unicode/#IsPrint.
// +optional
FieldManager string `json:"fieldManager,omitempty" protobuf:"bytes,2,name=fieldManager"`

// fieldValidation determines how the server should respond to
// unknown/duplicate fields.
// TODO: Do we still need a protobuf tag if protobuf is not supported?
// Valid values are:
// - Ignore: ignore's unknown/duplicate fields
// - Strict: fail the request on unknown/duplicate fields
// - Warn: respond with a warning, but successfully serve the request.
// +optional
FieldValidation string `json:"fieldValidation,omitempty"`
}

// Preconditions must be fulfilled before an operation (update, delete, etc.) is carried out.
Expand Down
Loading