Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fn framework: Enable validation using openAPI schema for functionConfig #4467

Merged

Conversation

KnVerey
Copy link
Contributor

@KnVerey KnVerey commented Feb 9, 2022

This introduces a new interface, ValidationSchemaProvider, and a new helper, SchemaFromFunctionDefinition(gvk resid.Gvk, data string) (*spec.Schema, error). Together, they enable a function author to use OpenAPI they have written to validate their inputs instead of manually writing the rules in the Validate() hook we already have (they can be used together, as the test show).

func (a v1alpha1JavaSpringBoot) Schema() (*spec.Schema, error) {
	schema, err := framework.SchemaFromFunctionDefinition(resid.NewGvk("example.com", "v1alpha1", "JavaSpringBoot"), javaSpringBootDefinition)
	return schema, errors.WrapPrefixf(err, "parsing JavaSpringBoot schema")
}

I used k8s.io/kube-openapi/pkg/validation/spec for this (instead of say, kubeval) for two reasons: we already depend on that package, and it is what the extensions API server appears to be using.

See also kubernetes/enhancements#3220, which makes the one change required for this to work for both KRMFunctionDefinition and CustomResourceDefinition. The advantage, as illustrated in one of the tests, is that you can use a tool like Kubebuilder to generate a CRD from your type structs, and then consume it directly with these new tools to get validation. Ideally we'd provide a bespoke KRMFunctionDefinition generator some day, but this should work nicely in the meantime.

This PR also defines the KRMFunctionDefinition type proposed in the KEP, since I need it for this feature. Alternatively, I could define only the subset of fields I actually need for this PR and/or make the type private for now--LMK if you want that.

/kind feature
/triage accepted
/cc @natasha41575 @mengqiy @jeremyrickard

@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. triage/accepted Indicates an issue or PR is ready to be actively worked on. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Feb 9, 2022
@KnVerey KnVerey force-pushed the fn-cfg-openapi-validation branch from 7454a75 to 0c483db Compare February 10, 2022 00:00

// SchemaFromFunctionDefinition extracts the schema for a particular GVK from the provided KRMFunctionDefinition
// Since the relevant fields of KRMFunctionDefinition exactly match the ones in CustomResourceDefinition,
// this helper can also load CRDs (e.g. produced by KubeBuilder) transparently.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this fact useful to users?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I came up with the PR while trying to use KubeBuilder to generate the schemas for me, so maybe? It's pretty neat: you annotate your types with Kubebuilder meta, run go generate, point an embed comment to the file, then point these new helpers to the embedded schema, and boom, super easy very thorough validations for your types.

`

func (a v1alpha1JavaSpringBoot) Schema() (*spec.Schema, error) {
schema, err := framework.SchemaFromFunctionDefinition(resid.NewGvk("example.com", "v1alpha1", "JavaSpringBoot"), javaSpringBootDefinition)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to support reading from a Catalog kind?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, we can have another helper for that once Catalog is in here / somewhere. I'd like to keep that out of scope of this PR though.

// Standard KRM object metadata
yaml.ObjectMeta `yaml:"metadata,omitempty" json:"metadata,omitempty"`
// APIVersion and Kind of the object. Must be config.kubernetes.io/v1alpha1 and KRMFunctionDefinition respectively.
yaml.TypeMeta `yaml:",inline" json:",inline"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we usually have this field on the top.

// The maintainers for this version of the function, if different from the primary maintainers.
Maintainers []string `yaml:"maintainers,omitempty" json:"maintainers,omitempty"`
// The runtime information describing how to execute this function.
Runtime runtimeutil.FunctionSpec
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

json and yaml tags are missing.
And FunctionSpec is not well aligned with the KRM fn schema. We should reconcile.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely. That's part of the Catalog-in-Kustomize work though, right? We never want two versions of this that are different, so that's why I just went ahead and used what is already available, for now. (this field isn't relevant to my PR and could also be left off)

"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/kio"
"sigs.k8s.io/kustomize/kyaml/kio/filters"
"sigs.k8s.io/kustomize/kyaml/openapi"
"sigs.k8s.io/kustomize/kyaml/yaml"
jyaml "sigs.k8s.io/yaml"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be aliased k8syaml for consistency

@KnVerey KnVerey force-pushed the fn-cfg-openapi-validation branch from 0c483db to cfbcf64 Compare February 22, 2022 22:02
@KnVerey KnVerey requested a review from mengqiy February 22, 2022 22:02
@natasha41575
Copy link
Contributor

/lgtm

CI is complaining about a missing go.sum entry. I can re-LGTM when you fix that.

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 23, 2022
@KnVerey KnVerey force-pushed the fn-cfg-openapi-validation branch from cfbcf64 to c90504a Compare February 24, 2022 18:20
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 24, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: KnVerey

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@KnVerey
Copy link
Contributor Author

KnVerey commented Feb 24, 2022

I was really confused until I finally realized the missing entry was in pluginator, not kyaml 😅
Updated!

@natasha41575
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 24, 2022
@k8s-ci-robot k8s-ci-robot merged commit 6950a0d into kubernetes-sigs:master Feb 24, 2022
// this helper can also load CRDs (e.g. produced by KubeBuilder) transparently.
func SchemaFromFunctionDefinition(gvk resid.Gvk, data string) (*spec.Schema, error) {
var def KRMFunctionDefinition
// need to use sigs yaml because spec.Schema type only has json tags
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI I'm expecting this to be fixed soon kubernetes/kube-openapi#279

So we can go back and clean up the places we're using sigs yaml after that PR is merged.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's great! There will still be a few places we need to do it, to support decoding into k/k types and types using metav1 metadata. But the fewer, the better!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants