Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: precompile cloudevent schema #471

Merged
merged 4 commits into from
Feb 15, 2022

Conversation

lance
Copy link
Member

@lance lance commented Feb 10, 2022

This commit modifies the build pipleline so that the cloudevent schema is
precompiled for runtime validation. This eliminates the need to compile the
schema at runtime, improving both performance and security.

Fixes: #423

Signed-off-by: Lance Ball lball@redhat.com

Note: There are a number of changes related to semi-dead code from spec version 0.3 that did not get removed when we dropped support for that version.

@lance lance added the type/fix A change that fixes something that is broken label Feb 10, 2022
@lance lance requested a review from a team February 10, 2022 20:03
This commit modifies the build pipleline so that the cloudevent schema is
precompiled for runtime validation. This eliminates the need to compile the
schema at runtime, improving both performance and security.

Fixes: cloudevents#423

Signed-off-by: Lance Ball <lball@redhat.com>
Signed-off-by: Lance Ball <lball@redhat.com>
@lance lance force-pushed the lance/423-precompile-schemas branch from 2235811 to 9e9c0f0 Compare February 11, 2022 13:51
@lance lance mentioned this pull request Feb 11, 2022
src/event/spec.ts Show resolved Hide resolved
if (!isValidAgainstSchemaV1(event)) {
throw new ValidationError("invalid payload", isValidAgainstSchemaV1.errors);
if (!validate(event)) {
throw new ValidationError("invalid payload", (validate as any).errors);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(validate as any) seems a bit off. Why do we need to cast to any?

I'm seeing validate as a function:

function validate(data: any, { instancePath, parentData, parentDataProperty, rootData }?: {
    instancePath?: string | undefined;
    parentData: any;
    parentDataProperty: any;
    rootData?: any;
}): boolean

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function that AJV generates has an errors object attached to it if there are errors found when validating with validate(data). It's a non-optimal API. Here you can see it in the generated validation code. I welcome suggestions on how to deal with this another way.

function validate20(data, { instancePath = "", parentData, parentDataProperty, rootData = data } = {}) {
  let vErrors = null;
  let errors = 0;
  if (errors === 0) {
    if (data && typeof data == "object" && !Array.isArray(data)) {
      let missing0;
      if (
        (data.id === undefined && (missing0 = "id")) ||
        (data.source === undefined && (missing0 = "source")) ||
        (data.specversion === undefined && (missing0 = "specversion")) ||
        (data.type === undefined && (missing0 = "type"))
      ) {
        validate20.errors = [
          {
            instancePath,
            schemaPath: "#/required",
            keyword: "required",
            params: { missingProperty: missing0 },
            message: `must have required property '${  missing0  }'`,
          },
        ];
        return false;
      }

}
},
"required": ["id", "source", "specversion", "type"],
"definitions": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the definitions need to have the suffix of def. Can you remove this suffix from the definitions?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is taken directly from the CloudEvent specification here: https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/formats/cloudevents.json. I would prefer it if we can use this document unchanged.

src/schema/v1.js Outdated Show resolved Hide resolved
Copy link
Member

@grant grant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to request a change at least for src/schema/v1.js to not be checked in, or to not be generated.

test/integration/constants_test.ts Outdated Show resolved Hide resolved
test/integration/message_test.ts Outdated Show resolved Hide resolved
src/schema/v1.js Outdated Show resolved Hide resolved
Signed-off-by: Lance Ball <lball@redhat.com>
Signed-off-by: Lance Ball <lball@redhat.com>
@lance
Copy link
Member Author

lance commented Feb 14, 2022

@grant requested changes have been made - ptal

Copy link
Member

@grant grant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@lance lance merged commit b13bde9 into cloudevents:main Feb 15, 2022
@lance lance deleted the lance/423-precompile-schemas branch February 15, 2022 19:06
@lance lance mentioned this pull request Jun 21, 2022
mcdurdin added a commit to keymanapp/keyman that referenced this pull request Oct 5, 2023
ajv seems to have some bugs around precompiling and formats, particularly
when using ESM:

* ajv-validator/ajv#1837
* ajv-validator/ajv-cli#200

Workaround for non-esm works for cloudevents/sdk-javascript#471

So if we want to work with this, we should look at how that is done, but
it is taking too many cycles to pursue for now.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/fix A change that fixes something that is broken
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AJV schema validation functions should be pre-compiled to avoid unsafe-eval CSP requirement
3 participants