Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for schema validators #2829

Merged
merged 3 commits into from
Jul 19, 2022

Conversation

kevin-bates
Copy link
Member

This pull request adds support for schema validators after learning that the default validate() method in jsonschema can leak memory when called repeatedly. (See #2826 for details)

Aside from using the Draft7Validator by default, we also expose the ability for SchemasProvider implementors to bring their own validators as part of their provider.

Rather than create a validator instance whenever validation is required (as was done in the results for #2826), the SchemaManager captures the validator's instance upon loading the SchemasProvider. By doing this, the overall test time went from 16 seconds to 2 seconds, thereby gaining an overall 50X performance improvement over using the default validate() method done originally.

Resolves: #2826

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

@kevin-bates kevin-bates added kind:enhancement New feature or request component:metadata metadata runtime area:back-end labels Jul 11, 2022
@elyra-bot
Copy link

elyra-bot bot commented Jul 11, 2022

Thanks for making a pull request to Elyra!

To try out this branch on binder, follow this link: Binder

elyra/metadata/manager.py Fixed Show fixed Hide fixed
@kiersten-stokes kiersten-stokes self-requested a review July 12, 2022 14:25
@ptitzler ptitzler added this to the 3.11.0 milestone Jul 12, 2022
Copy link
Member

@kiersten-stokes kiersten-stokes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea to allow custom validators - everything is working well for me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:back-end component:metadata metadata runtime kind:enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate possible memory leak issue using jsonschema.validate()
3 participants