Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(GH-96) Replace jsonschema crate with boon #154

Conversation

michaeltlombardi
Copy link
Collaborator

PR Summary

Prior to this change, the DSC library, CLI, and the Registry resource all relied on the jsonschema crate for schema compilation and validation.

As raised in #96, the jsonschema crate does not support newer versions of JSON Schema like drafts 2019-09 and 2020-12. Further, the implementation fails several of the tests designed by the JSON Schema organization to validate implementations for the newest dialect it does support (Draft 7).

This change prototypes the replacement of the jsonschema crate with the boon crate, which has full support for every released draft of JSON Schema and currently passes every test for implementation validation.

PR Context

In this initial prototype change, the code includes duplicated behaviors that can be cleaned up or abstracted into helper functions.

Another benefit and possible avenue going forward is to turn the decomposed schemas in the repository into the source of truth for validating the incoming data. With Boon, DSC could pre-load the schemas and only have to compile schemas for resources on-demand thereafter.

In that model, the JSON schemas in the repository could be embedded in the binary or shipped alongside it.

This change also updates the use of Schemars to output Draft 2019-09 schemas for DSC and the Registry resource, to make them closer to the definitions for the schemas in the repository.

This change adds documentation to the structs in the Registry resource, showing how they can be used to provide improved documentation for the schema at author-time.

Prior to this change, the DSC library, CLI, and the
Registry resource all relied on the `jsonschema` crate
for schema compilation and validation.

As raised in PowerShell#96, the `jsonschema` crate does not
support newer versions of JSON Schema like drafts
2019-09 and 2020-12. Further, the implementation
fails several of the tests designed by the JSON
Schema organization to validate implementations
for the newest dialect  it _does_ support (Draft 7).

This change prototypes the replacement of the
`jsonschema` crate with the `boon` crate, which
has full support for every released draft of
JSON Schema and currently passes every test
for implementation validation.

In this initial prototype change, the code
includes duplicated behaviors that can be
cleaned up or abstracted into helper
functions.

Another benefit and possible avenue going
forward is to turn the decomposed schemas
in the repository into the source of truth
for validating the incoming data. With
Boon, DSC could pre-load the schemas and
only have to compile schemas for resources
on-demand thereafter.

In that model, the JSON schemas in the
repository could be embedded in the binary
or shipped alongside it.

This change also updates the use of
Schemars to output Draft 2019-09 schemas
for DSC and the Registry resource, to
make them closer to the definitions for
the schemas in the repository.

This commit adds documentation to the
structs in the Registry resource, showing
how they can be used to provide improved
documentation for the schema at author-time.
@michaeltlombardi michaeltlombardi force-pushed the gh-96/main/schema-validation-and-docs branch from 5eb83ee to cde9eb5 Compare August 10, 2023 22:18
@SteveL-MSFT
Copy link
Member

It looks like jsonschema does (partially) support newer versions of jsonschema, but needs some features enabled in the cargo.toml file (see https://github.com/Stranger6667/jsonschema-rs). The concern I have moving to boon is that it appears to be a less active repo than jsonschema.

Copy link
Member

@SteveL-MSFT SteveL-MSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test if enabling the feature flags for jsonschema crate is sufficient

@michaeltlombardi
Copy link
Collaborator Author

The jsonschema maintainers are tracking progress for draft 2019 and 2020 support in these issues:

The following keywords aren't implemented:

  • $anchor
  • $recursiveRef (2019, replaced by $dynamicRef in 2020)
  • $recursiveAnchor (2019, replaced by $dynamicAnchor in 2020)
  • unevaluatedItems
  • contentSchema
  • deprecated

Regarding repository activity, I do generally prefer projects that are more active. But in this case, the recent activity I see for jsonschema is around making the library 2019+ compliant, whereas boon is quieter but fully compliant with the test suite maintained by the JSON schema maintainers.

I'll investigate using the crate feature flags to see if that is sufficient for our current use case, but it's worth noting that we can't support any resource instance schemas that use unsupported keywords. Of the not-implemented keywords, I think the only ones I expect to see in instance schemas are deprecated and unevaluatedItems, though contentSchema is possible.

Neither implementation seems to have full support for (non-annotation) custom keywords, which is okay, I think. Most of the keywords I've been able to think of as useful for DSC are annotation-type keywords.

@SteveL-MSFT
Copy link
Member

Closing for now, we can revisit later

@Stranger6667
Copy link

Stranger6667 commented Sep 29, 2024

Folks, FYI, in jsonschema==0.21.0 only unevaluatedItems & vocabulary are missing (other non-optional tests from the JSON Schema test suite depend on unevaluatedItems). I am going to implement unevaluatedItems in the next few releases. Thank you for your patience.

UPDATE 2024-10-24: jsonschema==0.25.0 is 100% compatible with Draft 4, 6, 7, 2019-09, and 2020-12 (bowtie builds are not yet updated)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants