[System.Text.Json] Json Schemas Support

[shaggygi]

Understanding this feature is not coming with .NET Core v3, are there current plans to begin reviewing/development soon after?

Good job on initial work on new API 👍

Any feedback is appreciated.

[ahsonkhan]

It is on our road-map for the future: https://github.com/dotnet/corefx/blob/master/src/System.Text.Json/roadmap/README.md#out-of-current-scope--future-considerations

[shmuelie]

Anything more concrete than future? Trying to decide on library usage

[ahsonkhan]

The JSON schema spec is not stable enough and iterating quite frequently for us to add support yet (and a feature like this would require a lot more design/thinking - draft-08 now). Hence, I think this has to remain as future for now (which is to say that it isn't planned for 5.0).

http://json-schema.org/

[shmuelie]

Reasonable. Sticking with JSON.NET for now then.

[andiemmadavieswilcox]

@ahsonkhan: “The JSON schema spec is not stable enough and iterating quite frequently for us to add support yet … ( … draft-08 now). …”

Would adding draft-07 support not still be a valid addition though, given a lot of the implementation could then be taken forward when draft-08 is stabilised, and at that point in time the majority of existing software that validates against a JSON schema will be doing so using draft-07 and older? I'm not contesting the decision by the way, just curious about how the roadmap items get prioritised 🙂

[andiemmadavieswilcox]

@SamuelEnglard : “Reasonable. Sticking with JSON.NET for now then.”

It's worth noting that Json.NET's schema validation functionality has been pulled-out of the main package and is now its own thing, which is no longer free. I found this out when I was recently working on some new validation stuff, and was a dealbreaker for what I was developing.

Fortunately, I stumbled across an alternative free library, called Manatee.Json, which provides much the same JSON schema validation functionality as Json.NET Schema; I'd highly recommend it based on my recent experiences with it, if free is something that's important to you or your project(s):

https://www.github.com/gregsdennis/Manatee.Json

[shmuelie]

It's worth noting that Json.NET's schema validation functionality has been pulled-out of the main package and is now its own thing, which is no longer free. I found this out when I was recently working on some new validation stuff, and was a dealbreaker for what I was developing.

Fortunately, I stumbled across an alternative free library, called Manatee.Json, which provides much the same JSON schema validation functionality as Json.NET Schema; I'd highly recommend it based on my recent experiences with it, if free is something that's important to you or your project(s):

Currently in a debate over if we can or cannot use Newtonsoft.Json.Schema and funny enough Manatee.Json was the suggested alternative!

Another alternative I found was JSchema. I liked that it does integrate nicely with JSON.NET but the API's lack of Stream support was a deal breaker for me.

[andiemmadavieswilcox]

@SamuelEnglard, ah no way, haha! Didn't know there was a current Microsoft solution for JSON schema stuff either. For me, the dealbreaker would be that it's only for draft-04 and nothing newer.

[shmuelie]

@andidavies92 as a quick update: end of the debate is we're using Newtonsoft.Json for creating the JSON and Manatee.Json for validation (not the best but time isn't with me)

[andiemmadavieswilcox]

Nice, I did the same, but mostly because .NET Core 3 wasn't in GA when I wanted to start work on it. Plan to migrate the Json.NET stuff to .NET Core when I get the chance, as I now have unit and integration tests to check for regressions and performance tests to see whether I actually get much boosted performance.

[wasabii]

Look at Cogito.Json.Schema.Validation. it's a open source valuator. It's 190 times faster than Newtonsofts. I'm the author. I've got it in my head to fork a second library for System.Text.Json.

It would be pretty easy. If anybody wanted to help.

[wasabii]

Update: I began porting Cogito.Json.Schema.Validation to run on top of System.Text.Json. I have most of the basics working. Need to implement a few things like $ref support, and some per-schema version nuances.

But it's a good start, and validates quite a bit. For those of you who want to validate System.Text.Json streams, Cogito.Text.Json.Schema should get you pretty far. Pull requests for missing stuff accepted, of course. There's an alpha package on NuGet.

This validator is quite different from Newtonsofts. And quite faster. It actually compiles an Expression tree that implements the validation for a given Schema instance. And then you can execute the expression tree against a JsonElement to return pass or fail.

@andidavies92 @SamuelEnglard

[shmuelie]

@wasabii looks interesting, though time says we're sticking with what we have for now.

[Felk]

@ahsonkhan

It is on our road-map for the future: https://github.com/dotnet/corefx/blob/master/src/System.Text.Json/roadmap/README.md#out-of-current-scope--future-considerations

Unfortunately that link is dead. Is there another link where people can track this? Or will any updates be posted here and subscribing to this issue should be good?

[shmuelie]

@Felk new link: https://github.com/dotnet/runtime/tree/master/src/libraries/System.Text.Json/roadmap#out-of-current-scope--future-considerations

[martimors]

This is my only reason for still using Json.NET. Would be a much appreciated feature indeed!

[gregsdennis]

I just released RC2 for JsonSchema.Net (on Nuget now). It has full support for drafts 6 through 2019-09 and is built 100% on System.Text.Json. (Working on streaming support. Stream supported as of 1.0.0.)

@andidavies92, thanks for the shout-out on Manatee.Json, but this one is so much faster.... and still free.

[ahdung]

Any news?

[gregsdennis]

Since my previous post, JsonSchema.Net has become a (if not the) premier package for JSON Schema support in STJ.

It supports draft 6 through 2020-12, and I intend to keep it supporting the latest version.

There were some significant changes between drafts 4 and 6, which is why I don't support 4. Additionally, given that draft 4 was released almost 9 years ago (6 was released 5 years ago), I would hope that any existing schemas that need to be used in new code will be updated.

[gregsdennis]

I have just released v4.1.0 of JsonSchema.Net that adds validation during deserialization via a custom converter (factory) and a [JsonSchema] attribute.

Basically, you expose a JsonSchema instance through a public static property and use the attribute on the type it describes to point to that schema instance. Then when you use the ValidatingJsonConverter in the serializer options, it will use that schema to validate the payload before deserializing it. If the validation fails, a JsonException is thrown with the JSON Schema output in the exception's .Data dictionary under the key "validation".

Read the docs for more info.

Maybe this will help people here until official support is again pushed out to .NET 9.

[vhatuncev]

I have just released v4.1.0 of JsonSchema.Net that adds validation during deserialization via a custom converter (factory) and a [JsonSchema] attribute.

Basically, you expose a JsonSchema instance through a public static property and use the attribute on the type it describes to point to that schema instance. Then when you use the ValidatingJsonConverter in the serializer options, it will use that schema to validate the payload before deserializing it. If the validation fails, a JsonException is thrown with the JSON Schema output in the exception's .Data dictionary under the key "validation".

Read the docs for more info.

Maybe this will help people here until official support is again pushed out to .NET 9.

Just thinking, does it makes sense to throw an exception in case of validation failed? Valid payload or not looks like a usual situation and nothing exceptinal here. Also it does affect how the client code will be structured, with exception approach you need to wrap validation call with try catch block. In case of exception you getting some additional, minor but additional work to do for the runtime. Maybe IsValid() method will look better here?

[gregsdennis]

does it makes sense to throw an exception in case of validation failed?

Why would it make sense to not throw an exception? Currently, if the serializer detects a required property that's not fulfilled in the data, it throws an exception. This converter is just doing the same.

Maybe IsValid() method will look better here?

You can forego the schema-validating converter I built and manually validate the data separately before deserializing it. (That's actually the primary purpose of the library.) Then you get the validation results without an exception. Once you know it's valid, then you can deserialize it.

My blog post on adding the converter goes into a bit more detail on the mechanics.