[API Proposal]: JsonSerializer.TryReadValue(ref Utf8JsonReader)

[davidfowl]

Today if the JSON passed into the JsonSerializer needs to be the fully formed, this means its basically impossible to let the caller handle buffering (which is unfortunate). This seems solvable if we add a TryReadValue which would return false if the Serializer failed to Read the full object from the Utf8JsonReader.

public static class JsonSerializer
{
    public static bool TryReadValue(ref Utf8JsonReader reader, Type returnType, out object value, JsonSerializerOptions options = null);
    public static bool TryReadValue<TValue>(ref Utf8JsonReader reader, out TValue value, JsonSerializerOptions options = null);
}

This currently wouldn't preserve the serializer state, but it would let the caller buffer an entire JSON payload from a Stream of bytes without a surrounding envelope.

[mqudsi]

I agree, this would make things easier.

this means its basically impossible to let the caller handle buffering (which is unfortunate)

I thought so too, but then I realized there was sufficient low-level state available to work around the limitations.

I actually just finished (technically it was at 3am, but who's keeping track? 😂) implementing a non-blocking, asynchronous Stream (or anything else, I guess) deserializer on top of the presently exposed System.Text.Json API surface. That was actually the easy part, the difficulty was in creating an async non-blocking equivalent to (yet better than!) the old Newtonsoft JsonTextReader. It wasn't fun (especially because Stream is ornery and ugly (why oh why can I not read from a Stream to a byte *?) and I need .NET Standard support, and System.IO.Pipelines doesn't help here at all in its latest non-preview state), but it's working with more-or-less zero copy and zero large-object allocations regardless of payload length while still allowing for caller-controlled buffering (and being called from an async context, no less).

There are some issues I ran into (which I've been filing along the way), but it's certainly doable!

[frankhaugen]

Would name this TryDeserialize()

And include a TrySerialize(), because suddenly you have something too complex or self recursive references that will cause a crash unless wrapped in try/catch. when creating a library you have no control over what is being serialized, it results in so many bug reports

[ghost]

Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.

This process is part of the experimental issue cleanup initiative we are currently trialing in a limited number of areas. Please share any feedback you might have in the linked issue.

[davidfowl]

😢

[udlose]

Sad to see this being killed off

[frankhaugen]

What is this missing to get the attention of the API reviewers? I really want "Try" -methods on the serializer

[eiriktsarpalis]

@davidfowl I realize that this is a fairly old issue, but don't the async deserialization methods address that particular use case today?

FWIW this API shape is precisely how async serialization is implemented internally, but you still need to somehow marshall the deserialization stack (this is done using the ReadState struct internally). We plan on possibly exposing this API in 7.0.0, but only on the converter level (cf. #36785)

[ghost]

This issue has been automatically marked no recent activity because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will remove no recent activity.

[davidfowl]

I'd still love a PipeReader overload on DeserializeAsync. This was a workaround for that missing

[eiriktsarpalis]

I see. We probably wouldn't be able to take a Pipes dependency on STJ, so presumably you'd need the ability to roll your own method.

We should consider offering this as part of #60904. Closing in favor of that issue.

[steveharter]

Note that PipeReader\Writer async were in the initial commit for the serializer back in 3.0. However, due to concerns it was pulled:

read
write
tests

Original issue: #28325
Older prototype: dotnet/corefxlab#2647 (review)

[davidfowl]

Bring it back 🥲

[steveharter]

Bring it back 🥲

Yeah we'd have to re-evaluate the dependency from STJ to System.IO.Pipelines. Some options:

• Take the dependency
  • We'd have to account for lack of it being in NetStandard so in-box only? If part of the package, users will have additional assembly(s) that they may not use.
  • Trimming should work fine for Blazor client and stand-alone apps.
• Add an extension assembly like STJ.PipeLines.
  • Since C# doesn't support static extension methods to add the new methods to JsonSerializer class, there would likely need to be a new class (e.g. JsonSerializerPipelines), a new extension model, or we add Serializer\Deserialize extension methods to PipeReader\Writer.
  • We'd also need a public way to call into the currently internal serializer methods that deal with internal classes like ReadState etc (we already have some other reasons to do this).