-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose instance methods in JsonSerializer #74492
Comments
Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis Issue DetailsBackground & MotivationThe current design of the
API ProposalThis issue proposes we expose instance methods in namespace System.Text.Json;
public partial class JsonSerializer // Changed from static class
{
public JsonSerializerOptions Options { get; }
public JsonSerializer(JsonSerializerOptions options); // linker-safe constructor, throws if options.TypeInfoResolver == null;
// we might consider adding a linker-unsafe factory that populates TypeInfoResolver with the reflection resolver, like the existing serialization APIs do.
[RequiresUnreferencedCode]
public static JsonSerializer Default { get; } // serializer wrapping JsonSerializerOptions.Default
/* Serialization APIs */
public string Serialize<TValue>(TValue value);
public byte[] SerializeToUtf8Bytes<TValue>(TValue value);
public void Serialize<TValue>(Utf8JsonWriter utf8Json, TValue value);
public void Serialize<TValue>(Stream utf8Json, TValue value);
public void SerializeAsync<TValue>(Stream utf8Json, TValue value);
public JsonDocument SerializeToDocument<TValue>(TValue value);
public JsonElement SerializeToElement<TValue>(TValue value);
public JsonNode SerializeToNode<TValue>(TValue value);
public string Serialize(object? value, Type inputType);
public byte[] SerializeToUtf8Bytes(object? value, Type inputType);
public void Serialize(Utf8JsonWriter writer, object? value, Type inputType);
public void Serialize(Stream utf8Json, object? value, Type inputType);
public void SerializeAsync(Stream utf8Json, object? value, Type inputType);
public JsonDocument SerializeToDocument(object? value, Type inputType);
public JsonElement SerializeToElement(object? value, Type inputType);
public JsonNode SerializeToNode(object? value, Type inputType);
/* Deserialization APIs */
public TValue? Deserialize<TValue>(string json);
public TValue? Deserialize<TValue>(ReadOnlySpan<char> json);
public TValue? Deserialize<TValue>(ReadOnlySpan<byte> utf8Json);
public TValue? Deserialize<TValue>(ref Utf8JsonReader reader);
public TValue? Deserialize<TValue>(Stream utf8Json);
public ValueTask<TValue?> DeserializeAsync<TValue>(Stream utf8Json);
public IAsyncEnumerable<TValue?> DeserializeAsyncEnumerable<TValue>(Stream utf8Json);
public TValue? Deserialize<TValue>(JsonDocument document);
public TValue? Deserialize<TValue>(JsonElement element);
public TValue? Deserialize<TValue>(JsonNode node);
public object? Deserialize(ReadOnlySpan<byte> utf8Json, Type returnType);
public object? Deserialize(ReadOnlySpan<char> json, Type returnType);
public object? Deserialize(string json, Type returnType);
public object? Deserialize(JsonDocument document, Type returnType);
public object? Deserialize(JsonElement element, Type returnType);
public object? Deserialize(JsonNode node, Type returnType);
public object? Deserialize(ref Utf8JsonReader reader, Type returnType);
public object? Deserialize(Stream utf8Json, Type returnType);
public ValueTask<object?> DeserializeAsync(Stream utf8Json, Type returnType);
}
namespace System.Text.Json.Serialization;
public partial class JsonSerializerContext
{
public JsonSerializer Serializer { get; }
} Usage ExamplesUsing the reflection-based serializer /* composition root */
var options = new JsonSerializerOptions
{
Converters = new JsonStringEnumConverter(),
TypeInfoResolver = new DefaultJsonTypeInfoResolver()
};
var serializer = new JsonSerializer(options);
/* usage */
serializer.Serialize(value); Using the source generator JsonSerializer serializer = MyContext.Default.Serializer;
serializer.Serialize(new MyPoco()); // Serializes using the source generator
[JsonSerializable(typeof(MyPoco))]
public partial class MyContext : JsonSerializerContext
{} Open Questions
Related to #65396, #31094, #64646 cc @eerhardt @davidfowl @ericstj
|
IMO while it could be a good idea when we first created this project I'm not so convinced to do it now and creating second set of the same APIs elsewhere just feels weird. I think it might make more sense to create code analyzer to hint people they're using it wrong. I'm also tempted to say if anything maybe put those methods on the options directly but then it would "weird really read": |
Why would we obsolete this? |
We don't need to, strictly speaking, but it seems to be too much of a pit of failure at the moment. Obsoletion could be one way to guide users to the new APIs. |
Or an analyzer? |
This is pretty compelling, but if we're not going to deprecate the existing static APIs, then it seems moot given that there are existing patterns to avoid the linker warnings. New APIs would add to an increasingly high concept count for using the serializer. New fixers to help users to help users detect and switch bad patterns would perhaps be sufficient to solve these problems. |
@eiriktsarpalis can you show how your problem examples would change? This might help explain how exposing these methods will solve the problems those scenarios present. Also, if we've already documented that people shouldn't do these things, how will these changes alter users' behavior? |
Unfortunately, that is not the case when it comes to using customized contract resolvers. These can only be consumed via the serialization methods that accept the optional
I think the "Usage Examples" section might clarify how these could be refactored, but ultimately the idea is that all serialization configuration is encapsulated behind a materialized "JsonSerializer" instance, so passing the right configuration becomes the responsibility of the composition root, rather than the callsite's, which is more conducive to applying DI patterns: public class MyClass
{
// Use constructor injection to determine serialization configuration; could be reflection or sourcegen or a mix of both.
// Currently we need to call into separate sets of APIs in order to avoid linker warnings.
private readonly JsonSerializer _serializer;
public MyClass(JsonSerializer serializer) => _serializer = serializer;
public string GetJson() => _serializer.Serialize(_data); // instance methods do not accept JsonSerializerOptions and do not differentiate between "reflection" and "sourcegen" flavors. There is no need for any linker annotations here.
} |
The primary original reason for static methods was to be zero-alloc at least for simpler scenarios. Also instantiating a "serializer" class may end up having the same problem we have today for those who just new up the "options" class on every use. IMO adding the Serialize() methods should be done on |
It's definitely useful for quickly producing JSON when configuration is not a concern, but it definitely suffers from ergonomic concerns in less trivial scenaria. Arguably this can also be achieved via the new
Agree that this is still a theoretical concern, but even though an optional parameter in a static method is (for most intents and purposes) equivalent to exposing an instance method on the parameter itself, the two approaches present themselves very differently to users not familiar with the library. Arguably the latter approach communicates more clearly that we're dealing with a serialization context and primes users to think of the type in terms of DI.
Agree in principle, but that ship has sailed when it comes to The work in #61734 doubles down on that design: starting with .NET 7 Regarding the relationship of
|
Here's a sketch of how one could define a serializer instance using .NET 7 APIs that is completely AOT/linker-safe: public partial class JsonSerializerInstance
{
private readonly JsonSerializerOptions _options;
public JsonSerializerInstance(JsonSerializerOptions options)
{
if (options.TypeInfoResolver is null)
{
// This is a departure from default semantics in JsonSerializer methods that accept JsonSerializerOptions,
// but it's essential for preserving AOT/linker safety in the serializer instance constructor.
throw new ArgumentException("The options parameter must specify a TypeInfoResolver value.", nameof(options));
}
_options = options;
}
// Serialization/Deserialization methods call into JsonTypeInfo<T> overloads that are marked as linker-safe.
public string Serialize<T>(T value) => JsonSerializer.Serialize(value, GetTypeInfo<T>());
public T? Deserialize<T>(string json) => JsonSerializer.Deserialize(json, GetTypeInfo<T>());
public Task SerializeAsync<T>(Stream utf8Json, T value, CancellationToken cancellationToken = default)
=> JsonSerializer.SerializeAsync(utf8Json, value, GetTypeInfo<T>(), cancellationToken);
public ValueTask<T?> DeserializeAsync<T>(Stream utf8Json, CancellationToken cancellationToken = default)
=> JsonSerializer.DeserializeAsync(utf8Json, GetTypeInfo<T>(), cancellationToken);
private JsonTypeInfo<T> GetTypeInfo<T>() => (JsonTypeInfo<T>)_options.GetTypeInfo(typeof(T)); Any AOT/linker warning annotations are now exclusively the purview of factories configuring serializer instances: public partial class JsonSerializerInstance
{
[RequiresDynamicCode("Method uses the default reflection-based resolver which requires dynamic code.")]
[RequiresUnreferencedCode("Method uses the default reflection-based resolver which requires unreferenced code.")]
public static JsonSerializerInstance Default { get; } = new(JsonSerializerOptions.Default);
[RequiresDynamicCode("Method uses the default reflection-based resolver which requires dynamic code.")]
[RequiresUnreferencedCode("Method uses the default reflection-based resolver which requires unreferenced code.")]
public static CreateUsingJsonSerializerSemantics(JsonSerializerOptions options)
{
options.TypeInfoResolver ??= new DefaultJsonTypeInfoResolver();
return new(options);
}
// No annotation needed since JsonSerializerContext instances are linker safe
public static JsonSerializerInstance Create(JsonSerializerContext jsonSerializerContext) => new(jsonSerializerContext.Options);
} As soon as a "JsonSerializerInstance" is materialized, it can be used without concern for linker warnings and is agnostic w.r.t. whether it uses source gen or reflection serialization. |
why can't you provide a way to set the options globally just like For example, It will be annoying that if we have to take care of this kind of thing everywhere we uses it. |
That approach has its own set of problems, see #31094 (comment) |
I'm not sure I fully understand how this solves the problem, but I may just not understand Json source generators well enough. Let's say I want to create an HttpClient with JsonSerialization var client = new HttpClient(new JsonSerializer(new MyContext.Default.Serializer));
client.WriteAsJson(new MyType1());
client.WriteAsJson(new MyType2()); How does this work? Does that MyTypeContext contain the JsonSerializer for all of the types automatically? Or do you need to specify which types it contains via the Also, does this mean that each type may get a source-gen'd implementation generated multiple times in each assembly? |
This is the answer.
You can use this new API to combine contexts.
I think it depends on where the |
And that should be a fairly common scenario, given that each |
OK, this makes sense then, thanks.
This is good to know. Given that code size is a metric we're tracking for Native AOT we should keep an eye on how this scales out with larger apps. No action for now, though. |
Generally speaking it should contribute to code size increases, see #77897 as an example. |
I love this proposal so far! Have a few questions as well 😄
I read the previous conversation here and didn't see anyone commenting on this yet - looking at that example I'm a bit worried about the fact you don't have an explicit SomeModel model = MicrosoftStoreJsonSerializerContext.Default.SomeModel.Deserialize(jsonText); This makes it very easy to see that:
In the example above for the new API instead, there's no way to be sure just by looking at the callsite that the
Just in case there's anyone else curious about that and/or following the issue, we're working on adjusting things on our end as well to help reduce the binary size impact (as a separate effort than the planned improvements for the JSON generator), so we'll share what we've learnt and what numbers we got once we're done with that. I'm hoping we'll be able to have a noticeable binary size reduction compared to that whopping +17MB increase by just tweaking our JSON models 🙂 And of course, all the planned improvements to reuse generate stubs should also help a ton for folks in these situations. |
@eiriktsarpalis we could design API to attach a custom resolver to a context instance. Thought would have to into making it as clean as possible, but it should be just a wrapping. |
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
The off-topic conversation was about reducing the size footprint of src-gen'd code. It was moved to the appropriate issue: #77897 (comment). |
The proposal concerns exposing functionality that supersedes the |
Ah, gotcha, yeah that makes sense, thank you for clarifying! 🙂 Somewhat related, this also makes me think whether people currently suffering from the issues described here (eg. creating options every time or forgetting to pass them) couldn't already solve the issue by just defining multiple option accessors in the context and then just statically accessing that like with the default one 🤔 To make an example, one thing we're doing in the Store is, we have some cases where we want to serialize/deserialize with case invariant properties (but it could be any example of "some statically known set of options different than the default ones to customize a given serialization operation"). What we did is to just add a separate property to our JSON context, like so: public sealed partial class MicrosoftStoreJsonSerializerContext : JsonSerializerContext
{
private static MicrosoftStoreJsonSerializerContext? _camelCase;
public static MicrosoftStoreJsonSerializerContext CamelCase => _camelCase ??= new(new JsonSerializerOptions(s_defaultOptions) { PropertyNamingPolicy = JsonNamingPolicy.CamelCase });
} This way whenever we need to use this, we simply use SomeModel model = MicrosoftStoreJsonSerializerContext.CamelCase.SomeModel.Deserialize(json); I wonder whether at least for cases where the options being used only have statically known properties (eg. like the naming policy here) it wouldn't be simpler for people to use this approach instead, which also doesn't need any new APIs. I guess I'm just curious because by just looking at the examples, for cases where again you only have statically known options, it's not immediately clear to me why should one use the new instance method approach to replace using |
The other reasons may be sufficient to add such APIs, but I continue to believe adding them to avoid the pitfalls of creating options every time is misguided. We have plenty of evidence developers are just as happy to create a new instance of a serializer every time they need to serialize/deserialize something (see practically every use of BinaryFormatter), and even outside of the serializers (e.g. HttpClient), even when guidance strongly urges otherwise. |
We need runtime analyzers 😄 |
I think people are a lot less likely to write |
Any update on this ? I personally don't want to add extra attribute on every single Properties of my custom type. |
This design is not being pursued for the near term. |
I may have misunderstood your words, but I have something like the following:
I have defined I don't want to be worry about missing I believe this requirement is meaningful, and I have searched for information on it (Note: I found this Issue to be the closest one. |
This appears to be unrelated to what is being discussed in this issue. You can apply a custom converter for your |
@eiriktsarpalis At #31094 (comment), you hinted that you'd prefer a design without an interface, and that is what you are proposing here as well. I'm curious, what was the reasoning behind the team choosing not to define an abstract interface for this? Why have only the static API? If we had an interface, then we'd at least have a standard way to encapsulate a serializer and for libraries to depend on that. |
Also, that issue remains, to this day, one of the top liked issues in this repo, even though it has been locked -- thus freezing reactions -- for two years now, so there seems to be clear interest from the community. Could the team share any insights as to why this is not currently a priority? Is there anything the community can do to help? |
Background & Motivation
The current design of the
JsonSerializer
class exposes all serialization functionality as static methods accepting configuration parametrically (and optionally). In hindsight, this design has contributed to a few ergonomic problems when using this API:Forgetting to pass
JsonSerializerOptions
to the serialization methods:This is probably the most common issue -- I've personally fallen for this too many times.
Creating a new
JsonSerializerOptions
instance on each serialization:Even though this anti-pattern has been explicitly documented as a potential performance bug, users keep falling for it. We've made attempts to mitigate the problem by implementing shared metadata caches but ultimately the underlying issue still affects users. See also Add an analyzer that warns on single-use JsonSerializerOptions instances #65396 for plans an adding an analyzer in this space.
Pervasive
RequiresUnreferenceCode
/RequiresDynamicCode
annotations: all serialization methods acceptingJsonSerializerOptions
have been annotated as linker/AOT-unsafe even though legitimate scenaria exist where this is not the case:Bifurcation of API surface between reflection and source generation: users need to call into distinct methods depending on whether they use sourcegen or reflection. The distinction between
JsonSerializerOptions
andJsonSerializerContext
has always been tenuous and has been rendered obsolete with the infrastructural changes introduced by Developers can customize the JSON serialization contracts of their types #63686. Arguably, the source of JSON contracts is a configuration detail that should be dealt with at the composition root and not concern any serialization methods.API Proposal
This issue proposes we expose instance methods in
JsonSerializer
(or some different class):Usage Examples
Using the reflection-based serializer
Using the source generator
Open Questions
JsonSerializer
class or introduce a new type?JsonSerializerOptions
?Related to #65396, #31094, #64646
cc @eerhardt @davidfowl @ericstj
The text was updated successfully, but these errors were encountered: