From d2843320cd717c2431e422ef64100ca4444be754 Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Tue, 8 Jun 2021 18:50:19 +0300 Subject: [PATCH 01/13] add JSON polymorphic serialization design --- accepted/2021/json-polymorphism.md | 273 +++++++++++++++++++++++++++++ 1 file changed, 273 insertions(+) create mode 100644 accepted/2021/json-polymorphism.md diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md new file mode 100644 index 000000000..0d6f8598d --- /dev/null +++ b/accepted/2021/json-polymorphism.md @@ -0,0 +1,273 @@ +# System.Text.Json polymorphic serialization + +**Owner** [Eirik Tsarpalis](https://github.com/eiriktsarpalis) + +This documents describes the proposed design for extending [polymorphism support](https://github.com/dotnet/runtime/issues/45189) in System.Text.Json. + +## Background + +By default, System.Text.Json will serialize a value using a converter derived from its declared type, +regardless of what the runtime type of the value might be. This behavior is in line with the +Liskov substitution principle, in that the serialization contract is unique (or "monomorphic") for a given type `T`, +regardless of what subtype of `T` we end up serializing at runtime. + +A notable exception to this rule is members of type `object`, in which case the runtime type of the value +is looked up and serialization is dispatched to the converter corresponding to that runtime type. +This is an instance of _polymorphic serialization_, in the sense that the schema might vary depending on +the runtime type a given `object` instance might have. + +Conversely, in _polymorphic deserialization_ the runtime type of a deserialized value might vary depending on the +shape of the input encoding. Currently, System.Text.Json does not offer any form of support for polymorphic +deserialization. + +We have received a number of user requests to add polymorphic serialization and deserialization support +to System.Text.Json. This can be a useful feature in domains where exporting type hierarchies is desirable, +for example when serializing tree-like data structures or discriminated unions. + +It should be noted however that polymorphic serialization comes with a few security risks: + +* Polymorphic serialization applied indiscriminately can result in unintended data leaks, + since properties of unexpected derived types may end up written on the wire. +* Polymorphic deserialization can be vulnerable when deserializing untrusted data, + in certain cases leading to remote code execution attacks. + +## Introduction + +The proposed design for polymorphic serialization in System.Text.Json can be split into two +largely orthogonal features: + +1. Simple Polymorphic serialization: extends the existing serialization infrastructure for `object` types to + arbitrary classes that can be specified by the user. It trivially dispatches to the converter corresponding + to the runtime type without emitting any metadata on the wire and does not provide any provision for + polymorphic deserialization. +2. Polymorphism with type discriminators ("tagged polymorphism"): classes can be serialized and deserialized + polymorphically by emitting a type discriminator ("tag") on the wire. Users must explicitly associate each + supported subtype of a given declared type with a string identifier. + +## Simple Polymorphic Serialization + +Consider the following type hierarchy: +```csharp +public class Foo +{ + public int A { get; set; } +} + +public class Bar : Foo +{ + public int B { get; set; } +} + +public class Baz : Bar +{ + public int C { get; set; } +} +``` +Currently, when serializing a `Bar` instance as type `Foo` +the serializer will apply the JSON schema derived from the type `Foo`: +```csharp +Foo foo1 = new Foo { A = 1 }; +Foo foo2 = new Bar { A = 1, B = 2 }; +Foo foo3 = new Baz { A = 1, B = 2, C = 3 }; + +JsonSerializer.Serialize(foo1); // { "A" : 1 } +JsonSerializer.Serialize(foo2); // { "A" : 1 } +JsonSerializer.Serialize(foo3); // { "A" : 1 } +``` +Under the new proposal we can change this behaviour by annotating +the base class (or interface) with the `JsonPolymorphicType` attribute: +```csharp +[JsonPolymorhicType] +public class Foo +{ + ... +} +``` +which will result in the above values now being serialized as follows: +```csharp +JsonSerializer.Serialize(foo1); // { "A" : 1 } +JsonSerializer.Serialize(foo1); // { "A" : 1, "B" : 2 } +JsonSerializer.Serialize(foo2); // { "A" : 1, "B" : 2, "C" : 3 } +``` +Note that the `JsonPolymorphic` attribute is not inherited by derived types. +In the above example `Bar` inherits from `Foo` yet is not polymorphic in its own right: +```csharp +Bar bar = new Baz { A = 1, B = 2, C = 3 }; +JsonSerializer.Serialize(bar); // { "A" : 1, "B" : 2 } +``` +If annotating the base class with an attribute is not possible, +polymorphism can alternatively be opted in for a type using the +new `JsonSerializerOptions.SupportedPolymorphicTypes` predicate: +```csharp +public class JsonSerializerOptions +{ + public Func SupportedPolymorphicTypes { get; set; } +} +``` +Applied to the example above: +```csharp +var options = new JsonSerializerOptions { SupportedPolymorphicTypes = type => type == typeof(Foo) }; +JsonSerializer.Serialize(foo1, options); // { "A" : 1, "B" : 2 } +JsonSerializer.Serialize(foo2, options); // { "A" : 1, "B" : 2, "C" : 3 } +``` +It is always possible to use this setting to enable polymorphism _for every_ serialized type: +```csharp +var options = new JsonSerializerOptions { SupportedPolymorphicTypes = _ => true }; + +// `options` treats both `Foo` and `Bar` members as polymorphic +Baz baz = new Baz { A = 1, B = 2, C = 3 }; +JsonSerializer.Serialize(baz, options); // { "A" : 1, "B" : 2, "C" : 3 } +JsonSerializer.Serialize(baz, options); // { "A" : 1, "B" : 2, "C" : 3 } +``` +As mentioned previously, this feature provides no provision for deserialization. +If deserialization is a requirement, users would need to opt for the +polymorphic serialization with type discriminators feature. + +## Polymorphism with type discriminators + +This feature allows users to opt in to polymorphic serialization for a given type +by associating string identifiers with particular subtypes in the hierarchy. +These identifiers are written to the wire so this brand of polymorphism is roundtrippable. + +At the core of the design is the introduction of `JsonKnownType` attribute that can +be applied to type hierarchies like so: +```csharp +[JsonKnownType(typeof(Derived1), "derived1")] +[JsonKnownType(typeof(Derived2), "derived2")] +public class Base +{ + public int X { get; set; } +} + +public class Derived1 : Base +{ + public int Y { get; set; } +} + +public class Derived2 : Base +{ + public int Z { get; set; } +} +``` +This allows roundtrippable polymorphic serialization using the following schema: +```csharp +var json1 = JsonSerializer.Serialize(new Derived1()); // { "$type" : "derived1", "X" : 0, "Y" : 0 } +var json2 = JsonSerializer.Serialize(new Derived2()); // { "$type" : "derived2", "X" : 0, "Z" : 0 } + +JsonSerializer.Deserialize(json1); // uses Derived1 as runtime type +JsonSerializer.Deserialize(json2); // uses Derived2 as runtime type +``` +Alternatively, users can specify known type configuration using the +`JsonSerializerOptions.TypeDiscriminatorConfigurations` property: +```csharp +public class JsonSerializerOptions +{ + public IList TypeDiscriminatorConfigurations { get; } +} +``` +which can be used as follows: +```csharp +var options = new JsonSerializerOptions +{ + TypeDiscriminatorConfigurations = + { + new TypeDiscriminatorConfiguration() + .WithKnownType("derived1") + .WithKnownType("derived2") + } +}; +``` +or alternatively +```csharp +var options = new JsonSerializerOptions +{ + PolymorphicTypeDiscriminators = + { + new TypeDiscriminatorConfiguration(typeof(Base)) + .WithKnownType(typeof(Derived1), "derived1") + .WithKnownType(typeof(Derived2), "derived2") + } +}; +``` + +### Open Questions + +The type discriminator feature has two possible design alternatives to be considered, +which for the purposes of this document I will be calling "strict mode" and "lax mode". +Each approach comes with its own sets of trade-offs. + +#### Strict mode + +"Strict mode" requires that any runtime type used during serialization must explicitly specify a type discriminator. +For example: +```csharp +[JsonKnownType(typeof(Derived1),"derived1")] +[JsonKnownType(typeof(Derived2),"derived2")] +public class Base { } + +public class Derived1 : Base { } +public class Derived2 : Base { } +public class Derived3 : Base { } + +public class OtherDerived1 : Derived1 { } + +JsonSerializer.Serialize(new Derived1()); // { "$type" : "derived1" } +JsonSerializer.Serialize(new Derived2()); // { "$type" : "derived2" } +JsonSerializer.Serialize(new Derived3()); // throws NotSupportedException +JsonSerializer.Serialize(new OtherDerived1()); // throws NotSupportedException +JsonSerializer.Serialize(new Base()); // throws NotSupportedException +``` +Any runtime type that is not associated with a type discriminator will be rejected, +including instances of the base type itself. This approach has a few drawbacks: + +* Does not work well with open hierarchies: any new derived types will have to be explicitly opted in. +* Each runtime type must use a separate type identifier. +* Interfaces or abstract classes cannot specify type discriminators. + +#### Lax mode + +"Lax mode" as the name suggests is more permissive, and runtime types without discriminators +are serialized using the nearest type ancestor that does specify a discriminator. +Using the previous example: +```csharp +[JsonKnownType(typeof(Derived1),"derived1")] +[JsonKnownType(typeof(Derived2),"derived2")] +public class Base { } + +public class Derived1 : Base { } +public class Derived2 : Base { } +public class Derived3 : Base { } + +public class OtherDerived1 : Derived1 { } + +JsonSerializer.Serialize(new Derived1()); // { "$type" : "derived1" } +JsonSerializer.Serialize(new Derived2()); // { "$type" : "derived2" } +JsonSerializer.Serialize(new Derived3()); // { } serialized as `Base` +JsonSerializer.Serialize(new OtherDerived1()); // { "$type" : "derived1" } inherits schema from `Derived1` +JsonSerializer.Serialize(new Base()); // { } serialized as `Base` +``` +This approach is more flexible and supports interface and abstract type hierarchies: +```csharp +[JsonKnownType(typeof(Foo), "foo")] +[JsonKnownType(typeof(IBar), "bar")] +public interface IFoo { } +public abstract class Foo : IFoo { } +public interface IBar : IFoo { } + +public class FooImpl : Foo {} + +JsonSerializer.Serialize(new FooImpl()); // { "$type" : "foo" } +``` +However it does come with its own set of problems: +```csharp +[JsonKnownType(typeof(Foo), "foo")] +[JsonKnownType(typeof(IBar), "bar")] +public interface IFoo { } +public class Foo : IFoo { } +public interface IBar : IFoo { } + +public Baz : Foo, IBar { } + +JsonSerializer.Serialize(new Baz()); // diamond ambiguity, could either be "foo" or "bar", + // throws NotSupportedException. +``` \ No newline at end of file From e38c7c54a67d61fd4d519b034494a797cccfab8b Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Tue, 8 Jun 2021 19:37:25 +0300 Subject: [PATCH 02/13] add link to draft implementation PR --- accepted/2021/json-polymorphism.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md index 0d6f8598d..5e3c4e97e 100644 --- a/accepted/2021/json-polymorphism.md +++ b/accepted/2021/json-polymorphism.md @@ -4,6 +4,8 @@ This documents describes the proposed design for extending [polymorphism support](https://github.com/dotnet/runtime/issues/45189) in System.Text.Json. +[Draft Implementation PR](https://github.com/dotnet/runtime/pull/53882). + ## Background By default, System.Text.Json will serialize a value using a converter derived from its declared type, From 14428002fe8f152cd5b73c9d3a896cbb2b819259 Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Tue, 8 Jun 2021 19:43:58 +0300 Subject: [PATCH 03/13] fix typo --- accepted/2021/json-polymorphism.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md index 5e3c4e97e..a5b967f80 100644 --- a/accepted/2021/json-polymorphism.md +++ b/accepted/2021/json-polymorphism.md @@ -183,7 +183,7 @@ or alternatively ```csharp var options = new JsonSerializerOptions { - PolymorphicTypeDiscriminators = + TypeDiscriminatorConfigurations = { new TypeDiscriminatorConfiguration(typeof(Base)) .WithKnownType(typeof(Derived1), "derived1") From dca1d712574dfffef5f77975b919f98dfdd8eb41 Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Tue, 8 Jun 2021 18:51:22 +0100 Subject: [PATCH 04/13] Update accepted/2021/json-polymorphism.md Co-authored-by: Christopher Watford <83599748+watfordsuzy@users.noreply.github.com> --- accepted/2021/json-polymorphism.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md index a5b967f80..dfbcb69cf 100644 --- a/accepted/2021/json-polymorphism.md +++ b/accepted/2021/json-polymorphism.md @@ -57,7 +57,7 @@ public class Foo public class Bar : Foo { - public int B { get; set; } + public int B { get; set; } } public class Baz : Bar @@ -272,4 +272,4 @@ public Baz : Foo, IBar { } JsonSerializer.Serialize(new Baz()); // diamond ambiguity, could either be "foo" or "bar", // throws NotSupportedException. -``` \ No newline at end of file +``` From 8729383f48003483b6322d9b658cecc9ae15a906 Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Tue, 8 Jun 2021 18:51:28 +0100 Subject: [PATCH 05/13] Update accepted/2021/json-polymorphism.md Co-authored-by: Christopher Watford <83599748+watfordsuzy@users.noreply.github.com> --- accepted/2021/json-polymorphism.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md index dfbcb69cf..36085f4a9 100644 --- a/accepted/2021/json-polymorphism.md +++ b/accepted/2021/json-polymorphism.md @@ -42,7 +42,7 @@ largely orthogonal features: arbitrary classes that can be specified by the user. It trivially dispatches to the converter corresponding to the runtime type without emitting any metadata on the wire and does not provide any provision for polymorphic deserialization. -2. Polymorphism with type discriminators ("tagged polymorphism"): classes can be serialized and deserialized +2. Polymorphism with type discriminators ("tagged polymorphism"): classes can be serialized and deserialized polymorphically by emitting a type discriminator ("tag") on the wire. Users must explicitly associate each supported subtype of a given declared type with a string identifier. From 21818402b99ef952c1bb61146f0d1180330dec41 Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Tue, 8 Jun 2021 18:51:33 +0100 Subject: [PATCH 06/13] Update accepted/2021/json-polymorphism.md Co-authored-by: Christopher Watford <83599748+watfordsuzy@users.noreply.github.com> --- accepted/2021/json-polymorphism.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md index 36085f4a9..21c7c6c4c 100644 --- a/accepted/2021/json-polymorphism.md +++ b/accepted/2021/json-polymorphism.md @@ -62,7 +62,7 @@ public class Bar : Foo public class Baz : Bar { - public int C { get; set; } + public int C { get; set; } } ``` Currently, when serializing a `Bar` instance as type `Foo` From e342f346a031fd22d4858ada7642fb8bae20fe7f Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Wed, 9 Jun 2021 01:00:34 +0300 Subject: [PATCH 07/13] address feedback --- accepted/2021/json-polymorphism.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md index 21c7c6c4c..e3448798c 100644 --- a/accepted/2021/json-polymorphism.md +++ b/accepted/2021/json-polymorphism.md @@ -88,8 +88,8 @@ public class Foo which will result in the above values now being serialized as follows: ```csharp JsonSerializer.Serialize(foo1); // { "A" : 1 } -JsonSerializer.Serialize(foo1); // { "A" : 1, "B" : 2 } -JsonSerializer.Serialize(foo2); // { "A" : 1, "B" : 2, "C" : 3 } +JsonSerializer.Serialize(foo2); // { "A" : 1, "B" : 2 } +JsonSerializer.Serialize(foo3); // { "A" : 1, "B" : 2, "C" : 3 } ``` Note that the `JsonPolymorphic` attribute is not inherited by derived types. In the above example `Bar` inherits from `Foo` yet is not polymorphic in its own right: @@ -194,7 +194,7 @@ var options = new JsonSerializerOptions ### Open Questions -The type discriminator feature has two possible design alternatives to be considered, +The type discriminator semantics could be implemented following two possible alternatives, which for the purposes of this document I will be calling "strict mode" and "lax mode". Each approach comes with its own sets of trade-offs. From 491648d6c03ea71ba89c30201813af5078daa645 Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Wed, 9 Jun 2021 14:20:55 +0300 Subject: [PATCH 08/13] make CI happy --- INDEX.md | 1 + 1 file changed, 1 insertion(+) diff --git a/INDEX.md b/INDEX.md index c28679d0d..422939372 100644 --- a/INDEX.md +++ b/INDEX.md @@ -75,6 +75,7 @@ Use update-index to regenerate it: | 2021 | [Preview Features](accepted/2021/preview-features/preview-features.md) | [Immo Landwerth](https://github.com/terrajobst) | | 2021 | [TFM for .NET nanoFramework](accepted/2021/nano-framework-tfm/nano-framework-tfm.md) | [Immo Landwerth](https://github.com/terrajobst), [Laurent Ellerbach](https://github.com/Ellerbach), [José Simões](https://github.com/josesimoes) | | 2021 | [Tracking Platform Dependencies](accepted/2021/platform-dependencies/platform-dependencies.md) | [Matt Thalman](https://github.com/mthalman) | +| 2021 | [System.Text.Json polymorphic serialization](accepted/2021/json-polymorphism.md) | [Eirik Tsarpalis]](https://github.com/eiriktsarpalis) | ## Drafts From 3b813ec0336669865260ad29b4f78d82023e816a Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Wed, 9 Jun 2021 15:20:53 +0100 Subject: [PATCH 09/13] update index the proper way --- INDEX.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/INDEX.md b/INDEX.md index 422939372..923b381a6 100644 --- a/INDEX.md +++ b/INDEX.md @@ -73,9 +73,9 @@ Use update-index to regenerate it: | 2021 | [Compile-time source generation for strongly-typed logging messages](accepted/2021/logging-generator.md) | [Maryam Ariyan](https://github.com/maryamariyan), [Martin Taillefer](https://github.com/geeknoid) | | 2021 | [Objective-C interoperability](accepted/2021/objectivec-interop.md) | [Aaron Robinson](https://github.com/AaronRobinsonMSFT) | | 2021 | [Preview Features](accepted/2021/preview-features/preview-features.md) | [Immo Landwerth](https://github.com/terrajobst) | +| 2021 | [System.Text.Json polymorphic serialization](accepted/2021/json-polymorphism.md) | [Eirik Tsarpalis](https://github.com/eiriktsarpalis) | | 2021 | [TFM for .NET nanoFramework](accepted/2021/nano-framework-tfm/nano-framework-tfm.md) | [Immo Landwerth](https://github.com/terrajobst), [Laurent Ellerbach](https://github.com/Ellerbach), [José Simões](https://github.com/josesimoes) | | 2021 | [Tracking Platform Dependencies](accepted/2021/platform-dependencies/platform-dependencies.md) | [Matt Thalman](https://github.com/mthalman) | -| 2021 | [System.Text.Json polymorphic serialization](accepted/2021/json-polymorphism.md) | [Eirik Tsarpalis]](https://github.com/eiriktsarpalis) | ## Drafts From c6cd93de68f57d8511a0798c878e0c315fa975c7 Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Fri, 11 Jun 2021 08:50:15 +0100 Subject: [PATCH 10/13] Update accepted/2021/json-polymorphism.md Co-authored-by: Jeff Handley --- accepted/2021/json-polymorphism.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md index e3448798c..7c394134d 100644 --- a/accepted/2021/json-polymorphism.md +++ b/accepted/2021/json-polymorphism.md @@ -79,7 +79,7 @@ JsonSerializer.Serialize(foo3); // { "A" : 1 } Under the new proposal we can change this behaviour by annotating the base class (or interface) with the `JsonPolymorphicType` attribute: ```csharp -[JsonPolymorhicType] +[JsonPolymorphicType] public class Foo { ... From 5f6d394461c94b2ab9dc19eec60448c9d097452d Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Fri, 11 Jun 2021 08:50:24 +0100 Subject: [PATCH 11/13] Update accepted/2021/json-polymorphism.md Co-authored-by: Jeff Handley --- accepted/2021/json-polymorphism.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md index 7c394134d..d51c972a5 100644 --- a/accepted/2021/json-polymorphism.md +++ b/accepted/2021/json-polymorphism.md @@ -91,7 +91,7 @@ JsonSerializer.Serialize(foo1); // { "A" : 1 } JsonSerializer.Serialize(foo2); // { "A" : 1, "B" : 2 } JsonSerializer.Serialize(foo3); // { "A" : 1, "B" : 2, "C" : 3 } ``` -Note that the `JsonPolymorphic` attribute is not inherited by derived types. +Note that the `JsonPolymorphicType` attribute is not inherited by derived types. In the above example `Bar` inherits from `Foo` yet is not polymorphic in its own right: ```csharp Bar bar = new Baz { A = 1, B = 2, C = 3 }; From 01b1f27e9b2c6478358e6b9fea4787277e984b69 Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Fri, 11 Jun 2021 19:43:08 +0100 Subject: [PATCH 12/13] address feedback --- accepted/2021/json-polymorphism.md | 76 +++++++++--------------------- 1 file changed, 21 insertions(+), 55 deletions(-) diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md index d51c972a5..17a5cc730 100644 --- a/accepted/2021/json-polymorphism.md +++ b/accepted/2021/json-polymorphism.md @@ -8,43 +8,25 @@ This documents describes the proposed design for extending [polymorphism support ## Background -By default, System.Text.Json will serialize a value using a converter derived from its declared type, -regardless of what the runtime type of the value might be. This behavior is in line with the -Liskov substitution principle, in that the serialization contract is unique (or "monomorphic") for a given type `T`, -regardless of what subtype of `T` we end up serializing at runtime. +By default, System.Text.Json will serialize a value using a converter based on its declared type, regardless of what the runtime type of the value might be. This behavior is in line with the Liskov substitution principle, in that the serialization contract is unique (or "monomorphic") for a given type `T`, regardless of what subtype of `T` we end up serializing at runtime. -A notable exception to this rule is members of type `object`, in which case the runtime type of the value -is looked up and serialization is dispatched to the converter corresponding to that runtime type. -This is an instance of _polymorphic serialization_, in the sense that the schema might vary depending on -the runtime type a given `object` instance might have. +A notable exception to this rule is members of type `object`, in which case the runtime type of the value is looked up and serialization is dispatched to the converter corresponding to that runtime type. This is an instance of _polymorphic serialization_, in the sense that the schema might vary depending on the runtime type a given `object` instance might have. -Conversely, in _polymorphic deserialization_ the runtime type of a deserialized value might vary depending on the -shape of the input encoding. Currently, System.Text.Json does not offer any form of support for polymorphic -deserialization. +Conversely, in _polymorphic deserialization_ the runtime type of a deserialized value might vary depending on the shape of the input encoding. Currently, System.Text.Json does not offer any form of support for polymorphic deserialization. It should be noted that deserialization `object` _is_ currently supported, however that is implemented via the `JsonElement` or `JsonNode` types rather than using a true form of polymorphic deserialization. -We have received a number of user requests to add polymorphic serialization and deserialization support -to System.Text.Json. This can be a useful feature in domains where exporting type hierarchies is desirable, -for example when serializing tree-like data structures or discriminated unions. +We have received a number of user requests to add polymorphic serialization and deserialization support to System.Text.Json. This can be a useful feature in domains where exporting type hierarchies is desirable, for example when serializing tree-like data structures or discriminated unions. It should be noted however that polymorphic serialization comes with a few security risks: -* Polymorphic serialization applied indiscriminately can result in unintended data leaks, - since properties of unexpected derived types may end up written on the wire. -* Polymorphic deserialization can be vulnerable when deserializing untrusted data, - in certain cases leading to remote code execution attacks. +* Polymorphic serialization applied indiscriminately can result in unintended data leaks, since properties of unexpected derived types may end up written on the wire. +* Polymorphic deserialization can be vulnerable when deserializing untrusted data, in certain cases leading to remote code execution attacks. ## Introduction -The proposed design for polymorphic serialization in System.Text.Json can be split into two -largely orthogonal features: +The proposed design for polymorphic serialization in System.Text.Json can be split into two largely orthogonal features: -1. Simple Polymorphic serialization: extends the existing serialization infrastructure for `object` types to - arbitrary classes that can be specified by the user. It trivially dispatches to the converter corresponding - to the runtime type without emitting any metadata on the wire and does not provide any provision for - polymorphic deserialization. -2. Polymorphism with type discriminators ("tagged polymorphism"): classes can be serialized and deserialized - polymorphically by emitting a type discriminator ("tag") on the wire. Users must explicitly associate each - supported subtype of a given declared type with a string identifier. +1. Simple Polymorphic serialization: extends the existing serialization infrastructure for `object` types to arbitrary classes that can be specified by the user. It trivially dispatches to the converter corresponding to the runtime type without emitting any metadata on the wire and does not provide any provision for polymorphic deserialization. +2. Polymorphism with type discriminators ("tagged polymorphism"): classes can be serialized and deserialized polymorphically by emitting a type discriminator ("tag") on the wire. Users must explicitly associate each supported subtype of a given declared type with a string identifier. ## Simple Polymorphic Serialization @@ -65,8 +47,7 @@ public class Baz : Bar public int C { get; set; } } ``` -Currently, when serializing a `Bar` instance as type `Foo` -the serializer will apply the JSON schema derived from the type `Foo`: +Currently, when serializing a `Bar` instance as type `Foo` the serializer will apply the JSON schema derived from the type `Foo`: ```csharp Foo foo1 = new Foo { A = 1 }; Foo foo2 = new Bar { A = 1, B = 2 }; @@ -76,8 +57,7 @@ JsonSerializer.Serialize(foo1); // { "A" : 1 } JsonSerializer.Serialize(foo2); // { "A" : 1 } JsonSerializer.Serialize(foo3); // { "A" : 1 } ``` -Under the new proposal we can change this behaviour by annotating -the base class (or interface) with the `JsonPolymorphicType` attribute: +Under the new proposal we can change this behaviour by annotating the base class (or interface) with the `JsonPolymorphicType` attribute: ```csharp [JsonPolymorphicType] public class Foo @@ -91,15 +71,12 @@ JsonSerializer.Serialize(foo1); // { "A" : 1 } JsonSerializer.Serialize(foo2); // { "A" : 1, "B" : 2 } JsonSerializer.Serialize(foo3); // { "A" : 1, "B" : 2, "C" : 3 } ``` -Note that the `JsonPolymorphicType` attribute is not inherited by derived types. -In the above example `Bar` inherits from `Foo` yet is not polymorphic in its own right: +Note that the `JsonPolymorphicType` attribute is not inherited by derived types. In the above example `Bar` inherits from `Foo` yet is not polymorphic in its own right: ```csharp Bar bar = new Baz { A = 1, B = 2, C = 3 }; JsonSerializer.Serialize(bar); // { "A" : 1, "B" : 2 } ``` -If annotating the base class with an attribute is not possible, -polymorphism can alternatively be opted in for a type using the -new `JsonSerializerOptions.SupportedPolymorphicTypes` predicate: +If annotating the base class with an attribute is not possible, polymorphism can alternatively be opted in for a type using the new `JsonSerializerOptions.SupportedPolymorphicTypes` predicate: ```csharp public class JsonSerializerOptions { @@ -121,18 +98,13 @@ Baz baz = new Baz { A = 1, B = 2, C = 3 }; JsonSerializer.Serialize(baz, options); // { "A" : 1, "B" : 2, "C" : 3 } JsonSerializer.Serialize(baz, options); // { "A" : 1, "B" : 2, "C" : 3 } ``` -As mentioned previously, this feature provides no provision for deserialization. -If deserialization is a requirement, users would need to opt for the -polymorphic serialization with type discriminators feature. +As mentioned previously, this feature provides no provision for deserialization. If deserialization is a requirement, users would need to opt for the polymorphic serialization with type discriminators feature. ## Polymorphism with type discriminators -This feature allows users to opt in to polymorphic serialization for a given type -by associating string identifiers with particular subtypes in the hierarchy. -These identifiers are written to the wire so this brand of polymorphism is roundtrippable. +This feature allows users to opt in to polymorphic serialization for a given type by associating string identifiers with particular subtypes in the hierarchy. These identifiers are written to the wire so this brand of polymorphism is roundtrippable. -At the core of the design is the introduction of `JsonKnownType` attribute that can -be applied to type hierarchies like so: +At the core of the design is the introduction of `JsonKnownType` attribute that can be applied to type hierarchies like so: ```csharp [JsonKnownType(typeof(Derived1), "derived1")] [JsonKnownType(typeof(Derived2), "derived2")] @@ -194,14 +166,11 @@ var options = new JsonSerializerOptions ### Open Questions -The type discriminator semantics could be implemented following two possible alternatives, -which for the purposes of this document I will be calling "strict mode" and "lax mode". -Each approach comes with its own sets of trade-offs. +The type discriminator semantics could be implemented following two possible alternatives, which for the purposes of this document I will be calling "strict mode" and "lax mode". Each approach comes with its own sets of trade-offs. #### Strict mode -"Strict mode" requires that any runtime type used during serialization must explicitly specify a type discriminator. -For example: +"Strict mode" requires that any runtime type used during serialization must explicitly specify a type discriminator. For example: ```csharp [JsonKnownType(typeof(Derived1),"derived1")] [JsonKnownType(typeof(Derived2),"derived2")] @@ -219,18 +188,15 @@ JsonSerializer.Serialize(new Derived3()); // throws NotSupportedException JsonSerializer.Serialize(new OtherDerived1()); // throws NotSupportedException JsonSerializer.Serialize(new Base()); // throws NotSupportedException ``` -Any runtime type that is not associated with a type discriminator will be rejected, -including instances of the base type itself. This approach has a few drawbacks: +Any runtime type that is not associated with a type discriminator will be rejected, including instances of the base type itself. This approach has a few drawbacks: * Does not work well with open hierarchies: any new derived types will have to be explicitly opted in. * Each runtime type must use a separate type identifier. -* Interfaces or abstract classes cannot specify type discriminators. +* It is not possible to specify identifiers for subtypes that are interfaces or abstract classes. #### Lax mode -"Lax mode" as the name suggests is more permissive, and runtime types without discriminators -are serialized using the nearest type ancestor that does specify a discriminator. -Using the previous example: +"Lax mode" as the name suggests is more permissive, and runtime types without discriminators are serialized using the nearest type ancestor that does specify a discriminator. Using the previous example: ```csharp [JsonKnownType(typeof(Derived1),"derived1")] [JsonKnownType(typeof(Derived2),"derived2")] From 9c6ca2cb0ad09bf2bde71ee75343c30e9fe13080 Mon Sep 17 00:00:00 2001 From: Eirik Tsarpalis Date: Tue, 15 Jun 2021 14:17:56 +0100 Subject: [PATCH 13/13] improve code samples --- accepted/2021/json-polymorphism.md | 32 ++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/accepted/2021/json-polymorphism.md b/accepted/2021/json-polymorphism.md index 17a5cc730..35a8686f3 100644 --- a/accepted/2021/json-polymorphism.md +++ b/accepted/2021/json-polymorphism.md @@ -49,13 +49,13 @@ public class Baz : Bar ``` Currently, when serializing a `Bar` instance as type `Foo` the serializer will apply the JSON schema derived from the type `Foo`: ```csharp -Foo foo1 = new Foo { A = 1 }; -Foo foo2 = new Bar { A = 1, B = 2 }; -Foo foo3 = new Baz { A = 1, B = 2, C = 3 }; +Foo foo = new Foo { A = 1 }; +Bar bar = new Bar { A = 1, B = 2 }; +Baz baz = new Baz { A = 1, B = 2, C = 3 }; -JsonSerializer.Serialize(foo1); // { "A" : 1 } -JsonSerializer.Serialize(foo2); // { "A" : 1 } -JsonSerializer.Serialize(foo3); // { "A" : 1 } +JsonSerializer.Serialize(foo); // { "A" : 1 } +JsonSerializer.Serialize(bar); // { "A" : 1 } +JsonSerializer.Serialize(baz); // { "A" : 1 } ``` Under the new proposal we can change this behaviour by annotating the base class (or interface) with the `JsonPolymorphicType` attribute: ```csharp @@ -67,14 +67,24 @@ public class Foo ``` which will result in the above values now being serialized as follows: ```csharp -JsonSerializer.Serialize(foo1); // { "A" : 1 } -JsonSerializer.Serialize(foo2); // { "A" : 1, "B" : 2 } -JsonSerializer.Serialize(foo3); // { "A" : 1, "B" : 2, "C" : 3 } +JsonSerializer.Serialize(foo); // { "A" : 1 } +JsonSerializer.Serialize(bar); // { "A" : 1, "B" : 2 } +JsonSerializer.Serialize(baz); // { "A" : 1, "B" : 2, "C" : 3 } +``` +Polymorphism applies to nested values as well, for example: +```csharp +public class MyPoco +{ + public Foo Value { get; set; } +} + +JsonSerializer.Serialize(new MyPoco { Value = foo }); // { "Value" : { "A" : 1 } } +JsonSerializer.Serialize(new MyPoco { Value = bar }); // { "Value" : { "A" : 1, "B" : 2 } } +JsonSerializer.Serialize(new MyPoco { Value = baz }); // { "Value" : { "A" : 1, "B" : 2, "C" : 3 } } ``` Note that the `JsonPolymorphicType` attribute is not inherited by derived types. In the above example `Bar` inherits from `Foo` yet is not polymorphic in its own right: ```csharp -Bar bar = new Baz { A = 1, B = 2, C = 3 }; -JsonSerializer.Serialize(bar); // { "A" : 1, "B" : 2 } +JsonSerializer.Serialize(baz); // { "A" : 1, "B" : 2 } ``` If annotating the base class with an attribute is not possible, polymorphism can alternatively be opted in for a type using the new `JsonSerializerOptions.SupportedPolymorphicTypes` predicate: ```csharp