System.Text.Json Deserialization throws NotSupportedException when implementing IEnumerable<T>

[christiannagel]

JsonSerializer.Deserialize throws NotSupportedException when IEnumerable<T> is implemented.

The Category and Card records implement the interface IEnumerable<T>:

public record Item(string Title, string Text, decimal Price);
public record Category(string Title) : IEnumerable<Item>
{
    public IList<Item> Items { get; init; } = new List<Item>();

    public IEnumerator<Item> GetEnumerator() => Items.GetEnumerator();

    IEnumerator IEnumerable.GetEnumerator() => ((IEnumerable)Items).GetEnumerator();

    public void Add(Item item) => Items.Add(item);
}
public record Card(string Title) : IEnumerable<Category>
{
    public IList<Category> Categories { get; init; } = new List<Category>();

    public IEnumerator<Category> GetEnumerator() => Categories.GetEnumerator();

    IEnumerator IEnumerable.GetEnumerator() => ((IEnumerable)Categories).GetEnumerator();

    public void Add(Category category) => Categories.Add(category);
}

This allows creating objects with an collection initializer:

Card card = new("The Restaurant")
{
    new Category("Appetizers")
    {
        new Item("Dungeon Crab Cocktail", "Classic cocktail sauce", 27M),
        new Item("Almond Crusted Scallops", "Almonds, Parmesan, chive beurre blanc", 19M)
    },
    new Category("Dinner")
    {
        new Item("Grilled King Salmon", "Lemon chive beurre blanc", 49M)
    }
};

Using a Card object, serialization with the JsonSerializer succeeds. Deserialization throws the NotSupportedException because the type is abstract, an interface, or is read only which is not the case.

Call stack:

System.NotSupportedException: The collection type 'Card' is abstract, an interface, or is read only, and could not be instantiated and populated. Path: $ | LineNumber: 0 | BytePositionInLine: 1.
 ---> System.NotSupportedException: The collection type 'Card' is abstract, an interface, or is read only, and could not be instantiated and populated.
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ThrowNotSupportedException(ReadStack& state, Utf8JsonReader& reader, NotSupportedException ex)
   at System.Text.Json.ThrowHelper.ThrowNotSupportedException_CannotPopulateCollection(Type type, Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.Serialization.Converters.IEnumerableOfTConverter`2.CreateCollection(Utf8JsonReader& reader, ReadStack& state, JsonSerializerOptions options)
   at System.Text.Json.Serialization.Converters.IEnumerableDefaultConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadCore[TValue](JsonConverter jsonConverter, Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadCore[TValue](Utf8JsonReader& reader, Type returnType, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, Type returnType, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at <Program>$.<<Main>$>g__DeserializeJson|0_1(String json) in C:\github\jsonserializerwithienumerable\JsonSerializerIssue\JsonSerializerIssue\Program.cs:line 48
   at <Program>$.<Main>$(String[] args) in C:\github\jsonserializerwithienumerable\JsonSerializerIssue\JsonSerializerIssue\Program.cs:line 31

If the implementation of the IEnumerable<T> interface is removed, and all other code stays the same, deserialization works as expected. The issue only occurs if the interface is implemented. Without this interface, the collection initializer cannot be used.

Sample project is available here: https://github.com/christiannagel/jsonserializerwithienumerable

Thanks!

[Symbai]

Like it says: Not supported. Track #45189

[christiannagel]

@Symbai I think this is a different scenario. The exception says NotSupportedException because the type is abstract, an interface, or is read only. The Card type is neither abstract, nor an interface, and not read-only. Removing the implementation of IEnumerable, deserialization is fully working.

[Symbai]

IEnumerable is an interface and interfaces are not supported because of missing polymorphic deserialization support. See #41749 and #45189

[huoyaoyuan]

This isn't the polymorphic case if you uses Deserialize<Card>.

[christiannagel]

@huoyaoyuan , yes, this is using Deserialize<Card>:
https://github.com/christiannagel/jsonserializerwithienumerable/blob/629d71f5eaa8e8c5aab1cf00ad7221b24b175098/JsonSerializerIssue/JsonSerializerIssue/Program.cs#L48

[christiannagel]

@Symbai , with the model type I can implement other interfaces, and this does not give any problem with deserialization.
This code is not deserializing IEnumerable, it's deserializing the Card: JsonSerializer.Deserialize<Card>(json)

[pinkfloydx33]

I ran into the very same problem. Deserializing to the concrete type throws an exception simply because that type implements IEnumerable. It seems the deserializer checks if the type implements the interface at all even when it shouldn't matter. Removing the interface worked, but was not what I wanted (for similar reasons to the above).

[wzchua]

actually, it looks like the serialization is also wrong here.

[
  [
    {
      "Title": "Dungeon Crab Cocktail",
      "Text": "Classic cocktail sauce",
      "Price": 27
    },
    {
      "Title": "Almond Crusted Scallops",
      "Text": "Almonds, Parmesan, chive beurre blanc",
      "Price": 19
    }
  ],
  [
    {
      "Title": "Grilled King Salmon",
      "Text": "Lemon chive beurre blanc",
      "Price": 49
    }
  ]
]

[christiannagel]

@wzchua, yes, you're right - serialization is also wrong. Card and Category types which implement IEnumerable are not correctly serialized, although there's no exception on serialization.
Now I tried the JSON that was correctly serialized with deserialization using the IEnumerable objects, and there's the same issue on deserialization.

The current status:

• incorrectly serialization of types implementing IEnumerable, but there's no exception
• NotSupportedException with deserialization of types implementing IEnumerable

[wzchua]

I'm not sure how this can be handled besides a custom converter.
You have an IEnumerable that you want to be treated as an object and not an IEnumerable.
The IEnumerable convertor comes before the Object converter because all IEnumerables are also objects

[christiannagel]

@wzchua, thanks for this information. Creating a custom converter that's just using the object converter could be a solution. I'll look into how to create this on using the object converter.
As the converters are all declared with the access modifier internal, it might not be possible to specify to use the object converter instead of the IEnumerableOfTConverter directly with a custom converter.

Anyway, I think this scenario should result into a feature request:

• The IEnumerableOfTConverter to also serialize the additional state and not just the collection available via the iterator
• At least better error information on Deserialize (the error message does not apply at all)

[JohnYoungers]

Is this documentation incorrect?
https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-supported-collection-types?pivots=dotnet-5-0

I've created a simple class that implements IReadOnlyList<T>, and based on that documentaiton it should be de-serializable; however, I'm getting this same message.

It's a bit frustrating that all of the System.Text.Json.Serialization.Converters are internal: for my purposes I can get away with reusing the ArrayConverter, but I'm having a hard time determining the cleanest way of going about that

[christiannagel]

Thanks for the information and the link @jayoungers - I missed this documentation.

The list of types and interfaces here https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-supported-collection-types?pivots=dotnet-5-0#systemcollectionsgeneric-namespace only list the members that are serializable (including IReadOnlyList<T>.

With the section on custom collections, IReadOnlyList<T> is not in the list of base classes/interfaces to support deserialization.

Because IEnumerable<T> does not show up with deserialization support here, in the sample code I created a new branch to implement IList<T> with the Card and Category classes: https://github.com/christiannagel/jsonserializerwithienumerable/tree/IListInsteadOfIEnumerable
Deserialization now succeeds, but of course the properties of the class are still not serialized.

I'm not thinking about these feature requests:

• The IEnumerableOfTConverter should also serialize additional properties and not just the collection that's accessed via IEnumerable<T>
• The documentation at https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-supported-collection-types?pivots=dotnet-5-0 should explicitly tell that properties are not serialized for collection types
• The Deserialize method when used with IEnumerable<T> types should give a better exception information - non of the cases described with the error apply to the exception in this case
• At least some of the built-in converters should be public instead of internal to allow using them e.g. for custom conversion to serialize properties with collections

[JohnYoungers]

@christiannagel - regarding the public vs internal part, I ended up pulling it from the JsonSerializerOptions, which may have been the intended route: I'm not sure I could have simplified this more if I had direct access to the internal converter.

My collection class is RecordArray<T>:

internal class RecordArrayConverterFactory : JsonConverterFactory
{
    public override bool CanConvert(Type typeToConvert) 
        => typeToConvert.IsGenericType && typeToConvert.GetGenericTypeDefinition() == typeof(RecordArray<>);

    public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options) 
        => (JsonConverter)Activator.CreateInstance(typeof(RecordArrayConverter<>).MakeGenericType(typeToConvert.GetGenericArguments()[0]));

    private class RecordArrayConverter<T> : JsonConverter<RecordArray<T>>
    {
        public override RecordArray<T> Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) 
            => new RecordArray<T>(((JsonConverter<T[]>)options.GetConverter(typeof(T[]))).Read(ref reader, typeof(T[]), options));

        public override void Write(Utf8JsonWriter writer, RecordArray<T> value, JsonSerializerOptions options) 
            => ((JsonConverter<IEnumerable<T>>)options.GetConverter(typeof(IEnumerable<T>))).Write(writer, value, options);
    }
}

[ugumba]

There are 2 use cases for a class implementing a collection interface:

1. The class desires System.Text.Json's collection semantics. @christiannagel, your workaround deals with this case. Your custom converter could be made redundant by future support for a [JsonConstructor] with a single collection argument which matches the implemented interface.
2. The class desires System.Text.Json's object semantics (e.g. properties, optionally with [JsonConstructor], or other non-collection converter). This is the case I posted about in System.Text.Json ignores [JsonConstructor] when class implements a generic read-only collection #49699. Basically, System.Text.Json should be able to completely ignore the collection aspect of the class, preferably by implicit inference, otherwise by explicit instruction. @christiannagel's workaround does not work for this case, as there is no type you can request a converter for. Since the core converters are internal, our only option is very hand-crafted converters!

These use cases are not mutually exclusive - a class could desire collection semantics in addition to object semantics. This is the currently unsupported case mentioned in docs#22402.

The lack of support for these "families" of classes makes for some very cumbersome hacks (custom converters etc.) for otherwise very powerful and expressive .NET data types.

[layomia]

@christiannagel - the serializer treats all types that are assignable to IEnumerable as collections. It seems that in this case, you want the Card type to be treated as a POCO, i.e with JSON object notation, serializing each property on the type.

There is a feature request to do just this: #1808. One solution is to provide a new attribute, say [JsonObject], which can be placed on a given type. The serializer would then treat that type as object with members. Would having this feature satisfy your needs?

[ghost]

This issue has been automatically marked no recent activity because it has been marked as needs author feedback but has not had any activity for 14 days. It will be closed if no further activity occurs within 7 more days. Any new comment (by anyone, not necessarily the author) will remove no recent activity