Using Queue, Queue<T>, Stack, and ConcurrentStack<T> as properties in classes deserialized by System.Text.Json does not work when trimmed

[eerhardt]

Publish and run the following application with -p:PublishTrimmed=true

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Text.Json;

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            var json = @"{""Name"":""Test"",""Sizes"":[0, 256, 256]}";

            var s = (Shape)JsonSerializer.Deserialize(json, typeof(Shape));
            Console.WriteLine(s.Sizes.Dequeue());
            Console.WriteLine(s.Sizes.Dequeue());
            Console.WriteLine(s.Sizes.Dequeue());
        }
    }

    public class Shape
    {
        public string Name { get; set; }

        //public ConcurrentStack<int> Sizes { get; set; }
        public Queue<int> Sizes { get; set; }
    }
}

1. dotnet publish -r win-x64 -p:PublishTrimmed=true
2. bin\Debug\net6.0\win-x64\publish\HelloWorld.exe

Expected results

The application should run, it does when I use List<T> for the property.

Actual results

Unhandled exception. System.NotSupportedException: The type 'System.Collections.Generic.Queue`1[System.Int32]' is not supported. Path: $.Sizes | LineNumber: 0 | BytePositionInLine: 24.
 ---> System.NotSupportedException: The type 'System.Collections.Generic.Queue`1[System.Int32]' is not supported.
   at System.Text.Json.ThrowHelper.ThrowNotSupportedException_SerializationNotSupported(Type ) in System.Text.Json.dll:token 0x60000d6+0x10
   at System.Text.Json.Serialization.Converters.QueueOfTConverter`2.CreateCollection(Utf8JsonReader& , ReadStack& , JsonSerializerOptions ) in System.Text.Json.dll:token 0x600070c+0x12
   at System.Text.Json.Serialization.Converters.IEnumerableDefaultConverter`2.OnTryRead(Utf8JsonReader& , Type , JsonSerializerOptions , ReadStack& , TCollection& ) in System.Text.Json.dll:token 0x60006d9+0x32
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& , Type , JsonSerializerOptions , ReadStack& , T& ) in System.Text.Json.dll:token 0x6000582+0x197
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object , ReadStack& , Utf8JsonReader& ) in System.Text.Json.dll:token 0x600061c+0xdb
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& , Type , JsonSerializerOptions , ReadStack& , T& ) in System.Text.Json.dll:token 0x6000738+0x8f
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& , Type , JsonSerializerOptions , ReadStack& , T& ) in System.Text.Json.dll:token 0x6000582+0x197
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& , JsonSerializerOptions , ReadStack& ) in System.Text.Json.dll:token 0x600056e+0xbf
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ThrowNotSupportedException(ReadStack& , Utf8JsonReader& , NotSupportedException ) in System.Text.Json.dll:token 0x60000fc+0xcb
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& , JsonSerializerOptions , ReadStack& ) in System.Text.Json.dll:token 0x600056e+0x1c0
   at System.Text.Json.Serialization.JsonConverter`1.ReadCoreAsObject(Utf8JsonReader& , JsonSerializerOptions , ReadStack& ) in System.Text.Json.dll:token 0x600056d+0x0
   at System.Text.Json.JsonSerializer.ReadCore[TValue](JsonConverter , Utf8JsonReader& , JsonSerializerOptions , ReadStack& ) in System.Text.Json.dll:token 0x60002c9+0x14
   at System.Text.Json.JsonSerializer.ReadUsingMetadata[TValue](ReadOnlySpan`1 , JsonTypeInfo , Nullable`1 ) in System.Text.Json.dll:token 0x60002ca+0x2f
   at System.Text.Json.JsonSerializer.ReadUsingMetadata[TValue](ReadOnlySpan`1 , JsonTypeInfo ) in System.Text.Json.dll:token 0x60002cd+0x59
   at System.Text.Json.JsonSerializer.ReadUsingOptions[TValue](ReadOnlySpan`1 , Type , JsonSerializerOptions ) in System.Text.Json.dll:token 0x60002cc+0x0
   at System.Text.Json.JsonSerializer.Deserialize(String , Type , JsonSerializerOptions ) in System.Text.Json.dll:token 0x60002cb+0x22
   at ConsoleApp1.Program.Main(String[] args) in C:\DotNetTest\HelloWorld\Program.cs:line 14

Notes

1. This seems to occur for Queue, Queue<T>, Stack, and ConcurrentStack<T> types.
2. This doesn't happen for List<T> nor Stack<T>, because I think other things are rooting their constructors.
3. These types may be esoteric enough that we don't want to root even more code in the application with System.Text.Json serialization, and instead tell people to use the source generator if they need this behavior. The user already is getting a warning by using JsonSerializer without the source generated information.

[ghost]

Tagging subscribers to 'linkable-framework': @eerhardt, @vitek-karas, @LakshanF, @sbomer
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Publish and run the following application with -p:PublishTrimmed=true

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Text.Json;

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            var json = @"{""Name"":""Test"",""Sizes"":[0, 256, 256]}";

            var s = (Shape)JsonSerializer.Deserialize(json, typeof(Shape));
            Console.WriteLine(s.Sizes.Dequeue());
            Console.WriteLine(s.Sizes.Dequeue());
            Console.WriteLine(s.Sizes.Dequeue());
        }
    }

    public class Shape
    {
        public string Name { get; set; }

        //public ConcurrentStack<int> Sizes { get; set; }
        public Queue<int> Sizes { get; set; }
    }
}

1. dotnet publish -r win-x64 -p:PublishTrimmed=true
2. bin\Debug\net6.0\win-x64\publish\HelloWorld.exe

Expected results

The application should run, it does when I use List<T> for the property.

Actual results

Unhandled exception. System.NotSupportedException: The type 'System.Collections.Generic.Queue`1[System.Int32]' is not supported. Path: $.Sizes | LineNumber: 0 | BytePositionInLine: 24.
 ---> System.NotSupportedException: The type 'System.Collections.Generic.Queue`1[System.Int32]' is not supported.
   at System.Text.Json.ThrowHelper.ThrowNotSupportedException_SerializationNotSupported(Type ) in System.Text.Json.dll:token 0x60000d6+0x10
   at System.Text.Json.Serialization.Converters.QueueOfTConverter`2.CreateCollection(Utf8JsonReader& , ReadStack& , JsonSerializerOptions ) in System.Text.Json.dll:token 0x600070c+0x12
   at System.Text.Json.Serialization.Converters.IEnumerableDefaultConverter`2.OnTryRead(Utf8JsonReader& , Type , JsonSerializerOptions , ReadStack& , TCollection& ) in System.Text.Json.dll:token 0x60006d9+0x32
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& , Type , JsonSerializerOptions , ReadStack& , T& ) in System.Text.Json.dll:token 0x6000582+0x197
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object , ReadStack& , Utf8JsonReader& ) in System.Text.Json.dll:token 0x600061c+0xdb
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& , Type , JsonSerializerOptions , ReadStack& , T& ) in System.Text.Json.dll:token 0x6000738+0x8f
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& , Type , JsonSerializerOptions , ReadStack& , T& ) in System.Text.Json.dll:token 0x6000582+0x197
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& , JsonSerializerOptions , ReadStack& ) in System.Text.Json.dll:token 0x600056e+0xbf
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ThrowNotSupportedException(ReadStack& , Utf8JsonReader& , NotSupportedException ) in System.Text.Json.dll:token 0x60000fc+0xcb
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& , JsonSerializerOptions , ReadStack& ) in System.Text.Json.dll:token 0x600056e+0x1c0
   at System.Text.Json.Serialization.JsonConverter`1.ReadCoreAsObject(Utf8JsonReader& , JsonSerializerOptions , ReadStack& ) in System.Text.Json.dll:token 0x600056d+0x0
   at System.Text.Json.JsonSerializer.ReadCore[TValue](JsonConverter , Utf8JsonReader& , JsonSerializerOptions , ReadStack& ) in System.Text.Json.dll:token 0x60002c9+0x14
   at System.Text.Json.JsonSerializer.ReadUsingMetadata[TValue](ReadOnlySpan`1 , JsonTypeInfo , Nullable`1 ) in System.Text.Json.dll:token 0x60002ca+0x2f
   at System.Text.Json.JsonSerializer.ReadUsingMetadata[TValue](ReadOnlySpan`1 , JsonTypeInfo ) in System.Text.Json.dll:token 0x60002cd+0x59
   at System.Text.Json.JsonSerializer.ReadUsingOptions[TValue](ReadOnlySpan`1 , Type , JsonSerializerOptions ) in System.Text.Json.dll:token 0x60002cc+0x0
   at System.Text.Json.JsonSerializer.Deserialize(String , Type , JsonSerializerOptions ) in System.Text.Json.dll:token 0x60002cb+0x22
   at ConsoleApp1.Program.Main(String[] args) in C:\DotNetTest\HelloWorld\Program.cs:line 14

Notes

1. This seems to occur for Queue, Queue<T>, Stack, and ConcurrentStack<T> types.
2. This doesn't happen for List<T> nor Stack<T>, because I think other things are rooting their constructors.
3. These types may be esoteric enough that we don't want to root even more code in the application with System.Text.Json serialization, and instead tell people to use the source generator if they need this behavior. The user already is getting a warning by using JsonSerializer without the source generated information.

Author:	eerhardt
Assignees:	-
Labels:	area-System.Text.Json, linkable-framework
Milestone:	-

[marek-safar]

Should we use the same approach as with #53317 and not keep them all the time (e.g. ConcurrentStack sounds like rarely used for serialization) ?

[layomia]

Should we use the same approach as with #53317 and not keep them all the time (e.g. ConcurrentStack sounds like rarely used for serialization) ?

+1
This idea was also captured in #51311 (comment).

We should have a trim-safe way to support these types - #53393.

[eerhardt]

Should we use the same approach as with #53317 and not keep them all the time (e.g. ConcurrentStack sounds like rarely used for serialization) ?

I did some investigation, and I don't think this is worth it. Looking through the collections that are hard-coded in JSON, I only find 4 that seem to be rarely used for serialization:

• Queue<T>
• Stack<T>
• ConcurrentQueue<T>
• ConcurrentStack<T>

In a default Blazor WASM app, the first three are all used elsewhere: Queue<T> and Stack<T> are used by ASP.NET and Microsoft.Extensions. ConcurrentQueue<T> is used by ThreadPool. The only one that isn't used elsewhere is ConcurrentStack<T>, but the linker is smart enough to see that this class isn't instantiated, so the trimmed code looks like:

public class ConcurrentStack<T> : IEnumerable<T>, IEnumerable, ICollection, IReadOnlyCollection<T>
{
	public int Count
	{
		[MethodImpl(MethodImplOptions.NoInlining)]
		get
		{
			throw new NotSupportedException("Linked away");
		}
	}

	[MethodImpl(MethodImplOptions.NoInlining)]
	public void Push(T P_0)
	{
		throw new NotSupportedException("Linked away");
	}

	[MethodImpl(MethodImplOptions.NoInlining)]
	public IEnumerator<T> GetEnumerator()
	{
		throw new NotSupportedException("Linked away");
	}

	[MethodImpl(MethodImplOptions.NoInlining)]
	IEnumerator IEnumerable.GetEnumerator()
	{
		throw new NotSupportedException("Linked away");
	}
}

Thus, doing the proposed work would only trim the ConcurrentStack<T> class and its 4 "throw NSE" methods, which isn't a lot of IL to trim.

For other applications that don't use ASP.NET or Microsoft.Extensions (and thus Stack<T> and Queue<T> can be trimmed), the linker will do the same optimization since it will see those classes are not instantiated. If the application only uses the JsonSerializer with the Source Generator, and not the "RequiresUnreferencedCode" JsonSerializer APIs, the JsonConverters for these unused collections will be trimmed, which means the unused collection classes will be trimmed as well.

[marek-safar]

I think excluding Concurrent* from the implicit set of dependencies is still a good hygiene approach.

[eiriktsarpalis]

@layomia has this been addressed in your recent collections support PR?

[layomia]

Yes with #55566 we now have a trim-safe way to support these types, and now we make no guarantees about trim-safety when the dynamic methods of the serializer are used. Closing this issue. The next item would be to re-write the trimming tests using the source-generator - #53437.