Use a better serialization/deserialization mechanism for data discovery

[Evangelink]

Describe the bug

Today, MSTest uses the DataContractJsonSerializer serializer in order to perform the serialization and deserialization of the data during discovery and execution.

The problem with current solution is that it requires custom objects to be marked with DataContractAttribute/DataMemberAttribute which isn't what user expects to do or to be doing for the sake of test.

At the moment, the BinaryFormatter seems to be the best option but it's not ideal as this formatter has a lot of security concerns associated (see https://learn.microsoft.com/dotnet/api/system.runtime.serialization.formatters.binary.binaryformatter#remarks)

Additional context

Related issues:

• MS Test 2.2.4 - 2.2.10: Tests fail because DynamicData gets only structs with default values (MS Test 2.2.4 - 2.2.10: Tests fail because DynamicData gets only structs with default values #1094)
• wrong value received by test method with ITestDataSource and values of struct with property of nullable primitive type (wrong value received by test method with ITestDataSource and values of struct with property of nullable primitive type #1022)
• DynamicData test is not passing exception objects correctly (DynamicData test is not passing exception objects correctly #1037)
• Using an interface in dynamic data causes a de-/serialization exception (Using an interface in dynamic data causes a de-/serialization exception #908)
• DateTime equality test fails after upgrading from 2.2.3 ([MSTest][Discovery] MSTestPlayground.exe Failed to discover tests from assembly (After upgrade from 2.1.2 to 2.2.8) #1019)
• Using dynamic data source method fails to work (Using dynamic data source method fails to work #1588)

[poizan42]

Is there really any issue with using BinaryFormatter here? The data doesn't cross a security boundary, does it?

[Evangelink]

Technically I don't see much issues but we would be flagged by internal MS compliance checks + there are many discussions toward removing this formatter in some new version of .NET (nothing acted) so I am not sure if best is to be using that and wait for some "break" to change mechanism.

[jnyrup]

Given BinaryFormatter Obsoletion Strategy it doesn't seem that using BinaryFormatter would be a good path forward.

[mungojam]

How about switching to System.Text.Json based serialization?

[Evangelink]

I would need to make some tests to see how many issues this would fix. It might also be problematic for .NET fx users.

Having some more experience with mstest, I think that there is no silver bullet solution and I would probably need to check if the data can be serialized (or to be exact if serializing/deserializing data gives me the same data as input) in which case we can "expand" the data otherwise we should not expand and so don't try to serialize/deserialize data.

[mungojam]

It might also be problematic for .NET fx users.

To answer that specific thought, it's supported in .net standard 2.0 and v4.6.2

https://www.nuget.org/packages/System.Text.Json/