DataTable serialisation is broken between 2.1 and previous versions or .NET framework

[marek-safar]

The fix dotnet/corefx#27286 broke DataTable serialization between the previous versions and 2.1 version netcore. It fails with the following error when deserializing data stored using the previous version on netcore 2.1.

 System.Runtime.Serialization.SerializationException : Member 'DataTable.DataColumn_0.DataType_AssemblyQualifiedName' was not found.

[karelz]

cc @Anipik @ViktorHofer @danmosemsft

[ViktorHofer]

Thanks for reporting Stephen. It's obvious why this is failing (GetValue on a string that's not available in blobs before the change) and it seems we have a test whole as we don't have a test case in BinaryFormatterTestData therefore it doesn't roundtrip across frameworks.

[Anipik]

@ViktorHofer i am able to understand why it is failing on .NET Framework but can you explain why it is failing for previous versions of .NET Core.
dotnet/corefx#27286 handles the case when the format is binary and the number of columns are non-zero. Earlier in this case it was throwing serialization exception because the dataColumn was not serializable.

[divega]

@Anipik @ViktorHofer @danmosemsft what do you think the plan should be for this issue?

From reading the comments at dotnet/corefx#27286 and https://github.com/dotnet/corefx/issues/26472#issuecomment-359152205 it seems that this was broken for .NET Core before the change and it was known that the change would break compatibility with other frameworks.

Unless something has changed and we can have a different fix that preserves compatibility, it is hard to think of anything actionable for .NET Core 2.1.

[danmoseley]

I agree, the DataType field was changed to DataType_AssemblyQualifiedName because we do not want to make Type binary serializable. That means DataTable when serialized with dt.RemotingFormat = SerializationFormat.Binary cannot be exchanged between .NET Core and .NET Framework. (Nor with .NET Core 2.0). As noted in dotnet/corefx#26742, if you want to exchange with .NET Framework you have to serialize DataTable with SerializationFormat.Xml, which is DataTable's default.

@marek-safar this is therefore unfortunately expected, and to workaround it you must avoid serializing with SerializationFormat.Binary set on the DataTable.

If I understand right then this issue can be closed.

[divega]

@danmosemsft @marek-safar should we consider increasing compatibility moving forward by updating .NET Framework and Mono to understand and produce DataType_AssemblyQualifiedName in addition to DataType, and by introducing some sort of flag to control the behavior on all implementations?

[marek-safar]

@divega @danmosemsft @karelz The MSDN page explicitly lists DataTable to be

guaranteed to be serializable between .NET Framework 4.5.1 and later versions and .NET Core 2.0 and later versions

which I think is no longer true after the change. We should either fix the code to support what we promised or not give such promises.

[danmoseley]

@ViktorHofer can you please get the doc fixed? It should say "unless you set RemotingFormat to SerializationFormat.Binary in which case it can only be exchanged with .NET Core 2.1 and later versions."

[ViktorHofer]

Right, please ignore my initial comment here, I was confused. Sure, I'll send a PR to docs to refine the message.

[danmoseley]

@ViktorHofer since this is 2.1 can you please get it closed...

[ViktorHofer]

Fixed in dotnet/docs#4956. Closing.