Inconsistency in TRX logging between .NET Core 3.1.103 and 3.1.300

[GeorchW]

Description

The TRX logs are different between .NET Core 3.1.103 and 3.1.300. We noticed while relying on the output in the VSCode .NET Test Explorer extension: formulahendry/vscode-dotnet-test-explorer#267

Steps to reproduce

1. Clone this repo: https://github.com/formulahendry/vscode-dotnet-test-explorer/
2. cd to this folder: https://github.com/formulahendry/vscode-dotnet-test-explorer/tree/master/test/xunittests
3. Run dotnet test --no-build --logger "trx;LogFileName=testResultsXml.trx" --filter "FullyQualifiedName=XunitTests.TestClass3.Fail" in both versions of the .NET Core SDK

Expected behavior

The generated TRX file follows some kind of specification and is consistent.

Actual behavior

The TRX file differs. The output for .NET Core 3.1.103 is:

<TestDefinitions>
  <UnitTest name="XunitTests.TestClass3.Fail(value: 2)" storage="/path/to/the/repo/vscode-dotnet-test-explorer/test/xunittests/bin/debug/netcoreapp3.1/xunittests.dll" id="5d23175e-8479-779f-39f4-23aa2608428b">
    <Execution id="285b15ac-175b-4204-8d35-32dab901e73d" />
    <TestMethod codeBase="/path/to/the/repo/vscode-dotnet-test-explorer/test/xunittests/bin/Debug/netcoreapp3.1/XunitTests.dll" adapterTypeName="executor://xunit/VsTestRunner2/netcoreapp" className="XunitTests.TestClass3" name="XunitTests.TestClass3.Fail(value: 2)" />
  </UnitTest>
</TestDefinitions>

while .NET Core 3.1.300 returns:

<TestDefinitions>
    <UnitTest name="XunitTests.TestClass3.Fail(value: 2)" storage="d:\SomePath\vscode-dotnet-test-explorer\test\xunittests\bin\debug\netcoreapp3.1\xunittests.dll" id="5d23175e-8479-779f-39f4-23aa2608428b">
      <Execution id="ceaf151e-e91c-4f92-ba85-4af475de7c28" />
      <TestMethod codeBase="d:\SomePath\vscode-dotnet-test-explorer\test\xunittests\bin\Debug\netcoreapp3.1\XunitTests.dll" adapterTypeName="executor://xunit/VsTestRunner2/netcoreapp" className="XunitTests.TestClass3" name="Fail" />
    </UnitTest>
  </TestDefinitions>

Note the differences in TestDefinitions/UnitTest/TestMethod/@name.

Environment

Happens under both Windows and Linux.

Actual question

On which behaviour should we rely in the future? What string is supposed to be contained in UnitTest/TestMethod/@name and UnitTest/@name? Is there documentation about this somewhere?

[Sanan07]

@GeorchW Rely on .NET Core 3.1.300 version. There is no documentation on it, but you can look at the change : #2355 @nohwnd

[GeorchW]

Okay, can you tell me what value TestDefinitions/UnitTest/@name contains? Is it just the un-split test method name? Or is it possible for a UnitTest node to contain multiple TestMethods?

[Sanan07]

TestDefinitions/UnitTest/@name - contains name of the test method. UnitTest contains only one TestMethod.

[Sanan07]

@GeorchW is everything clear now?

[GeorchW]

Yeah, it works now