[Urgent] TRX logger produces invalid XML file #320

natemcmaster · 2017-01-05T22:09:09Z

Description

The TRX logger can produce invalid XML

Steps to reproduce

Create an xunit test project
Add this test code:

using System;
using System.Collections.Generic;
using System.IO;
using Xunit;
using Xunit.Abstractions;

namespace XUnitTestProject1
{
    public class UnitTest1
    {
        private ITestOutputHelper _output;

        public UnitTest1(ITestOutputHelper output)
        {
            _output = output;
        }

        [Fact]
        public void Test1()
        {
            foreach (var character in Path.GetInvalidFileNameChars())
            {
                _output.WriteLine($"Invalid char '{character}'");
            }
        }

        public static TheoryData TestValues
            => new TheoryData<IEnumerable<KeyValuePair<string, string>>>
            {
                { new []{ new KeyValuePair<string,string>("key", "string with \0 in it") } },
                { new []{ new KeyValuePair<string,string>("key2", "second string with \0 in it") } }
            };

        [Theory]
        [MemberData(nameof(TestValues))]
        public void Test2(IEnumerable<KeyValuePair<string, string>> values)
        {
        }
    }
}

Execute dotnet test --logger:trx

Expected behavior

TRX logger should sanitize the characters send to StdOut

Actual behavior

XML is invalid. Incorrect escape codes are used.

Character reference "&#x0" is an invalid XML character.

 <UnitTestResult executionId="862c473e-6f2f-4136-bd1c-5d9fcfb206e8" testId="662a0d34-144b-4917-aa1c-0ec240fbef84" testName="XUnitTestProject1.UnitTest1.Test2(values: [[key, string with &#x0; in it]])" computerName="EMPIRE-Z" duration="00:00:00.0060000" startTime="2017-01-05T18:44:59.6276259-08:00" endTime="2017-01-05T18:44:59.6276259-08:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="862c473e-6f2f-4136-bd1c-5d9fcfb206e8" />

Character reference "&#xB" is an invalid XML character.

<Results>
    <UnitTestResult executionId="1c1acadd-03d8-4af1-9462-b3856bac0fa1" testId="54931488-5172-41a4-b6e0-cab422b61256" testName="XUnitTestProject1.UnitTest1.Test1" computerName="EMPIRE-Z" duration="00:00:00.0080000" startTime="2017-01-05T14:07:06.3572382-08:00" endTime="2017-01-05T14:07:06.3572382-08:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="1c1acadd-03d8-4af1-9462-b3856bac0fa1">
      <Output>
        <StdOut>Invalid char '"'
Invalid char '&lt;'
Invalid char '&gt;'
Invalid char '|'
Invalid char '\0'
Invalid char '\x01'
Invalid char '\x02'
Invalid char '\x03'
Invalid char '\x04'
Invalid char '\x05'
Invalid char '\x06'
Invalid char '\x07'
Invalid char '\x08'
Invalid char '	'
Invalid char '
'
Invalid char '&#xB;'
Invalid char '&#xC;'
Invalid char '
'
Invalid char '\x0e'
Invalid char '\x0f'
Invalid char '\x10'
Invalid char '\x11'
Invalid char '\x12'
Invalid char '\x13'
Invalid char '\x14'
Invalid char '\x15'
Invalid char '\x16'
Invalid char '\x17'
Invalid char '\x18'
Invalid char '\x19'
Invalid char '\x1a'
Invalid char '\x1b'
Invalid char '\x1c'
Invalid char '\x1d'
Invalid char '\x1e'
Invalid char '\x1f'
Invalid char ':'
Invalid char '*'
Invalid char '?'
Invalid char '\'
Invalid char '/'</StdOut>
      </Output>
    </UnitTestResult>

Environment

dotnet-CLI 1.0.0-preview4-004233

The text was updated successfully, but these errors were encountered:

Workaround microsoft/vstest#320

Eilon · 2017-01-31T00:36:56Z

vstest team - any update on this issue? This is blocking our ability to get test results on our CI machine. All we get is pass/fail, with no info on what tests ran.

natemcmaster · 2017-01-31T04:47:33Z

cc @codito

* Issue: #320 Fix: Replace invalid xml character with ?

Faizan2304 · 2017-02-03T05:14:52Z

@Eilon @natemcmaster : We are sanitizing the character by replacing it by "?"
Any thoughts??

After the fix the trx will look like

<?xml version="1.0" encoding="UTF-8"?>
<TestRun id="7efb1637-5204-4b84-9d19-205de9f42c60" name="faahmad@FAAHMAD-REL113 2017-01-31 04:23:34" runUser="FAREAST\faahmad" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2017-01-31T04:23:34.9288418-08:00" queuing="2017-01-31T04:23:34.9288418-08:00" start="2017-01-31T04:23:32.2258134-08:00" finish="2017-01-31T04:23:34.9418414-08:00" />
  <TestSettings name="default" id="3fb75b39-566e-4240-9a00-13c9b459dd4a">
    <Deployment runDeploymentRoot="faahmad_FAAHMAD-REL113 2017-01-31 04_23_34" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="cfe1bba8-ca10-4635-a486-92a6fca0083a" testId="4975e323-e9af-42a2-ac66-58fde7510691" testName="XUnitTestProject1.UnitTest1.Test2(values: [[key, string with ? in it]])" computerName="FAAHMAD-REL113" duration="00:00:00.0130000" startTime="2017-01-31T04:23:34.9398416-08:00" endTime="2017-01-31T04:23:34.9398416-08:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="cfe1bba8-ca10-4635-a486-92a6fca0083a" />
    <UnitTestResult executionId="cdcd7900-7710-4063-adc3-40ad5a247f73" testId="4d9c5aa4-c332-4d9a-9969-a4fb64e4867e" testName="XUnitTestProject1.UnitTest1.Test2(values: [[key2, second string with ? in it]])" computerName="FAAHMAD-REL113" duration="00:00:00.0010000" startTime="2017-01-31T04:23:34.9408436-08:00" endTime="2017-01-31T04:23:34.9408436-08:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="cdcd7900-7710-4063-adc3-40ad5a247f73" />
    <UnitTestResult executionId="21259b9c-663f-4aab-a7d7-2967936ff03e" testId="e55bf2ce-7bdf-4d15-9588-e482428b961a" testName="XUnitTestProject1.UnitTest1.Test1" computerName="FAAHMAD-REL113" duration="00:00:00.0040000" startTime="2017-01-31T04:23:34.9408436-08:00" endTime="2017-01-31T04:23:34.9408436-08:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="21259b9c-663f-4aab-a7d7-2967936ff03e">
      <Output>
        <StdOut>Invalid char '"'
Invalid char '&lt;'
Invalid char '&gt;'
Invalid char '|'
Invalid char '\0'
Invalid char '\x01'
Invalid char '\x02'
Invalid char '\x03'
Invalid char '\x04'
Invalid char '\x05'
Invalid char '\x06'
Invalid char '\x07'
Invalid char '\x08'
Invalid char '	'
Invalid char '
'
Invalid char '?'
Invalid char '?'
Invalid char '
'
Invalid char '\x0e'
Invalid char '\x0f'
Invalid char '\x10'
Invalid char '\x11'
Invalid char '\x12'
Invalid char '\x13'
Invalid char '\x14'
Invalid char '\x15'
Invalid char '\x16'
Invalid char '\x17'
Invalid char '\x18'
Invalid char '\x19'
Invalid char '\x1a'
Invalid char '\x1b'
Invalid char '\x1c'
Invalid char '\x1d'
Invalid char '\x1e'
Invalid char '\x1f'
Invalid char ':'
Invalid char '*'
Invalid char '?'
Invalid char '\'
Invalid char '/'</StdOut>
      </Output>
    </UnitTestResult>
  </Results>
  <TestDefinitions>
    <UnitTest name="XUnitTestProject1.UnitTest1.Test2(values: [[key, string with ? in it]])" storage="c:\users\faahmad\documents\visual studio 2017\projects\xunittestproject1\xunittestproject1\bin\debug\netcoreapp1.0\xunittestproject1.dll" id="4975e323-e9af-42a2-ac66-58fde7510691">
      <Execution id="cfe1bba8-ca10-4635-a486-92a6fca0083a" />
      <TestMethod codeBase="C:\Users\faahmad\Documents\visual studio 2017\Projects\XUnitTestProject1\XUnitTestProject1\bin\Debug\netcoreapp1.0\XUnitTestProject1.dll" executorUriOfAdapter="executor://xunit/VsTestRunner2" className="XUnitTestProject1.UnitTest1" name="XUnitTestProject1.UnitTest1.Test2(values: [[key, string with ? in it]])" />
    </UnitTest>
    <UnitTest name="XUnitTestProject1.UnitTest1.Test2(values: [[key2, second string with ? in it]])" storage="c:\users\faahmad\documents\visual studio 2017\projects\xunittestproject1\xunittestproject1\bin\debug\netcoreapp1.0\xunittestproject1.dll" id="4d9c5aa4-c332-4d9a-9969-a4fb64e4867e">
      <Execution id="cdcd7900-7710-4063-adc3-40ad5a247f73" />
      <TestMethod codeBase="C:\Users\faahmad\Documents\visual studio 2017\Projects\XUnitTestProject1\XUnitTestProject1\bin\Debug\netcoreapp1.0\XUnitTestProject1.dll" executorUriOfAdapter="executor://xunit/VsTestRunner2" className="XUnitTestProject1.UnitTest1" name="XUnitTestProject1.UnitTest1.Test2(values: [[key2, second string with ? in it]])" />
    </UnitTest>
    <UnitTest name="XUnitTestProject1.UnitTest1.Test1" storage="c:\users\faahmad\documents\visual studio 2017\projects\xunittestproject1\xunittestproject1\bin\debug\netcoreapp1.0\xunittestproject1.dll" id="e55bf2ce-7bdf-4d15-9588-e482428b961a">
      <Execution id="21259b9c-663f-4aab-a7d7-2967936ff03e" />
      <TestMethod codeBase="C:\Users\faahmad\Documents\visual studio 2017\Projects\XUnitTestProject1\XUnitTestProject1\bin\Debug\netcoreapp1.0\XUnitTestProject1.dll" executorUriOfAdapter="executor://xunit/VsTestRunner2" className="XUnitTestProject1.UnitTest1" name="XUnitTestProject1.UnitTest1.Test1" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="4975e323-e9af-42a2-ac66-58fde7510691" executionId="cfe1bba8-ca10-4635-a486-92a6fca0083a" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="4d9c5aa4-c332-4d9a-9969-a4fb64e4867e" executionId="cdcd7900-7710-4063-adc3-40ad5a247f73" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="e55bf2ce-7bdf-4d15-9588-e482428b961a" executionId="21259b9c-663f-4aab-a7d7-2967936ff03e" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Completed">
    <Counters total="3" executed="3" passed="3" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
  </ResultSummary>
</TestRun>```

natemcmaster · 2017-02-03T05:35:07Z

Certainly better than producing invalid XML. Since '?' could be a wide range of characters, it might make it hard to distinguish tests.

Would it be possible to make it a readable format, like \xNNNN? As a bonus, it would be nice-to-have if it did that for all whitespace characters.

Faizan2304 · 2017-02-07T14:43:11Z

Thanks for your feedback. Yes we can replace invalid character with escape sequence like \uNNNN.
This fix has been checked-in in master (#437)

natemcmaster · 2017-02-07T16:13:14Z

Awesome! Thanks @Faizan2304.

* Issue: microsoft#320 Fix: Replace invalid xml character with ?

* Replace invalid xml character with ? (#425) * Issue: #320 Fix: Replace invalid xml character with ? * Testhost std error messages should pass to vstest.console (#417) * Log testhost std error message if exit with error * Add exit call back before process start * add unit test and acceptance test * Address PR comments * xlf changes * Replacing invalid xml character by unicode escape character (#437) * Replace invalid xml character by unicode escape character * Add auto-generated header to generated Program (#424) Add auto-generated header comment to generated Program.cs and Program.vb files to resolve #420. * Show IFrameworkHandle.SendMessage messages from test adapters. (#429) * Change the type name of the class generated in the GenerateProgramFile target (#441) * Build TestPlatform.vsix with RC3 VSSDK tooling (#438) * Build vsix using RC3 VSSDK tooling. * Issue: https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_workitems?id=366417&_a=edit Fix: Update manifest.json to have entry for all the files and theit sha256 which are going to be part of vsix. Testing: Manually validated by installing new vsix. The old folder is getting deleted after installing new one. * Fix vsix folder name in extension folder with name "TestPlatform" * Log test host standard error (#458) * Log test host standard error * Update xlf files * Update acceptance test * add 'Reason: ' prefix

natemcmaster changed the title ~~TRX logger should santize StdOut to ensure it is valid XML~~ TRX logger should santize StdOut to ensure trx file is valid XML Jan 5, 2017

natemcmaster pushed a commit to aspnet/Configuration that referenced this issue Jan 5, 2017

Remove output logging from test that causes invalid TRX format.

aff0c1d

Workaround microsoft/vstest#320

natemcmaster changed the title ~~TRX logger should santize StdOut to ensure trx file is valid XML~~ TRX logger produces invalid XML file Jan 6, 2017

NTaylorMullen mentioned this issue Jan 6, 2017

[Tracker] MSBuild conversion issues aspnet/Mvc#5482

Closed

55 tasks

codito added bug exp: cli labels Jan 6, 2017

codito added the blocking-partner label Jan 31, 2017

codito changed the title ~~TRX logger produces invalid XML file~~ [Urgent] TRX logger produces invalid XML file Jan 31, 2017

codito assigned Faizan2304 Jan 31, 2017

Faizan2304 mentioned this issue Feb 1, 2017

Replace invalid xml character with ? #425

Merged

Faizan2304 added a commit that referenced this issue Feb 2, 2017

Replace invalid xml character with ? (#425)

01a1abf

* Issue: #320 Fix: Replace invalid xml character with ?

Faizan2304 closed this as completed Feb 2, 2017

codito pushed a commit to codito/vstest that referenced this issue Feb 9, 2017

Replace invalid xml character with ? (microsoft#425)

79e9a42

* Issue: microsoft#320 Fix: Replace invalid xml character with ?

codito mentioned this issue Feb 9, 2017

Port of fixes for RTW #462

Merged

This was referenced Feb 10, 2017

Insert TestPlatform v15.0.0-preview-20170210-02. dotnet/sdk#855

Merged

Insert TestPlatform v15.0.0-preview-20170210-02. dotnet/cli#5657

Merged

This was referenced Feb 15, 2017

Update Microsoft.NET.Test.Sdk version in test templates to 15.0.0-preview-20170210-02 dotnet/templating#325

Closed

Update testplatform to 15.0.0-preview-20170210-02 dotnet/templating#326

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Urgent] TRX logger produces invalid XML file #320

[Urgent] TRX logger produces invalid XML file #320

natemcmaster commented Jan 5, 2017 •

edited

Loading

Eilon commented Jan 31, 2017

natemcmaster commented Jan 31, 2017

Faizan2304 commented Feb 3, 2017 •

edited

Loading

natemcmaster commented Feb 3, 2017

Faizan2304 commented Feb 7, 2017

natemcmaster commented Feb 7, 2017

[Urgent] TRX logger produces invalid XML file #320

[Urgent] TRX logger produces invalid XML file #320

Comments

natemcmaster commented Jan 5, 2017 • edited Loading

Description

Steps to reproduce

Expected behavior

Actual behavior

Environment

Eilon commented Jan 31, 2017

natemcmaster commented Jan 31, 2017

Faizan2304 commented Feb 3, 2017 • edited Loading

natemcmaster commented Feb 3, 2017

Faizan2304 commented Feb 7, 2017

natemcmaster commented Feb 7, 2017

natemcmaster commented Jan 5, 2017 •

edited

Loading

Faizan2304 commented Feb 3, 2017 •

edited

Loading