Skip to content

[Bug]: Documentation: runsettings is not actually supported #4687

@dosolkowski-work

Description

@dosolkowski-work

Description

The documentation for code coverage (under extensions, as well as for individual frameworks e.g. migrating from NUnit) says that code coverage behavior can be customized using a .runsettings file. However, .runsettings files do not appear to be supported by TUnit using Microsoft.Testing.Platform; instead, MTP uses testconfig.json files with basically the same options, although some defaults are different. The documentation should show testconfig.json examples instead.

Expected Behavior

Using a .runsettings file would customize the behavior of the code coverage tool.

Actual Behavior

The code coverage report is always completely blank, with the message "Empty results generated: No binaries were instrumented. Make sure the tests ran, required binaries were loaded, had matching symbol files, and were not excluded through custom settings. For native C++ code coverage you have to link with /PROFILE option enabled. For more information see https://docs.microsoft.com/visualstudio/test/troubleshooting-code-coverage"

Steps to Reproduce

Example .runsettings file:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <DataCollectionRunSettings>
        <DataCollectors>
            <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
                <Configuration>
                    <CodeCoverage>
                        <Attributes>
                            <Exclude>
                                <Attribute>^System\.Diagnostics\.DebuggerHiddenAttribute$</Attribute>
                                <Attribute>^System\.Diagnostics\.DebuggerNonUserCodeAttribute$</Attribute>
                                <Attribute>^System\.CodeDom\.Compiler\.GeneratedCodeAttribute$</Attribute>
                                <Attribute>^System\.Diagnostics\.CodeAnalysis\.ExcludeFromCodeCoverageAttribute$</Attribute>
                            </Exclude>
                        </Attributes>
                    </CodeCoverage>
                </Configuration>
            </DataCollector>
        </DataCollectors>
    </DataCollectionRunSettings>
</RunSettings>

TUnit Version

1.12.139

.NET Version

.NET 10

Operating System

Windows

IDE / Test Runner

Visual Studio

Error Output / Stack Trace

Additional Context

There isn't actually an option to pass a .runsettings file to dotnet test, it only accepts testconfig.json files. The only way to even try to supply a .runsettings file is in Visual Studio, because it doesn't support testconfig.json yet, but this doesn't work for me. Even if there is a way to get it working, it would only be an option in Visual Studio, and everyone not using Visual Studio would still have to use testconfig.json.

IDE-Specific Issue?

  • I've confirmed this issue occurs when running via dotnet test or dotnet run, not just in my IDE

Metadata

Metadata

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions