Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Common exclude include assembly #377

Merged
merged 2 commits into from
Jan 13, 2024
Merged

Common exclude include assembly #377

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e475ed1
add functionality and update readme
tonyhallett Jan 13, 2024
451e157
readme table
tonyhallett Jan 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions FineCodeCoverageTests/AppOptionsProvider_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,8 @@ internal void Should_Use_Deseralized_String_From_Store_For_AppOption_Property(Fu
{ nameof(IAppOptions.ShowPartiallyCoveredInOverviewMargin),true},
{ nameof(IAppOptions.ShowUncoveredInOverviewMargin),true},
{ nameof(IAppOptions.ShowToolWindowToolbar),true},
{nameof(IAppOptions.ExcludeAssemblies),new string[]{ "Exclude"} },
{nameof(IAppOptions.IncludeAssemblies),new string[]{ "Include"} },
};
var mockJsonConvertService = autoMocker.GetMock<IJsonConvertService>();
mockJsonConvertService.Setup(
Expand Down
32 changes: 31 additions & 1 deletion FineCodeCoverageTests/CoverageProject_Settings_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
using AutoMoq;
using FineCodeCoverage.Core.Utilities;
using FineCodeCoverage.Engine.Model;
using FineCodeCoverage.Options;
Expand All @@ -12,7 +13,7 @@
using System.Threading.Tasks;
using System.Xml.Linq;

namespace Test
namespace FineCodeCoverageTests
{
public class CoverageProject_Settings_Tests
{
Expand Down Expand Up @@ -675,5 +676,34 @@ public async Task Should_Provide_The_Merged_Result_Using_Project_Settings()
var coverageProjectSettings = await coverageProjectSettingsManager.GetSettingsAsync(coverageProject);
Assert.AreSame(mergedSettings, coverageProjectSettings);
}

[Test]
public async Task Should_Add_Common_Assembly_Excludes_Includes()
{
var mockAppOptions = new Mock<IAppOptions>();
mockAppOptions.SetupAllProperties();
var appOptions = mockAppOptions.Object;
appOptions.Exclude = new string[] { "oldexclude" };
appOptions.Include = new string[] { "oldinclude" };
appOptions.ModulePathsExclude = new string[] { "msexclude" };
appOptions.ModulePathsInclude = new string[] { "msinclude" };
appOptions.ExcludeAssemblies = new string[] { "excludeassembly" };
appOptions.IncludeAssemblies = new string[] { "includeassembly" };

var autoMoqer = new AutoMoqer();
var coverageProjectSettingsManager = autoMoqer.Create<CoverageProjectSettingsManager>();
autoMoqer.GetMock<ISettingsMerger>().Setup(settingsMerger => settingsMerger.Merge(
It.IsAny<IAppOptions>(),
It.IsAny<List<XElement>>(),
It.IsAny<XElement>()
)).Returns(appOptions);

var settings = await coverageProjectSettingsManager.GetSettingsAsync(new Mock<ICoverageProject>().Object);

Assert.That(settings.Exclude, Is.EquivalentTo(new string[] { "oldexclude", "[excludeassembly]*" }));
Assert.That(settings.Include, Is.EquivalentTo(new string[] { "oldinclude", "[includeassembly]*" }));
Assert.That(settings.ModulePathsExclude, Is.EquivalentTo(new string[] { "msexclude", ".*\\excludeassembly.dll$" }));
Assert.That(settings.ModulePathsInclude, Is.EquivalentTo(new string[] { "msinclude", ".*\\includeassembly.dll$" }));
}
}
}
5 changes: 5 additions & 0 deletions FineCodeCoverageTests/MsCodeCoverage/RunSettingsTemplateReplacementsFactory_Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ internal class TestMsCodeCoverageOptions : IMsCodeCoverageOptions

public bool IncludeTestAssembly { get; set; }
public bool IncludeReferencedProjects { get; set; }
public string[] ExcludeAssemblies { get; set; }
public string[] IncludeAssemblies { get; set; }
}

internal static class ReplacementsAssertions
Expand All @@ -51,6 +53,7 @@ public static void AssertAllEmpty(IRunSettingsTemplateReplacements replacements)
}
}


internal class RunSettingsTemplateReplacementsFactory_UserRunSettings_Tests
{
private RunSettingsTemplateReplacementsFactory runSettingsTemplateReplacementsFactory;
Expand Down Expand Up @@ -694,5 +697,7 @@ internal class TestCoverageProjectOptions : IAppOptions
public bool ShowToolWindowToolbar { get; set; }
public bool Hide0Coverable { get; set; }
public bool Hide0Coverage { get; set; }
public string[] ExcludeAssemblies { get; set; }
public string[] IncludeAssemblies { get; set; }
}
}
161 changes: 84 additions & 77 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ tool for most developers. It is currently in Beta.

With the old coverage it was possible for FCC to provide an abstraction over each tool's exclusion / inclusion options. This abstraction does not work for MS code coverage.
Thus you will find that there are separate configuration options for Ms coverage vs old coverage and options that are common to the two.
Assembly level exclusions and inclusions can be achieved - see ExcludeAssemblies and IncludeAssemblies.
Configuration is ( mostly ) determined from Visual Studio options, finecodecoverage-settings.xml files and project msbuild properties. All of these settings are optional.
For options that have a project scope, these settings form a hierarchy where lower levels override or, for collections, override or merge with the level above. This is described in detail further on.

Expand Down Expand Up @@ -227,101 +228,107 @@ If you are using option 1) then project and global options will only be used whe


#### Options
```
*** Common
CoverageColoursFromFontsAndColours Specify true to use Environment / Fonts and Colors / Text Editor for editor Coverage colouring ( if present).
Coverage Touched Area / Coverage Not Touched Area / Coverage Partially Touched Area.
When false colours used are Green, Red and Gold.
|Option |Description|
|--|---|
|**Common**||
|CoverageColoursFromFontsAndColours|Specify true to use Environment / Fonts and Colors / Text Editor for editor Coverage colouring ( if present). Coverage Touched Area / Coverage Not Touched Area / Coverage Partially Touched Area. When false colours used are Green, Red and Gold.|
|ShowCoverageInOverviewMargin|Set to false to prevent coverage marks in the overview margin|
|ShowCoveredInOverviewMargin|Set to false to prevent covered marks in the overview margin|
|ShowUncoveredInOverviewMargin|Set to false to prevent uncovered marks in the overview margin|
|ShowPartiallyCoveredInOverviewMargin|Set to false to prevent partially covered marks in the overview margin|
|ShowToolWindowToolbar|Set to false to hide the toolbar on the tool window. Requires restarting Visual Studio. The toolbar has buttons for viewing the Cobertura xml and the risk hotspots.|
|FCCSolutionOutputDirectoryName|To have fcc output visible in a sub folder of your solution provide this name|
|ToolsDirectory|Folder to which copy tools subfolder. Must alredy exist. Requires restart of VS.|
|ThresholdForCyclomaticComplexity| When [cyclomatic complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity) exceeds this value for a method then the method will be present in the risk hotspots tab. |
|StickyCoverageTable|Set to true for coverage table to have a sticky thead.|
|NamespacedClasses|Set to false to show classes in report in short form. Affects grouping.|
|HideFullyCovered|Set to true to hide classes, namespaces and assemblies that are fully covered.|
|Hide0Coverage|Set to true to hide classes, namespaces and assemblies that have 0% coverage.|
|Hide0Coverable|Set to false to show classes, namespaces and assemblies that are not coverable.|
|Enabled|Specifies whether or not coverage output is enabled|
|RunWhenTestsFail|By default coverage runs when tests fail. Set to false to prevent this. **Cannot be used in conjunction with RunInParallel**|
|RunWhenTestsExceed|Specify a value to only run coverage based upon the number of executing tests. **Cannot be used in conjunction with RunInParallel**|
|RunMsCodeCoverage|Change to IfInRunSettings to only collect with configured runsettings. Yes for runsettings generation.|
|IncludeTestAssembly|Specifies whether to report code coverage of the test assembly|
|IncludeReferencedProjects|Set to true to add all directly referenced projects to Include.|
|IncludeAssemblies|Provide a list of assemblies to include in coverage. The dll name without extension is used for matching.|
|ExcludeAssemblies| Provide a list of assemblies to exclude from coverage. The dll name without extension is used for matching.|
|<br>||
|**OpenCover / Coverlet**||
|AdjacentBuildOutput|If your tests are dependent upon their path set this to true.|
|Exclude|Filter expressions to exclude specific modules and types (multiple values)|
|Include|Filter expressions to include specific modules and types (multiple values)|
|ExcludeByFile|Glob patterns specifying source files to exclude e.g. **/Migrations/* (multiple values)|
|ExcludeByAttribute|Attributes to exclude from code coverage (multiple values)|
|RunInParallel|By default OpenCover / Coverlet tests run and then coverage is performed. Set to true to run coverage immediately|
|<br>||
|**Ms code coverage**|Each of below is an array of regexes to be transformed into runsettings elements [see](https://learn.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2022#include-or-exclude-assemblies-and-members)|
|ModulePathsExclude|Exclude - Matches assemblies specified by assembly name or file path.|
|ModulePathsInclude|Include - Matches assemblies specified by assembly name or file path.|
|CompanyNamesExclude|Exclude - Matches assemblies by the Company attribute.|
|CompanyNamesInclude|Include - Matches assemblies by the Company attribute.|
|PublicKeyTokensExclude|Exclude - Matches signed assemblies by the public key token.|
|PublicKeyTokensInclude|Include - Matches signed assemblies by the public key token.|
|SourcesExclude|Exclude - Matches elements by the path name of the source file in which they're defined.|
|SourcesInclude|Include - Matches elements by the path name of the source file in which they're defined.|
|AttributesExclude|Exclude - Matches elements that have the specified attribute. Specify the full name of the attribute|
|AttributesInclude|Include - Matches elements that have the specified attribute. Specify the full name of the attribute|
|FunctionsExclude|Exclude - Matches procedures, functions, or methods by fully qualified name, including the parameter list.|
|FunctionsInclude|Include - Matches procedures, functions, or methods by fully qualified name, including the parameter list.|
|<br>||
|**Coverlet**||
|RunSettingsOnly|Specify false for global and project options to be used for coverlet data collector configuration elements when not specified in runsettings|
|CoverletCollectorDirectoryPath|Specify path to directory containing coverlet collector files if you need functionality that the FCC version does not provide.|
|CoverletConsoleLocal|Specify true to use your own dotnet tools local install of coverlet console.|
|CoverletConsoleCustomPath|Specify path to coverlet console exe if you need functionality that the FCC version does not provide.|
|CoverletConsoleGlobal|Specify true to use your own dotnet tools global install of coverlet console.|
|**The "CoverletConsole" settings have precedence Local / CustomPath / Global.**||
|<br>||
|**OpenCover**||
|OpenCoverCustomPath|Specify path to open cover exe if you need functionality that the FCC version does not provide.|
|ThresholdForNPathComplexity|When [npath complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity) exceeds this value for a method then the method will be present in the risk hotspots tab.|
|ThresholdForCrapScore|When [crap score](https://testing.googleblog.com/2011/02/this-code-is-crap.html) exceeds this value for a method then the method will be present in the risk hotspots tab.|


ShowCoverageInOverviewMargin Set to false to prevent coverage marks in the overview margin
ShowCoveredInOverviewMargin Set to false to prevent covered marks in the overview margin
ShowUncoveredInOverviewMargin Set to false to prevent uncovered marks in the overview margin
ShowPartiallyCoveredInOverviewMargin Set to false to prevent partially covered marks in the overview margin
## Exclusions and inclusions
You probably want to set IncludeReferencedProjects to true. This will ensure that you do not get coverage for testing frameworks - only your code.

ShowToolWindowToolbar Set to false to hide the toolbar on the tool window. Requires restarting Visual Studio. The toolbar has buttons for viewing
the Cobertura xml and the risk hotspots.
Coverlet and OpenCover use filter expressions.
Filter expressions

FCCSolutionOutputDirectoryName To have fcc output visible in a sub folder of your solution provide this name
Wildcards

ToolsDirectory Folder to which copy tools subfolder. Must alredy exist. Requires restart of VS.
\* => matches zero or more characters

Examples

ThresholdForCyclomaticComplexity When [cyclomatic complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity) exceeds this value for a method then the method will be present in the risk hotspots tab.
[\*]* => All types in all assemblies.

StickyCoverageTable Set to true for coverage table to have a sticky thead.
NamespacedClasses Set to false to show classes in report in short form. Affects grouping.
HideFullyCovered Set to true to hide classes, namespaces and assemblies that are fully covered.
Hide0Coverage Set to true to hide classes, namespaces and assemblies that have 0% coverage.
Hide0Coverable Set to false to show classes, namespaces and assemblies that are not coverable.
[coverlet\.\*]Coverlet.Core.Coverage => The Coverage class in the Coverlet.Core namespace belonging to any assembly that matches coverlet.* (e.g coverlet.core)

Enabled Specifies whether or not coverage output is enabled
RunWhenTestsFail By default coverage runs when tests fail. Set to false to prevent this. **Cannot be used in conjunction with RunInParallel**
RunWhenTestsExceed Specify a value to only run coverage based upon the number of executing tests. **Cannot be used in conjunction with RunInParallel**
RunMsCodeCoverage Change to IfInRunSettings to only collect with configured runsettings. Yes for runsettings generation.
[\*\]Coverlet.Core.Instrumentation.* => All types belonging to Coverlet.Core.Instrumentation namespace in any assembly

IncludeTestAssembly Specifies whether to report code coverage of the test assembly
IncludeReferencedProjects Set to true to add all referenced projects to Include.
[coverlet\.\*.tests]* => All types in any assembly starting with coverlet. and ending with .tests

*** OpenCover / Coverlet
AdjacentBuildOutput If your tests are dependent upon their path set this to true.
Both 'Exclude' and 'Include' options can be used together but 'Exclude' takes precedence.

Exclude Filter expressions to exclude specific modules and types (multiple values)
Include Filter expressions to include specific modules and types (multiple values)
ExcludeByFile Glob patterns specifying source files to exclude e.g. **/Migrations/* (multiple values)
ExcludeByAttribute Attributes to exclude from code coverage (multiple values)
RunInParallel By default OpenCover / Coverlet tests run and then coverage is performed. Set to true to run coverage immediately
Ms code coverage uses [regexes](https://learn.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2022#regular-expressions).
You can include or exclude assemblies or specific types and members from code coverage analysis. If the Include section is empty or omitted, then all assemblies that are loaded and have associated PDB files are included. If an assembly or member matches a clause in the Exclude section, then it is excluded from code coverage. The Exclude section takes precedence over the Include section: if an assembly is listed in both Include and Exclude, it will not be included in code coverage.

Filter expressions
Wildcards
* => matches zero or more characters
Examples
[*]* => All types in all assemblies (nothing is instrumented)
[coverlet.*]Coverlet.Core.Coverage => The Coverage class in the Coverlet.Core namespace belonging to any assembly that matches coverlet.* (e.g coverlet.core)
[*]Coverlet.Core.Instrumentation.* => All types belonging to Coverlet.Core.Instrumentation namespace in any assembly
[coverlet.*.tests]* => All types in any assembly starting with coverlet. and ending with .tests

You can ignore a method or an entire class from code coverage by applying the [ExcludeFromCodeCoverage] attribute present in the System.Diagnostics.CodeAnalysis namespace.

Both 'Exclude' and 'Include' options can be used together but 'Exclude' takes precedence.
For .Net Framework this attribute cannot be applied at the assembly level. See FCCExcludeFromCodeCoverage above for similar functinality.

You can ignore a method or an entire class from code coverage by creating and applying the [ExcludeFromCodeCoverage] attribute present in the System.Diagnostics.CodeAnalysis namespace.
You can also ignore additional attributes by adding to the 'ExcludeByAttributes' list (short name or full name supported) e.g. :
[GeneratedCode] => Present in System.CodeDom.Compiler namespace
[MyCustomExcludeFromCodeCoverage] => Any custom attribute that you may define
You can also ignore additional attributes by adding to the 'ExcludeByAttributes' list for Coverlet/OpenCover (short name or full name supported)

*** MS Code Coverage each multiple regexes to be transformed into runsettings elements
ModulePathsExclude
ModulePathsInclude
CompanyNamesExclude
CompanyNamesInclude
PublicKeyTokensExclude
PublicKeyTokensInclude
SourcesExclude
SourcesInclude
AttributesExclude
AttributesInclude
FunctionsExclude
FunctionsInclude
*** Coverlet
RunSettingsOnly Specify false for global and project options to be used for coverlet data collector configuration elements when not specified in runsettings
CoverletCollectorDirectoryPath Specify path to directory containing coverlet collector files if you need functionality that the FCC version does not provide.
CoverletConsoleLocal Specify true to use your own dotnet tools local install of coverlet console.
CoverletConsoleCustomPath Specify path to coverlet console exe if you need functionality that the FCC version does not provide.
CoverletConsoleGlobal Specify true to use your own dotnet tools global install of coverlet console.
The "CoverletConsole" settings have precedence Local / CustomPath / Global.
*** OpenCover
OpenCoverCustomPath Specify path to open cover exe if you need functionality that the FCC version does not provide.
ThresholdForNPathComplexity When [npath complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity) exceeds this value for a method then the method will be present in the risk hotspots tab. OpenCover only.
ThresholdForCrapScore When [crap score](https://testing.googleblog.com/2011/02/this-code-is-crap.html) exceeds this value for a method then the method will be present in the risk hotspots tab. OpenCover only.
e.g. :

[GeneratedCode] => Present in System.CodeDom.Compiler namespace

```
## Exclusions and inclusions
You probably want to set IncludeReferencedProjects to true. This will ensure that you do not get coverage for testing frameworks - only your code.
[MyCustomExcludeFromCodeCoverage] => Any custom attribute that you may define

or for ms code coverage - AttributesExclude

## FCC Output
FCC outputs, by default, inside each test project's Debug folder.
Expand Down
Loading
Loading