Skip to content

use .NET 8.0 target framework for coverlet.core and remove Newtonsoft.Json #1733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .config/dotnet-tools.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
"isRoot": true,
"tools": {
"dotnet-reportgenerator-globaltool": {
"version": "5.4.3",
"version": "5.4.7",
"commands": [
"reportgenerator"
],
"rollForward": false
}
}
}
}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -318,3 +318,4 @@ FolderProfile.pubxml
/NuGet.config
nuget.config
*.dmp
Playground*/
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Contributions are highly welcome, however, except for very small changes, kindly

## Requirements

[.NET SDK 6.0](https://dotnet.microsoft.com/en-us/download/dotnet/6.0)
[.NET SDK 7.0](https://dotnet.microsoft.com/en-us/download/dotnet/7.0)
[.NET SDK 8.0](https://dotnet.microsoft.com/en-us/download/dotnet/8.0)
STS version [.NET SDK 9.0](https://dotnet.microsoft.com/en-us/download/dotnet/9.0) for development environment

LTS version [.NET SDK 8.0](https://dotnet.microsoft.com/en-us/download/dotnet/8.0) for runtime environment

## Building the Project

Expand Down
31 changes: 0 additions & 31 deletions Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -48,35 +48,4 @@
<VSTestLogger>@(VSTestLogger)</VSTestLogger>
</PropertyGroup>

<PropertyGroup>
<!--
Do not change versions since we need to support VSTest DataCollectors. We need to load assembly version 1.6.0 to properly work
We can check minimum supported package version here https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Microsoft.TestPlatform.ObjectModel.csproj#L34
-->
<!-- VSTest dependency versions -->
<!-- https://github.com/microsoft/vstest/blob/fde8bf79d3f0f80e3548f873a56ffb4100c0ae49/eng/Versions.props#L22C1-L54C2 -->
<MicrosoftBuildUtilitiesCorePackageVersion>15.9.20</MicrosoftBuildUtilitiesCorePackageVersion>
<SystemReflectionMetadataVersion>1.6.0</SystemReflectionMetadataVersion>
<SystemCollectionsImmutableVersion>1.5.0</SystemCollectionsImmutableVersion>

<!-- some additional information about dependency management limitations for MSBuild tasks (coverlet.msbuild.tasks)
- msBuild task with dependencies https://natemcmaster.com/blog/2017/11/11/msbuild-task-with-dependencies/
- Aligning versions with MSBuild
- If you need a dependency that is also used in MSBuild itself, you have to align with the version MSBuild uses.
- This includes System.Reflection.Metadata, System.Collections.Immutable, NuGet, Newtonsoft.Json, and others.
- See example: dependencies.props (https://github.com/aspnet/BuildTools/blob/1f3f14382764e06b7e691e5ee89d12a280249284/build/dependencies.props#L19-L29)"

.NET 8.0.3xx (MSBuild) core-sdk-tasks version details https://github.com/dotnet/installer/blob/release/8.0.3xx/src/core-sdk-tasks/core-sdk-tasks.csproj

ALIGN PACKAGE VERSIONS of coverlet.core with coverlet.collector (vstest) and coverlet.msbuild.tasks (.NET SDK MSBuild)

e.g. candidates for version conflicts:
<MicrosoftBuildTaskVersion>15.7.179</MicrosoftBuildTaskVersion>
<MicrosoftBuildTaskUtilitiesCoreVersion>15.7.179</MicrosoftBuildTaskUtilitiesCoreVersion>
<NuGetBuildTasksPackageVersion>6.9.0-rc.86</NuGetBuildTasksPackageVersion>
<MicrosoftBuildTaskSystemReflectionMetaData>1.4.2</MicrosoftBuildTaskSystemReflectionMetaData>
<MicrosoftBuildTaskSystemCollectionImmutable>1.5.0</MicrosoftBuildTaskSystemCollectionImmutable> >= 1.3.1
-->
</PropertyGroup>

</Project>
23 changes: 12 additions & 11 deletions Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,20 @@
<GlobalPackageReference Include="Nerdbank.GitVersioning" Version="3.7.115" />
</ItemGroup>
<PropertyGroup>
<MicrosoftBuildVersion>17.13.9</MicrosoftBuildVersion>
<MicrosoftCodeAnalysisVersion>4.12.0</MicrosoftCodeAnalysisVersion>
<MicrosoftBuildVersion>17.13.26</MicrosoftBuildVersion>
<MicrosoftCodeAnalysisVersion>4.13.0</MicrosoftCodeAnalysisVersion>
<NugetPackageVersion>6.14.0</NugetPackageVersion>
<!-- Test Platform, .NET Test SDK and Object Model -->
<MicrosoftNETTestSdkVersion>17.13.0</MicrosoftNETTestSdkVersion>
<NugetPackageVersion>6.13.2</NugetPackageVersion>
<XunitV3Version>2.0.0</XunitV3Version>
<XunitRunnerVisualstudioVersion>3.0.2</XunitRunnerVisualstudioVersion>
<XunitRunnerVisualstudioVersion>3.1.0</XunitRunnerVisualstudioVersion>
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="DotNetConfig" Version="1.2.0" />
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="8.0.0" />
<PackageVersion Include="Microsoft.Build.Utilities.Core" Version="$(MicrosoftBuildVersion)" />
<PackageVersion Include="Microsoft.Build.Framework" Version="$(MicrosoftBuildVersion)" />
<PackageVersion Include="Microsoft.Build.Locator" Version="1.7.8" />
<PackageVersion Include="Microsoft.Build.Locator" Version="1.9.1" />
<PackageVersion Include="Microsoft.CodeAnalysis.CSharp" Version="$(MicrosoftCodeAnalysisVersion)" />
<PackageVersion Include="Microsoft.Extensions.DependencyModel" Version="8.0.2" />
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="8.0.1" />
Expand All @@ -39,7 +39,6 @@
vstest 17.8 version
NuGetFrameworksVersion is defined here https://github.com/microsoft/vstest/blob/9a0c41811637edf4afe0e265e08fdd1cb18109ed/eng/Versions.props#L94C1-L94C1
-->
<PackageVersion Include="Newtonsoft.Json" Version="13.0.3" />
<PackageVersion Include="NuGet.Frameworks" Version="$(NugetPackageVersion)" />
<PackageVersion Include="NuGet.Packaging" Version="$(NugetPackageVersion)" />
<PackageVersion Include="NuGet.Versioning" Version="$(NugetPackageVersion)" />
Expand All @@ -54,18 +53,20 @@
<PackageVersion Include="Tmds.ExecFunction" Version="0.8.0" />
<PackageVersion Include="xunit.v3" Version="$(XunitV3Version)" />
<PackageVersion Include="xunit.runner.visualstudio" Version="$(XunitRunnerVisualstudioVersion)" />
<PackageVersion Include="System.Buffers" Version="4.6.0" />
<PackageVersion Include="System.Buffers" Version="4.6.1" />
<PackageVersion Include="System.Collections.Immutable" Version="8.0.0" />
<PackageVersion Include="System.Configuration.ConfigurationManager" Version="8.0.0" />
<PackageVersion Include="System.Diagnostics.EventLog" Version="8.0.2" />
<PackageVersion Include="System.Linq.Async" Version="6.0.1" />
<PackageVersion Include="System.Memory" Version="4.6.0" />
<PackageVersion Include="System.Memory" Version="4.6.3" />
<PackageVersion Include="System.Net.Http" Version="4.3.4" />
<PackageVersion Include="System.Reflection.Metadata" Version="8.0.1" />
<PackageVersion Include="System.Runtime.CompilerServices.Unsafe" Version="6.1.0" />
<PackageVersion Include="System.Security.Cryptography.Pkcs" Version="6.0.5" />
<PackageVersion Include="System.Runtime.CompilerServices.Unsafe" Version="6.1.2" />
<PackageVersion Include="System.Security.AccessControl" Version="6.0.1" />
<PackageVersion Include="System.Security.Cryptography.Pkcs" Version="8.0.1" />
<PackageVersion Include="System.Text.Encoding.CodePages" Version="8.0.0" />
<PackageVersion Include="System.Text.Json" Version="8.0.5" />
<PackageVersion Include="System.Text.RegularExpressions" Version="4.3.1" />
<PackageVersion Include="System.Threading.Tasks.Extensions" Version="4.6.0" />
<PackageVersion Include="System.Threading.Tasks.Extensions" Version="4.6.3" />
</ItemGroup>
</Project>
5 changes: 4 additions & 1 deletion Documentation/Changelog.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
Expand All @@ -7,12 +8,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## Unreleased

### Fixed

- Fix branchpoint exclusion for sdk 8.0.407 [#1741](https://github.com/coverlet-coverage/coverlet/issues/1741)
- Use `<TargetFramework>netstandard2.0</TargetFramework>` for _coverlet.collector_ and _coverlet.msbuild.tasks_ Packages´

### Improvements
- Use [xunit.v3](https://xunit.net/docs/getting-started/v3/whats-new) for tests and example code

- Upgraded minimum required .NET SDK and runtime to .NET 8.0 LTS (Long Term Support) (**Breaking Change**)
- Use [xunit.v3](https://xunit.net/docs/getting-started/v3/whats-new) for tests and example code

## Release date 2024-01-20
### Packages
Expand Down
100 changes: 26 additions & 74 deletions Documentation/Examples/MSBuild/DeterministicBuild/HowTo.md
Original file line number Diff line number Diff line change
@@ -1,57 +1,34 @@
To run test we need to generates packages to reference in on test project.
Run from repo root
# Using Deterministic Builds with Coverlet MSBuild Integration

## Prerequisites

Before running tests with deterministic builds, you need to generate the local NuGet packages:

```shell
C:\git\coverlet
λ dotnet pack
Microsoft (R) Build Engine version 17.7.4+3ebbd7c49 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

Restore completed in 73,36 ms for C:\git\coverlet\src\coverlet.core\coverlet.core.csproj.
Restore completed in 73,41 ms for C:\git\coverlet\test\coverlet.testsubject\coverlet.testsubject.csproj.
Restore completed in 73,33 ms for C:\git\coverlet\test\coverlet.tests.projectsample.excludedbyattribute\coverlet.tests.projectsample.excludedbyattribute.csproj.
Restore completed in 73,34 ms for C:\git\coverlet\src\coverlet.collector\coverlet.collector.csproj.
Restore completed in 73,35 ms for C:\git\coverlet\test\coverlet.tests.xunit.extensions\coverlet.tests.xunit.extensions.csproj.
Restore completed in 75,92 ms for C:\git\coverlet\test\coverlet.integration.tests\coverlet.integration.tests.csproj.
Restore completed in 73,41 ms for C:\git\coverlet\src\coverlet.console\coverlet.console.csproj.
Restore completed in 73,36 ms for C:\git\coverlet\test\coverlet.tests.projectsample.empty\coverlet.tests.projectsample.empty.csproj.
Restore completed in 73,47 ms for C:\git\coverlet\src\coverlet.msbuild.tasks\coverlet.msbuild.tasks.csproj.
Restore completed in 73,37 ms for C:\git\coverlet\test\coverlet.core.tests.samples.netstandard\coverlet.core.tests.samples.netstandard.csproj.
Restore completed in 76,37 ms for C:\git\coverlet\test\coverlet.collector.tests\coverlet.collector.tests.csproj.
Restore completed in 77,05 ms for C:\git\coverlet\test\coverlet.integration.template\coverlet.integration.template.csproj.
Restore completed in 77,2 ms for C:\git\coverlet\test\coverlet.core.performancetest\coverlet.core.performancetest.csproj.
Restore completed in 87,7 ms for C:\git\coverlet\test\coverlet.core.tests\coverlet.core.tests.csproj.
coverlet.core -> C:\git\coverlet\src\coverlet.core\bin\Debug\netstandard2.0\coverlet.core.dll
coverlet.collector -> C:\git\coverlet\src\coverlet.collector\bin\Debug\netcoreapp2.0\coverlet.collector.dll
coverlet.msbuild.tasks -> C:\git\coverlet\src\coverlet.msbuild.tasks\bin\Debug\netstandard2.0\coverlet.msbuild.tasks.dll
coverlet.console -> C:\git\coverlet\src\coverlet.console\bin\Debug\net6.0\coverlet.console.dll
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.collector.6.0.1-preview.6.g918cd179e0.nupkg'.
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.collector.6.0.1-preview.6.g918cd179e0.snupkg'.
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.msbuild.6.0.1-preview.6.g918cd179e0.nupkg'.
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.msbuild.6.0.1-preview.6.g918cd179e0.snupkg'.
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.console.6.0.1-preview.6.g918cd179e0.nupkg'.
Successfully created package 'C:\git\coverlet\bin\Debug\Packages\coverlet.console.6.0.1-preview.6.g918cd179e0.snupkg'.
# Run from repository root
dotnet pack
```

Add msbuild package version generated to `"..\Documentation\Examples\MSBuild\DeterministicBuild\XUnitTestProject1\XUnitTestProject1.csproj"`
## Project Setup

Update your test project file `XUnitTestProject1.csproj`:

```xml
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<TargetFramework>net8.0</TargetFramework>
<IsPackable>false</IsPackable>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />
<PackageReference Include="xunit.v3" Version="1.1.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.0.2">
<PackageReference Include="xunit.v3" Version="2.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
<!-- version comes from coverlet.msbuild.6.0.1-preview.8.gcb9b802a5f.nupkg -->
<PackageReference Include="coverlet.msbuild" Version="coverlet.msbuild.6.0.1-preview.8.gcb9b802a5f.nupkg" >
<!-- Use the version from your locally built package -->
<PackageReference Include="coverlet.msbuild" Version="8.0.1-preview.8.gcb9b802a5f">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
Expand All @@ -60,50 +37,25 @@ Add msbuild package version generated to `"..\Documentation\Examples\MSBuild\Det
<ItemGroup>
<ProjectReference Include="..\ClassLibrary1\ClassLibrary1.csproj" />
</ItemGroup>

</Project>
```

Go to test project folder and run
## Running Tests

Navigate to your test project directory and run:

```shell
C:\git\coverlet\Documentation\Examples\MSBuild\DeterministicBuild (detbuilddocs -> origin)
λ dotnet test /p:CollectCoverage=true /p:DeterministicSourcePaths=true
Test run for C:\git\coverlet\Documentation\Examples\MSBuild\DeterministicBuild\XUnitTestProject1\bin\Debug\net6.0\XUnitTestProject1.dll(.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.5.0
Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.

Test Run Successful.
Total tests: 1
Passed: 1
Total time: 1,1471 Seconds

Calculating coverage result...
Generating report 'C:\git\coverlet\Documentation\Examples\MSBuild\DeterministicBuild\XUnitTestProject1\coverage.json'

+---------------+------+--------+--------+
| Module | Line | Branch | Method |
+---------------+------+--------+--------+
| ClassLibrary1 | 100% | 100% | 100% |
+---------------+------+--------+--------+

+---------+------+--------+--------+
| | Line | Branch | Method |
+---------+------+--------+--------+
| Total | 100% | 100% | 100% |
+---------+------+--------+--------+
| Average | 100% | 100% | 100% |
+---------+------+--------+--------+
dotnet test /p:CollectCoverage=true /p:DeterministicSourcePaths=true
```

You should see on output folder the coverlet source root mapping file generated. The filename starts with 'CoverletSourceRootsMapping_'. Do not use `--no-build` option
This is the confirmation that you're running coverage on deterministic build e.g. `Documentation\Examples\MSBuild\DeterministicBuild\XUnitTestProject1\bin\Debug\net6.0\CoverletSourceRootsMapping_XUnitTestProject1`
> **Important**: Do not use the `--no-build` option as it will prevent the generation of deterministic build artifacts.

## Verification

After running the tests, verify the deterministic build by checking for the source root mapping file:

```text
Documentation\Examples\MSBuild\DeterministicBuild\XUnitTestProject1\bin\Debug\net6.0\CoverletSourceRootsMapping_XUnitTestProject1
bin\Debug\net8.0\CoverletSourceRootsMapping_XUnitTestProject1
```

The presence of this file confirms that your coverage report was generated with deterministic build settings.
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<TargetFramework>net8.0</TargetFramework>
<IsPackable>false</IsPackable>
<ManagePackageVersionsCentrally>false</ManagePackageVersionsCentrally>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />
<PackageReference Include="xunit.v3" Version="1.1.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.0.2">
<PackageReference Include="xunit.v3" Version="2.0.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
Expand Down
61 changes: 47 additions & 14 deletions Documentation/Examples/MSBuild/MergeWith/HowTo.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,57 @@
**Run from solution root sln**
# Merging Coverage Reports

To merge report together you need to run separate test and merge in one `json` format file.
Last command will join and create final needed format file.
## Running Tests Separately

```shell
dotnet test XUnitTestProject1\XUnitTestProject1.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/
dotnet test XUnitTestProject2\XUnitTestProject2.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json"
dotnet test XUnitTestProject3\XUnitTestProject3.csproj /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat="opencover"
To merge coverage reports, run tests for each project and combine them into a single file:

```bash
# Generate coverage for first project
dotnet test XUnitTestProject1/XUnitTestProject1.csproj \
/p:CollectCoverage=true \
/p:CoverletOutput=../CoverageResults/

# Merge coverage from second project
dotnet test XUnitTestProject2/XUnitTestProject2.csproj \
/p:CollectCoverage=true \
/p:CoverletOutput=../CoverageResults/ \
/p:MergeWith="../CoverageResults/coverage.json"

# Merge coverage from third project and generate final OpenCover report
dotnet test XUnitTestProject3/XUnitTestProject3.csproj \
/p:CollectCoverage=true \
/p:CoverletOutput=../CoverageResults/ \
/p:MergeWith="../CoverageResults/coverage.json" \
/p:CoverletOutputFormat="opencover"
```

You can merge also running `dotnet test` and merge with single command from a solution file, but you need to ensure that tests will run sequentially(`-m:1`). This slow down testing but avoid invalid coverage result.
## Running Tests from Solution

```shell
dotnet test /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat=\"opencover,json\" -m:1
To merge coverage using a single command (requires sequential execution):

```bash
dotnet test \
/p:CollectCoverage=true \
/p:CoverletOutput=../CoverageResults/ \
/p:MergeWith="../CoverageResults/coverage.json" \
/p:CoverletOutputFormat="opencover,json" \
-m:1
```

N.B. You need to specify `json` format plus another format(the final one), because Coverlet can only merge proprietary format. At the end you can delete temporary `coverage.json` file.
> **Note**: Sequential execution (`-m:1`) ensures accurate coverage but increases test duration.

## Important Considerations

- Include `json` format alongside your desired output format
- Coverlet only merges its proprietary JSON format
- The temporary `coverage.json` file can be deleted after merging

You can also merge the coverage result and generate another valid format to export the content than opencover, like cobertura.
## Example with Cobertura Output

```shell
dotnet test /p:CollectCoverage=true /p:CoverletOutput=../CoverageResults/ /p:MergeWith="../CoverageResults/coverage.json" /p:CoverletOutputFormat=\"cobertura,json\" -m:1
```bash
dotnet test \
/p:CollectCoverage=true \
/p:CoverletOutput=../CoverageResults/ \
/p:MergeWith="../CoverageResults/coverage.json" \
/p:CoverletOutputFormat="cobertura,json" \
-m:1
```
Loading