Skip to content

Commit 30f586b

Browse files
authored
feat!: MTP integration for spekt loggers
- Support command line params `--report-spekt-junit --report-spekt-junit-filename results.xml` or if you'd like to pass custom options `--report-spekt-junit "StoreConsoleOutput=false"` - Behavior differences in MTP reporters compared to VSTest loggers - Inconclusive tests are now reported as Skipped - Run level test messages are not available - Xunit: skip reason is not captured - Nunit: some properties are reported differently or skipped (seed, category, description)
1 parent 5677c45 commit 30f586b

File tree

58 files changed

+2058
-564
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

58 files changed

+2058
-564
lines changed

.claude/settings.local.json

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"permissions": {
3+
"allow": [
4+
"Bash(dotnet:*)",
5+
"Bash(find:*)",
6+
"Bash(./build.sh:*)"
7+
],
8+
"deny": [],
9+
"ask": []
10+
}
11+
}

.github/workflows/dotnet.yml

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -76,19 +76,25 @@ jobs:
7676
dotnet test -p:PackageVersion=${{ env.APP_BUILD_VERSION }} test/JUnit.Xml.TestLogger.UnitTests/JUnit.Xml.TestLogger.UnitTests.csproj -p:CollectCoverage=true -p:CoverletOutputFormat=opencover
7777
- name: Acceptance test
7878
run: |
79-
dotnet test -p:PackageVersion=${{ env.APP_BUILD_VERSION }} test/TestLogger.AcceptanceTests/TestLogger.AcceptanceTests.csproj
80-
dotnet test -p:PackageVersion=${{ env.APP_BUILD_VERSION }} test/Xunit.Xml.TestLogger.AcceptanceTests/Xunit.Xml.TestLogger.AcceptanceTests.csproj
81-
dotnet test -p:PackageVersion=${{ env.APP_BUILD_VERSION }} test/NUnit.Xml.TestLogger.AcceptanceTests/NUnit.Xml.TestLogger.AcceptanceTests.csproj
82-
dotnet test -p:PackageVersion=${{ env.APP_BUILD_VERSION }} test/JUnit.Xml.TestLogger.AcceptanceTests/JUnit.Xml.TestLogger.AcceptanceTests.csproj
83-
- name: Upload Verification Files on Fail
79+
echo "Running acceptance tests with verbose output..."
80+
dotnet test -p:PackageVersion=${{ env.APP_BUILD_VERSION }} test/TestLogger.AcceptanceTests/TestLogger.AcceptanceTests.csproj --logger "console;verbosity=detailed" --blame
81+
dotnet test -p:PackageVersion=${{ env.APP_BUILD_VERSION }} test/Xunit.Xml.TestLogger.AcceptanceTests/Xunit.Xml.TestLogger.AcceptanceTests.csproj --logger "console;verbosity=detailed" --blame
82+
dotnet test -p:PackageVersion=${{ env.APP_BUILD_VERSION }} test/NUnit.Xml.TestLogger.AcceptanceTests/NUnit.Xml.TestLogger.AcceptanceTests.csproj --logger "console;verbosity=detailed" --blame
83+
dotnet test -p:PackageVersion=${{ env.APP_BUILD_VERSION }} test/JUnit.Xml.TestLogger.AcceptanceTests/JUnit.Xml.TestLogger.AcceptanceTests.csproj --logger "console;verbosity=detailed" --blame
84+
- name: Upload Debug Files on Fail
8485
if: ${{ failure() }}
8586
uses: actions/upload-artifact@v4
8687
with:
87-
name: Acceptance Test Failure
88+
name: Acceptance Test Failure Debug Files
8889
retention-days: 5
8990
path: |
9091
**/*.verified.txt
9192
**/*.received.txt
93+
**/test-results-*.xml
94+
**/test-results-*.json
95+
**/*.log
96+
test/assets/**/bin/Debug/netcoreapp3.1/
97+
test/assets/**/bin/Debug/net9.0/
9298
- name: Codecov
9399
uses: codecov/codecov-action@v3.1.0
94100
with:

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ bld/
2525
[Ll]og/
2626

2727
# Test results
28-
**/*test-results.xml
28+
**/*test-results*.xml
2929
**/test-results.json
3030

3131
# Visual Studio 2015/2017 cache/options directory

CLAUDE.md

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
This repository contains .NET test loggers for Visual Studio Test Platform (VSTest) and Microsoft.Testing.Platform (MTP) that generate JUnit, NUnit, and Xunit XML reports. The project enables integration with CI/CD systems like Circle CI and Gitlab.
8+
9+
## Development Commands
10+
11+
### Building and Testing
12+
13+
- **Full build and test**: `./build.sh` (Linux/macOS) or `.\build.ps1` (Windows)
14+
- **Package only**: `dotnet pack`
15+
- **Run specific unit test**: `dotnet test test/TestLogger.UnitTests/TestLogger.UnitTests.csproj`
16+
- **Run specific acceptance test**: `dotnet test test/TestLogger.AcceptanceTests/TestLogger.AcceptanceTests.csproj`
17+
18+
### Debugging Acceptance Tests
19+
20+
If acceptance tests are failing, run the test asset with detailed output. Example below:
21+
22+
```sh
23+
DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 # prefix below commands with this on Linux
24+
25+
# Reproduce VSTest logger issue
26+
dotnet test --logger:"json;LogFilePath=test-results-vstest.json" test/assets/Json.TestLogger.MSTest.NetCore.Tests/Json.TestLogger.MSTest.NetCore.Tests.csproj
27+
28+
# Reproduce MTP logger issue
29+
dotnet test test/assets/Json.TestLogger.MSTest.NetCore.Tests/Json.TestLogger.MSTest.NetCore.Tests.csproj -p:IsMTP=true -- --report-spekt-json --report-spekt-json test-results-mtp.json
30+
```
31+
32+
## Architecture
33+
34+
### Core Components
35+
36+
- **TestLogger**: Core library with interfaces and workflows in `src/TestLogger/Core/` supporting both VSTest and MTP
37+
- **Format-specific loggers**: JUnit, NUnit, and Xunit implementations in their respective directories
38+
- **Package projects**: Distributable NuGet packages in `src/*.Package/`
39+
40+
### Key Interfaces and Workflows
41+
42+
- `ITestRunBuilder`: Builds test run information
43+
- `ITestResultSerializer`: Serializes test results to different formats
44+
- `TestRunMessageWorkflow`: Handles test run messages
45+
- `TestRunCompleteWorkflow`: Handles test run completion
46+
47+
### Testing Strategy
48+
49+
- **Unit tests**: Core functionality testing in `test/*UnitTests/`
50+
- **Acceptance tests**: End-to-end testing with test projects in `test/*AcceptanceTests/`
51+
- **Package tests**: NuGet package validation in `test/TestLogger.PackageTests/`
52+
53+
## Configuration
54+
55+
### Build Properties
56+
57+
- Version is managed via `version.txt` and CI workflow
58+
- Uses .NET 9.0 for development, supports netstandard2.0+ for packages
59+
- Code analysis with StyleCop enforced via `Directory.Build.props`
60+
61+
### Code Style
62+
63+
- C# latest language version
64+
- XML documentation required for public APIs
65+
- StyleCop analyzers enabled with custom ruleset
66+
- Warnings treated as errors
67+
68+
## Project Structure
69+
70+
```
71+
src/
72+
├── TestLogger/ # Core library
73+
├── JUnit.Xml.TestLogger/ # JUnit format implementation
74+
├── NUnit.Xml.TestLogger/ # NUnit format implementation
75+
├── Xunit.Xml.TestLogger/ # Xunit format implementation
76+
└── *.Package/ # NuGet package projects
77+
78+
test/
79+
├── *UnitTests/ # Unit tests for each logger
80+
├── *AcceptanceTests/ # End-to-end tests
81+
├── TestLogger.UnitTests/ # Core library tests
82+
└── assets/ # Test project assets
83+
```
84+
85+
## CI/CD
86+
87+
- Uses GitHub Actions with multi-OS testing (Ubuntu, Windows)
88+
- Automated versioning via release-please action
89+
- Coverage reporting to Codecov
90+
- Pre-release packages published to MyGet
91+
92+
## Development Guidelines
93+
94+
- Respect DRY, SOLID and similar clean code practices.
95+
- Code must be usable, correct and performant.
96+
- Add documentation for all public methods.
97+
- Add unit tests for every change. Follow the existing test conventions in the project.
98+
- Do not add unnecessary comments.

Directory.Packages.props

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,13 @@
55
</PropertyGroup>
66
<ItemGroup>
77
<PackageVersion Include="Microsoft.TestPlatform.ObjectModel" Version="15.5.0" />
8-
<PackageVersion Include="Microsoft.Testing.Platform" Version="1.8.3" />
8+
<PackageVersion Include="Microsoft.Testing.Platform" Version="1.8.4" />
99
<PackageVersion Include="Newtonsoft.Json" Version="13.0.3" />
1010

1111
<PackageVersion Include="Stylecop.Analyzers" Version="1.2.0-beta.556" />
1212

13-
<PackageVersion Include="MSTest.TestFramework" Version="3.8.3" />
14-
<PackageVersion Include="MSTest.TestAdapter" Version="3.8.3" />
13+
<PackageVersion Include="MSTest.TestFramework" Version="3.10.1" />
14+
<PackageVersion Include="MSTest.TestAdapter" Version="3.10.1" />
1515
<PackageVersion Include="xunit" Version="2.9.3" />
1616
<PackageVersion Include="xunit.runner.visualstudio" Version="3.0.2" />
1717

nuget.config

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
<?xml version="1.0" encoding="utf-8"?>
2+
<configuration>
3+
<packageSources>
4+
<!--To inherit the global NuGet package sources remove the <clear/> line below -->
5+
<clear />
6+
<add key="nuget" value="https://api.nuget.org/v3/index.json" />
7+
</packageSources>
8+
</configuration>

src/JUnit.Xml.TestLogger/JUnitReporterCommandLineOptionsProvider.cs

Lines changed: 0 additions & 20 deletions
This file was deleted.

src/JUnit.Xml.TestLogger/JUnitTestReporter.cs

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,11 @@ namespace Spekt.TestReporter.JUnit
1010
internal sealed class JUnitTestReporter : TestReporter
1111
{
1212
public JUnitTestReporter(JUnitTestReporterExtension extension, IServiceProvider serviceProvider)
13-
: base(serviceProvider, extension)
13+
: base(serviceProvider, extension, "junit")
1414
{
1515
}
1616

17-
protected override string FileNameOption => JUnitReporterCommandLineOptionsProvider.ReportJUnitFileNameOption;
18-
19-
protected override string ReportOption => JUnitReporterCommandLineOptionsProvider.ReportJUnitOption;
17+
protected override string DefaultFileName => "TestResults.xml";
2018

2119
protected override ITestResultSerializer CreateTestResultSerializer()
2220
=> new JunitXmlSerializer();

src/JUnit.Xml.TestLogger/JUnitTestReporterExtensions.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ public static void AddJUnitReportProvider(this ITestApplicationBuilder testAppli
1616
testApplicationBuilder.TestHost.AddDataConsumer(compositeExtension);
1717
testApplicationBuilder.TestHost.AddTestSessionLifetimeHandle(compositeExtension);
1818

19-
testApplicationBuilder.CommandLine.AddProvider(() => new JUnitReporterCommandLineOptionsProvider(extension));
19+
testApplicationBuilder.CommandLine.AddProvider(() => new TestReporterCommandLineProvider(extension, "junit"));
2020
}
2121
}
2222
}

src/NUnit.Xml.TestLogger/NUnitReporterCommandLineOptionsProvider.cs

Lines changed: 0 additions & 20 deletions
This file was deleted.

0 commit comments

Comments
 (0)