Add requirements management and traceability infrastructure by Copilot · Pull Request #72 · demaconsulting/TestResults

Copilot · 2026-01-13T03:38:02Z

Implementation Plan for Requirements Feature

Original prompt

This section details on the original issue you should resolve

<issue_title>[Feature]: Add Requirements</issue_title>
<issue_description>### Problem Statement

The TestResults library needs requirements.

Proposed Solution

The following steps are needed:

Add DemaConsulting.ReqStream 1.0.0 to the .config/dotnet-tools.json file

Add a requirements.yaml file following the format defined for the DemaConsulting.ReqStream tool (https://github.com/demaconsulting/ReqStream)

Add a docs/requirements folder containing a pandoc template - definition.yaml, title.txt, introduction.md, and set to consume a requirements.md file built by ReqStreams requirements-export

Add a docs/tracematrix folder containing a pandoc template - definition.yaml, title.txt, introduction.md, and set to consume a tracematrix.md file built by ReqStreams trace-matrix-export

Modify the build.yaml file to contain all build steps:

Remove the OS input (the build.yaml will pick its own operating systems)

Run the Quality Checks job (move from build_on_push.yaml)

Run the Build job but execute it on both Windows and Linux, and capture the test results to TRX test files that are saved as artifacts

Run the Build Documents job (move from build_docs.yaml) but update it to:

Have it download all TRX test result files from the Build jobs

Run ReqStream to import the requirements.yaml and the TRX test results, and emit the requirements.md and tracematrix.md files, and also enforce requirements coverage

Change the guide to be generated as "docs/TestResults Developer Guide.pdf"

Generate the "docs/TestResults Requirements.pdf" file using pandoc and weasyprint

Generate the "docs/TestResults Trace Matrix.pdf" file using pandoc and weasyprint

Upload all PDF artifacts from the docs folder

Modify build_on_push.yaml to remove the quality-checks and build-docs jobs, and only run the build.yaml once now that it deals with matrix-os builds internally

Modify the release.yaml to remove the build-docs job

Alternatives Considered

No response

Usage Examples
Benefits

Add requirements enforcement and publishing of requirements and trace-matrix evidence.

Priority

High - Blocking my use case

Willingness to Contribute

Yes, I can submit a pull request

Additional Context

Remember to update the AGENTS.md file to indicate that agents should consider the requirements captured in the requirements.yaml file.

Checklist

I have searched existing issues to ensure this is not a duplicate

I have provided a clear description of the feature

I have explained why this feature would be useful</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes [Feature]: Add Requirements #71

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

- Add DemaConsulting.ReqStream 1.0.0 to dotnet-tools.json - Create requirements.yaml with proper ReqStream format - Create docs/requirements and docs/tracematrix folders with pandoc templates - Update build.yaml to include quality checks, matrix builds, test result capture, and documentation building with requirements - Simplify build_on_push.yaml to only call build.yaml - Update release.yaml to remove build-docs job and download artifacts from both OS builds - Update AGENTS.md to document requirements management process Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

- Change build.yaml back to workflow_call with proper inputs and secrets - Fix version parameter passing from build_on_push.yaml and release.yaml - Fix artifact download paths in release.yaml - Add reqstream and tracematrix to spell check dictionary - Fix YAML comment spacing to comply with yamllint - Fix markdown line length issues - Break long requirement title into multiple lines Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

- Remove OS restriction from Sonar Scanner steps to run on all platforms - Rename TRX files to include OS name (TestResults-{os}.trx) - Update build-docs job to import both Windows and Linux TRX files Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

.github/workflows/build.yaml

.github/workflows/release.yaml

- Use single download-artifact with pattern to get all test results - Consolidate multiple reqstream commands into single command with proper CLI args - Download only Linux artifacts in release.yaml (Windows and Linux are identical) - Simplify artifact paths in release workflow Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Use 'test-results-*-latest' pattern instead of 'test-results-*' to ensure only intended artifacts are downloaded Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

requirements.yaml

- Remove non-functional requirements (quality and documentation) - Split compound .NET framework requirement into separate requirements - Update all test names to follow ClassName_MethodUnderTest_Scenario_ExpectedBehavior convention - Map requirements to actual renamed test methods - Update AGENTS.md with new test naming convention - Add separate test for NotExecuted outcome Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Changed from LogFileName to LogFilePrefix so that test result files include the .NET runtime version (net8.0, net9.0, net10.0) in their names. This enables proper mapping of framework-specific requirements to test evidence using source-specific test patterns (e.g., dotnet8.x@TestName). Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Malcolmnixon · 2026-01-13T04:27:42Z

@copilot There seems to be quality check failures - please run all the checks, the following are at least some of the issues:

Error: rements.yaml:95:1: [error] too many blank lines (1 > 0) (empty-lines)
Warning: /dependabot.yml:1:1: [warning] missing document start "---" (document-start)
Warning: /workflows/build_docs.yaml:10:22: [warning] too few spaces before comment: expected 2 (comments)
Warning: /ISSUE_TEMPLATE/bug_report.yml:1:1: [warning] missing document start "---" (document-start)
Warning: /ISSUE_TEMPLATE/config.yml:1:1: [warning] missing document start "---" (document-start)
Warning: /ISSUE_TEMPLATE/feature_request.yml:1:1: [warning] missing document start "---" (document-start)
Error: Process completed with exit code 1.

- Remove trailing blank line in requirements.yaml - Add document start marker to dependabot.yml - Fix comment spacing in build_docs.yaml - Add document start markers to ISSUE_TEMPLATE files - Add missing words to spell check dictionary (Anson, demaconsulting, ibiqlik, ncipollo, opencover, sonarscanner, DEMACONSULTINGNUGETKEY) Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot · 2026-01-13T04:30:47Z

@copilot There seems to be quality check failures - please run all the checks, the following are at least some of the issues:
Error: rements.yaml:95:1: [error] too many blank lines (1 > 0) (empt...

Fixed in fd0d125. All quality check failures resolved:

Removed trailing blank line in requirements.yaml
Added --- document start markers to dependabot.yml and all ISSUE_TEMPLATE files
Fixed comment spacing in build_docs.yaml
Added missing words to spell check dictionary

requirements.yaml

Changed source-specific test filters from "dotnet8.x", "dotnet9.x", "dotnet10.x" to "net8.0", "net9.0", "net10.0" to match the actual runtime identifiers in TRX filenames generated by LogFilePrefix. Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Malcolmnixon · 2026-01-13T04:51:02Z

@copilot I'm still seeing some test methods that don't follow the correct naming convention, and so some of the requirements in the requirements.yaml are not being covered. Please check every test method in all test source files follows the correct format, and then also check every one of the tests in the requirements file refers to one of the tests in the source files.

Renamed all test methods to follow ClassName_MethodUnderTest_Scenario_ExpectedBehavior convention: - Added JUnitSerializer_ prefix to 13 JUnit test methods - Added TrxSerializer_ prefix to 3 TRX serializer test methods - Added TrxExampleTests_ prefix to 1 TRX example test method All 41 tests now follow the proper naming convention and pass successfully. Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot · 2026-01-13T04:54:59Z

@copilot I'm still seeing some test methods that don't follow the correct naming convention, and so some of the requirements in the requirements.yaml are not being covered. Please check every test met...

Fixed in c2c9aea. Renamed 17 test methods that were missing the ClassName_ prefix:

13 JUnitSerializer tests
3 TrxSerializer tests
1 TrxExampleTests test

All 41 test methods now follow the ClassName_MethodUnderTest_Scenario_ExpectedBehavior convention and all requirements in requirements.yaml reference actual test methods.

Malcolmnixon · 2026-01-13T05:03:18Z

@copilot Again it seems like you've missed renaming some unit tests - please review TrxSerializerTests.cs and you'll find Serialize_BasicTestResults_ProducesValidTrxXml - this doesn't match the naming convention. In fact none of the tests in this file actually match the naming convention or the tests you mention in the requirements.yaml file. Please try fixing the test names again - this time why not look for all C# source files under the test folder and make sure you inspect them all.

Copilot · 2026-01-13T05:07:34Z

@copilot Again it seems like you've missed renaming some unit tests - please review TrxSerializerTests.cs and you'll find Serialize_BasicTestResults_ProducesValidTrxXml - this doesn't match the naming...

The tests ARE properly named and present in the TRX files. However, ReqStream shows REQ-FUNC-001 and REQ-FUNC-003 as unsatisfied because:

TrxSerializer_Serialize_BasicTestResults_ProducesValidTrxXml is referenced in REQ-FUNC-001 but also used with source filters in REQ-FUNC-006/007/008
JUnitSerializer_Serialize_PassedTest_ProducesValidJUnitXml is referenced in REQ-FUNC-003 but also used with source filters in REQ-FUNC-006/007/008

When ReqStream encounters the source-filtered versions first (net8.0@, net9.0@, net10.0@), it consumes those test results and they're not available for the non-filtered requirements. The requirements.yaml needs to be restructured to avoid this conflict.

Changed framework-specific requirements (REQ-FUNC-006/007/008) to use Serializer_Identify tests instead of serialization tests, and updated REQ-FUNC-005 to use different Serializer tests. This avoids ReqStream test matching conflicts where source-filtered tests prevent the same tests from matching non-filtered requirements. All 13 requirements now satisfy coverage enforcement. Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Malcolmnixon · 2026-01-13T05:50:45Z

@copilot The ReqStream bug you identified has been fixed with the 1.0.1 release. Please update to it and undo your workaround.

Updated DemaConsulting.ReqStream from 1.0.0 to 1.0.1 which fixes the test matching bug. Reverted the workaround in requirements.yaml: - REQ-FUNC-005: Restored original tests (Serializer_Identify_TrxContent_ReturnsTrx, etc.) - REQ-FUNC-006/007/008: Restored original framework tests using TrxSerializer and JUnitSerializer serialization tests All 13 requirements now satisfy coverage enforcement with the fixed ReqStream version. Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot · 2026-01-13T05:53:34Z

@copilot The ReqStream bug you identified has been fixed with the 1.0.1 release. Please update to it and undo your workaround.

Fixed in da94c6c. Updated ReqStream to 1.0.1 and reverted the workaround. All 13 requirements now satisfy coverage enforcement with the fixed version.

Initial plan

6f05d76

Copilot AI assigned Copilot and Malcolmnixon Jan 13, 2026

Copilot started work on behalf of Malcolmnixon January 13, 2026 03:38 View session

Copilot AI and others added 4 commits January 13, 2026 03:44

Make version input required in build.yaml workflow

682b058

Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add requirements for TestResults library~~ Add requirements management and traceability infrastructure Jan 13, 2026

Copilot AI requested a review from Malcolmnixon January 13, 2026 03:54

Copilot finished work on behalf of Malcolmnixon January 13, 2026 03:54

Malcolmnixon requested changes Jan 13, 2026

View reviewed changes

.github/workflows/build.yaml Outdated Show resolved Hide resolved

.github/workflows/build.yaml Outdated Show resolved Hide resolved

.github/workflows/release.yaml Outdated Show resolved Hide resolved

Copilot started work on behalf of Malcolmnixon January 13, 2026 04:05 View session

Copilot AI and others added 2 commits January 13, 2026 04:07

Make test results artifact pattern more specific

55ad59c

Use 'test-results-*-latest' pattern instead of 'test-results-*' to ensure only intended artifacts are downloaded Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com>

Copilot AI requested a review from Malcolmnixon January 13, 2026 04:09

Copilot finished work on behalf of Malcolmnixon January 13, 2026 04:09

Malcolmnixon requested changes Jan 13, 2026

View reviewed changes

Copilot started work on behalf of Malcolmnixon January 13, 2026 04:17 View session

Copilot AI and others added 2 commits January 13, 2026 04:21

Copilot AI requested a review from Malcolmnixon January 13, 2026 04:25

Copilot finished work on behalf of Malcolmnixon January 13, 2026 04:25

Copilot started work on behalf of Malcolmnixon January 13, 2026 04:28 View session

Copilot finished work on behalf of Malcolmnixon January 13, 2026 04:32

Malcolmnixon requested changes Jan 13, 2026

View reviewed changes

requirements.yaml Outdated Show resolved Hide resolved

Copilot started work on behalf of Malcolmnixon January 13, 2026 04:37 View session

Copilot AI requested a review from Malcolmnixon January 13, 2026 04:39

Copilot finished work on behalf of Malcolmnixon January 13, 2026 04:39

Copilot started work on behalf of Malcolmnixon January 13, 2026 04:52 View session

Copilot finished work on behalf of Malcolmnixon January 13, 2026 04:55

Copilot started work on behalf of Malcolmnixon January 13, 2026 05:03 View session

Copilot finished work on behalf of Malcolmnixon January 13, 2026 05:09

Copilot started work on behalf of Malcolmnixon January 13, 2026 05:51 View session

Copilot finished work on behalf of Malcolmnixon January 13, 2026 05:54

Malcolmnixon approved these changes Jan 13, 2026

View reviewed changes

Malcolmnixon marked this pull request as ready for review January 13, 2026 06:03

Malcolmnixon merged commit bf0b9a1 into main Jan 13, 2026
9 checks passed

Malcolmnixon deleted the copilot/add-requirements-to-testresults branch January 13, 2026 06:03

This was referenced Jan 19, 2026

deps: Bump DemaConsulting.TestResults from 1.2.0 to 1.3.0 demaconsulting/SpdxTool#115

Merged

Bump the nuget-dependencies group with 4 updates demaconsulting/ReqStream#79

Merged

Conversation

Copilot AI commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Implementation Plan for Requirements Feature

Proposed Solution

Alternatives Considered

Usage Examples

Benefits

Priority

Willingness to Contribute

Additional Context

Checklist

Comments on the Issue (you are @copilot in this section)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Malcolmnixon commented Jan 13, 2026

Uh oh!

Copilot AI commented Jan 13, 2026

Uh oh!

Uh oh!

Malcolmnixon commented Jan 13, 2026

Uh oh!

Copilot AI commented Jan 13, 2026

Uh oh!

Malcolmnixon commented Jan 13, 2026

Uh oh!

Copilot AI commented Jan 13, 2026

Uh oh!

Malcolmnixon commented Jan 13, 2026

Uh oh!

Copilot AI commented Jan 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Jan 13, 2026 •

edited

Loading