Skip non-executed tests in trace matrix processing

[Copilot]

Pull Request

Description

Test results containing TestOutcome.NotExecuted (e.g., tests filtered by OS/Runtime conditions) were incorrectly counted in execution statistics, skewing requirement satisfaction metrics.

Changes

• TraceMatrix.cs: Skip non-executed tests using IsExecuted() extension method

  // Skip non-executed tests (e.g., filtered by OS/Runtime conditions)
  if (!result.Outcome.IsExecuted())
  {
      continue;
  }

• TraceMatrixTests.cs: Added 3 unit tests covering mixed executed/non-executed scenarios
• ARCHITECTURE.md: Updated documentation to reflect new behavior

Requirements with only non-executed tests are now correctly treated as having no test coverage.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Code quality improvement

Related Issues

Addresses the issue where tests filtered by OS or Runtime conditions incorrectly impact requirement satisfaction.

Pre-Submission Checklist

Before submitting this pull request, ensure you have completed the following:

Build and Test

• Code builds successfully: dotnet build --configuration Release
• All tests pass: dotnet test --configuration Release
• Code produces zero warnings

Code Quality

• Code formatting is correct: dotnet format --verify-no-changes
• New code has appropriate XML documentation comments
• Static analyzer warnings have been addressed

Quality Checks

Please run the following checks before submitting:

• Spell checker passes: cspell "**/*.{md,cs}"
• Markdown linter passes: markdownlint "**/*.md"
• YAML linter passes: yamllint '**/*.{yml,yaml}'

Testing

• Added unit tests for new functionality
• Updated existing tests if behavior changed
• All tests follow the AAA (Arrange, Act, Assert) pattern
• Test coverage is maintained or improved

Documentation

• Updated README.md (if applicable)
• Updated ARCHITECTURE.md (if applicable)
• Added code examples for new features (if applicable)
• Updated requirements.yaml (if applicable)

Additional Notes

Uses IsExecuted() and IsPassed() extension methods from DemaConsulting.TestResults library for cleaner code. All 112 tests pass (including 3 new tests). CodeQL security scan shows zero vulnerabilities.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[Feature]: Ignore non-executed tests</issue_title>
<issue_description>### Problem Statement

Some test results may contain tests that could not be executed - for example if the tests are filtered based on OS or Runtime conditions. These non-executed tests should not be considered failures.

Proposed Solution

The TestResults IsExecuted method may be useful to reject non-executed test results.

Additional tests should be added containing non-executed tests to make sure non-executed tests don't impact results positively or negatively.

Alternatives Considered

No response

Usage Examples

Benefits

Better handling of requirements covered by complex conditional testing.

Priority

Medium - Would improve my workflow

Willingness to Contribute

Yes, I can submit a pull request

Additional Context

No response

Checklist

• I have searched existing issues to ensure this is not a duplicate
• I have provided a clear description of the feature
• I have explained why this feature would be useful</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes [Feature]: Ignore non-executed tests #91

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.