Skip to content

[Feature]: Simplify Trace Matrix #95

@Malcolmnixon

Description

@Malcolmnixon

Problem Statement

The Trace Matrix logic is a little convoluted.

Proposed Solution

The TraceMatrix needs to provide the following capabilities:

  • Export documentation showing for each requirement the test metrics (linked, passed, failed, not-executed)
  • Export documentation showing for each test the requirement and metrics (passed, failed)
  • Provide the list of unsatisfied requirements

It should satisfy these capabilities in the clearest and most concise way - with a design so obvious that its correct behavior is immediately obvious. Firstly we can recognize some basic facts:

  • Every requirement ID is unique
  • Any TestOutcome that is not 'IsExecuted' should be ignored
  • The combination of TestFileBaseName and TestResult.Name may not be unique because they may come from different TestResult.ClassName, but these duplicate results can be collapsed to a single record counting the passes and fails.

The implementation should therefore be to create a record of TestExecution consisting of:

  • FileNameBase (the base name of the test file)
  • Name (the test name)
  • Passes (number of passes in the file matching the test name)
  • Fails (number of fails in the file matching the test name)

For fast access it's possible to assemble a Dictionary[TestName,List[TestExecution]]. This then becomes exceedingly easy to process - a simple FindTestExecutions method given the test name (with or without test source filter) can be:

  1. Split into test source and test name
  2. Look up the test name in the dictionary
  3. Iterate through the list of TestExecution gathering those tests matching the test source

The three core capabilities can be built on top of this core FindTestExecutions functionality.

Alternatives Considered

No response

Usage Examples

Benefits

Exceedingly easy processing, and using a dictionary by test name should improve performance. In almost all projects the number of unique tests is far larger than the number of test files, so the implementation optimizes in the important area.

Priority

Medium - Would improve my workflow

Willingness to Contribute

Yes, I can submit a pull request

Additional Context

Remember to update the architecture.

Checklist

  • I have searched existing issues to ensure this is not a duplicate
  • I have provided a clear description of the feature
  • I have explained why this feature would be useful

Metadata

Metadata

Labels

enhancementNew feature or request

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions