Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report TestType in XML reports for templated test cases #2567

Draft
wants to merge 1 commit into
base: devel
Choose a base branch
from

Conversation

FoamScience
Copy link

Description

This tiny PR tries to allow XML reporter to add the actual TestType as an arg. to TestCase if it's a TEMPLATE_TEST_CASE.
Proof of concept atm; if the idea attracts enough interest; I'll add it to other reports too as well as maybe generalizing this to arbitrary number of case properties.

Current behavior: Catch reports append the TestType to the case name; separated by a dash.
Reason for this improvement: Want to see the actual failing TestType (if some fail and some pass) in my editor. But parsing the name is error prone (as I usually have dashes in case names, and there is no way to distinguish if it's a template test case - unless parsing the expression for TestType - which can be just mentioned there as a string value or variable ... you can see how this can get confusing quickly).
Solution: As TestType is a property of the test case; adding it as a parameter to the corresponding TestCase tag avoids all confusion; No effect should be observed in regular test cases.

This quick test:

TEST_CASE( "Regular - test case") {
    REQUIRE( true );
}

template<class T>
bool isTrue() {
    return true;
}

template<>
bool isTrue<double>() {
    return false;
}

TEMPLATE_TEST_CASE( "Template - test case", "", double, int) {
    REQUIRE( isTrue<TestType>() );
}

results in this XML report (notice the testType="..." when it's a TEMPLATE_TEST_CASE)

<?xml version="1.0" encoding="UTF-8"?>
<Catch2TestRun name="test" rng-seed="3252414589" xml-format-version="2" catch2-version="3.1.1">
  <TestCase name="Regular - test case" filename="test.cpp" line="3">
    <OverallResult success="true"/>
  </TestCase>
  <TestCase name="Template - test case - double" testType="double" filename="test.cpp" line="17">
    <Expression success="false" type="REQUIRE" filename="test.cpp" line="18">
      <Original>
        isTrue&lt;TestType>()
      </Original>
      <Expanded>
        false
      </Expanded>
    </Expression>
    <OverallResult success="false"/>
  </TestCase>
  <TestCase name="Template - test case - int" testType="int" filename="test.cpp" line="17">
    <OverallResult success="true"/>
  </TestCase>
  <OverallResults successes="2" failures="1" expectedFailures="0"/>
  <OverallResultsCases successes="2" failures="1" expectedFailures="0"/>
</Catch2TestRun>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant