Skip to content

Commit

Permalink
Modify generator tracking to allow GENERATEs between SECTIONs
Browse files Browse the repository at this point in the history
This means that code such as

```cpp
TEST_CASE() {
    SECTION("first") { SUCCEED(); }
    auto _ = GENERATE(1, 2);
    SECTION("second") { SUCCEED(); }
}
```

will run and report 3 assertions, 1 from section "first" and 2
from section "second". This also applies for greater and potentially
more confusing nesting, but fundamentally it is up to the user to
avoid overly complex and confusing nestings, just as with `SECTION`s.

The old behaviour of `GENERATE` as first thing in a `TEST_CASE`,
`GENERATE` not followed by a `SECTION`, etc etc should be unchanged.

Closes #1938
  • Loading branch information
horenmar committed Jul 26, 2020
1 parent 250d9b9 commit 4565b82
Show file tree
Hide file tree
Showing 14 changed files with 1,420 additions and 18 deletions.
27 changes: 23 additions & 4 deletions src/catch2/internal/catch_run_context.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ namespace Catch {
currentTracker.addChild( tracker );
}

if( !ctx.completedCycle() && !tracker->isComplete() ) {
if( !tracker->isComplete() ) {
tracker->open();
}

Expand All @@ -66,8 +66,28 @@ namespace Catch {
}
void close() override {
TrackerBase::close();
// Generator interface only finds out if it has another item on atual move
if (m_runState == CompletedSuccessfully && m_generator->next()) {
// If a generator has a child (it is followed by a section)
// and none of its children have started, then we must wait
// until later to start consuming its values.
// This catches cases where `GENERATE` is placed between two
// `SECTION`s.
// **The check for m_children.empty cannot be removed**.
// doing so would break `GENERATE` _not_ followed by `SECTION`s.
const bool should_wait_for_child =
!m_children.empty() &&
std::find_if( m_children.begin(),
m_children.end(),
[]( TestCaseTracking::ITrackerPtr tracker ) {
return tracker->hasStarted();
} ) == m_children.end();

// This check is a bit tricky, because m_generator->next()
// has a side-effect, where it consumes generator's current
// value, but we do not want to invoke the side-effect if
// this generator is still waiting for any child to start.
if ( should_wait_for_child ||
( m_runState == CompletedSuccessfully &&
m_generator->next() ) ) {
m_children.clear();
m_runState = Executing;
}
Expand Down Expand Up @@ -200,7 +220,6 @@ namespace Catch {
using namespace Generators;
GeneratorTracker& tracker = GeneratorTracker::acquire(m_trackerContext,
TestCaseTracking::NameAndLocation( static_cast<std::string>(generatorName), lineInfo ) );
assert( tracker.isOpen() );
m_lastAssertionInfo.lineInfo = lineInfo;
return tracker;
}
Expand Down
3 changes: 2 additions & 1 deletion src/catch2/internal/catch_test_case_tracker.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,8 @@ namespace TestCaseTracking {
bool SectionTracker::isComplete() const {
bool complete = true;

if ((m_filters.empty() || m_filters[0] == "")
if (m_filters.empty()
|| m_filters[0] == ""
|| std::find(m_filters.begin(), m_filters.end(), m_trimmed_name) != m_filters.end()) {
complete = TrackerBase::isComplete();
}
Expand Down
5 changes: 4 additions & 1 deletion src/catch2/internal/catch_test_case_tracker.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ namespace TestCaseTracking {
virtual bool isSuccessfullyCompleted() const = 0;
virtual bool isOpen() const = 0; // Started but not complete
virtual bool hasChildren() const = 0;
virtual bool hasStarted() const = 0;

virtual ITracker& parent() = 0;

Expand Down Expand Up @@ -121,7 +122,9 @@ namespace TestCaseTracking {
bool isSuccessfullyCompleted() const override;
bool isOpen() const override;
bool hasChildren() const override;

bool hasStarted() const override {
return m_runState != NotStarted;
}

void addChild( ITrackerPtr const& child ) override;

Expand Down
5 changes: 5 additions & 0 deletions tests/SelfTest/Baselines/automake.sw.approved.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ Nor would this
:test-result: PASS #1912 -- test spec parser handles escaping
:test-result: PASS #1913 - GENERATE inside a for loop should not keep recreating the generator
:test-result: PASS #1913 - GENERATEs can share a line
:test-result: PASS #1938 - GENERATE after a section
:test-result: PASS #1938 - Section followed by flat generate
:test-result: PASS #1938 - flat generate
:test-result: PASS #1938 - mixed sections and generates
:test-result: PASS #1938 - nested generate
:test-result: PASS #1954 - 7 arg template test case sig compiles - 1, 1, 1, 1, 1, 0, 0
:test-result: PASS #1954 - 7 arg template test case sig compiles - 5, 1, 1, 1, 1, 0, 0
:test-result: PASS #1954 - 7 arg template test case sig compiles - 5, 3, 1, 1, 1, 0, 0
Expand Down
42 changes: 42 additions & 0 deletions tests/SelfTest/Baselines/compact.sw.approved.txt
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,48 @@ Generators.tests.cpp:<line number>: passed: i != j for: 1 != 3
Generators.tests.cpp:<line number>: passed: i != j for: 1 != 4
Generators.tests.cpp:<line number>: passed: i != j for: 2 != 3
Generators.tests.cpp:<line number>: passed: i != j for: 2 != 4
PartTracker.tests.cpp:<line number>: passed: with 1 message: 'A'
PartTracker.tests.cpp:<line number>: passed: m for: 1
PartTracker.tests.cpp:<line number>: passed: m for: 2
PartTracker.tests.cpp:<line number>: passed: m for: 3
PartTracker.tests.cpp:<line number>: passed: 1
PartTracker.tests.cpp:<line number>: passed: m for: 2
PartTracker.tests.cpp:<line number>: passed: m for: 3
PartTracker.tests.cpp:<line number>: passed: m for: 1
PartTracker.tests.cpp:<line number>: passed: m for: 2
PartTracker.tests.cpp:<line number>: passed: m for: 3
PartTracker.tests.cpp:<line number>: passed: with 1 message: 'A'
PartTracker.tests.cpp:<line number>: passed: with 3 messages: 'i := 1' and 'j := 3' and 'k := 5'
PartTracker.tests.cpp:<line number>: passed: with 1 message: 'B'
PartTracker.tests.cpp:<line number>: passed: with 3 messages: 'i := 1' and 'j := 3' and 'k := 6'
PartTracker.tests.cpp:<line number>: passed: with 1 message: 'B'
PartTracker.tests.cpp:<line number>: passed: with 3 messages: 'i := 1' and 'j := 4' and 'k := 5'
PartTracker.tests.cpp:<line number>: passed: with 3 messages: 'i := 1' and 'j := 4' and 'k := 6'
PartTracker.tests.cpp:<line number>: passed: with 1 message: 'A'
PartTracker.tests.cpp:<line number>: passed: with 3 messages: 'i := 2' and 'j := 3' and 'k := 5'
PartTracker.tests.cpp:<line number>: passed: with 1 message: 'B'
PartTracker.tests.cpp:<line number>: passed: with 3 messages: 'i := 2' and 'j := 3' and 'k := 6'
PartTracker.tests.cpp:<line number>: passed: with 1 message: 'B'
PartTracker.tests.cpp:<line number>: passed: with 3 messages: 'i := 2' and 'j := 4' and 'k := 5'
PartTracker.tests.cpp:<line number>: passed: with 3 messages: 'i := 2' and 'j := 4' and 'k := 6'
PartTracker.tests.cpp:<line number>: passed: m for: 1
PartTracker.tests.cpp:<line number>: passed: n for: 1
PartTracker.tests.cpp:<line number>: passed: m for: 1
PartTracker.tests.cpp:<line number>: passed: n for: 2
PartTracker.tests.cpp:<line number>: passed: m for: 1
PartTracker.tests.cpp:<line number>: passed: n for: 3
PartTracker.tests.cpp:<line number>: passed: m for: 2
PartTracker.tests.cpp:<line number>: passed: n for: 1
PartTracker.tests.cpp:<line number>: passed: m for: 2
PartTracker.tests.cpp:<line number>: passed: n for: 2
PartTracker.tests.cpp:<line number>: passed: m for: 2
PartTracker.tests.cpp:<line number>: passed: n for: 3
PartTracker.tests.cpp:<line number>: passed: m for: 3
PartTracker.tests.cpp:<line number>: passed: n for: 1
PartTracker.tests.cpp:<line number>: passed: m for: 3
PartTracker.tests.cpp:<line number>: passed: n for: 2
PartTracker.tests.cpp:<line number>: passed: m for: 3
PartTracker.tests.cpp:<line number>: passed: n for: 3
Misc.tests.cpp:<line number>: passed:
Misc.tests.cpp:<line number>: passed:
Misc.tests.cpp:<line number>: passed:
Expand Down
4 changes: 2 additions & 2 deletions tests/SelfTest/Baselines/console.std.approved.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1380,6 +1380,6 @@ due to unexpected exception with message:
Why would you throw a std::string?

===============================================================================
test cases: 343 | 269 passed | 70 failed | 4 failed as expected
assertions: 1941 | 1789 passed | 131 failed | 21 failed as expected
test cases: 348 | 274 passed | 70 failed | 4 failed as expected
assertions: 1983 | 1831 passed | 131 failed | 21 failed as expected

Loading

0 comments on commit 4565b82

Please sign in to comment.