Skip to content

Commit

Permalink
Added approvals for “failed as expected” tests
Browse files Browse the repository at this point in the history
  • Loading branch information
@philsquared
philsquared committed Apr 7, 2017
1 parent 09e4830 commit 250f0ee
Show file tree
Hide file tree
Showing 5 changed files with 245 additions and 242 deletions.
79 changes: 15 additions & 64 deletions projects/SelfTest/Baselines/console.std.approved.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,19 @@
<exe-name> is a <version> host application.
Run with -? for options

-------------------------------------------------------------------------------
#748 - captures with unexpected exceptions
inside REQUIRE_NOTHROW
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
REQUIRE_NOTHROW( thisThrows() )
due to unexpected exception with messages:
answer := 42
expected exception

-------------------------------------------------------------------------------
#835 -- errno should not be touched by Catch
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -102,17 +115,6 @@ TrickyTests.cpp:<line number>: FAILED:
with expansion:
{?} == {?}

-------------------------------------------------------------------------------
An unchecked exception reports the line of the last assertion
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
{Unknown expression after the reported line}
due to unexpected exception with message:
unexpected exception

-------------------------------------------------------------------------------
Contains string matcher
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -146,16 +148,6 @@ ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom exception - not std

-------------------------------------------------------------------------------
Custom std-exceptions can be custom translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom std exception

-------------------------------------------------------------------------------
EndsWith string matcher
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -421,16 +413,6 @@ MiscTests.cpp:<line number>:
warning:
This one ran

-------------------------------------------------------------------------------
Non-std exceptions can be translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom exception

-------------------------------------------------------------------------------
Ordering comparison checks that should fail
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -617,16 +599,6 @@ with expansion:
}
"

-------------------------------------------------------------------------------
Unexpected exceptions can be translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
3.14

-------------------------------------------------------------------------------
Vector matchers that fail
Contains (element)
Expand Down Expand Up @@ -688,16 +660,6 @@ MatchersTests.cpp:<line number>: FAILED:
with expansion:
{ 1, 2, 3 } Equals: { }

-------------------------------------------------------------------------------
When unchecked exceptions are thrown directly they are always failures
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
unexpected exception

-------------------------------------------------------------------------------
When unchecked exceptions are thrown during a CHECK the test should continue
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -732,17 +694,6 @@ ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
expected exception

-------------------------------------------------------------------------------
When unchecked exceptions are thrown from sections they are always failures
section name
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
unexpected exception

-------------------------------------------------------------------------------
Where the LHS is not a simple value
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -915,6 +866,6 @@ with expansion:
"first" == "second"

===============================================================================
test cases: 166 | 119 passed | 44 failed | 3 failed as expected
assertions: 961 | 855 passed | 87 failed | 19 failed as expected
test cases: 167 | 125 passed | 38 failed | 4 failed as expected
assertions: 960 | 859 passed | 81 failed | 20 failed as expected

140 changes: 67 additions & 73 deletions projects/SelfTest/Baselines/console.sw.approved.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,38 +15,84 @@ with message:
yay

-------------------------------------------------------------------------------
#542
#748 - captures with unexpected exceptions
inside REQUIRE_NOTHROW
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
REQUIRE_NOTHROW( thisThrows() )
due to unexpected exception with messages:
answer := 42
expected exception

-------------------------------------------------------------------------------
#748 - captures with unexpected exceptions
inside REQUIRE_THROWS
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>:
PASSED:
REQUIRE_THROWS( thisThrows() )
with message:
answer := 42

-------------------------------------------------------------------------------
#809
-------------------------------------------------------------------------------
CompilationTests.cpp:<line number>
...............................................................................

CompilationTests.cpp:<line number>:
PASSED:
REQUIRE( 42 == f )
with expansion:
42 == {?}

-------------------------------------------------------------------------------
#833
-------------------------------------------------------------------------------
CompilationTests.cpp:<line number>
...............................................................................

CompilationTests.cpp:<line number>:
PASSED:
CHECK_THROWS_AS( throws_int(true), int )
REQUIRE( a == t )
with expansion:
3 == 3

CompilationTests.cpp:<line number>:
PASSED:
CHECK_THROWS_AS( throws_int(true), int& )
CHECK( a == t )
with expansion:
3 == 3

CompilationTests.cpp:<line number>:
PASSED:
CHECK_THROWS_AS( throws_int(true), const int )
REQUIRE_THROWS( throws_int(true) )

CompilationTests.cpp:<line number>:
PASSED:
CHECK_THROWS_AS( throws_int(true), const int& )

-------------------------------------------------------------------------------
#809
-------------------------------------------------------------------------------
CompilationTests.cpp:<line number>
...............................................................................
CompilationTests.cpp:<line number>:
PASSED:
REQUIRE_NOTHROW( throws_int(false) )

CompilationTests.cpp:<line number>:
PASSED:
REQUIRE( 42 == f )
REQUIRE_THAT( "aaa", Catch::EndsWith("aaa") )
with expansion:
42 == {?}
"aaa" ends with: "aaa"

CompilationTests.cpp:<line number>:
PASSED:
REQUIRE( templated_tests<int>(3) )
with expansion:
true

-------------------------------------------------------------------------------
#835 -- errno should not be touched by Catch
Expand Down Expand Up @@ -398,11 +444,6 @@ ExceptionTests.cpp:<line number>:
PASSED:
CHECK( 1 == 1 )

ExceptionTests.cpp:<line number>: FAILED:
{Unknown expression after the reported line}
due to unexpected exception with message:
unexpected exception

-------------------------------------------------------------------------------
Anonymous test case 1
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -957,16 +998,6 @@ ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom exception - not std

-------------------------------------------------------------------------------
Custom std-exceptions can be custom translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom std exception

-------------------------------------------------------------------------------
Demonstrate that a non-const == is not used
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -4477,16 +4508,6 @@ MiscTests.cpp:<line number>:
warning:
This one ran

-------------------------------------------------------------------------------
Non-std exceptions can be translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
custom exception

-------------------------------------------------------------------------------
NotImplemented exception
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -6807,31 +6828,35 @@ PASSED:
with expansion:
"error: tag alias, "[@zzz]" already registered.
First seen at file:2
Redefined at file:10" contains: "[@zzz]"
Redefined at file:10
" contains: "[@zzz]"

TagAliasTests.cpp:<line number>:
PASSED:
CHECK_THAT( what, Contains( "file" ) )
with expansion:
"error: tag alias, "[@zzz]" already registered.
First seen at file:2
Redefined at file:10" contains: "file"
Redefined at file:10
" contains: "file"

TagAliasTests.cpp:<line number>:
PASSED:
CHECK_THAT( what, Contains( "2" ) )
with expansion:
"error: tag alias, "[@zzz]" already registered.
First seen at file:2
Redefined at file:10" contains: "2"
Redefined at file:10
" contains: "2"

TagAliasTests.cpp:<line number>:
PASSED:
CHECK_THAT( what, Contains( "10" ) )
with expansion:
"error: tag alias, "[@zzz]" already registered.
First seen at file:2
Redefined at file:10" contains: "10"
Redefined at file:10
" contains: "10"

-------------------------------------------------------------------------------
Tag alias can be registered against tag patterns
Expand Down Expand Up @@ -7920,16 +7945,6 @@ PASSED:
with expansion:
true

-------------------------------------------------------------------------------
Unexpected exceptions can be translated
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
3.14

-------------------------------------------------------------------------------
Use a custom approx
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -8150,16 +8165,6 @@ ExceptionTests.cpp:<line number>:
PASSED:
REQUIRE_THROWS( thisThrows() )

-------------------------------------------------------------------------------
When unchecked exceptions are thrown directly they are always failures
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
unexpected exception

-------------------------------------------------------------------------------
When unchecked exceptions are thrown during a CHECK the test should continue
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -8194,17 +8199,6 @@ ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
expected exception

-------------------------------------------------------------------------------
When unchecked exceptions are thrown from sections they are always failures
section name
-------------------------------------------------------------------------------
ExceptionTests.cpp:<line number>
...............................................................................

ExceptionTests.cpp:<line number>: FAILED:
due to unexpected exception with message:
unexpected exception

-------------------------------------------------------------------------------
Where the LHS is not a simple value
-------------------------------------------------------------------------------
Expand Down Expand Up @@ -9404,6 +9398,6 @@ MiscTests.cpp:<line number>:
PASSED:

===============================================================================
test cases: 166 | 118 passed | 45 failed | 3 failed as expected
assertions: 963 | 855 passed | 89 failed | 19 failed as expected
test cases: 167 | 124 passed | 39 failed | 4 failed as expected
assertions: 962 | 859 passed | 83 failed | 20 failed as expected

Loading

0 comments on commit 250f0ee

Please sign in to comment.