Use CATCH_TRANSLATE_EXCEPTION for printing failures in *_THROWS_MATCHES

[Osse]

Description

Using CATCH_TRANSLATE_EXCEPTION is convenient, but sometimes we want to test the concrete type and message. Using *_THROWS_MATCHES does not use the function provided by CATCH_TRANSLATE_EXCEPTION and just uses std::exception::what() (as documented). The output of what() prefixes the output of describe() without quotes or other annotation making the error harder to read and understand.

It may be that this is intentional and thus neither a bug nor a wanted feature. If so then I wonder if there is some way to customize the generated error messages that I may have missed in the docs.

Additional context

We're using version 3.6.0 here, but I will test 3.7.0 although I don't see any relevant changes.

The following program produces the output below it. I tried my best to make it as simple but complete as possible. It defines two exceptions, one of which does not inherit from std::exception, along with three matchers: one old matcher for each type and new generic matcher for both.

#include <catch2/catch_test_macros.hpp>
#include <catch2/catch_translate_exception.hpp>
#include <catch2/matchers/catch_matchers_exception.hpp>
#include <catch2/matchers/catch_matchers_string.hpp>
#include <catch2/matchers/catch_matchers.hpp>
#include <catch2/matchers/catch_matchers_templated.hpp>

#include <stdexcept>
#include <string>

class Error : public std::runtime_error
{
public:
    Error(std::string what, std::string extra)
        : std::runtime_error(what)
        , extra(extra)
    {}

    std::string toString() const
    {
        return std::string(what()) + ": " + extra;
    }

private:
    std::string extra;
};

CATCH_TRANSLATE_EXCEPTION(Error const& ex)
{
    return ex.toString();
}

class CustomError
{
public:
    CustomError(std::string msg)
        : message(msg)
    {}

    std::string toString() const
    {
        return std::string("I don't inherit: ") + message;
    }

private:
    std::string message;
};

CATCH_TRANSLATE_EXCEPTION(CustomError const& ex)
{
    return ex.toString();
}

using namespace Catch::Matchers;

struct OldErrorMatcher : MatcherBase<Error>
{
    OldErrorMatcher(std::string m)
        : m(m)
    {}

    bool match(const Error &ex) const override
    {
        auto s = ex.toString();
        auto pos = s.find(": ");
        return s.substr(0, pos) == m;
    }

    std::string describe() const override
    {
        return "describe()";
    }

    std::string m;
};

struct OldCustomErrorMatcher : MatcherBase<CustomError>
{
    OldCustomErrorMatcher(std::string m)
        : m(m)
    {}

    bool match(const CustomError &ex) const override
    {
        auto s = ex.toString();
        auto pos = s.find(": ");
        return s.substr(0, pos) == m;
    }

    std::string describe() const override
    {
        return "describe()";
    }

    std::string m;
};

struct NewErrorMatcher : MatcherGenericBase
{
    NewErrorMatcher(std::string m)
        : m(m)
    {}

    template<typename Ex>
    bool match(const Ex &ex) const
    {
        auto s = ex.toString();
        auto pos = s.find(": ");
        return s.substr(0, pos) == m;
    }

    std::string describe() const override
    {
        return "describe()";
    }

    std::string m;
};

void throwError()
{
    throw Error("Oh no", "Bad luck");
}

void throwCustomError()
{
    throw CustomError("Oh no");
}

TEST_CASE("Error") {
    CHECK_THROWS_WITH(throwError(), StartsWith("Ahh yes"));
    CHECK_THROWS_MATCHES(throwError(), Error, MessageMatches(StartsWith("Ahh yes")));
    CHECK_THROWS_MATCHES(throwError(), Error, OldErrorMatcher("Ahh yes"));
    CHECK_THROWS_MATCHES(throwError(), Error, NewErrorMatcher("Ahh yes"));
}

TEST_CASE("CustomError") {
    CHECK_THROWS_WITH(throwCustomError(), StartsWith("Ahh yes"));
    CHECK_THROWS_MATCHES(throwCustomError(), CustomError, OldCustomErrorMatcher("Ahh yes"));
    CHECK_THROWS_MATCHES(throwCustomError(), CustomError, NewErrorMatcher("Ahh yes"));
}

The following is the output of the above program. Note the double space when using MessageMatcher (bug?).

Randomness seeded to: 1082738197

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tests is a Catch2 v3.6.0 host application.
Run with -? for options

-------------------------------------------------------------------------------
Error
-------------------------------------------------------------------------------
/tmp/catch2experiment/test.cpp:128
...............................................................................

/tmp/catch2experiment/test.cpp:129: FAILED:
  CHECK_THROWS_WITH( throwError(), StartsWith("Ahh yes") )
with expansion:
  "Oh no: Bad luck" starts with: "Ahh yes"

/tmp/catch2experiment/test.cpp:130: FAILED:
  CHECK_THROWS_MATCHES( throwError(), Error, MessageMatches(StartsWith("Ahh yes")) )
with expansion:
  Oh no  matches "starts with: "Ahh yes""

/tmp/catch2experiment/test.cpp:131: FAILED:
  CHECK_THROWS_MATCHES( throwError(), Error, OldErrorMatcher("Ahh yes") )
with expansion:
  Oh no describe()

/tmp/catch2experiment/test.cpp:132: FAILED:
  CHECK_THROWS_MATCHES( throwError(), Error, NewErrorMatcher("Ahh yes") )
with expansion:
  Oh no describe()

-------------------------------------------------------------------------------
CustomError
-------------------------------------------------------------------------------
/tmp/catch2experiment/test.cpp:135
...............................................................................

/tmp/catch2experiment/test.cpp:136: FAILED:
  CHECK_THROWS_WITH( throwCustomError(), StartsWith("Ahh yes") )
with expansion:
  "I don't inherit: Oh no" starts with: "Ahh yes"

/tmp/catch2experiment/test.cpp:137: FAILED:
  CHECK_THROWS_MATCHES( throwCustomError(), CustomError, OldCustomErrorMatcher("Ahh yes") )
with expansion:
  {?} describe()

/tmp/catch2experiment/test.cpp:138: FAILED:
  CHECK_THROWS_MATCHES( throwCustomError(), CustomError, NewErrorMatcher("Ahh yes") )
with expansion:
  {?} describe()

===============================================================================
test cases: 2 | 2 failed
assertions: 7 | 7 failed

[Osse]

It turns out that both std::ostream& operator << ( std::ostream& os, Error const& value ) and Catch::StringMaker<Error> as mentioned in tostring.md work just fine for my purposes 😄 I think this could made a little clearer in the documentation, if it is indeed intentional that string matchers use the configured exception translators while other matchers do not, but I don't have an immediate suggestion as to how.