Add support for multiple reporters

docs/command-line.md

@@ -123,7 +123,9 @@ Test names containing special characters, such as `,` or `[` can specify them on
 <a id="choosing-a-reporter-to-use"></a>
 ## Choosing a reporter to use
 
-<pre>-r, --reporter &lt;reporter></pre>
+<pre>-r, --reporter &lt;reporter[::output-file]&gt;</pre>
+
+> Support for providing output-file through the `-r`, `--reporter` flag was [introduced](https://github.com/catchorg/Catch2/pull/2183) in Catch2 X.Y.Z
 
 A reporter is an object that formats and structures the output of running tests, and potentially summarises the results. By default a console reporter is used that writes, IDE friendly, textual output. Catch comes bundled with some alternative reporters, but more can be added in client code.<br />
 The bundled reporters are:
@@ -136,6 +138,15 @@ The bundled reporters are:
 
 The JUnit reporter is an xml format that follows the structure of the JUnit XML Report ANT task, as consumed by a number of third-party tools, including Continuous Integration servers such as Jenkins. If not otherwise needed, the standard XML reporter is preferred as this is a streaming reporter, whereas the Junit reporter needs to hold all its results until the end so it can write the overall results into attributes of the root node.
 
+This option may be passed multiple times to use multiple (different) reporters  at the same time. See [Reporters](reporters.md#multiple-reporters) for details.
+
+As with the `--out` flag, `-` means writing to stdout.
+
+_Note: There is currently no way to escape `::` in the reporter spec,
+and thus reporter/file names with `::` in them will not work properly.
+As `::` in paths is relatively obscure (unlike `:`), we do not consider
+this an issue._
+
 <a id="breaking-into-the-debugger"></a>
 ## Breaking into the debugger
 <pre>-b, --break</pre>
@@ -178,11 +189,16 @@ If one or more test-specs have been supplied too then only the matching tests wi
 
 <a id="sending-output-to-a-file"></a>
 ## Sending output to a file
-<pre>-o, --out &lt;filename>
+<pre>-o, --out &lt;filename&gt;
 </pre>
 
 Use this option to send all output to a file. By default output is sent to stdout (note that uses of stdout and stderr *from within test cases* are redirected and included in the report - so even stderr will effectively end up on stdout).
 
+Using `-` as the filename sends the output to stdout.
+
+> Support for `-` as the filename was introduced in Catch2 X.Y.Z
+
+
 <a id="naming-a-test-run"></a>
 ## Naming a test run
 <pre>-n, --name &lt;name for test run></pre>
@@ -414,15 +430,15 @@ There are some limitations of this feature to be aware of:
 - Code outside of sections being skipped will still be executed - e.g. any set-up code in the TEST_CASE before the
 start of the first section.</br>
 - At time of writing, wildcards are not supported in section names.
-- If you specify a section without narrowing to a test case first then all test cases will be executed 
+- If you specify a section without narrowing to a test case first then all test cases will be executed
 (but only matching sections within them).
 
 
 <a id="filenames-as-tags"></a>
 ## Filenames as tags
 <pre>-#, --filenames-as-tags</pre>
 
-When this option is used then every test is given an additional tag which is formed of the unqualified 
+When this option is used then every test is given an additional tag which is formed of the unqualified
 filename it is found in, with any extension stripped, prefixed with the `#` character.
 
 So, for example,  tests within the file `~\Dev\MyProject\Ferrets.cpp` would be tagged `[#Ferrets]`.

docs/reporters.md

@@ -19,13 +19,13 @@ There are four reporters built in to the single include:
 * `console` writes as lines of text, formatted to a typical terminal width, with colours if a capable terminal is detected.
 * `compact` similar to `console` but optimised for minimal output - each entry on one line
 * `junit` writes xml that corresponds to Ant's [junitreport](http://help.catchsoftware.com/display/ET/JUnit+Format) target. Useful for build systems that understand Junit.
-Because of the way the junit format is structured the run must complete before anything is written. 
+Because of the way the junit format is structured the run must complete before anything is written.
 * `xml` writes an xml format tailored to Catch. Unlike `junit` this is a streaming format so results are delivered progressively.
 
 There are a few additional reporters, for specific build systems, in the Catch repository (in `include\reporters`) which you can `#include` in your project if you would like to make use of them.
 Do this in one source file - the same one you have `CATCH_CONFIG_MAIN` or `CATCH_CONFIG_RUNNER`.
 
-* `teamcity` writes the native, streaming, format that [TeamCity](https://www.jetbrains.com/teamcity/) understands. 
+* `teamcity` writes the native, streaming, format that [TeamCity](https://www.jetbrains.com/teamcity/) understands.
 Use this when building as part of a TeamCity build to see results as they happen ([code example](../examples/207-Rpt-TeamCityReporter.cpp)).
 * `tap` writes in the TAP ([Test Anything Protocol](https://en.wikipedia.org/wiki/Test_Anything_Protocol)) format.
 * `automake` writes in a format that correspond to [automake  .trs](https://www.gnu.org/software/automake/manual/html_node/Log-files-generation-and-test-results-recording.html) files
@@ -35,6 +35,19 @@ You see what reporters are available from the command line by running with `--li
 
 By default all these reports are written to stdout, but can be redirected to a file with [`-o` or `--out`](command-line.md#sending-output-to-a-file)
 
+<a id="multiple-reporters"></a>
+## Using multiple reporters
+
+> Support for having multiple parallel reporters was [introduced](https://github.com/catchorg/Catch2/pull/2183) in Catch2 X.Y.Z
+
+Multiple reporters may be used at the same time, e.g. to save a machine-readable output to a file but still print the human-readable output to the console:
+```
+-r console -r xml::result.xml -r junit::result-junit.xml
+```
+
+The output file name is given after the reporter name, delimited by a colon. If omitted, it defaults to the file name specified by `-o` (or stdout). Only one reporter may use the default output.
+
+
 ## Writing your own reporter
 
 You can write your own custom reporter and register it with Catch.

src/catch2/catch_config.cpp

@@ -13,13 +13,39 @@
 #include <catch2/internal/catch_test_spec_parser.hpp>
 #include <catch2/interfaces/catch_interfaces_tag_alias_registry.hpp>
 
+#include <ostream>
 
 namespace Catch {
+    namespace Detail {
+        namespace {
+            class RDBufStream : public IStream {
+                mutable std::ostream m_os;
+
+            public:
+                //! The streambuf `sb` must outlive the constructed object.
+                RDBufStream( std::streambuf* sb ): m_os( sb ) {}
+                ~RDBufStream() override = default;
+
+            public: // IStream
+                std::ostream& stream() const override { return m_os; }
+            };
+        } // unnamed namespace
+    }     // namespace Detail
+
+    std::ostream& operator<<( std::ostream& os,
+                              ConfigData::ReporterAndFile const& reporter ) {
+        os << "{ " << reporter.reporterName << ", ";
+        if ( reporter.outputFileName ) {
+            os << *reporter.outputFileName;
+        } else {
+            os << "<default-output>";
+        }
+        return os << " }";
+    }
 
-    Config::Config( ConfigData const& data )
-    :   m_data( data ),
-        m_stream( Catch::makeStream(m_data.outputFilename) )
-    {
+    Config::Config( ConfigData const& data ):
+        m_data( data ),
+        m_defaultStream( openStream( data.defaultOutputFilename ) ) {
         // We need to trim filter specs to avoid trouble with superfluous
         // whitespace (esp. important for bdd macros, as those are manually
         // aligned with whitespace).
@@ -39,33 +65,46 @@ namespace Catch {
             }
         }
         m_testSpec = parser.testSpec();
+
+        m_reporterStreams.reserve( m_data.reporterSpecifications.size() );
+        for ( auto const& reporterAndFile : m_data.reporterSpecifications ) {
+            if ( reporterAndFile.outputFileName.none() ) {
+                m_reporterStreams.emplace_back( new Detail::RDBufStream(
+                    m_defaultStream->stream().rdbuf() ) );
+            } else {
+                m_reporterStreams.emplace_back(
+                    openStream( *reporterAndFile.outputFileName ) );
+            }
+        }
     }
 
     Config::~Config() = default;
 
 
-    std::string const& Config::getFilename() const {
-        return m_data.outputFilename ;
-    }
-
     bool Config::listTests() const          { return m_data.listTests; }
     bool Config::listTags() const           { return m_data.listTags; }
     bool Config::listReporters() const      { return m_data.listReporters; }
 
-    std::string const& Config::getReporterName() const { return m_data.reporterName; }
-
     std::vector<std::string> const& Config::getTestsOrTags() const { return m_data.testsOrTags; }
     std::vector<std::string> const& Config::getSectionsToRun() const { return m_data.sectionsToRun; }
 
+    std::vector<ConfigData::ReporterAndFile> const& Config::getReportersAndOutputFiles() const {
+        return m_data.reporterSpecifications;
+    }
+
+    std::ostream& Config::getReporterOutputStream(std::size_t reporterIdx) const {
+        return m_reporterStreams.at(reporterIdx)->stream();
+    }
+
     TestSpec const& Config::testSpec() const { return m_testSpec; }
     bool Config::hasTestFilters() const { return m_hasTestFilters; }
 
     bool Config::showHelp() const { return m_data.showHelp; }
 
     // IConfig interface
     bool Config::allowThrows() const                   { return !m_data.noThrow; }
-    std::ostream& Config::stream() const               { return m_stream->stream(); }
-    StringRef Config::name() const                     { return m_data.name.empty() ? m_data.processName : m_data.name; }
+    std::ostream& Config::defaultStream() const        { return m_defaultStream->stream(); }
+    StringRef Config::name() const { return m_data.name.empty() ? m_data.processName : m_data.name; }
     bool Config::includeSuccessfulResults() const      { return m_data.showSuccessfulTests; }
     bool Config::warnAboutMissingAssertions() const {
         return !!( m_data.warnings & WarnAbout::NoAssertions );
@@ -92,4 +131,8 @@ namespace Catch {
     unsigned int Config::benchmarkResamples() const               { return m_data.benchmarkResamples; }
     std::chrono::milliseconds Config::benchmarkWarmupTime() const { return std::chrono::milliseconds(m_data.benchmarkWarmupTime); }
 
+    Detail::unique_ptr<IStream const> Config::openStream(std::string const& outputFileName) {
+        return Catch::makeStream(outputFileName);
+    }
+
 } // end namespace Catch

src/catch2/catch_config.hpp

@@ -14,6 +14,7 @@
 #include <catch2/internal/catch_optional.hpp>
 #include <catch2/internal/catch_random_seed_generation.hpp>
 
+#include <iosfwd>
 #include <vector>
 #include <string>
 
@@ -22,6 +23,18 @@ namespace Catch {
     struct IStream;
 
     struct ConfigData {
+        struct ReporterAndFile {
+            std::string reporterName;
+
+            // If none, the output goes to the default output.
+            Optional<std::string> outputFileName;
+
+            friend bool operator==(ReporterAndFile const& lhs, ReporterAndFile const& rhs) {
+                return lhs.reporterName == rhs.reporterName && lhs.outputFileName == rhs.outputFileName;
+            }
+            friend std::ostream& operator<<(std::ostream &os, ReporterAndFile const& reporter);
+        };
+
         bool listTests = false;
         bool listTags = false;
         bool listReporters = false;
@@ -55,13 +68,17 @@ namespace Catch {
         UseColour useColour = UseColour::Auto;
         WaitForKeypress::When waitForKeypress = WaitForKeypress::Never;
 
-        std::string outputFilename;
+        std::string defaultOutputFilename;
         std::string name;
         std::string processName;
 #ifndef CATCH_CONFIG_DEFAULT_REPORTER
 #define CATCH_CONFIG_DEFAULT_REPORTER "console"
 #endif
-        std::string reporterName = CATCH_CONFIG_DEFAULT_REPORTER;
+        std::vector<ReporterAndFile> reporterSpecifications = {
+            {CATCH_CONFIG_DEFAULT_REPORTER, {}}
+        };
+        // Internal: used as parser state
+        bool _nonDefaultReporterSpecifications = false;
 #undef CATCH_CONFIG_DEFAULT_REPORTER
 
         std::vector<std::string> testsOrTags;
@@ -76,13 +93,12 @@ namespace Catch {
         Config( ConfigData const& data );
         ~Config() override; // = default in the cpp file
 
-        std::string const& getFilename() const;
-
         bool listTests() const;
         bool listTags() const;
         bool listReporters() const;
 
-        std::string const& getReporterName() const;
+        std::vector<ConfigData::ReporterAndFile> const& getReportersAndOutputFiles() const;
+        std::ostream& getReporterOutputStream(std::size_t reporterIdx) const;
 
         std::vector<std::string> const& getTestsOrTags() const override;
         std::vector<std::string> const& getSectionsToRun() const override;
@@ -94,7 +110,7 @@ namespace Catch {
 
         // IConfig interface
         bool allowThrows() const override;
-        std::ostream& stream() const override;
+        std::ostream& defaultStream() const override;
         StringRef name() const override;
         bool includeSuccessfulResults() const override;
         bool warnAboutMissingAssertions() const override;
@@ -118,13 +134,14 @@ namespace Catch {
         std::chrono::milliseconds benchmarkWarmupTime() const override;
 
     private:
+        Detail::unique_ptr<IStream const> openStream(std::string const& outputFileName);
         ConfigData m_data;
 
-        Detail::unique_ptr<IStream const> m_stream;
+        Detail::unique_ptr<IStream const> m_defaultStream;
+        std::vector<Detail::unique_ptr<IStream const>> m_reporterStreams;
         TestSpec m_testSpec;
         bool m_hasTestFilters = false;
     };
-
 } // end namespace Catch
 
 #endif // CATCH_CONFIG_HPP_INCLUDED

src/catch2/catch_session.cpp

@@ -34,24 +34,34 @@ namespace Catch {
     namespace {
         const int MaxExitCode = 255;
 
-        IStreamingReporterPtr createReporter(std::string const& reporterName, IConfig const* config) {
+        IStreamingReporterPtr createReporter(std::string const& reporterName, ReporterConfig const& config) {
             auto reporter = Catch::getRegistryHub().getReporterRegistry().create(reporterName, config);
             CATCH_ENFORCE(reporter, "No reporter registered with name: '" << reporterName << '\'');
 
             return reporter;
         }
 
         IStreamingReporterPtr makeReporter(Config const* config) {
-            if (Catch::getRegistryHub().getReporterRegistry().getListeners().empty()) {
-                return createReporter(config->getReporterName(), config);
+            if (Catch::getRegistryHub().getReporterRegistry().getListeners().empty()
+                    && config->getReportersAndOutputFiles().size() == 1) {
+                auto& stream = config->getReporterOutputStream(0);
+                return createReporter(config->getReportersAndOutputFiles()[0].reporterName, ReporterConfig(config, stream));
             }
 
             auto multi = Detail::make_unique<ListeningReporter>(config);
+
             auto const& listeners = Catch::getRegistryHub().getReporterRegistry().getListeners();
             for (auto const& listener : listeners) {
-                multi->addListener(listener->create(Catch::ReporterConfig(config)));
+                multi->addListener(listener->create(Catch::ReporterConfig(config, config->defaultStream())));
+            }
+
+            std::size_t reporterIdx = 0;
+            for (auto const& reporterAndFile : config->getReportersAndOutputFiles()) {
+                auto& stream = config->getReporterOutputStream(reporterIdx);
+                multi->addReporter(createReporter(reporterAndFile.reporterName, ReporterConfig(config, stream)));
+                reporterIdx++;
             }
-            multi->addReporter(createReporter(config->getReporterName(), config));
+
             return multi;
         }
 

src/catch2/interfaces/catch_interfaces_config.hpp

@@ -61,7 +61,7 @@ namespace Catch {
         virtual ~IConfig();
 
         virtual bool allowThrows() const = 0;
-        virtual std::ostream& stream() const = 0;
+        virtual std::ostream& defaultStream() const = 0;
         virtual StringRef name() const = 0;
         virtual bool includeSuccessfulResults() const = 0;
         virtual bool shouldDebugBreak() const = 0;

src/catch2/interfaces/catch_interfaces_reporter.cpp

@@ -20,9 +20,6 @@
 
 namespace Catch {
 
-    ReporterConfig::ReporterConfig( IConfig const* _fullConfig )
-    :   m_stream( &_fullConfig->stream() ), m_fullConfig( _fullConfig ) {}
-
     ReporterConfig::ReporterConfig( IConfig const* _fullConfig, std::ostream& _stream )
     :   m_stream( &_stream ), m_fullConfig( _fullConfig ) {}
 

src/catch2/interfaces/catch_interfaces_reporter.hpp

@@ -32,8 +32,6 @@ namespace Catch {
     struct IConfig;
 
     struct ReporterConfig {
-        explicit ReporterConfig( IConfig const* _fullConfig );
-
         ReporterConfig( IConfig const* _fullConfig, std::ostream& _stream );
 
         std::ostream& stream() const;

src/catch2/interfaces/catch_interfaces_reporter_registry.hpp

@@ -23,13 +23,14 @@ namespace Catch {
     using IStreamingReporterPtr = Detail::unique_ptr<IStreamingReporter>;
     struct IReporterFactory;
     using IReporterFactoryPtr = Detail::unique_ptr<IReporterFactory>;
+    struct ReporterConfig;
 
     struct IReporterRegistry {
         using FactoryMap = std::map<std::string, IReporterFactoryPtr, Detail::CaseInsensitiveLess>;
         using Listeners = std::vector<IReporterFactoryPtr>;
 
         virtual ~IReporterRegistry(); // = default
-        virtual IStreamingReporterPtr create( std::string const& name, IConfig const* config ) const = 0;
+        virtual IStreamingReporterPtr create( std::string const& name, ReporterConfig const& config ) const = 0;
         virtual FactoryMap const& getFactories() const = 0;
         virtual Listeners const& getListeners() const = 0;
     };