Multiple reporters support (continuation of #930)

[benvinegar]

This is a continuation of @misterdjules PR #930 – multiple reporters support.

On top of @misterdjules work, it has:

• Merged up w/ master (original patch was > 1 year old)
• Restores 'spec' as the default reporter
• Fixed unit tests
• An additional unit test for multiple reporters (tests/reporters/multiple.js)
• Renames variables to be closer to original mocha.js code (e.g. _reporter -> _reporters)
• Keeps original Mocha.prototype.reporter interface (vs the plural reporters) for API compatibility. IMHO anybody using Mocha directly shouldn't have to change their code to remain compatible.

It does not implement Growl as a separate reporter, as was discussed here. I think it could be done as a follow-up patch just fine, but if you'd prefer that was done before accepting then that's cool too.

[benvinegar]

Hmm, should this come back? Mocha.prototype.reporter also throws if the reporter isn't found.

[dasilvacontin]

Checking if the reporters exist is still necessary. Ignoring those which are not found is not the way to go, since there is a reason why they have been specified.

[benvinegar]

Looking at this closer ... I think there's a fundamental flaw. Even though, yes, this invokes multiple reporters – they still all go to stdout.

e.g. if you specify --reporters spec,json you will have the spec output to stdout, followed by JSON output to stdout. If you use json-stream as one of your reporters, I think the output could also intermingle.

So, I'm not sure if this patch is useful unless you can additionally specify a target output file per reporter, something like:

mocha test.js --reporter:spec,json=tmp/out.json

... which seems kind of gross. Thoughts?

[travisjeffery]

ya that's bit gross. what's your need/use case for multiple reporters btw?

[boneskull]

@travisjeffery lcov generation + arbitrary console logger. As it stands if I want this, I need to run my tests 2x.

[demmer]

@travisjeffery Please consider #1218 as part of pushing this multiple reporter work through.

From my standpoint, being able to include the xunit reporter writing to a file and a spec reporter to console would be great, but I agree with @benvinegar that having multiple reporters going to console doesn't seem very useful.

[benvinegar]

@travisjeffery We use both json and xunit at the same time. The former for some scripts that generate reports on our unit tests. The latter for integrating with our particular CI system.

I was working on a little converter – pretty trivial – but I saw this patch was nearly accepted and thought I should help push it through.

[jbnicolai]

+1 from me :)

[Zensavona]

Is this going in any time soon?

[lpinca]

I would like to have this feature. 👍

[dasilvacontin]

mocha --reporters [spec],[json:path/to/file],[nyan:path/to/file]
mocha --reporters spec, json:path/to/file, nyan:path/to/file

I'm not sure how possible these are with commander, I should investigate the possibilities.

[benvinegar]

@dasilvacontin That format looks good to me (and certainly better than whatever the hell I suggested above). I'd be glad to move forward with it.

[dasilvacontin]

I really like the colon to define the file path for a given reporter. It would be nice to hear someone else's opinion too.

Very excited about landing this PR on master soon. 😄

[benvinegar]

Okay, just tried merging up with master and noticed that mocha introduced --reporter-options (7ce102b) ... which appears to only be used by the XUnit reporter and is not documented on mochajs.org.

@dasilvacontin – any thoughts on how you'd like multiple reporters to play nicely with --reporter-options?

e.g. something like:

mocha --reporter spec,json:tmp/out.json --reporter-options json:prettyprint

[dasilvacontin]

Reporter options currently have the following format:

--reporter-options <k=v,k2=v2,...>'

Since you can set multiple options for each reporter, what about:

mocha --reporter spec,json:tmp/out.json --reporter-options spec{style:prettyprint,color:blue},json{prettify,indent:tabs}
// or
mocha --reporter spec,json:tmp/out.json --reporter-options spec{style=prettyprint,color=blue},json{prettify,indent=tabs}

I kind of prefer the {x:y} syntax since it resembles object definition.

[benvinegar]

That works for me. I think we should keep the default behavior the same as today though, for backwards compatibility. Meaning, if you don't specify multiple reporters, this should work fine:

mocha --reporter json --reporter-options prettyprint

That make sense?

[dasilvacontin]

Sure. We could change it in the future, if necessary, or requested. Current reporter-options code needs to be updated to support multiple options, and also to support options without value. Like the prettyprint one, which should set prettyprint to true in the options object.

[travisjeffery]

mocha --reporter spec,json:tmp/out.json --reporter-options spec{style:prettyprint,color:blue},json{prettify,indent:tabs}

this looks good to me

[dasilvacontin]

@travisjeffery, glad to hear that. :)

[jaredly]

+1
What's the status here? once merge conflicts are fixed, is this ready to go?

[benvinegar]

Picking this back up.

[boneskull]

I think we need to close this one, as I imagine we'd want this in branch v3.0.0

[boneskull]

please create new PR and squash when ready

[darkn3rd]

👍

[e-cloud]

any progress? mocha v3.0.0 has been released for a while

[kay-ramme]

would be great to have ..

[azachar]

+1

[pmarreck]

Literally just used mocha for the first time and one of the first things I noticed was the total absence of this feature (having used multiple test reporters on Rails tests in the past).

FFS, guys...

[boneskull]

@pmarreck Please keep your entitled attitude off of this issue tracker.

[Anthony-Mckale]

guessing the developers of mocha have good reasons for not supporting multiple reporters out the box

for anyone else coming to this ticket looking for multiple reports, i found writing a wee simple custom reporter which pulled in multiple reports from mocha really worked for me

As I found a lot of old node modules with combo reporters, on the web copy and pasted the reports, which I found a bit dangerous for my liking (would worry about have a old version of the reporters)

my ~20 line reporter

`let mocha = require('mocha');

exports = module.exports = MultiReporter;

function MultiReporter(runner, options) {
this.reports = [];
if (!options.reporterOptions.reporters) {
console.log('\nneeds --reporter-options reporters="SPACE_SEPARATED_MOCHA_REPORTS"');
return;
}
let that = this;
options.reporterOptions.reporters.split(' ').forEach(function(report) {
let ReportClass = mocha.reporters[report];
if (!ReportClass) {
console.log('\ninvalid report class available: ' + Object.keys(mocha.reporters).join(','));
return;
}
let reportInstance = new ReportClass(runner, options);
that.reports.push(reportInstance);
});
}
MultiReporter.prototype.epilogue = function() {
this.reports.forEach(function(reportInstance) {
reportInstance.epilogue();
});
};
`

which you can just reference with

mocha STUFF_FLAGS TEST_FILES --reporter PATH_TO_REPORTER/REPORTER_FILE.js --reporter-options reporters=REPORTERS,OTHER_OPTS

aka for my project (with my project flags)

node node_modules\mocha\bin\mocha --require babel-register --require test/mocha/bootstrap.js --ui chai-style-jasmine src\**\*.spec.js --reporter test/mocha/multi-reporter.js --reporter-options reporters="XUnit Spec",output=reports/mocha-xunit/test-results.xml

[benjamingr]

Hey @boneskull 👋

Is doing #1360 (comment) generally considered fine?