Modify runner and reporters to use JSON serializable results

[ForbesLindesay]

The first step down the road to turn reporters into separate modules

Extremely useful for writing CI systems as you can serialize the complete state of the world and let people view it in a local command line, the browser etc. using whichever reporter they want. This would make tj/mocha-cloud#5 much simpler as it wouldn't need to be able to serialize circular data structures with prototype chains.

I've specified the resulting API here so you can read through the basics of what's changed in plain English.

[tj]

I dont see a ton of immediate value in having CI related stuff utilizing regular reporters, if you're running tests in 8+ browsers whatever the least noisy solution is is the best IMO

[ForbesLindesay]

I think they'd be useless as the only reporter. But really usefull as a drildown. For example I don't have easy access to an ie 7.0 browser so if it fails for that browser I want all the info that the browser reporter provides.

[ForbesLindesay]

The other thing I think would be really useful is getting output from the more special reporters such as html-coverage and markdown documentation. Markdown documentation as a pre-made website that's always up to date would be a nice thing to be able to link to (and more people might find out that the feature exists to generate it themselves). Code coverage would be nice mostly because it's a pain to set up the build yourself (especially if your on windows), so having it set up for you online would be great. I do realise that getting code-coverage to work the same would require some extra work, but I think this represents the first step.

[ForbesLindesay]

It would also helpfully let someone re-use the same reporters with a completely different test framework.

[tj]

the only thing I would want to see from a CI is which browsers failed and what the error(s) are, the rest doesn't matter at all really

[ForbesLindesay]

I find it helpful to be able to view which tests failed and passed as well as often that narrows down what could've caused the error a lot. I think if people in general were happy with that we'd have far fewer and far simpler reporters.

[ForbesLindesay]

Perhaps the ci thing is leaving us with a little bit of a red herring here (I think we use them in quite different ways).

I think it presents a cleaner simpler abstraction to reporters and is advantageous because of that, and I think it would be nice to have the ability to run the reporters as separate command line apps and pipe data from the test runner to the reporters (e.g. For aggregating multiple tests or using non-mocha test frameworks with mocha reporters) and I can't see any way of doing that without simplifying the format to be JSON serializable.

[tj]

yeah I dont disagree there, I'd like to do that eventually anyway, but we already have json-stream working just fine, they would just be built on that or really similar

[ForbesLindesay]

OK, but JSON stream doesn't output anywhere near enough information to build the other reporters on top of it.

[tj]

yeah fair enough, at a glance I dont totally get what this pull-request adds, it seems to be a lot of noise from adding (time whatever that's for and a few additional props in the clean thing lower. I'll try and pull it down soon and have a better look

[ForbesLindesay]

The time thing is so that all the timings are available as part of the test results (from the runner) and therefore the timings computed bor the test summary are correct even if the results were stored or processed in one go rather than streamed.

All the other properties are used by at least one reporter at some point.

[jbnicolai]

@ForbesLindesay cleaning this PR up, as I believe it's no longer of interest. Sorry for the long silence, and feel free to add an explanation if you feel that this is still relevant.

Thanks for the effort 👍 and sorry for the long silence.