Internal test feedback/report UX

[plocket]

Currently, our internal test outcomes are a bit confusing. We have tests that test failure messages. The summary report shows what the author would see, so those tests look like they've failed. That's useful, but it's impossible for us to see our own failing tests in there. The cucumberjs output is what we usually use to hunt down the problem. Is that enough?

There's an issue somewhere in our repo about wrapping all our cucumber-related code so we can meta-test it. Another library has done this in the past. Looks like a ton of work. Is it worth it?

Needs:

• We need to see what test authors see.
• We need to see the actual status of our tests.
• Separate the signal from the noise.

Are there other improvements we can make? A couple ideas:

• Make different icons for internal test failures to save in our debug files
• Note all the log codes that showed up in one Scenario's report