Don't generate coverage reports when tests fail.

[olafurw]

Clear and concise description of the problem

A test coverage report (either text or generated html/json) does not have a lot of value if the test run itself has failed. The text output (if large enough) can cover the entire screen and hide the fact that the test has failed.

Suggested solution

If the test failed, don't show or generate any coverage report.

Alternative

A configuration flag could be useful as well, for those that want the older behavior (or the other way around).

test: {
  coverage: {
    generateOnFailure: true
  }
}

Additional context

No response

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[AriPerkkio]

Personally I agree that coverage should not be reported at all when tests fail. There should be no use for coverage report when tests fail, as the test execution stops where ever the error happens. Even if someone is posting the results to some 3rd party coverage service, the overall coverage is not likely to meet the thresholds when tests fail.

The text output (if large enough) can cover the entire screen and hide the fact that the test has failed.

Especially in watch-mode this must be frustrating as users have to scroll up past the coverage report to see the actual error.

But it seems that Jest, NYC and c8 report the coverage even when test execution fails. We might want to keep same behaviour as default, and introduce a new configuration flag for this. Instead of generateOnFailure, maybe name it reportOnFailure?

[olafurw]

If it's the standard across other tools to generate then that's fine and a reportOnFailure would be very helpful.