neutrino test always exits with 0

[okonet]

When using Jest preset, even if my test suit fails the exit code is always 0 which causes CI to always pass.

neutrino version 7.2.1+ (also tested on 7.4.0)

It works as expected on v6.2.0

[okonet]

Also the coverage report is gone between v6 and v7

[okonet]

Looks like 3d7ea84 broke it since Jest runCLI returns a promise...

[eliperelman]

Trying to dig into the Jest changelog, but nowhere did it mention this breaking change. I found the part of the commit that broke this:

https://github.com/facebook/jest/pull/3862/files#diff-281a28872bb5a78b747f7cd3d114a0edR47

Unfortunately the commit message is only "Refactor CLI", and didn't mention at all how the API had changed here, so this was missed. Apologies.

[okonet]

Oh wow! Nothing to apologize. Let's try to cover this with tests to prevent in the future

[eliperelman]

Just to address the issue title: the neutrino test command always exits with a 0 code unless a Promise gets rejected, like with a test runner, in which case it should exit with 1.

https://github.com/mozilla-neutrino/neutrino-dev/blob/abdcbeb20098531a74a6d39198e878de29309e8d/packages/neutrino/bin/test.js#L10-L14

https://github.com/mozilla-neutrino/neutrino-dev/blob/abdcbeb20098531a74a6d39198e878de29309e8d/packages/neutrino/bin/base.js#L78