Mocha continue after failure

[allen-tong]

Is it just me, or does Mocha terminate completely once it fails a single test? This breaks the "run all diagnostics" functionality since it won't run any Amazon tests after failing any Google test. Mocha is supposed to run all tests and report numbers of passes and failures, right?

[allen-tong]

Attached screenshot of error, let me know if you also get this and/or know how to fix it!

[nmalkin]

This is why:

smart-speakers/src/chrome/pages/diagnostics.ts

Line 29 in 0187d62

bail: true,

(see --bail at https://mochajs.org/#usage)

This was the intended behavior because I assumed that, once an early test fails (e.g., you're logged out), the subsequent ones would fail too. You're totally right, though, that this assumption doesn't hold if we're trying to run both Amazon and Google tests.

I see the two easiest options as:

1. Go ahead and allow subsequent tests, even if they'll probably fail
2. Only allow you to run one of the test suites at a time
   • Then you'd have to refresh the page in-between. That's annoying, but not horrible, since most people will only be testing one or the other.
   • We could take this further and move the diagnostics to separate pages, so they don't interact at all.

[nmalkin]

As a side note, the --bail flag on Jest behaves differently from Mocha: there it works across test suites, not individual tests. This threw me off for a bit.

[allen-tong]

Ahhh. Then I think we can just remove the "run all diagnostics" functionality and be done with it. Not sure if by having to refresh the page in-between you mean what was fixed in 252b668? You should be able to just click the appropriate button to run/rerun whichever test you want.

[nmalkin]

Ahhh. Then I think we can just remove the "run all diagnostics" functionality and be done with it.

Sounds good. Done in 8bc4704

Not sure if by having to refresh the page in-between you mean what was fixed in 252b668? You should be able to just click the appropriate button to run/rerun whichever test you want.

Cool! Yeah, I'm not sure exactly what I had in mind, but glad no further action is necessary. 😃