Error thrown in beforeEach causes tests to be reported as "passed".

[edhager]

I am using intern 4.0.2.

When I run my unit tests in the browser and an error is thrown in a beforeEach, the corresponding tests are reported as "passed" in the HTML report. If I run the unit tests in Node, I see the error in the console and the "TOTAL" line reports a suite error and indicates the test as "not run".

Steps to reproduce:

1. In an empty directory, run npm install intern
2. In that directory, create this test file and name it sample.js:

var assert = intern.getPlugin('chai').assert;
var test = intern.getInterface('tdd');

test.suite('sample', function () {
	test.beforeEach(function () {
		throw new Error('I am a beforeEach error!');
	});

	test.test('test 1', function () {
		assert.isTrue(false);
	});
});

3. Run the command: ./node_modules/.bin/intern suites=sample.js. This does what I expect.

Suite node - sample FAILED
Error: I am a beforeEach error!
  at <sample.js:6:9>
  at <src/lib/interfaces/tdd.ts:143:19>
  at <anonymous>
TOTAL: tested 1 platforms, 0 passed, 0 failed, 1 not run; suite error occurred

4. Run the command: ./node_modules/.bin/intern serveOnly.
5. Point browser at http://localhost:9000/__intern/index.html?suites=sample.js. Notice the test is reported as passed in the suite row. There is no row for the test itself. The browser's console reports the same info as the reporter does in Node.

Note: In these steps, I am not creating an intern.json. Even though the console reports the missing intern.json file as an error, its existence has no effect on this error.

I don't think the HTML report should make it look like the test passed. I think the test should be reported as not ran. It should appear as a row under the suite and that row should be a color other than green.

[jason0x43]

I actually fixed a similar issue with some of the other reporters fairly recently, so this won't be too bad. The problem is just that Intern's reporters have traditionally been focused on failures; if a test didn't fail, it was assumed to have passed. Luckily we have enough info to determined whether it passed or didn't run.