Nested test functions hide errors

[tp]

Description

I was just reviewing a PR where a colleague wrote tests in a nested way.

While the sub-tests never execute / show up in the test output they also did not create an error.

I would expect such code as the example below to at least throw an error.

Test Source

import test from 'ava';

test('outer', (t) => {

  test('innter', (t) => {
    t.fail();
    throw new Error('ERR');
  });

});

Console Output

=> ./node_modules/.bin/ava -v avaTest.js

  ✔ outer

  1 test passed [16:35:53]

Config

No config in package.json

Command-Line Arguments

/ava -v avaTest.js

Relevant Links

• If your project is public, link to the repo so we can investigate directly.
• BONUS POINTS: Create a minimal reproduction and upload it to GitHub. This will get you the fastest support.

Environment

Tell us which operating system you are using, as well as which versions of Node.js, npm, and AVA. Run the following to get it quickly:

Node.js v5.9.0
darwin 15.5.0
ava 0.15.2
npm 3.8.6

[novemberborn]

Yea, seems like the test runner should fail if a test is declared after it's started running tests (or a beforeEach hook etc).

[GramParallelo]

I'm looking to contribute my first PR. Can I have a go at this? (might need a little helping hand :-/ ) Thanks

[sindresorhus]

@GramParallelo Yes, go ahead. We're happy to handhold you through it :)

Jump into https://gitter.im/avajs/ava if you have any questions.

[pinwen]

Can I start looking into this too?

[GramParallelo]

@pinwen Yes, please. I'm a little lost and also can't put in the time at the moment. Thanks.

[asafigan]

Hey, I've never contributed to an open-source project before. It doesn't look like there has been that much progress on this issue. I was hoping I could help out with this issue?

[asafigan]

I was thinking that a property, like hasBegunRunning, could be set to true in the runner's run method and than checked when the test method was was called. Would that be a good approach?

[asafigan]

@pinwen Are you still working on this issue?

[jfmengels]

Actually very surprised we didn't think of this for the ESLint plugin earlier. Created an issue over there avajs/eslint-plugin-ava#131