test_runner: fix delete test file cause dependency file change not rerun the tests

[jakecastelli]

There is an edge case left from #53114

Reproduce step:

// create dependency index.js
export const a = 1;
export const b = 2;
export const c = 3;

// create test-a.mjs
import { a } from "./index.mjs";

import { test } from "node:test";
import assert from "node:assert";

test("test One", () => {
  assert.equal(a, 1);
});

// create test-b.mjs
import { a, b } from "./index.mjs";

import { test } from "node:test";
import assert from "node:assert";

test("test Two", () => {
  assert.equal(a, 1);
  assert.equal(b, 2);
});

// create test-c.mjs
import { a, b, c } from "./index.mjs";

import { test } from "node:test";
import assert from "node:assert";

test("test Three", () => {
  assert.equal(a, 1);
  assert.equal(b, 2);
  assert.equal(c, 3);
});

node --watch --test

# now delete test-a.mjs
# then make any change to index.mjs e.g. change const a = 1 to const a = 2

You would observe that test-b.mjs and test-c.mjs are not being rerun. The expected behaviour should be test-b.mjs, test-c.mjs being rerun and failed (since assert.strictEqual(a, 1) is not correct anymore).

Change explain:

When a watched test file is being deleted, the test runner will rerun the test(s) and because the test file is no longer there but the dependencyOwners wouldn't be able to know it, so it failed in silence. Since a test file is being deleted, we don't need to rerun anything so we can just safely return.

notes:

The test is quite identical to test/parallel/test-runner-watch-mode.mjs but with a more complicated setup, I am thinking to spend some time to see how I can refactor the test so we can write more complicated test cases in the future, I am committed and would like to do a separate PR for it.

Ref: #53114

[nodejs-github-bot]

Review requested:

• @nodejs/test_runner

[atlowChemi]

Is this used only for util.createDeferredPromise?

[jakecastelli]

yes, it only used for util.createDeferredPromise

[jakecastelli]

maybe we can get CI started again? 👀

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/60140/

[benjamingr]

This could just be .toArray()on child.stdout which would also save the util.createDeferredPromise and would make it easier to listen to stdout

[benjamingr]

(just make sure to not await it before you trigger the watch)

[jakecastelli]

Can I get a little bit of help here? I tried to use child.stdout.toArray() - but I cannot seem to be able to control the test runs, I cannot manage to get 2 runs (get result for initial run 3/3 pass, delete a test file modify another test to get the result of second run 2/2 pass) successfully, I can only get result for 1 run

I must be doing something incorrectly.

Would you mind explaining not await to what?

(just make sure to not await it before you trigger the watch)

[jakecastelli]

I think I may have understood what you mean - use child.stdout.toArray() to get a promise then delete a test file and update a file in order to trigger the rerun (watch), after that resolve the promise to get all the std out results, am I understanding correctly? 👀

[jakecastelli]

Haven't got back from @benjamingr yet on #53533 (comment), shall we land this bug fix first and I can try to create a follow up PR to improve it.

[benjamingr]

hall we land this bug fix first and I can try to create a follow up PR to improve it.

SGTM

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/60929/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/60934/

[jakecastelli]

I figured out the watch capability is limited in AIX which caused the test in CI on that platform to fail.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.

Project coverage is 87.10%. Comparing base (e0634f5) to head (b1e06fc).
Report is 58 commits behind head on main.

Files	Patch %	Lines
lib/internal/test_runner/runner.js	0.00%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #53533      +/-   ##
==========================================
- Coverage   87.10%   87.10%   -0.01%     
==========================================
  Files         647      647              
  Lines      181739   181759      +20     
  Branches    34887    34889       +2     
==========================================
+ Hits       158310   158323      +13     
- Misses      16738    16740       +2     
- Partials     6691     6696       +5     

Files	Coverage Δ
lib/internal/test_runner/runner.js	84.86% <0.00%> (-0.69%)	⬇️

... and 27 files with indirect coverage changes

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/60973/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/60975/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/60976/

[jakecastelli]

Green CI now 🥳 would appreciate someone can take a look again and give a green tick so this PR is able to land (my latest change was skipping the test on AIX platform as the watch capability is restricted)

[jakecastelli]

@mcollina do you mind taking a quick look 🙏 as you recently did some work in this area as well

[mcollina]

lgtm

[nodejs-github-bot]

Landed in 1212eca