test: print failed JS/parallel tests

[GeoffreyBooth]

This is a small productivity improvement. When running make test, or specifically make test-parallel or make jstest, if there any failed JavaScript (or parallel-run) tests, the error output is printed but the only summary is a line like this:

[02:24|% 100|+ 3110|-   2]: Done

With this PR, in addition to the previous output the test runner will also print a list of the failed tests, like so:

[02:32|% 100|+ 3110|-   2]: Done
Failed tests:
out/Release/node --experimental-loader ./test/fixtures/es-module-loaders/loader-this-value-inside-hook-functions.mjs /node/test/parallel/test-loaders-this-value-inside-hook-functions.mjs
out/Release/node --experimental-loader ./test/fixtures/es-module-loaders/loader-unknown-builtin-module.mjs node/test/parallel/test-loaders-unknown-builtin-module.mjs

Again, this is in addition to the current output; the present full output for each test is preserved.

My reasons for adding this:

• When there are many failing tests, it’s helpful to see a summary of all the failing tests to discern patterns of which groups of tests are failing. This cannot be done in the current output, where each failing test gets something like 10 to 100 lines of output.
• The summary of commands makes it quick to re-run particular failed tests.

cc @nodejs/testing @nodejs/test_runner @nodejs/python

[JakobJingleheimer]

Huzzah!

[aduh95]

/Cc @nodejs/testing

[aduh95]

Can you add a test failure in this PR, and run a light CI to see it impacts it negatively?

[JakobJingleheimer]

Another tiny tweak that I think would make a huge improvement would be to add an empty line and/or horizontal line between test outputs (as it currently is, they are back to back, each block is nigh indistinguishable from the next).

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/48725/

[GeoffreyBooth]

Another tiny tweak that I think would make a huge improvement would be to add an empty line and/or horizontal line between test outputs (as it currently is, they are back to back, each block is nigh indistinguishable from the next).

I just tried it and I think two blank lines are ideal, because many of the tests have single blank lines within them (like after a stack trace, say) but rarely if ever two blank lines, so visually scanning down a lot of results the two blank lines help make the breaks stand out.

[GeoffreyBooth]

Can you add a test failure in this PR, and run a light CI to see it impacts it negatively?

This is a CI run with an intentionally failing test: https://ci.nodejs.org/job/node-test-pull-request/48725/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/48728/

[GeoffreyBooth]

@cclauss I’ve added the type hint and responded to all of your suggestions. I believe this resolves your concerns; do you mind please removing your block?

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/48740/

Made requested changes

[GeoffreyBooth]

@cclauss Please let me know if you have any other notes.

[nodejs-github-bot]

Commit Queue failed

- Loading data for nodejs/node/pull/45960
✔  Done loading data for nodejs/node/pull/45960
----------------------------------- PR info ------------------------------------
Title      test: print failed JS/parallel tests (#45960)
   ⚠  Could not retrieve the email or name of the PR author's from user's GitHub profile!
Branch     GeoffreyBooth:print-failures -> nodejs:main
Labels     test, tools, python, author ready, commit-queue-squash
Commits    5
 - test: print failed JS/parallel tests
 - add type hint
 - Add newline between failed test outputs
 - Match style for variable names
 - Add import for backward compatibility
Committers 1
 - Geoffrey Booth 
PR-URL: https://github.com/nodejs/node/pull/45960
Reviewed-By: Jacob Smith 
Reviewed-By: Christian Clauss 
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/45960
Reviewed-By: Jacob Smith 
Reviewed-By: Christian Clauss 
--------------------------------------------------------------------------------
   ℹ  This PR was created on Sat, 24 Dec 2022 01:50:48 GMT
   ✔  Approvals: 2
   ✔  - Jacob Smith (@JakobJingleheimer): https://github.com/nodejs/node/pull/45960#pullrequestreview-1229549661
   ✔  - Christian Clauss (@cclauss): https://github.com/nodejs/node/pull/45960#pullrequestreview-1230212097
   ✖  Last GitHub CI failed
   ℹ  Green GitHub CI is sufficient
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu

https://github.com/nodejs/node/actions/runs/3783385909

[nodejs-github-bot]

Landed in eb4c83f