Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Bug: Unhandled rejections cause parallel tests to succeed when they should fail #5184

Open
3 of 4 tasks
cinder-fish opened this issue Aug 2, 2024 · 2 comments
Open
3 of 4 tasks
Labels
status: accepting prs Mocha can use your help with this one! type: bug a defect, confirmed by a maintainer

Comments

@cinder-fish
Copy link

cinder-fish commented Aug 2, 2024

Bug Report Checklist

  • I have read and agree to Mocha's Code of Conduct and Contributing Guidelines
  • I have searched for related issues and issues with the faq label, but none matched my issue.
  • I have 'smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, my usage of Mocha, or Mocha itself.
  • I want to provide a PR to resolve this

Expected

mocha to run the rests and exit with a non-0 exit code

Actual

mocha exits with 0

and prints the following output

causes an unhandled rejection
    ✔ throws after 3000ms

file:///Users/Alex.Brown/projects/scratch/mocha-parallel-bug/test/unhandled.rejection.test.js:4
      throw new Error("rejected after 3000ms");
      ^

Error: rejected after 3000ms
    at Timeout._onTimeout (file:///Users/Alex.Brown/projects/scratch/mocha-parallel-bug/test/unhandled.rejection.test.js:4:13)
    at listOnTimeout (node:internal/timers:573:17)
    at process.processTimers (node:internal/timers:514:7)

Node.js v22.2.0

  long running test
    1) fails after 5000ms

which includes the failing test but not the final summary

Minimal, Reproducible Example

https://github.com/cinder-fish/mocha-parallel-bug

Essentially one test file sets up a timeout which will throw an unhandled error

And another has a failing test that delays long enough that this fires

When run with parallel: true this passes despite the failing test

When run with parallel: false this fails as expected

Versions

$ mocha --version
zsh: command not found: mocha

$ node_modules/.bin/mocha --version
10.7.0

$ node --version
v22.2.0

Additional Info

No response

@cinder-fish cinder-fish added status: in triage a maintainer should (re-)triage (review) this issue type: bug a defect, confirmed by a maintainer labels Aug 2, 2024
@JoshuaKGoldberg
Copy link
Member

JoshuaKGoldberg commented Aug 3, 2024

See also:

...however, per comments in #2797, it looks like Mocha isn't respecting Node unhandled-rejections anymore?

$ npx mocha -n unhandled-rejections=strict; echo "mocha status: $?"                                 


  causes an unhandled rejection
    ✔ throws after 3000ms


  1 passing (84ms)

mocha status: 0

Perhaps this is a regression? Needs investigation.

@JoshuaKGoldberg
Copy link
Member

Ok I played around with this more: yes, this is a bug.

  • When running in the default mode (without parallel), mocha status: 7 is logged: indicating Mocha failing
  • When running with parallel, mocha status: 0 is logged: indicating no failing exit code

The behavior between the default mode & parallel should be the same. An unhandled rejection should be logged & reported.

@JoshuaKGoldberg JoshuaKGoldberg added status: accepting prs Mocha can use your help with this one! and removed status: in triage a maintainer should (re-)triage (review) this issue labels Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: accepting prs Mocha can use your help with this one! type: bug a defect, confirmed by a maintainer
Projects
None yet
Development

No branches or pull requests

2 participants