[CI Failing]: Github Action passing even on test failures

[vaibhavsingh97]

Describe the bug

We use 'mocha' for unit tests. Currently, we have a Github action setup and whenever a failure occurs, Mocha exits with a 0 status code, so the GitHub action passes instead of failure.

Reason for failure

Seeing mocha exit with a 0 exit code after failing a test.

Related issues

🐛 mochajs/mocha#2713

Reference

Failed Build: https://github.com/nightwatchjs/nightwatch/runs/4854125616 (PR: #2995)
PR which fixes the issue: https://github.com/cypress-io/cypress/pull/5779/files

[swrdfish]

When ever there is a failure, mocha sets an eventlistener on process.on(‘exit’) event to update the process.exitCode of the global process. Which normally, works but nightwatch also attaches an eventlistner on the exit event. This eventlistner in process-listners.js#L55 file terminates the process forcefully calling process.exit() hence the listener set by mocha does not even get a chance to execute.

Possible solutions:

1. check if there are additional listeners and execute them before calling exit.
2. try to attach the nightwatch listener after the mocha one is attached.
3. do not call process.exit if not necessary let the process exit gracefully after the event queue is empty