process: allow monitoring uncaughtException #31257

Flarna · 2020-01-08T09:23:53Z

Installing an uncaughtException listener has a side effect that process won't exit. This is quite bad for monitoring/logging tools which tend to be interested in errors but don't want to cause side effects like swallow an exception or change the output on console.

There are some workarounds in the wild like monkey patching emit or rethrow in the exception if monitoring tool detects that it is the only listener but this is error prone and risky.

This PR allows to install a listener to monitor uncaughtException without the side effect to consider the exception has handled.

Refs: #30932

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Installing an uncaughtException listener has a side effect that process is not aborted. This is quite bad for monitoring/logging tools which tend to be interested in errors but don't want to cause side effects like swallow an exception or change the output on console. There are some workarounds in the wild like monkey patching emit or rethrow in the exception if monitoring tool detects that it is the only listener but this is error prone and risky. This PR allows to install a listener to monitor uncaughtException without the side effect to consider the exception has handled. To avoid conflicts with other events it exports a symbol on process which owns this special meaning.

test/parallel/test-process-uncaught-exception-monitor.js

lib/internal/process/execution.js

Flarna · 2020-01-08T11:38:41Z

I tried also to add a test which verifies that process still exits if no uncaughtException listener is installed but failed. Is there anywhere a sample/test which verifies something like this?

BridgeAR · 2020-01-08T11:47:40Z

If you only want to check the stack, you could write a message test similar to e.g., test/message/events_unhandled_error_common_trace.js. Otherwise it's possible to maybe use the on('exit') hook or a child process.

test/fixtures/uncaught-exceptions/uncaught-monitor.js

lib/internal/process/execution.js

addaleax

The code LGTM, although it feels like this issue isn’t really specific to uncaught exceptions – maybe there should be a general way of attaching event listeners to emitters that doesn’t affect the return value of .emit()?

Also, it feels like monkey-patching process.emit to intercept uncaught exceptions would actually be just fine and should be considered supported by Node.js.

doc/api/process.md

Flarna · 2020-01-09T19:30:09Z

The code LGTM, although it feels like this issue isn’t really specific to uncaught exceptions – maybe there should be a general way of attaching event listeners to emitters that doesn’t affect the return value of .emit()?

That sounds interessting. The guaranteed sequence of first calling monitoring listeners could be also implemented.
But besides the return value of emit() also APIs like emitter.listeners() and maybe even emitter.removeAllListeners() needs to handle such listeners in a special way.
In case of uncaughtException there is process.setUncaughtExceptionCaptureCallback() which avoids that uncaughtException is emitted at all.

Also, it feels like monkey-patching process.emit to intercept uncaught exceptions would actually be just fine and should be considered supported by Node.js.

In general I would avoid monkey patching if easily possible. There might be serveral users patching the same api and some try to unpatch which ends up in a mess if done in the wrong order.
Patching process.emit means also that inheritance tree is changed and users may call the wrong original function. I know these are quite corner cases but I got used to corner cases...
Another option is monkey patching process._fatalException but I assume this will not increase the happiness looking into the comments around this function.

Another idea would be to have some more generic fatal error event which signals also other types of errors like OOM to allow sync reporting. But I fear it may be not possible to call JS functions in a save way at this time.

bnoordhuis

Can you add a regression test that ensures an uncaughtExceptionMonitor listener that itself throws an exception terminates the process? (I.e., doesn't result in an infinite loop or anything like that.)

Flarna · 2020-01-10T07:13:53Z

@bnoordhuis sure.

But thinking once more about this I'm not sure if we should add a try/catch around this specific emit(uncaughtExceptionMonitor) and swallow any exceptions from these listeners.
In general throwing in a listener is a bad idea and most of the time it ends up in an uncaught exception. This is different here as there was already an uncaught exception. Throwing in a monitoring listener will result in ending the process no calling regular uncaughtException handlers nor an UncaughtExceptionCaptureCallback.

Update: Added a test. I have not added a try catch as an exception within the fatal error handling should result in exit code 7 and this is the case now.

exit code 7

bnoordhuis

LGTM with a suggestion.

test/fixtures/uncaught-exceptions/uncaught-monitor2.js

nodejs-github-bot · 2020-01-10T17:23:58Z

CI: https://ci.nodejs.org/job/node-test-pull-request/28352/

doc/api/process.md

Co-Authored-By: Colin Ihrig <cjihrig@gmail.com>

nodejs-github-bot · 2020-01-10T23:07:43Z

CI: https://ci.nodejs.org/job/node-test-pull-request/28355/

Trott · 2020-01-10T23:08:50Z

The initial commit message still mentions the symbol, but that has been removed, right? Whoever lands this should update the commit message (unless @Flarna does it before that time and force-pushes).

Trott · 2020-01-11T02:35:34Z

Landed in f4797ff

Installing an uncaughtException listener has a side effect that process is not aborted. This is quite bad for monitoring/logging tools which tend to be interested in errors but don't want to cause side effects like swallow an exception or change the output on console. There are some workarounds in the wild like monkey patching emit or rethrow in the exception if monitoring tool detects that it is the only listener but this is error prone and risky. This PR allows to install a listener to monitor uncaughtException without the side effect to consider the exception has handled. PR-URL: #31257 Refs: #30932 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

Notable changes: * **deps**: * upgrade npm to 6.13.6 (Ruy Adorno) [#31304](#31304) * **doc**: * add GeoffreyBooth to collaborators (Geoffrey Booth) [#31306](#31306) * **process**: * allow monitoring uncaughtException (Gerhard Stoebich) [#31257](#31257)

Notable changes: * **deps**: * upgrade npm to 6.13.6 (Ruy Adorno) [nodejs#31304](nodejs#31304) * **doc**: * add GeoffreyBooth to collaborators (Geoffrey Booth) [nodejs#31306](nodejs#31306) * **process**: * allow monitoring uncaughtException (Gerhard Stoebich) [nodejs#31257](nodejs#31257) PR-URL: nodejs#31382

Notable changes: * deps: * upgrade to libuv 1.34.1 (cjihrig) #31332 * upgrade npm to 6.13.6 (Ruy Adorno) #31304 * doc: * add GeoffreyBooth to collaborators (Geoffrey Booth) #31306 * module * add API for interacting with source maps (bcoe) #31132 * loader getSource, getFormat, transform hooks (Geoffrey Booth) #30986 * logical conditional exports ordering (Guy Bedford) #31008 * unflag conditional exports (Guy Bedford) #31001 * process: * allow monitoring uncaughtException (Gerhard Stoebich) #31257 PR-URL: #31382

mmarchini · 2020-01-20T18:24:54Z

@Flarna thank you for working on this! It seems like the right direction to monitor uncaught exceptions on third-party monitoring tools.

Just a minor nit on the PR description: Installing an uncaughtException listener has a side effect that process is not aborted, when I first read it, I thought this PR was referring to --abort-on-uncaught-exception. Do you mind changing the description to Installing an uncaughtException listener has a side effect that process won't exit to avoid confusion when others read it?

Flarna · 2020-01-20T19:34:41Z

@mmarchini Done, I removed also the final sentence that a symbol has been added as this was removed during the review phase.

Notable changes: * deps: * upgrade to libuv 1.34.1 (cjihrig) #31332 * upgrade npm to 6.13.6 (Ruy Adorno) #31304 * module * add API for interacting with source maps (bcoe) #31132 * loader getSource, getFormat, transform hooks (Geoffrey Booth) #30986 * logical conditional exports ordering (Guy Bedford) #31008 * unflag conditional exports (Guy Bedford) #31001 * process: * allow monitoring uncaughtException (Gerhard Stoebich) #31257 * Added new collaborators: * [GeoffreyBooth](https://github.com/GeoffreyBooth) - Geoffrey Booth. #31306 PR-URL: #31382

Installing an uncaughtException listener has a side effect that process is not aborted. This is quite bad for monitoring/logging tools which tend to be interested in errors but don't want to cause side effects like swallow an exception or change the output on console. There are some workarounds in the wild like monkey patching emit or rethrow in the exception if monitoring tool detects that it is the only listener but this is error prone and risky. This PR allows to install a listener to monitor uncaughtException without the side effect to consider the exception has handled. PR-URL: nodejs#31257 Refs: nodejs#30932 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

Installing an uncaughtException listener has a side effect that process is not aborted. This is quite bad for monitoring/logging tools which tend to be interested in errors but don't want to cause side effects like swallow an exception or change the output on console. There are some workarounds in the wild like monkey patching emit or rethrow in the exception if monitoring tool detects that it is the only listener but this is error prone and risky. This PR allows to install a listener to monitor uncaughtException without the side effect to consider the exception has handled. PR-URL: #31257 Refs: #30932 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com>

nodejs-github-bot added the process Issues and PRs related to the process subsystem. label Jan 8, 2020

Flarna force-pushed the uncaughtExceptionMonitor branch from 18704d4 to 85d940c Compare January 8, 2020 09:25

Flarna force-pushed the uncaughtExceptionMonitor branch from 85d940c to 9039128 Compare January 8, 2020 09:40

BridgeAR reviewed Jan 8, 2020

View reviewed changes

test/parallel/test-process-uncaught-exception-monitor.js Outdated Show resolved Hide resolved

lib/internal/process/execution.js Outdated Show resolved Hide resolved

fixup! crlf => lf

df03daa

use arrow functions

b58dd7c

BridgeAR approved these changes Jan 8, 2020

View reviewed changes

add test to verify default exit behavior is unchanged

6f53695

BridgeAR reviewed Jan 8, 2020

View reviewed changes

test/fixtures/uncaught-exceptions/uncaught-monitor.js Outdated Show resolved Hide resolved

test/fixtures/uncaught-exceptions/uncaught-monitor.js Outdated Show resolved Hide resolved

fixup! improve fixture

acee3de

joyeecheung reviewed Jan 8, 2020

View reviewed changes

lib/internal/process/execution.js Outdated Show resolved Hide resolved

patilharshal16 approved these changes Jan 9, 2020

View reviewed changes

addaleax reviewed Jan 9, 2020

View reviewed changes

doc/api/process.md Outdated Show resolved Hide resolved

bnoordhuis reviewed Jan 9, 2020

View reviewed changes

Flarna added 2 commits January 10, 2020 09:09

replace symbol uncaughtExceptionMonitor by a string

5daab4d

add test to verify that throwing in monitoring listener results in

c3b5f2d

exit code 7

bnoordhuis approved these changes Jan 10, 2020

View reviewed changes

test/fixtures/uncaught-exceptions/uncaught-monitor2.js Show resolved Hide resolved

addaleax approved these changes Jan 10, 2020

View reviewed changes

BridgeAR added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jan 10, 2020

cjihrig approved these changes Jan 10, 2020

View reviewed changes

doc/api/process.md Outdated Show resolved Hide resolved

doc/api/process.md Outdated Show resolved Hide resolved

doc/api/process.md Outdated Show resolved Hide resolved

Flarna and others added 3 commits January 10, 2020 19:38

Update doc/api/process.md

9db2da6

Co-Authored-By: Colin Ihrig <cjihrig@gmail.com>

Update doc/api/process.md

4206027

Co-Authored-By: Colin Ihrig <cjihrig@gmail.com>

Update doc/api/process.md

efedbb5

Co-Authored-By: Colin Ihrig <cjihrig@gmail.com>

Trott approved these changes Jan 10, 2020

View reviewed changes

Trott added the semver-minor PRs that contain new features and should be released in the next minor version. label Jan 11, 2020

Trott closed this Jan 11, 2020

Trott added the notable-change PRs with changes that should be highlighted in changelogs. label Jan 11, 2020

Flarna deleted the uncaughtExceptionMonitor branch January 11, 2020 07:17

codebytere mentioned this pull request Jan 16, 2020

v13.7.0 proposal #31382

Merged

targos removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Apr 25, 2020

targos mentioned this pull request May 2, 2020

v12.17.0 release proposal #33197

Merged

Linkgoron mentioned this pull request Apr 19, 2021

if uncaughtExceptionMonitor handler throws, the process dies without chance for recovery by uncaughtException or uncaughtRejection #38280

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

process: allow monitoring uncaughtException #31257

process: allow monitoring uncaughtException #31257

Flarna commented Jan 8, 2020 •

edited

Loading

Flarna commented Jan 8, 2020

BridgeAR commented Jan 8, 2020

addaleax left a comment

Flarna commented Jan 9, 2020

bnoordhuis left a comment

Flarna commented Jan 10, 2020 •

edited

Loading

bnoordhuis left a comment

nodejs-github-bot commented Jan 10, 2020

nodejs-github-bot commented Jan 10, 2020

Trott commented Jan 10, 2020 •

edited

Loading

Trott commented Jan 11, 2020

mmarchini commented Jan 20, 2020

Flarna commented Jan 20, 2020

process: allow monitoring uncaughtException #31257

process: allow monitoring uncaughtException #31257

Conversation

Flarna commented Jan 8, 2020 • edited Loading

Checklist

Flarna commented Jan 8, 2020

BridgeAR commented Jan 8, 2020

addaleax left a comment

Choose a reason for hiding this comment

Flarna commented Jan 9, 2020

bnoordhuis left a comment

Choose a reason for hiding this comment

Flarna commented Jan 10, 2020 • edited Loading

bnoordhuis left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented Jan 10, 2020

nodejs-github-bot commented Jan 10, 2020

Trott commented Jan 10, 2020 • edited Loading

Trott commented Jan 11, 2020

mmarchini commented Jan 20, 2020

Flarna commented Jan 20, 2020

Flarna commented Jan 8, 2020 •

edited

Loading

Flarna commented Jan 10, 2020 •

edited

Loading

Trott commented Jan 10, 2020 •

edited

Loading