Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

worker: fix interaction of terminate() with messaging port #37319

Closed

Conversation

addaleax
Copy link
Member

When a Worker is terminated, its own handle and the public
MessagePort are .ref()’ed, so that all relevant events,
including the 'exit' events, end up being received.

However, this is problematic if messages end up being queued
from the Worker between the beginning of the .terminate() call
and its completion, and there are no 'message' event handlers
present at that time. In that situation, currently the messages
would not end up being processed, and since the MessagePort
is still .ref()’ed, it would keep the event loop alive
indefinitely.

To fix this:

  • Make sure that all messages end up being received by
    drainMessagePort(), including cases in which the port had
    been stopped (i.e. there are no 'message' listeners) and
    cases in which we exceed the limit for messages being processed
    in one batch.
  • Unref the Worker’s internal ports manually after the Worker
    has exited.

Either of these solutions should be solving this on its own,
but I think it makes sense to make sure that both of them
happen during cleanup.

@nodejs/workers

When a Worker is terminated, its own handle and the public
`MessagePort` are `.ref()`’ed, so that all relevant events,
including the `'exit'` events, end up being received.

However, this is problematic if messages end up being queued
from the Worker between the beginning of the `.terminate()` call
and its completion, and there are no `'message'` event handlers
present at that time. In that situation, currently the messages
would not end up being processed, and since the MessagePort
is still `.ref()`’ed, it would keep the event loop alive
indefinitely.

To fix this:

- Make sure that all messages end up being received by
  `drainMessagePort()`, including cases in which the port had
  been stopped (i.e. there are no `'message'` listeners) and
  cases in which we exceed the limit for messages being processed
  in one batch.
- Unref the Worker’s internal ports manually after the Worker
  has exited.

Either of these solutions should be solving this on its own,
but I think it makes sense to make sure that both of them
happen during cleanup.
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. worker Issues and PRs related to Worker support. labels Feb 11, 2021
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@addaleax
Copy link
Member Author

Landed in 5968c54

@addaleax addaleax closed this Feb 27, 2021
addaleax added a commit that referenced this pull request Feb 27, 2021
When a Worker is terminated, its own handle and the public
`MessagePort` are `.ref()`’ed, so that all relevant events,
including the `'exit'` events, end up being received.

However, this is problematic if messages end up being queued
from the Worker between the beginning of the `.terminate()` call
and its completion, and there are no `'message'` event handlers
present at that time. In that situation, currently the messages
would not end up being processed, and since the MessagePort
is still `.ref()`’ed, it would keep the event loop alive
indefinitely.

To fix this:

- Make sure that all messages end up being received by
  `drainMessagePort()`, including cases in which the port had
  been stopped (i.e. there are no `'message'` listeners) and
  cases in which we exceed the limit for messages being processed
  in one batch.
- Unref the Worker’s internal ports manually after the Worker
  has exited.

Either of these solutions should be solving this on its own,
but I think it makes sense to make sure that both of them
happen during cleanup.

PR-URL: #37319
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
@addaleax addaleax deleted the worker-terminate-ref-public-port branch February 27, 2021 16:26
targos pushed a commit that referenced this pull request Feb 28, 2021
When a Worker is terminated, its own handle and the public
`MessagePort` are `.ref()`’ed, so that all relevant events,
including the `'exit'` events, end up being received.

However, this is problematic if messages end up being queued
from the Worker between the beginning of the `.terminate()` call
and its completion, and there are no `'message'` event handlers
present at that time. In that situation, currently the messages
would not end up being processed, and since the MessagePort
is still `.ref()`’ed, it would keep the event loop alive
indefinitely.

To fix this:

- Make sure that all messages end up being received by
  `drainMessagePort()`, including cases in which the port had
  been stopped (i.e. there are no `'message'` listeners) and
  cases in which we exceed the limit for messages being processed
  in one batch.
- Unref the Worker’s internal ports manually after the Worker
  has exited.

Either of these solutions should be solving this on its own,
but I think it makes sense to make sure that both of them
happen during cleanup.

PR-URL: #37319
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. worker Issues and PRs related to Worker support.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants