Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib: convert transfer sequence to array in js #55317

Merged
merged 1 commit into from
Oct 13, 2024

Conversation

jazelly
Copy link
Member

@jazelly jazelly commented Oct 8, 2024

This PR utilizes the webidl sequence converter to convert transfer in JS layer. Although this is kind of like undoing some efforts made to move the entire strucutredClone to C++, it fixes the inconsistency of structuredClone with other runtimes with webidl compliance.

Additionally, this reduces the C++ to JS overhead, as the iterator function is no longer invoked at C++ layer, when transfer is an iterator object.

Fixes: #55280
Refs: #50330

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/startup
  • @nodejs/web-standards

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Oct 8, 2024
@jazelly jazelly force-pushed the js-structured-clone branch 3 times, most recently from 5f3c150 to fa84f31 Compare October 8, 2024 09:25
@aduh95
Copy link
Contributor

aduh95 commented Oct 8, 2024

Tests are failing:

TypeError: webidl.converters.sequence<transfer> is not a function
    at structuredClone (node:internal/structured_clone:35:62)

Copy link

codecov bot commented Oct 8, 2024

Codecov Report

Attention: Patch coverage is 86.36364% with 9 lines in your changes missing coverage. Please review.

Project coverage is 88.40%. Comparing base (8dbca2d) to head (c9f5904).
Report is 40 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/webidl.js 50.00% 6 Missing ⚠️
src/node_messaging.cc 70.00% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #55317      +/-   ##
==========================================
- Coverage   88.41%   88.40%   -0.02%     
==========================================
  Files         652      652              
  Lines      186759   186827      +68     
  Branches    36097    36048      -49     
==========================================
+ Hits       165130   165165      +35     
- Misses      14898    14911      +13     
- Partials     6731     6751      +20     
Files with missing lines Coverage Δ
...internal/bootstrap/web/exposed-window-or-worker.js 93.70% <100.00%> (+0.15%) ⬆️
lib/internal/perf/usertiming.js 94.14% <100.00%> (ø)
lib/internal/webstreams/readablestream.js 98.32% <100.00%> (-0.01%) ⬇️
lib/internal/worker/js_transferable.js 98.54% <100.00%> (+0.54%) ⬆️
src/node_messaging.cc 83.56% <70.00%> (-0.50%) ⬇️
lib/internal/webidl.js 95.04% <50.00%> (-1.74%) ⬇️

... and 58 files with indirect coverage changes

@jazelly
Copy link
Member Author

jazelly commented Oct 8, 2024

Should be fixed now. Thanks for the heads up.

@RedYetiDev RedYetiDev added web-standards Issues and PRs related to Web APIs and removed lib / src Issues and PRs related to general changes in the lib or src directory. labels Oct 10, 2024
@jazelly
Copy link
Member Author

jazelly commented Oct 10, 2024

cc @joyeecheung This is more like a RFC right now. I'm keen to hear your thoughts on this approach since you initiated the work moving structuredClone to C++.

@joyeecheung
Copy link
Member

The work wasn't really started to move it entirely to C++, it was about getting rid of the unnecessary MessageChannel + MessagePort abstraction and doing the Message::Serialize/Message::Deserialize directly, which you couldn't do with what was exposed to JS. The most essential part was this

node/src/node_messaging.cc

Lines 1609 to 1613 in b5fb2ff

if (msg->Serialize(env, context, value, transfer_list, Local<Object>())
.IsNothing() ||
!msg->Deserialize(env, context, nullptr).ToLocal(&result)) {
return;
}

And this PR is not touching it. The argument conversion should've been done in JS in the first place IMO, just not in the middle of binding, but should be done before the binding is invoked. In fact I actually left a TODO comment in there:

node/src/node_messaging.cc

Lines 1600 to 1601 in b5fb2ff

// TODO(joyeecheung): implement this in JS land to avoid the C++ -> JS
// cost to convert a sequence into an array.

Which I think is exactly what this PR is doing?

lib/internal/structured_clone.js Outdated Show resolved Hide resolved
lib/internal/structured_clone.js Outdated Show resolved Hide resolved
@jazelly
Copy link
Member Author

jazelly commented Oct 10, 2024

Yes, exactly what this PR is trying to achieve : )

Copy link
Member

@joyeecheung joyeecheung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a nit

lib/internal/worker/js_transferable.js Outdated Show resolved Hide resolved
This commit lets `tranfer` passed to `structuredClone` get validated at
JS layer by doing webidl conversion. This avoids the C++ to JS function
call overhead in the native implementaiton of `structuredClone`
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jakecastelli jakecastelli added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. labels Oct 13, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 13, 2024
@nodejs-github-bot
Copy link
Collaborator

Copy link
Member

@KhafraDev KhafraDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not going to block this PR, especially with a green CI, but having an object converter is not correct here. As noted in the PR, a dictionary converter is correct - as an idl dictionary can be represented in js, whilst an idl object cannot.

There are some small issues like hitting object.transfer's getter twice, which is observable via a proxy, and checking the type of options multiple times, but nothing major.

@KhafraDev KhafraDev added the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 13, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 13, 2024
@nodejs-github-bot nodejs-github-bot merged commit 50b4ada into nodejs:main Oct 13, 2024
63 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in 50b4ada

marco-ippolito pushed a commit to marco-ippolito/node that referenced this pull request Oct 14, 2024
This commit lets `tranfer` passed to `structuredClone` get validated at
JS layer by doing webidl conversion. This avoids the C++ to JS function
call overhead in the native implementaiton of `structuredClone`

PR-URL: nodejs#55317
Fixes: nodejs#55280
Refs: nodejs#50330
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
aduh95 pushed a commit that referenced this pull request Oct 19, 2024
This commit lets `tranfer` passed to `structuredClone` get validated at
JS layer by doing webidl conversion. This avoids the C++ to JS function
call overhead in the native implementaiton of `structuredClone`

PR-URL: #55317
Fixes: #55280
Refs: #50330
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
@aduh95 aduh95 mentioned this pull request Oct 24, 2024
louwers pushed a commit to louwers/node that referenced this pull request Nov 2, 2024
This commit lets `tranfer` passed to `structuredClone` get validated at
JS layer by doing webidl conversion. This avoids the C++ to JS function
call overhead in the native implementaiton of `structuredClone`

PR-URL: nodejs#55317
Fixes: nodejs#55280
Refs: nodejs#50330
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
tpoisseau pushed a commit to tpoisseau/node that referenced this pull request Nov 21, 2024
This commit lets `tranfer` passed to `structuredClone` get validated at
JS layer by doing webidl conversion. This avoids the C++ to JS function
call overhead in the native implementaiton of `structuredClone`

PR-URL: nodejs#55317
Fixes: nodejs#55280
Refs: nodejs#50330
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
@jazelly jazelly deleted the js-structured-clone branch November 25, 2024 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. needs-ci PRs that need a full CI run. web-standards Issues and PRs related to Web APIs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Transfer should not be null or undefined in structuredClone
9 participants