Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

stream: improve Readable#from perf #50359

Merged
merged 1 commit into from
Oct 26, 2023
Merged

stream: improve Readable#from perf #50359

merged 1 commit into from
Oct 26, 2023

Conversation

rluvaton
Copy link
Member

@rluvaton rluvaton commented Oct 24, 2023 •
edited
Loading

Benchmarks

Benchmark URL

15:39:36                                                                             confidence improvement accuracy (*)    (**)   (***)
15:39:36 streams/readable-from.js type='array' n=10000000                                   ***     23.20 %       ±9.27% ±12.34% ±16.09%
15:39:36 streams/readable-from.js type='async-generator' n=10000000                           *      2.28 %       ±1.83%  ±2.44%  ±3.17%
15:39:36 streams/readable-from.js type='sync-generator-with-async-values' n=10000000                 0.48 %       ±1.82%  ±2.42%  ±3.15%
15:39:36 streams/readable-from.js type='sync-generator-with-sync-values' n=10000000         ***     18.26 %       ±2.01%  ±2.68%  ±3.49%

@rluvaton rluvaton added stream Issues and PRs related to the stream subsystem. performance Issues and PRs related to the performance of Node.js. labels Oct 24, 2023
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/streams

@nodejs-github-bot nodejs-github-bot added the needs-ci PRs that need a full CI run. label Oct 24, 2023
@rluvaton rluvaton added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 24, 2023
@rluvaton

This comment was marked as outdated.

Sign in to view
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 24, 2023
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55177/

@ronag
Copy link
Member

ronag commented Oct 24, 2023

Not sure I'm convinced the extra complexity is worth it. Let's see what the benchmark says.

@ronag
Copy link
Member

ronag commented Oct 24, 2023

@rluvaton when you start a benchmark ci in cases like this, add a filter e,g, from so not all benchmarks run.

@ronag
Copy link
Member

ronag commented Oct 24, 2023

https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1457/

@ronag
Copy link
Member

ronag commented Oct 24, 2023 

11:20:15 streams/readable-from.js type='array' n=10000000                                            7.03 %       ±7.37% ±9.91% ±13.11%
11:20:15 streams/readable-from.js type='async-generator' n=10000000                           *      1.64 %       ±1.41% ±1.88%  ±2.45%
11:20:15 streams/readable-from.js type='sync-generator-with-async-values' n=10000000                 1.62 %       ±1.68% ±2.24%  ±2.92%
11:20:15 streams/readable-from.js type='sync-generator-with-sync-values' n=10000000         ***     11.09 %       ±1.82% ±2.42%  ±3.16%

ronag
ronag reviewed Oct 24, 2023
View reviewed changes
lib/internal/streams/from.js Outdated Show resolved Hide resolved
ronag
ronag reviewed Oct 24, 2023
View reviewed changes
lib/internal/streams/from.js Outdated Show resolved Hide resolved
@ronag
Copy link
Member

ronag commented Oct 24, 2023

I'm -0. There is a perf improvement but the maintainability cost is quite high.

@rluvaton
Copy link
Member Author

due to the removal of the async modifier I rerun the benchmark:
https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1458/

I'm -0. There is a perf improvement but the maintainability cost is quite high.

I'm on the fence as well...

@rluvaton
Copy link
Member Author

@ronag Updated the benchmarks...

@rluvaton rluvaton added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 24, 2023
@H4ad H4ad added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Oct 24, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 24, 2023
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55183/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55193/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55194/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55195/

@benjamingr
Copy link
Member

I'm tired so if I don't make sense let me know - In the array case can't we just set the stream's buffer to ArrayPrototypeSlice(Input)? With ronag's bufferliist->array PR that could be ideal?

@rluvaton
Copy link
Member Author

I'm tired so if I don't make sense let me know - In the array case can't we just set the stream's buffer to ArrayPrototypeSlice(Input)? With ronag's bufferliist->array PR that could be ideal?

it would really improve the performance for sure but wouldn't it be dangerous as we need to make the stream reach the state where it would have data after init and everything

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55228/

@ronag
Copy link
Member

ronag commented Oct 25, 2023

Wouldn't we skip a lot of state transitions that the user expects?

Not really?

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55233/

@benjamingr
Copy link
Member

In that case this whole implementation would basically be:

Yeah that was my point in #50359 I think that would be way faster and probably semver-major

@rluvaton rluvaton added the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 25, 2023
@rluvaton
Copy link
Member Author

In that case this whole implementation would basically be:

Yeah that was my point in #50359 I think that would be way faster and probably semver-major

Because it needs a semver major, I will create a new PR with the new code after that

@rluvaton rluvaton added the commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. label Oct 25, 2023
@rluvaton rluvaton added request-ci Add this label to start a Jenkins CI on a PR. and removed commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. labels Oct 25, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 25, 2023
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55237/

@github-actions github-actions bot mentioned this pull request Oct 26, 2023
Open
20 tasks
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/55246/

@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 26, 2023
@nodejs-github-bot nodejs-github-bot merged commit 10d51e8 into nodejs:main Oct 26, 2023
50 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in 10d51e8

@rluvaton rluvaton deleted the improve-from branch October 26, 2023 09:04
This was referenced Oct 27, 2023
Open
Open
Open
Open
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
@rluvaton @alexfernandez
stream: improve from perf
d1a9379 
PR-URL: nodejs#50359
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
targos pushed a commit that referenced this pull request Nov 11, 2023
@rluvaton @targos
stream: improve from perf
e20b272 
PR-URL: #50359
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
@targos targos mentioned this pull request Nov 12, 2023
Merged
UlisesGascon pushed a commit that referenced this pull request Dec 11, 2023
@rluvaton @UlisesGascon
stream: improve from perf
08a0c6c 
PR-URL: #50359
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
@UlisesGascon UlisesGascon mentioned this pull request Dec 12, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@H4ad H4ad H4ad approved these changes

@ronag ronag ronag approved these changes

@benjamingr benjamingr benjamingr approved these changes

Assignees
No one assigned
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. needs-ci PRs that need a full CI run. performance Issues and PRs related to the performance of Node.js. stream Issues and PRs related to the stream subsystem.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@rluvaton @nodejs-github-bot @ronag @benjamingr @H4ad