Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

node22 performance degradation #52700

Open
FishOrBear opened this issue Apr 26, 2024 · 10 comments
Open

node22 performance degradation #52700

FishOrBear opened this issue Apr 26, 2024 · 10 comments
Labels
performance Issues and PRs related to the performance of Node.js. v22.x v22.x Issues that can be reproduced on v22.x or PRs targeting the v22.x-staging branch.

Comments

@FishOrBear
Copy link

Version

22.0.0

Platform

Linux debian 6.1.0-13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.55-1 (2023-09-29) x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

const COUNT1 = 1024;
const COUNT2 = 5120;
const COUNT3 = 20480;

function encrypt(buf, value = 199)
{
    let count = Math.min(buf.length, COUNT1);
    for (let i = 0; i < count; i++)
        buf[i] += value;

    count = Math.min(buf.length, COUNT2);
    for (let i = COUNT1; i < count; i += 2)
        buf[i] += value;
}

const buf = Buffer.alloc(65535);
function aa()
{
    console.time("j");
    for (let i = 0; i < 20 * 125; i++)
        encrypt(buf, 199);
    console.timeEnd("j");
}

for (let index = 0; index < 20; index++) {
    aa()
}

When I increase the number of operations, due to the optimization of v8, the performance will be shortened to 23ms, but it is still worse than the previous 21ms performance.

How often does it reproduce? Is there a required condition?

node perf.js

What is the expected behavior? Why is that the expected behavior?

in nodejs 20.12.2

j: 64.709ms
j: 54.228ms
j: 21.822ms
j: 21.83ms
j: 21.858ms
j: 21.851ms
j: 21.821ms
j: 21.841ms
j: 21.827ms
j: 21.887ms
j: 21.823ms
j: 21.863ms
j: 21.842ms
j: 21.857ms
j: 21.845ms
j: 21.851ms
j: 21.821ms
j: 21.843ms
j: 21.816ms
j: 21.847ms

in nodejs 22.0.0

j: 44.602ms
j: 34.992ms
j: 35.053ms
j: 34.981ms
j: 35.355ms
j: 35.013ms
j: 35.036ms
j: 34.979ms
j: 35.056ms
j: 35.135ms
j: 35.09ms
j: 34.989ms
j: 35.023ms
j: 34.974ms
j: 35.028ms
j: 35.076ms
j: 35.034ms
j: 34.973ms
j: 35.045ms
j: 35.013ms

What do you see instead?

1

Additional information

1
@RedYetiDev RedYetiDev added the performance Issues and PRs related to the performance of Node.js. label Apr 26, 2024
@climba03003
Copy link
Contributor

Maybe related to #51360

@RedYetiDev RedYetiDev added the v22.x v22.x Issues that can be reproduced on v22.x or PRs targeting the v22.x-staging branch. label Apr 27, 2024
@RedYetiDev
Copy link
Member

@nodejs/performance

@lemire
Copy link
Member

lemire commented Apr 27, 2024

@FishOrBear What are you running this on? Your numbers appear to be very, very high.

I do not see a regression. What I see on this test is that Node 20 had much better performance on this particular test, but it went away soon after.

Here are my results on a macbook...

Bun

[12.89ms] j
[8.05ms] j
[7.32ms] j
[6.83ms] j
[6.37ms] j
[5.95ms] j
[5.77ms] j
[5.52ms] j
[5.34ms] j
[5.16ms] j
[5.07ms] j
[4.90ms] j
[4.87ms] j
[4.79ms] j
[4.99ms] j
[4.76ms] j
[4.69ms] j
[4.81ms] j
[4.73ms] j
[4.70ms] j

Node 22

 node issue.js
j: 6.7ms
j: 4.991ms
j: 4.909ms
j: 4.694ms
j: 4.692ms
j: 4.47ms
j: 4.477ms
j: 4.377ms
j: 4.249ms
j: 4.314ms
j: 4.551ms
j: 4.311ms
j: 4.288ms
j: 4.378ms
j: 4.418ms
j: 4.246ms
j: 4.281ms
j: 4.372ms
j: 4.384ms
j: 4.248ms

Node 21...

node issue.js
j: 7.221ms
j: 5.245ms
j: 5.057ms
j: 5.02ms
j: 4.848ms
j: 4.645ms
j: 4.684ms
j: 4.57ms
j: 4.436ms
j: 4.489ms
j: 4.638ms
j: 4.475ms
j: 4.467ms
j: 4.599ms
j: 4.578ms
j: 4.405ms
j: 4.418ms
j: 4.651ms
j: 4.483ms
j: 4.405ms

Node 20

j: 9.254ms
j: 6.872ms
j: 4.039ms
j: 3.949ms
j: 3.861ms
j: 3.748ms
j: 3.727ms
j: 3.618ms
j: 3.599ms
j: 3.925ms
j: 3.544ms
j: 3.425ms
j: 3.521ms
j: 3.56ms
j: 3.522ms
j: 3.422ms
j: 3.47ms
j: 3.589ms
j: 3.559ms
j: 3.476ms

Node 19

j: 7.636ms
j: 5.37ms
j: 5.496ms
j: 5.431ms
j: 5.442ms
j: 4.993ms
j: 5.244ms
j: 4.796ms
j: 4.836ms
j: 4.578ms
j: 4.475ms
j: 4.59ms
j: 4.716ms
j: 4.499ms
j: 4.592ms
j: 4.571ms
j: 4.588ms
j: 4.454ms
j: 4.547ms
j: 4.9ms

Node 18

  node issue.js
j: 10.643ms
j: 6.794ms
j: 6.516ms
j: 6.421ms
j: 6.46ms
j: 4.704ms
j: 4.814ms
j: 4.631ms
j: 4.505ms
j: 4.637ms
j: 4.8ms
j: 4.584ms
j: 4.518ms
j: 4.754ms
j: 4.645ms
j: 4.492ms
j: 4.605ms
j: 4.763ms
j: 4.568ms
j: 4.51ms

Node 17

j: 9.578ms
j: 6.67ms
j: 6.423ms
j: 6.232ms
j: 6.258ms
j: 6.004ms
j: 4.501ms
j: 4.758ms
j: 4.603ms
j: 4.539ms
j: 4.605ms
j: 4.793ms
j: 4.521ms
j: 4.512ms
j: 4.73ms
j: 4.598ms
j: 4.524ms
j: 4.571ms
j: 4.682ms
j: 4.56ms

Node 16

j: 7.864ms
j: 5.231ms
j: 5.038ms
j: 4.87ms
j: 4.777ms
j: 4.594ms
j: 4.698ms
j: 4.709ms
j: 4.546ms
j: 4.444ms
j: 4.498ms
j: 4.701ms
j: 4.53ms
j: 4.7ms
j: 4.635ms
j: 4.676ms
j: 4.427ms
j: 4.504ms
j: 4.68ms
j: 4.429ms

Node 10

j: 14.909ms
j: 5.316ms
j: 4.624ms
j: 4.854ms
j: 4.679ms
j: 6.734ms
j: 5.198ms
j: 5.218ms
j: 5.150ms
j: 5.224ms
j: 4.984ms
j: 5.020ms
j: 5.276ms
j: 4.999ms
j: 5.492ms
j: 5.013ms
j: 5.031ms
j: 5.218ms
j: 5.084ms
j: 4.990ms

I think we need more tests to verify the issue.

cc @RafaelGSS @anonrig

@mcollina
Copy link
Member

Unless there is something specific about buffers, this has nothing to do with Node.js, but it's rather a V8 change.

@FishOrBear
Copy link
Author

@FishOrBear What are you running this on? Your numbers appear to be very, very high.

I do not see a regression. What I see on this test is that Node 20 had much better performance on this particular test, but it went away soon after.

Here are my results on a macbook...

Bun

[12.89ms] j
[8.05ms] j
[7.32ms] j
[6.83ms] j
[6.37ms] j
[5.95ms] j
[5.77ms] j
[5.52ms] j
[5.34ms] j
[5.16ms] j
[5.07ms] j
[4.90ms] j
[4.87ms] j
[4.79ms] j
[4.99ms] j
[4.76ms] j
[4.69ms] j
[4.81ms] j
[4.73ms] j
[4.70ms] j

Node 22

 node issue.js
j: 6.7ms
j: 4.991ms
j: 4.909ms
j: 4.694ms
j: 4.692ms
j: 4.47ms
j: 4.477ms
j: 4.377ms
j: 4.249ms
j: 4.314ms
j: 4.551ms
j: 4.311ms
j: 4.288ms
j: 4.378ms
j: 4.418ms
j: 4.246ms
j: 4.281ms
j: 4.372ms
j: 4.384ms
j: 4.248ms

Node 21...

node issue.js
j: 7.221ms
j: 5.245ms
j: 5.057ms
j: 5.02ms
j: 4.848ms
j: 4.645ms
j: 4.684ms
j: 4.57ms
j: 4.436ms
j: 4.489ms
j: 4.638ms
j: 4.475ms
j: 4.467ms
j: 4.599ms
j: 4.578ms
j: 4.405ms
j: 4.418ms
j: 4.651ms
j: 4.483ms
j: 4.405ms

Node 20

j: 9.254ms
j: 6.872ms
j: 4.039ms
j: 3.949ms
j: 3.861ms
j: 3.748ms
j: 3.727ms
j: 3.618ms
j: 3.599ms
j: 3.925ms
j: 3.544ms
j: 3.425ms
j: 3.521ms
j: 3.56ms
j: 3.522ms
j: 3.422ms
j: 3.47ms
j: 3.589ms
j: 3.559ms
j: 3.476ms

Node 19

j: 7.636ms
j: 5.37ms
j: 5.496ms
j: 5.431ms
j: 5.442ms
j: 4.993ms
j: 5.244ms
j: 4.796ms
j: 4.836ms
j: 4.578ms
j: 4.475ms
j: 4.59ms
j: 4.716ms
j: 4.499ms
j: 4.592ms
j: 4.571ms
j: 4.588ms
j: 4.454ms
j: 4.547ms
j: 4.9ms

Node 18

  node issue.js
j: 10.643ms
j: 6.794ms
j: 6.516ms
j: 6.421ms
j: 6.46ms
j: 4.704ms
j: 4.814ms
j: 4.631ms
j: 4.505ms
j: 4.637ms
j: 4.8ms
j: 4.584ms
j: 4.518ms
j: 4.754ms
j: 4.645ms
j: 4.492ms
j: 4.605ms
j: 4.763ms
j: 4.568ms
j: 4.51ms

Node 17

j: 9.578ms
j: 6.67ms
j: 6.423ms
j: 6.232ms
j: 6.258ms
j: 6.004ms
j: 4.501ms
j: 4.758ms
j: 4.603ms
j: 4.539ms
j: 4.605ms
j: 4.793ms
j: 4.521ms
j: 4.512ms
j: 4.73ms
j: 4.598ms
j: 4.524ms
j: 4.571ms
j: 4.682ms
j: 4.56ms

Node 16

j: 7.864ms
j: 5.231ms
j: 5.038ms
j: 4.87ms
j: 4.777ms
j: 4.594ms
j: 4.698ms
j: 4.709ms
j: 4.546ms
j: 4.444ms
j: 4.498ms
j: 4.701ms
j: 4.53ms
j: 4.7ms
j: 4.635ms
j: 4.676ms
j: 4.427ms
j: 4.504ms
j: 4.68ms
j: 4.429ms

Node 10

j: 14.909ms
j: 5.316ms
j: 4.624ms
j: 4.854ms
j: 4.679ms
j: 6.734ms
j: 5.198ms
j: 5.218ms
j: 5.150ms
j: 5.224ms
j: 4.984ms
j: 5.020ms
j: 5.276ms
j: 4.999ms
j: 5.492ms
j: 5.013ms
j: 5.031ms
j: 5.218ms
j: 5.084ms
j: 4.990ms

I think we need more tests to verify the issue.

cc @RafaelGSS @anonrig

I'm running it on an older cpu and your macbook performs well.
cpu:Intel(R) Celeron(R) CPU J3455 @ 1.50GHz

@lemire
Copy link
Member

lemire commented Apr 28, 2024

The JavaScript engine v8 was updated to 11.3 with Node 20, it was upgraded again to 11.8 with Node 20. Node 19 had v8 10.7.

Regarding the possibility that it could be #51360... maglev was introduced in Node 22, but you see a performance difference between Node 20 and Node 21. So my numbers do not point at a regression caused by maglev.

@joyeecheung
Copy link
Member

Unless there is something specific about buffers, this has nothing to do with Node.js, but it's rather a V8 change.

Buffers are just Uint8Arrays now, so if this reproduces with Uint8Array, this is more likely an V8 issue (especially if it reproduces with different versions of d8)

@wight-airmash wight-airmash mentioned this issue May 1, 2024
Closed
@FishOrBear
Copy link
Author

stay tuned

@FishOrBear
Copy link
Author

I have updated the nodejs version to 22.9.0 and I will update the benchmark.

nodejs20.17.0

j: 140.934ms
j: 125.321ms
j: 49.63ms
j: 49.295ms
j: 49.417ms
j: 49.363ms
j: 49.328ms
j: 49.411ms
j: 49.232ms
j: 49.446ms
j: 49.753ms
j: 49.99ms
j: 49.585ms
j: 49.632ms
j: 49.354ms
j: 49.356ms
j: 49.381ms
j: 49.337ms
j: 49.347ms
j: 49.299ms
j: 49.462ms
j: 49.331ms
j: 49.408ms
j: 49.27ms
j: 49.269ms
j: 49.37ms
j: 49.35ms
j: 49.326ms
j: 49.338ms
j: 49.303ms

nodejs.22.9.0

j: 86.212ms
j: 73.845ms
j: 73.809ms
j: 79.236ms
j: 53.444ms
j: 53.285ms
j: 53.054ms
j: 53.05ms
j: 53.324ms
j: 53.3ms
j: 53.27ms
j: 53.185ms
j: 53.375ms
j: 53.145ms
j: 53.381ms
j: 53.341ms
j: 53.144ms
j: 53.147ms
j: 53.412ms
j: 53.042ms
j: 53.241ms
j: 53.215ms
j: 53.236ms
j: 53.166ms
j: 53.061ms
j: 53.086ms
j: 53.187ms
j: 53.011ms
j: 53.1ms
j: 53.225ms

It is probably because nodejs has disabled Maglev in 22.9.0.
The performance is now similar to v20, but it is still slightly slower

@joyeecheung

@joyeecheung
Copy link
Member

Can you reproduce it with just Uint8Array? If it's also reproducible in different versions of d8 then it's probably a V8 bug. I'd say among different V8 versions some performance differences are to be expected, since they are always tweaking the architecture and also they optimize towards better macro-benchmark scores, not necessarily maintaining any numbers you could see in micro benchmarks like this because the former is more meaningful to real world applications.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance Issues and PRs related to the performance of Node.js. v22.x v22.x Issues that can be reproduced on v22.x or PRs targeting the v22.x-staging branch.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@mcollina @lemire @joyeecheung @FishOrBear @climba03003 @RedYetiDev