Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: http/2 benchmark #35

Merged
merged 41 commits into from
Jul 17, 2023
Merged

Conversation

mkaufmaner
Copy link

@mkaufmaner mkaufmaner commented May 30, 2023

Start the server:

PORT=3000 node --experimental-wasm-simd ./benchmarks/server-http2.js

Run the benchmark:

PORT=3000 ERROR_THRESHOLD=100 CONNECTIONS=10 PARALLEL=10 node --experimental-wasm-simd ./benchmarks/benchmark-http2.js 

Note: May have to run benchmark-http2.js multiple times, and in quick succession, in order to get a result.

Result:

│ Tests             │ Samples │          Result │ Tolerance │ Difference with slowest │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ http2 - request   │       1 │  449.75 req/sec │  ± 0.00 % │                       - │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ undici - fetch    │       1 │  509.34 req/sec │  ± 0.00 % │               + 13.25 % │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ undici - request  │       1 │ 1086.15 req/sec │  ± 0.00 % │              + 141.50 % │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ undici - stream   │       1 │ 1168.15 req/sec │  ± 0.00 % │              + 159.73 % │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ undici - pipeline │       1 │ 1200.61 req/sec │  ± 0.00 % │              + 166.95 % │

@mkaufmaner mkaufmaner mentioned this pull request May 30, 2023
20 tasks
mkaufmaner and others added 2 commits May 30, 2023 15:59
* chore: add http2 alpn test using fastify

* chore: update to test https 1 with http2

* chore: update alpn test to return server request alpn protocol and http version

* chore: add alpn with body

* fix: remove fastify from package json
@codecov
Copy link

codecov bot commented May 31, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.10 🎉

Comparison is base (e1d9717) 93.19% compared to head (7cd4203) 93.29%.

❗ Current head 7cd4203 differs from pull request most recent head 2e49522. Consider uploading reports for the commit 2e49522 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##            http2      #35      +/-   ##
==========================================
+ Coverage   93.19%   93.29%   +0.10%     
==========================================
  Files          41       41              
  Lines        3144     3147       +3     
==========================================
+ Hits         2930     2936       +6     
+ Misses        214      211       -3     
Impacted Files Coverage Δ
lib/client.js 95.23% <100.00%> (+0.34%) ⬆️
lib/core/connect.js 80.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@metcoder95
Copy link
Owner

The results indeed looks promising, though if the streams are better managed, the things can go even better as we'll be reusing them to the same origin.
Let me fix some stuff to make it work with dispatch and focus on the performance improvements later on. Great work btw 🚀

@mkaufmaner
Copy link
Author

The results indeed looks promising, though if the streams are better managed, the things can go even better as we'll be reusing them to the same origin.
Let me fix some stuff to make it work with dispatch and focus on the performance improvements later on. Great work btw 🚀

@metcoder95 Thank you! Looking forward to seeing the main PR in a mergable state. We 100% need more tests too especially around the closing of connections and sessions for the server and the client.

@metcoder95
Copy link
Owner

Hey @mkaufmaner, can you rebase with the branch and enable the benchmarks?
Feel free to open a new PR if the conflicts are complex to resolve 🙂

@mkaufmaner
Copy link
Author

mkaufmaner commented Jul 7, 2023

@metcoder95

Hey @mkaufmaner, can you rebase with the branch and enable the benchmarks? Feel free to open a new PR if the conflicts are complex to resolve 🙂

Merged down from your http2 branch, but I am not sure what you mean by "enable the benchmarks". Could you elaborate?

Secondly, I updated the benchmarks with allowH2: true, enabled the dispatch benchmark, and reran the http2 benchmark. I am still encountering the same issue before where sometimes the benchmark just doesn't print anything so you have to run it in succession to get a result.

Nonetheless,

│ Tests             │ Samples │          Result │ Tolerance │ Difference with slowest │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ http2 - request   │       1 │  459.30 req/sec │  ± 0.00 % │                       - │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ undici - fetch    │       1 │  518.96 req/sec │  ± 0.00 % │               + 12.99 % │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ undici - request  │       1 │  951.29 req/sec │  ± 0.00 % │              + 107.12 % │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ undici - pipeline │       1 │ 1116.78 req/sec │  ± 0.00 % │              + 143.15 % │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ undici - stream   │       1 │ 1169.42 req/sec │  ± 0.00 % │              + 154.61 % │
|───────────────────|─────────|─────────────────|───────────|─────────────────────────|
│ undici - dispatch │       1 │ 1230.43 req/sec │  ± 0.00 % │              + 167.89 % │

Copy link
Owner

@metcoder95 metcoder95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate?

Sure, I was meaning those that were commented out

Secondly, I updated the benchmarks with allowH2: true, enabled the dispatch benchmark, and reran the http2 benchmark. I am still encountering the same issue before where sometimes the benchmark just doesn't print anything so you have to run it in succession to get a result.

Those are good numbers, thanks! 🙇
I'm not sure what the error might be, I'm guessing that is due to how the request is done, as they differ from the way undici usually works, although not really important if we can get the numbers

Would you like to go further and do the cleanup to merge?

benchmarks/benchmark-http2.js Outdated Show resolved Hide resolved
benchmarks/server-http2.js Outdated Show resolved Hide resolved
@mkaufmaner
Copy link
Author

@metcoder95

Could you elaborate?

Sure, I was meaning those that were commented out

Secondly, I updated the benchmarks with allowH2: true, enabled the dispatch benchmark, and reran the http2 benchmark. I am still encountering the same issue before where sometimes the benchmark just doesn't print anything so you have to run it in succession to get a result.

Those are good numbers, thanks! 🙇 I'm not sure what the error might be, I'm guessing that is due to how the request is done, as they differ from the way undici usually works, although not really important if we can get the numbers

Would you like to go further and do the cleanup to merge?

Cleaned up the leftover comments so this should be ready to go.

@metcoder95
Copy link
Owner

Linter seems failing :(

@mkaufmaner
Copy link
Author

Linter seems failing :(

Fixed the lint errors.

@metcoder95 metcoder95 merged commit f6246f0 into metcoder95:http2 Jul 17, 2023
metcoder95 added a commit that referenced this pull request Jul 21, 2023
Co-authored-by: Carlos Fuentes <me@metcoder.dev>
metcoder95 added a commit that referenced this pull request Aug 20, 2023
Co-authored-by: Carlos Fuentes <me@metcoder.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants