Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PR #9835/32ccfc9a backport][3.10] Adjust client payload benchmarks to better represent real world cases #9836

Conversation

patchback[bot]
Copy link
Contributor

@patchback patchback bot commented Nov 12, 2024

This is a backport of PR #9835 as merged into master (32ccfc9).

We are interested in benchmarking various payload sizes at different buffering cut offs to see the memcpy and buffering impacts

2048 was too high as most small payloads are < 1024
32768 was two high as it always did two reads for this case so it wasn't giving us a case for a single large read.
1MiB was fine but 512KiB is also fine as it still gives us a benchmark for a multi-read case but there was no need to go that high as it didn't change the profile and only made the benchmark run longer

Note I'm testing on mac. I'm going to test on linux shortly since thats where the bulk of our users are. The breakpoints are going to be dictated by the kernel. If the linux buffer sizes are lower by default, I'll adjust the 2nd one down some more. Ideally we keep it as high as possible to see the memcpy effects but not so high that the message ends up in multiple reads

The 30000 size is good for linux as well as it results in a single read.

@bdraco bdraco enabled auto-merge (squash) November 12, 2024 16:35
Copy link

codspeed-hq bot commented Nov 12, 2024

CodSpeed Performance Report

Merging #9836 will not alter performance

Comparing patchback/backports/3.10/32ccfc9a19c63041c3e0f3166f9e14ba16b113e6/pr-9835 (3a277ff) with 3.10 (f3dd0f9)

Summary

✅ 17 untouched benchmarks

🆕 6 new benchmarks
⁉️ 6 (👁 6) dropped benchmarks

Benchmarks breakdown

Benchmark 3.10 patchback/backports/3.10/32ccfc9a19c63041c3e0f3166f9e14ba16b113e6/pr-9835 Change
🆕 test_one_hundred_get_requests_with_1024_chunked_payload[pyloop] N/A 44.3 ms N/A
🆕 test_one_hundred_get_requests_with_1024_content_length_payload[pyloop] N/A 44.1 ms N/A
👁 test_one_hundred_get_requests_with_1mib_chunked_payload[pyloop] 325.3 ms N/A N/A
👁 test_one_hundred_get_requests_with_1mib_content_length_payload[pyloop] 305.6 ms N/A N/A
👁 test_one_hundred_get_requests_with_2048_chunked_payload[pyloop] 44.4 ms N/A N/A
👁 test_one_hundred_get_requests_with_2048_content_length_payload[pyloop] 44.2 ms N/A N/A
🆕 test_one_hundred_get_requests_with_30000_chunked_payload[pyloop] N/A 53.5 ms N/A
🆕 test_one_hundred_get_requests_with_30000_content_length_payload[pyloop] N/A 48 ms N/A
👁 test_one_hundred_get_requests_with_32768_chunked_payload[pyloop] 54.4 ms N/A N/A
👁 test_one_hundred_get_requests_with_32768_content_length_payload[pyloop] 48.4 ms N/A N/A
🆕 test_one_hundred_get_requests_with_512kib_chunked_payload[pyloop] N/A 267.8 ms N/A
🆕 test_one_hundred_get_requests_with_512kib_content_length_payload[pyloop] N/A 175.2 ms N/A

@bdraco bdraco force-pushed the patchback/backports/3.10/32ccfc9a19c63041c3e0f3166f9e14ba16b113e6/pr-9835 branch from 17c9d7a to 3a277ff Compare November 12, 2024 17:10
Copy link

codecov bot commented Nov 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.92%. Comparing base (f3dd0f9) to head (3a277ff).
Report is 1 commits behind head on 3.10.

✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             3.10    #9836      +/-   ##
==========================================
+ Coverage   97.45%   97.92%   +0.46%     
==========================================
  Files         113      113              
  Lines       35816    35821       +5     
  Branches     4303     4314      +11     
==========================================
+ Hits        34903    35076     +173     
+ Misses        693      553     -140     
+ Partials      220      192      -28     
Flag Coverage Δ
CI-GHA 97.81% <100.00%> (+0.43%) ⬆️
OS-Linux 97.50% <100.00%> (+0.19%) ⬆️
OS-Windows 95.11% <100.00%> (?)
OS-macOS 97.21% <100.00%> (+0.07%) ⬆️
Py-3.10.11 97.06% <100.00%> (+0.28%) ⬆️
Py-3.10.15 97.00% <100.00%> (ø)
Py-3.11.10 97.08% <100.00%> (ø)
Py-3.11.9 97.13% <100.00%> (+0.29%) ⬆️
Py-3.12.7 97.57% <100.00%> (?)
Py-3.13.0 97.57% <100.00%> (+0.58%) ⬆️
Py-3.8.10 94.51% <100.00%> (?)
Py-3.8.18 96.84% <100.00%> (?)
Py-3.9.13 96.98% <100.00%> (?)
Py-3.9.20 96.96% <100.00%> (?)
Py-pypy7.3.16 96.53% <100.00%> (?)
VM-macos 97.21% <100.00%> (+0.07%) ⬆️
VM-ubuntu 97.50% <100.00%> (+0.19%) ⬆️
VM-windows 95.11% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@bdraco bdraco merged commit 4f4b90f into 3.10 Nov 12, 2024
39 of 40 checks passed
@bdraco bdraco deleted the patchback/backports/3.10/32ccfc9a19c63041c3e0f3166f9e14ba16b113e6/pr-9835 branch November 12, 2024 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant