bench(transfer): run with and without pacing #2006
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mxinden
requested review from
KershawChang,
martinthomson and
larseggert
as code owners
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2006 +/- ##
==========================================
- Coverage 95.00% 94.99% -0.01%
==========================================
Files 112 112
Lines 36371 36371
==========================================
- Hits 34554 34551 -3
- Misses 1817 1820 +3 ☔ View full report in Codecov by Sentry. |
Failed Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
All resultsSucceeded Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
Unsupported Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
|
Benchmark resultsPerformance differences relative to 608c0a7. coalesce_acked_from_zero 1+1 entries: No change in performance detected. time: [193.69 ns 194.13 ns 194.62 ns]
change: [-0.1709% +0.3416% +0.7615%] (p = 0.16 > 0.05)
No change
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severecoalesce_acked_from_zero 3+1 entries: No change in performance detected. time: [237.24 ns 237.86 ns 238.53 ns]
change: [-0.9237% -0.2223% +0.3655%] (p = 0.53 > 0.05)
No change
Found 16 outliers among 100 measurements (16.00%)
1 (1.00%) low mild
5 (5.00%) high mild
10 (10.00%) high severecoalesce_acked_from_zero 10+1 entries: No change in performance detected. time: [237.56 ns 238.46 ns 239.48 ns]
change: [-0.2078% +0.2500% +0.6814%] (p = 0.28 > 0.05)
No change
Found 7 outliers among 100 measurements (7.00%)
7 (7.00%) high severecoalesce_acked_from_zero 1000+1 entries: No change in performance detected. time: [217.01 ns 217.22 ns 217.46 ns]
change: [-0.3565% +0.3775% +1.1441%] (p = 0.34 > 0.05)
No change
Found 11 outliers among 100 measurements (11.00%)
4 (4.00%) high mild
7 (7.00%) high severeRxStreamOrderer::inbound_frame(): Change within noise threshold. time: [119.59 ms 119.73 ms 119.87 ms]
change: [-0.8027% -0.5599% -0.3648%] (p = 0.00 < 0.05)
Change withintransfer/pacing-false/varying-seeds: time: [40.606 ms 42.665 ms 44.776 ms]
thrpt: [89.333 MiB/s 93.753 MiB/s 98.508 MiB/s]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mildtransfer/pacing-true/varying-seeds: time: [53.904 ms 57.043 ms 60.251 ms]
thrpt: [66.389 MiB/s 70.123 MiB/s 74.206 MiB/s]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mildtransfer/pacing-false/same-seed: time: [47.705 ms 49.184 ms 50.619 ms]
thrpt: [79.022 MiB/s 81.328 MiB/s 83.848 MiB/s]
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) low mildtransfer/pacing-true/same-seed: time: [65.243 ms 72.132 ms 78.858 ms]
thrpt: [50.724 MiB/s 55.454 MiB/s 61.309 MiB/s]1-conn/1-100mb-resp (aka. Download)/client: No change in performance detected. time: [289.33 ms 296.78 ms 304.06 ms]
thrpt: [328.88 MiB/s 336.95 MiB/s 345.62 MiB/s]
change:
time: [-2.8352% +0.4152% +3.7902%] (p = 0.82 > 0.05)
thrpt: [-3.6518% -0.4135% +2.9179%]
No change1-conn/10_000-parallel-1b-resp (aka. RPS)/client: Change within noise threshold. time: [405.05 ms 408.35 ms 411.72 ms]
thrpt: [24.289 Kelem/s 24.489 Kelem/s 24.688 Kelem/s]
change:
time: [-2.9356% -1.8216% -0.7290%] (p = 0.00 < 0.05)
thrpt: [+0.7344% +1.8554% +3.0243%]
Change within1-conn/1-1b-resp (aka. HPS)/client: No change in performance detected. time: [68.225 ms 68.596 ms 69.017 ms]
thrpt: [14.489 elem/s 14.578 elem/s 14.657 elem/s]
change:
time: [-0.6471% +0.1703% +1.0195%] (p = 0.69 > 0.05)
thrpt: [-1.0092% -0.1700% +0.6514%]
No change
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low severe
1 (1.00%) low mild
8 (8.00%) high severeClient/server transfer resultsTransfer of 33554432 bytes over loopback.
|
KershawChang
approved these changes
Jul 24, 2024
larseggert
requested changes
Jul 24, 2024
larseggert
approved these changes
Jul 24, 2024
mxinden
commented
Jul 26, 2024
| @@ -200,7 +200,7 @@ jobs: | |||
| fi | |||
| sed -E -e 's/^ //gi' \ | |||
| -e 's/((change|time|thrpt):[^%]*% )([^%]*%)(.*)/\1<b>\3<\/b>\4/gi' results.txt |\ | |||
| perl -p -0777 -e 's/(.*?)\n(.*?)((No change|Change within|Performance has).*?)(\nFound .*?)?\n\n/<details><summary>$1: $3<\/summary><pre>\n$2$5<\/pre><\/details>\n/gs' |\ | |||
| perl -p -0777 -e 's/(.*?)\n(.*?)(((No change|Change within|Performance has).*?)(\nFound .*?)?)?\n\n/<details><summary>$1: $4<\/summary><pre>\n$2$5$6<\/pre><\/details>\n/gs' |\ | |||
There was a problem hiding this comment.
Thank you @larseggert!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Might help debug #1998.
Does reproduce regression described in #1998.