Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(transport): check local and remote GREASE_QUIC_BIT TP #2101

Merged
merged 2 commits into from
Sep 13, 2024

Conversation

mxinden
Copy link
Collaborator

@mxinden mxinden commented Sep 12, 2024

Follow-up to #2100.

Test succeeds when run on top of #2100.

@larseggert feel free to cherry-pick in case you find this useful.

Copy link

github-actions bot commented Sep 12, 2024

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

@larseggert
Copy link
Collaborator

Thanks, we can land this after #2100.

Copy link

github-actions bot commented Sep 12, 2024

Firefox builds for this PR

The following builds are available for testing. Crossed-out builds did not succeed.

Copy link

codecov bot commented Sep 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.38%. Comparing base (2ca29c1) to head (2a0492e).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2101      +/-   ##
==========================================
+ Coverage   95.35%   95.38%   +0.02%     
==========================================
  Files         112      112              
  Lines       36325    36325              
==========================================
+ Hits        34638    34647       +9     
+ Misses       1687     1678       -9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@larseggert larseggert added this pull request to the merge queue Sep 13, 2024
Merged via the queue into mozilla:main with commit d4978de Sep 13, 2024
56 checks passed
Copy link

Benchmark results

Performance differences relative to 0b106c2.

coalesce_acked_from_zero 1+1 entries: No change in performance detected.
       time:   [99.351 ns 99.683 ns 100.02 ns]
       change: [-0.1542% +0.6098% +1.5340%] (p = 0.17 > 0.05)

Found 15 outliers among 100 measurements (15.00%)
10 (10.00%) high mild
5 (5.00%) high severe

coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [117.23 ns 117.56 ns 117.92 ns]
       change: [-0.0962% +0.3521% +0.8000%] (p = 0.14 > 0.05)

Found 15 outliers among 100 measurements (15.00%)
2 (2.00%) low mild
13 (13.00%) high severe

coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [116.98 ns 118.23 ns 120.34 ns]
       change: [+0.0235% +1.0363% +2.4624%] (p = 0.09 > 0.05)

Found 17 outliers among 100 measurements (17.00%)
5 (5.00%) low severe
1 (1.00%) low mild
1 (1.00%) high mild
10 (10.00%) high severe

coalesce_acked_from_zero 1000+1 entries: No change in performance detected.
       time:   [97.378 ns 97.571 ns 97.807 ns]
       change: [-1.0251% -0.1066% +0.8405%] (p = 0.83 > 0.05)

Found 12 outliers among 100 measurements (12.00%)
3 (3.00%) high mild
9 (9.00%) high severe

RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [112.47 ms 112.61 ms 112.83 ms]
       change: [+0.9661% +1.1095% +1.3312%] (p = 0.00 < 0.05)

Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) low mild
1 (1.00%) high severe

transfer/pacing-false/varying-seeds: No change in performance detected.
       time:   [25.839 ms 26.842 ms 27.841 ms]
       change: [-9.2546% -4.2640% +0.6030%] (p = 0.10 > 0.05)
transfer/pacing-true/varying-seeds: No change in performance detected.
       time:   [34.514 ms 36.361 ms 38.253 ms]
       change: [-5.6946% +0.5461% +7.4262%] (p = 0.88 > 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

transfer/pacing-false/same-seed: No change in performance detected.
       time:   [31.305 ms 32.021 ms 32.727 ms]
       change: [-5.1059% -2.2080% +0.9687%] (p = 0.17 > 0.05)
transfer/pacing-true/same-seed: Change within noise threshold.
       time:   [38.192 ms 40.661 ms 43.139 ms]
       change: [-18.383% -10.608% -1.9496%] (p = 0.02 < 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild

1-conn/1-100mb-resp (aka. Download)/client: Change within noise threshold.
       time:   [113.08 ms 113.54 ms 114.04 ms]
       thrpt:  [876.90 MiB/s 880.77 MiB/s 884.31 MiB/s]
change:
       time:   [-1.8711% -1.3187% -0.7524%] (p = 0.00 < 0.05)
       thrpt:  [+0.7581% +1.3363% +1.9068%]

Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) low mild
1 (1.00%) high severe

1-conn/10_000-parallel-1b-resp (aka. RPS)/client: No change in performance detected.
       time:   [312.60 ms 316.45 ms 320.33 ms]
       thrpt:  [31.218 Kelem/s 31.600 Kelem/s 31.990 Kelem/s]
change:
       time:   [-2.2632% -0.6558% +1.0318%] (p = 0.45 > 0.05)
       thrpt:  [-1.0213% +0.6601% +2.3156%]

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

1-conn/1-1b-resp (aka. HPS)/client: No change in performance detected.
       time:   [33.948 ms 34.124 ms 34.316 ms]
       thrpt:  [29.141  elem/s 29.305  elem/s 29.456  elem/s]
change:
       time:   [-0.9045% -0.0607% +0.7617%] (p = 0.89 > 0.05)
       thrpt:  [-0.7559% +0.0608% +0.9127%]

Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe

Client/server transfer results

Transfer of 33554432 bytes over loopback.

Client Server CC Pacing Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 201.3 ± 94.5 100.6 355.0 1.00
neqo msquic reno on 268.1 ± 84.7 214.0 422.0 1.00
neqo msquic reno 241.1 ± 56.1 208.7 414.4 1.00
neqo msquic cubic on 290.5 ± 96.5 217.7 436.4 1.00
neqo msquic cubic 253.4 ± 75.5 210.1 462.3 1.00
msquic neqo reno on 155.1 ± 86.3 94.5 323.0 1.00
msquic neqo reno 158.4 ± 82.2 95.0 323.7 1.00
msquic neqo cubic on 153.0 ± 93.9 93.3 447.0 1.00
msquic neqo cubic 180.0 ± 109.5 82.3 346.7 1.00
neqo neqo reno on 314.1 ± 123.4 129.8 440.3 1.00
neqo neqo reno 231.8 ± 112.2 139.8 430.2 1.00
neqo neqo cubic on 223.7 ± 104.2 140.3 424.8 1.00
neqo neqo cubic 270.5 ± 138.5 133.4 606.0 1.00

⬇️ Download logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants