Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Add a test case for 0-RTT loss #2057

Merged
merged 10 commits into from
Aug 20, 2024
Merged

Conversation

larseggert
Copy link
Collaborator

This currently fails, because we have a bug where the server does not accept retransmitted 0-RTT packets when it should.

This currently fails, because we have a bug where the server does not
accept retransmitted 0-RTT packets when it should.
Copy link

github-actions bot commented Aug 19, 2024

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Copy link

Firefox builds for this PR

The following builds are available for testing. Crossed-out builds did not succeed.

@martinthomson
Copy link
Member

It's not a bug, but a test configuration issue. See larseggert#29.

@larseggert
Copy link
Collaborator Author

So this also happens in the interop runner though - that inspired the test...

@martinthomson
Copy link
Member

That probably means that we need to be more generous with our tolerance to delay. The default we have in our server is 10 seconds, which seems fine. Are those tests that fail doing so because of being outside that window?

@larseggert
Copy link
Collaborator Author

larseggert commented Aug 20, 2024

I'm attaching a log (taken locally) where this happens. The ZeroRtt packet is generated at 10s778ms and at 16s905ms the server fails to accept it (after saying AllowZeroRtt accepting 0-RTT).

The real bug here may be that the client never seems to retransmit the frame data in the 0-RTT packet (after printing WARN Unhandled event ZeroRttRejected). Is this something that might have crept in during the move to tokio, @mxinden ?

(Even if this is the real bug, I think the server should still have accepted the 0-RTT...)

output.txt

@larseggert larseggert marked this pull request as ready for review August 20, 2024 06:12
test-fixture/src/lib.rs Outdated Show resolved Hide resolved
Copy link

codecov bot commented Aug 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.35%. Comparing base (a9f707f) to head (f0d615f).
Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2057   +/-   ##
=======================================
  Coverage   95.35%   95.35%           
=======================================
  Files         112      112           
  Lines       36499    36500    +1     
=======================================
+ Hits        34805    34806    +1     
  Misses       1694     1694           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Aug 20, 2024

Benchmark results

Performance differences relative to a9f707f.

coalesce_acked_from_zero 1+1 entries: 💔 Performance has regressed.
       time:   [134.74 ns 135.16 ns 135.60 ns]
       change: [+34.420% +35.048% +35.583%] (p = 0.00 < 0.05)

Found 17 outliers among 100 measurements (17.00%)
15 (15.00%) high mild
2 (2.00%) high severe

coalesce_acked_from_zero 3+1 entries: 💔 Performance has regressed.
       time:   [166.03 ns 166.60 ns 167.19 ns]
       change: [+40.423% +41.248% +42.013%] (p = 0.00 < 0.05)

Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe

coalesce_acked_from_zero 10+1 entries: 💔 Performance has regressed.
       time:   [165.76 ns 166.36 ns 166.98 ns]
       change: [+40.396% +40.935% +41.450%] (p = 0.00 < 0.05)

Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe

coalesce_acked_from_zero 1000+1 entries: 💔 Performance has regressed.
       time:   [138.06 ns 138.25 ns 138.47 ns]
       change: [+23.159% +34.043% +42.063%] (p = 0.00 < 0.05)

Found 10 outliers among 100 measurements (10.00%)
6 (6.00%) high mild
4 (4.00%) high severe

RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [111.04 ms 111.11 ms 111.18 ms]
       change: [-0.4820% -0.3912% -0.2986%] (p = 0.00 < 0.05)

Found 11 outliers among 100 measurements (11.00%)
3 (3.00%) low mild
8 (8.00%) high mild

transfer/pacing-false/varying-seeds: No change in performance detected.
       time:   [26.473 ms 27.443 ms 28.407 ms]
       change: [-3.7658% +1.3027% +6.4133%] (p = 0.62 > 0.05)
transfer/pacing-true/varying-seeds: No change in performance detected.
       time:   [35.005 ms 36.516 ms 38.042 ms]
       change: [-0.1852% +5.8854% +12.092%] (p = 0.06 > 0.05)

Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) low mild
2 (2.00%) high mild

transfer/pacing-false/same-seed: No change in performance detected.
       time:   [31.651 ms 32.506 ms 33.322 ms]
       change: [-1.4171% +1.8578% +5.2168%] (p = 0.26 > 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) low mild

transfer/pacing-true/same-seed: No change in performance detected.
       time:   [42.924 ms 45.858 ms 48.770 ms]
       change: [-3.8406% +6.0339% +17.302%] (p = 0.25 > 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild

1-conn/1-100mb-resp (aka. Download)/client: 💔 Performance has regressed.
       time:   [115.07 ms 115.40 ms 115.72 ms]
       thrpt:  [864.13 MiB/s 866.56 MiB/s 869.05 MiB/s]
change:
       time:   [+1.0044% +1.5351% +2.0714%] (p = 0.00 < 0.05)
       thrpt:  [-2.0294% -1.5119% -0.9944%]

Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high mild

1-conn/10_000-parallel-1b-resp (aka. RPS)/client: No change in performance detected.
       time:   [310.82 ms 314.57 ms 318.38 ms]
       thrpt:  [31.409 Kelem/s 31.790 Kelem/s 32.173 Kelem/s]
change:
       time:   [-0.2459% +1.3835% +3.1446%] (p = 0.11 > 0.05)
       thrpt:  [-3.0487% -1.3646% +0.2465%]
1-conn/1-1b-resp (aka. HPS)/client: No change in performance detected.
       time:   [40.245 ms 40.938 ms 41.626 ms]
       thrpt:  [24.023  elem/s 24.427  elem/s 24.848  elem/s]
change:
       time:   [-2.0272% +0.2005% +2.7398%] (p = 0.87 > 0.05)
       thrpt:  [-2.6667% -0.2001% +2.0692%]

Client/server transfer results

Transfer of 33554432 bytes over loopback.

Client Server CC Pacing Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 126.8 ± 66.2 83.5 359.1 1.00
neqo msquic reno on 213.4 ± 7.7 203.0 231.2 1.00
neqo msquic reno 216.5 ± 12.8 202.4 239.3 1.00
neqo msquic cubic on 220.2 ± 12.0 206.0 238.2 1.00
neqo msquic cubic 220.3 ± 14.5 206.8 252.3 1.00
msquic neqo reno on 87.7 ± 21.4 74.4 158.6 1.00
msquic neqo reno 113.5 ± 69.0 74.5 343.8 1.00
msquic neqo cubic on 84.5 ± 17.3 73.7 163.6 1.00
msquic neqo cubic 110.7 ± 74.8 74.2 359.0 1.00
neqo neqo reno on 164.7 ± 68.1 124.4 418.8 1.00
neqo neqo reno 163.8 ± 71.1 121.0 423.4 1.00
neqo neqo cubic on 166.9 ± 67.3 122.8 440.4 1.00
neqo neqo cubic 179.4 ± 70.4 123.7 400.8 1.00

⬇️ Download logs

Copy link
Member

@martinthomson martinthomson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nits only.

neqo-transport/src/connection/tests/zerortt.rs Outdated Show resolved Hide resolved
neqo-transport/src/connection/tests/zerortt.rs Outdated Show resolved Hide resolved
neqo-transport/src/connection/tests/zerortt.rs Outdated Show resolved Hide resolved
larseggert and others added 3 commits August 20, 2024 10:20
Co-authored-by: Martin Thomson <mt@lowentropy.net>
Signed-off-by: Lars Eggert <lars@eggert.org>
Co-authored-by: Martin Thomson <mt@lowentropy.net>
Signed-off-by: Lars Eggert <lars@eggert.org>
Co-authored-by: Martin Thomson <mt@lowentropy.net>
Signed-off-by: Lars Eggert <lars@eggert.org>
@larseggert larseggert added this pull request to the merge queue Aug 20, 2024
Merged via the queue into mozilla:main with commit e3d8be4 Aug 20, 2024
56 checks passed
@larseggert larseggert deleted the test-0rtt-loss branch August 20, 2024 08:07
@mxinden
Copy link
Collaborator

mxinden commented Aug 20, 2024

Is this something that might have crept in during the move to tokio, @mxinden ?

Off the top of my head, I am not sure how, given that the move to tokio didn't touch the state machines itself, but only the IO paths on the outside. That said, it might have introduced a timing issue?

I can take a look. With which server does neqo-latest fail most reliably @larseggert?

@larseggert
Copy link
Collaborator Author

@mxinden I don't think it's tokio-related. See #2062.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants