Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HTTP/3] SendMoreThanStreamLimitRequests_Succeeds times out #55957

Closed
mdh1418 opened this issue Jul 19, 2021 · 9 comments · Fixed by #56064 or #69789
Closed

[HTTP/3] SendMoreThanStreamLimitRequests_Succeeds times out #55957

mdh1418 opened this issue Jul 19, 2021 · 9 comments · Fixed by #56064 or #69789
Assignees
Labels
area-System.Net.Http disabled-test The test is disabled in source code against the issue test-run-core Test failures in .NET Core test runs
Milestone

Comments

@mdh1418
Copy link
Member

mdh1418 commented Jul 19, 2021

Type:

  • System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http3_Mock -- Mock
  • System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http3_MsQuic -- MsQuic

Mock

System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http3_Mock.SendMoreThanStreamLimitRequests_Succeeds(streamLimit: 10) [FAIL]
      System.AggregateException : One or more errors occurred. (One or more errors occurred. (The operation has timed out.)) (One or more errors occurred. (Stream aborted by peer (268).))
      ---- System.AggregateException : One or more errors occurred. (The operation has timed out.)
      -------- System.TimeoutException : The operation has timed out.
      ---- System.AggregateException : One or more errors occurred. (Stream aborted by peer (268).)
      -------- System.Net.Quic.QuicStreamAbortedException : Stream aborted by peer (268).
      Stack Trace:
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(80,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(49,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout)
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs(117,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.SendMoreThanStreamLimitRequests_Succeeds(Int32 streamLimit)
        --- End of stack trace from previous location ---
        ----- Inner Stack Trace #1 (System.AggregateException) -----
        
        ----- Inner Stack Trace -----
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs(113,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.<>c__DisplayClass4_0.<<SendMoreThanStreamLimitRequests_Succeeds>b__1>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(63,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        ----- Inner Stack Trace #2 (System.AggregateException) -----
        
        ----- Inner Stack Trace -----
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Implementations/Mock/MockStream.cs(76,0): at System.Net.Quic.Implementations.Mock.MockStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(400,0): at System.Net.Test.Common.Http3LoopbackStream.ReadIntegerAsync()
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(370,0): at System.Net.Test.Common.Http3LoopbackStream.ReadFrameAsync()
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(200,0): at System.Net.Test.Common.Http3LoopbackStream.ReadRequestBodyAsync()
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(224,0): at System.Net.Test.Common.Http3LoopbackStream.ReadRequestDataAsync(Boolean readBody)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(52,0): at System.Net.Test.Common.Http3LoopbackStream.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs(96,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.<>c__DisplayClass4_0.<<SendMoreThanStreamLimitRequests_Succeeds>b__0>d.MoveNext()

log: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-54845-merge-281d2557555b40c185/System.Net.Http.Functional.Tests/console.b4270b74.log?sv=2019-07-07&se=2021-08-08T17%3A25%3A26Z&sr=c&sp=rl&sig=Ili2HVRMJUMjyhEZObd2yR6udqbRONrd6qRiNGOIOFc%3D
build: https://dev.azure.com/dnceng/public/_build/results?buildId=1246061&view=logs&j=71afe768-e493-597c-228c-037834193aeb&t=fe87d331-02fa-5ca6-e382-69b0a11bb0f6

MsQuic

System.TimeoutException : The operation has timed out.

   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.<>c__DisplayClass4_0.<<SendMoreThanStreamLimitRequests_Succeeds>b__1>d.MoveNext() in /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs:line 113
--- End of stack trace from previous location ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 55
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 82
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 49
   at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.SendMoreThanStreamLimitRequests_Succeeds(Int32 streamLimit) in /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs:line 117
--- End of stack trace from previous location ---
@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Jul 19, 2021
@ghost
Copy link

ghost commented Jul 19, 2021

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details
System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http3_Mock.SendMoreThanStreamLimitRequests_Succeeds(streamLimit: 10) [FAIL]
      System.AggregateException : One or more errors occurred. (One or more errors occurred. (The operation has timed out.)) (One or more errors occurred. (Stream aborted by peer (268).))
      ---- System.AggregateException : One or more errors occurred. (The operation has timed out.)
      -------- System.TimeoutException : The operation has timed out.
      ---- System.AggregateException : One or more errors occurred. (Stream aborted by peer (268).)
      -------- System.Net.Quic.QuicStreamAbortedException : Stream aborted by peer (268).
      Stack Trace:
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(80,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(49,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout)
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs(117,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.SendMoreThanStreamLimitRequests_Succeeds(Int32 streamLimit)
        --- End of stack trace from previous location ---
        ----- Inner Stack Trace #1 (System.AggregateException) -----
        
        ----- Inner Stack Trace -----
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs(113,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.<>c__DisplayClass4_0.<<SendMoreThanStreamLimitRequests_Succeeds>b__1>d.MoveNext()
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(63,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        ----- Inner Stack Trace #2 (System.AggregateException) -----
        
        ----- Inner Stack Trace -----
        /_/src/libraries/System.Net.Quic/src/System/Net/Quic/Implementations/Mock/MockStream.cs(76,0): at System.Net.Quic.Implementations.Mock.MockStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(400,0): at System.Net.Test.Common.Http3LoopbackStream.ReadIntegerAsync()
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(370,0): at System.Net.Test.Common.Http3LoopbackStream.ReadFrameAsync()
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(200,0): at System.Net.Test.Common.Http3LoopbackStream.ReadRequestBodyAsync()
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(224,0): at System.Net.Test.Common.Http3LoopbackStream.ReadRequestDataAsync(Boolean readBody)
        /_/src/libraries/Common/tests/System/Net/Http/Http3LoopbackStream.cs(52,0): at System.Net.Test.Common.Http3LoopbackStream.HandleRequestAsync(HttpStatusCode statusCode, IList`1 headers, String content)
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs(96,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.<>c__DisplayClass4_0.<<SendMoreThanStreamLimitRequests_Succeeds>b__0>d.MoveNext()

log: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-54845-merge-281d2557555b40c185/System.Net.Http.Functional.Tests/console.b4270b74.log?sv=2019-07-07&se=2021-08-08T17%3A25%3A26Z&sr=c&sp=rl&sig=Ili2HVRMJUMjyhEZObd2yR6udqbRONrd6qRiNGOIOFc%3D
build: https://dev.azure.com/dnceng/public/_build/results?buildId=1246061&view=logs&j=71afe768-e493-597c-228c-037834193aeb&t=fe87d331-02fa-5ca6-e382-69b0a11bb0f6

Author: mdh1418
Assignees: -
Labels:

area-System.Net.Http

Milestone: -

@karelz karelz changed the title Failing SocketsHttpHandlerTest_Http3_Mock.SendMoreThanStreamLimitRequests_Succeeds test on coreclr Linux x64 Debug [HTTP/3] Failing SocketsHttpHandlerTest_Http3_Mock.SendMoreThanStreamLimitRequests_Succeeds test on coreclr Linux x64 Debug Jul 19, 2021
@karelz
Copy link
Member

karelz commented Jul 19, 2021

Failures 5/19-7/30 (incl. PRs):

Day Build OS Type
7/16 PR #55844 Ubuntu.1604.Amd64.Open Mock
7/19 PR #55283 RedHat.7.Amd64.Open Mock
7/19 PR #54845 RedHat.7.Amd64.Open Mock
7/19 PR #55283 Fedora.32.Amd64.Open MsQuic -- TimeoutException
7/20 17x Official run + 47x PRs (many) Mock only
7/21 15x Official run + 56x PRs (many) Mock only
7/21 Mock test disabled
7/23 PR #56231 Fedora.32.Amd64.Open MsQuic -- AggregateException
7/26 PR #56025 SLES.15.Amd64.Open Mock (branched from before test disabled)
7/26 PR #56025 Fedora.32.Amd64.Open MsQuic -- TimeoutException
7/27 PR #56387 Fedora.32.Amd64.Open MsQuic -- AggregateException
7/27 PR #56404 Ubuntu.1804.Amd64.Open Mock (Preview 7 branch - before test disabled)
7/27 PR #56410 Fedora.32.Amd64.Open MsQuic -- AggregateException
7/28 PR #56443 Fedora.32.Amd64.Open MsQuic -- AggregateException
7/29 PR #56196 Fedora.32.Amd64.Open MsQuic -- AggregateException
7/29 Official run Fedora.32.Amd64.Open MsQuic -- AggregateException
7/29 PR #55871 Fedora.32.Amd64.Open MsQuic -- TimeoutException
7/29 PR #56470 Fedora.32.Amd64.Open MsQuic -- TimeoutException
7/29 PR #56558 Fedora.32.Amd64.Open MsQuic -- AggregateException
7/29 Official run Fedora.32.Amd64.Open MsQuic -- TimeoutException
7/30 PR #56559 Fedora.32.Amd64.Open MsQuic -- TimeoutException
7/30 PR #56056 Fedora.32.Amd64.Open MsQuic -- TimeoutException
7/30 PR #56595 Fedora.32.Amd64.Open MsQuic -- TimeoutException
7/30 MsQuic test also disabled

@karelz karelz changed the title [HTTP/3] Failing SocketsHttpHandlerTest_Http3_Mock.SendMoreThanStreamLimitRequests_Succeeds test on coreclr Linux x64 Debug [HTTP/3] SendMoreThanStreamLimitRequests_Succeeds times out Jul 20, 2021
@karelz
Copy link
Member

karelz commented Jul 20, 2021

Triage: Disable only the Mock test, then move to Future.

@karelz karelz added this to the 6.0.0 milestone Jul 20, 2021
@karelz karelz self-assigned this Jul 20, 2021
@karelz karelz removed the untriaged New issue has not been triaged by the area owner label Jul 20, 2021
@karelz karelz added the test-run-core Test failures in .NET Core test runs label Jul 20, 2021
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 21, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 21, 2021
@stephentoub stephentoub reopened this Jul 21, 2021
@stephentoub stephentoub added the disabled-test The test is disabled in source code against the issue label Jul 21, 2021
@karelz
Copy link
Member

karelz commented Jul 21, 2021

Test disabled for Mock only in PR #56064 on 7/21

Test disabled for all configurations (incl. MsQuic) in PR #56566 on 7/30

@karelz karelz removed their assignment Jul 21, 2021
@karelz karelz modified the milestones: 6.0.0, Future, 7.0.0 Jul 21, 2021
@ericstj

This comment has been minimized.

@karelz
Copy link
Member

karelz commented Jul 27, 2021

@ericstj the PR likely didn't have the test disabled for Mock yet (it was created prior to the disabling and I think it didn't merge main changes afterwards).
For MsQuic, the failure rate so far is small, so we didn't disable it yet.

@ericstj
Copy link
Member

ericstj commented Jul 27, 2021

Ok that makes sense. We reran after the commit went in to disable the test but rerun doesn’t pick up a new merge commit.

karelz added a commit that referenced this issue Jul 29, 2021
Disable the test also for MsQuic, not just Mock.

Test types:
- System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http3_MsQuic
- System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http3_Mock

Disabled test tracked by #55957
@karelz karelz modified the milestones: 7.0.0, 6.0.0 Jul 29, 2021
@wfurt wfurt self-assigned this Jul 29, 2021
karelz added a commit that referenced this issue Jul 30, 2021
…56566)

Disable the test also for MsQuic, not just Mock.

Test types:
- System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http3_MsQuic
- System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http3_Mock

Disabled test tracked by #55957
@karelz
Copy link
Member

karelz commented Jul 30, 2021

Test disabled for all configurations (incl. MsQuic) in PR #56566 on 7/30

@ManickaP
Copy link
Member

Triage: not actively worked on, punting.

@ManickaP ManickaP modified the milestones: 6.0.0, 7.0.0, Future Aug 10, 2021
@rzikm rzikm self-assigned this May 13, 2022
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label May 25, 2022
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label May 25, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Jun 24, 2022
@karelz karelz modified the milestones: Future, 7.0.0 Jul 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Net.Http disabled-test The test is disabled in source code against the issue test-run-core Test failures in .NET Core test runs
Projects
None yet
7 participants