[HTTP/3] SocketsHttpHandlerTest_Http3_MsQuic.ClientSettingsReceived_Success timing out

[jaredpar]

Failure query

https://runfo.azurewebsites.net/search/tests/?q=started%3A~7%20definition%3Aruntime%20name%3A%22system.net.http.functional.tests.socketshttphandlertest_http3_msquic.clientsettingsreceived_success%22

Test failures typically have the following call stack

    System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http3_MsQuic.ClientSettingsReceived_Success(headerSizeLimit: 10) [FAIL]
      System.TimeoutException : The operation has timed out.
      Stack Trace:
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(49,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout)
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs(78,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.ClientSettingsReceived_Success(Int32 headerSizeLimit)
        --- End of stack trace from previous location ---

Note: thus far this has only failed in Pull Requests so it's not definitively a bug. However the PRs aren't networking related to my understanding hence seems more likely it is a bug here. It's also hit 14 times now.

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Failure query

https://runfo.azurewebsites.net/search/tests/?q=started%3A~7%20definition%3Aruntime%20name%3A%22system.net.http.functional.tests.socketshttphandlertest_http3_msquic.clientsettingsreceived_success%22

Test failures typically have the following call stack

    System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http3_MsQuic.ClientSettingsReceived_Success(headerSizeLimit: 10) [FAIL]
      System.TimeoutException : The operation has timed out.
      Stack Trace:
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(49,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout)
        /_/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs(78,0): at System.Net.Http.Functional.Tests.HttpClientHandlerTest_Http3.ClientSettingsReceived_Success(Int32 headerSizeLimit)
        --- End of stack trace from previous location ---

Note: thus far this has only failed in Pull Requests so it's not definitively a bug. However the PRs aren't networking related to my understanding hence seems more likely it is a bug here. It's also hit 14 times now.

Author:	jaredpar
Assignees:	-
Labels:	area-System.Net.Http, untriaged
Milestone:	-

[ManickaP]

Triage: 27 occurrences so far, we might disable it. For 6.0 we need to at least rule out whether it's product bug

"Method": ClientSettingsReceived_Success,
"count_": 27,
"min_Queued": 2021-07-15T02:49:58.522Z,

[karelz]

Failures 5/17-7/16 (incl. PRs):

Date	Failures
7/15	20 failures -- 19x Fedora.32.Amd64.Open, 1x Debian.10.Amd64.Open (Mock) ... 2 official builds ... 2x Mock
7/16	12 failures (so far) - 11x Fedora.32.Amd64.Open, 1x Ubuntu.1804.ArmArch.Open ... 3 official builds ... 1x Mock

We should disable it at least on Fedora ASAP to avoid creating noise for everybody in PRs.
UPDATE: We run non-Mock tests only on Fedora now ... that's why we see mainly that. Real msquic implementation has likely sufficiently different timing that it started failing frequently. Though Mock test version started failing also only recently, so perhaps some product change introduced that?

runtime/src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientHandlerTest.Http3.cs

Line 78 in 9a9b105

await new[] { clientTask, serverTask }.WhenAllOrAnyFailed(20_000);

[karelz]

Test disabled in PR #55811 on 7/16

Failures from Preview 7 branch (which branched without the test disabled) 7/16-7/27 (incl. PRs):

Day	Run
7/21	PR #56039
7/23	PR #56013
7/23	PR #56240
7/23	PR #56250
7/23	PR #56235
7/23	PR #56255
7/24	PR #56255
7/26	PR #56255
7/27	PR #56330

[ManickaP]

Re-enabled in #67418, no occurrences in Kusto.