Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reenable Http2_ServerSendsInvalidSettingsValue_Error test #1581

Closed
geoffkizer opened this issue Jul 3, 2019 · 5 comments · Fixed by #2134 or #56552
Closed

Reenable Http2_ServerSendsInvalidSettingsValue_Error test #1581

geoffkizer opened this issue Jul 3, 2019 · 5 comments · Fixed by #2134 or #56552
Assignees
@alnikola
Labels
area-System.Net.Http disabled-test The test is disabled in source code against the issue test-run-core Test failures in .NET Core test runs
Milestone
6.0.0

Comments

@geoffkizer
Copy link
Contributor

This is marked as [ActiveIssue(35466)], but dotnet/corefx#35466 has been closed.

When I reenable and run this, I get:

      System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_ServerSendsInvalidSettingsValue_Error(settingId: MaxFrameSize, value: 16383, expectedError: PROTOCOL_ERROR, skipForWinHttp: True) [FAIL]
        Assert.Throws() Failure
        Expected: typeof(System.Net.Http.HttpRequestException)
        Actual:   typeof(System.Threading.Tasks.TaskCanceledException): The operation was canceled.
        ---- System.Threading.Tasks.TaskCanceledException : The operation was canceled.
        Stack Trace:
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\CancellationHelper.cs(34,0): at System.Net.Http.CancellationHelper.ThrowOperationCanceledException(Exception innerException, CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\CancellationHelper.cs(42,0): at System.Net.Http.CancellationHelper.ThrowIfCancellationRequested(CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http2Stream.cs(779,0): at System.Net.Http.Http2Connection.Http2Stream.<>c__DisplayClass51_0.<<GetWaiterTask>g__GetWaiterTaskCore|0>d.MoveNext()
          --- End of stack trace from previous location where exception was thrown ---
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http2Stream.cs(557,0): at System.Net.Http.Http2Connection.Http2Stream.ReadResponseHeadersAsync(CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http2Connection.cs(1540,0): at System.Net.Http.Http2Connection.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\HttpConnectionPool.cs(541,0): at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\RedirectHandler.cs(33,0): at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\HttpClient.cs(496,0): at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
          ----- Inner Stack Trace -----
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\CancellationHelper.cs(34,0): at System.Net.Http.CancellationHelper.ThrowOperationCanceledException(Exception innerException, CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\CancellationHelper.cs(42,0): at System.Net.Http.CancellationHelper.ThrowIfCancellationRequested(CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http2Stream.cs(779,0): at System.Net.Http.Http2Connection.Http2Stream.<>c__DisplayClass51_0.<<GetWaiterTask>g__GetWaiterTaskCore|0>d.MoveNext()
          --- End of stack trace from previous location where exception was thrown ---
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http2Stream.cs(557,0): at System.Net.Http.Http2Connection.Http2Stream.ReadResponseHeadersAsync(CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http2Connection.cs(1540,0): at System.Net.Http.Http2Connection.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\HttpConnectionPool.cs(541,0): at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\RedirectHandler.cs(33,0): at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
          C:\github\geoffkizer\corefx\src\System.Net.Http\src\System\Net\Http\HttpClient.cs(496,0): at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
@karelz
Copy link
Member

karelz commented Jul 3, 2019

@geoffkizer @davidsh do you think this is must have for 3.0?

@geoffkizer
Copy link
Contributor Author

Nope, kill it.

@danmoseley danmoseley transferred this issue from dotnet/corefx Jan 9, 2020
@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added area-System.Net.Http untriaged New issue has not been triaged by the area owner labels Jan 9, 2020
@danmoseley danmoseley added the disabled-test The test is disabled in source code against the issue label Jan 9, 2020
@wfurt wfurt mentioned this issue Jan 24, 2020
Merged
@karelz karelz added this to the 5.0 milestone Apr 15, 2020
@karelz karelz added test-bug Problem in test source code (most likely) and removed disabled-test The test is disabled in source code against the issue untriaged New issue has not been triaged by the area owner labels Apr 15, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Dec 12, 2020
@stephentoub
Copy link
Member

stephentoub commented Jul 20, 2021 •
edited
Loading

Failed again here:
https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-55955-merge-dcaf39925115489a8c/System.Net.Http.Functional.Tests/console.e3bf82e4.log?sv=2019-07-07&se=2021-08-09T14%3A07%3A07Z&sr=c&sp=rl&sig=VSIaF7BHFR84IF5IzWFOg%2FDd9Q%2B8OGlBTc0AQUrIW2U%3D

https://dev.azure.com/dnceng/public/_build/results?buildId=1248297&view=ms.vss-test-web.build-test-results-tab&runId=37078482&resultId=161002&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab

    System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_ServerSendsInvalidSettingsValue_Error(settingId: InitialWindowSize, value: 2147483648) [FAIL]
      Assert.Throws() Failure
      Expected: typeof(System.Net.Http.HttpRequestException)
      Actual:   typeof(System.Threading.Tasks.TaskCanceledException): The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
      ---- System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
      -------- System.TimeoutException : A task was canceled.
      ------------ System.Threading.Tasks.TaskCanceledException : A task was canceled.
      Stack Trace:
        /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs(623,0): at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
        /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs(546,0): at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
        ----- Inner Stack Trace -----
        /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs(623,0): at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
        /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs(546,0): at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
        ----- Inner Stack Trace -----
        
        ----- Inner Stack Trace -----
        /_/src/libraries/Common/src/System/Threading/Tasks/TaskCompletionSourceWithCancellation.cs(21,0): at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
        /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs(789,0): at System.Net.Http.HttpConnectionPool.GetHttp2ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
        /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs(954,0): at System.Net.Http.HttpConnectionPool.TrySendUsingHttp2Async(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
        /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs(1002,0): at System.Net.Http.HttpConnectionPool.DetermineVersionAndSendAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
        /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs(1019,0): at System.Net.Http.HttpConnectionPool.SendAndProcessAltSvcAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
        /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs(1038,0): at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
        /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs(30,0): at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
        /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs(527,0): at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)

@stephentoub stephentoub reopened this Jul 20, 2021
@stephentoub stephentoub removed this from the 5.0.0 milestone Jul 20, 2021
@stephentoub stephentoub added untriaged New issue has not been triaged by the area owner and removed test-bug Problem in test source code (most likely) labels Jul 20, 2021
@karelz
Copy link
Member

karelz commented Jul 20, 2021 •
edited
Loading

Test failures 5/20-7/20 (incl. PRs):

Day Run OS
7/15 PR #55703 Windows.10.Amd64.Server19H1.ES.Open
7/15 PR #55738 Windows.10.Amd64.Server19H1.ES.Open
7/15 PR #55678 Windows.10.Amd64.Server19H1.Open
7/16 Official run Debian.9.Amd64.Open
7/16 PR #55994 Windows.Nano.1809.Amd64.Open
7/16 PR #55726 Windows.10.Amd64.Server19H1.Open
7/16 PR #55836 Windows.10.Amd64.Server19H1.Open
7/16 PR #55756 Windows.10.Amd64.Server19H1.Open
7/17 PR #55756 Windows.81.Amd64.Open
7/18 PR #55877 Windows.10.Amd64.Server19H1.Open
7/19 PR #55283 Windows.Nano.1809.Amd64.Open
7/19 PR #55916 Debian.10.Amd64.Open
7/20 PR #55955 Windows.Nano.1809.Amd64.Open
7/20 PR #55997 Windows.81.Amd64.Open
7/20 PR #55955 Windows.Nano.1809.Amd64.Open

Looks like a recent regression ... @stephentoub we should probably create new issue next time instead of reopening old ones ...

Assert.Throws() Failure
Expected: typeof(System.Net.Http.HttpRequestException)
Actual:   typeof(System.Threading.Tasks.TaskCanceledException): The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---- System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
-------- System.TimeoutException : A task was canceled.
------------ System.Threading.Tasks.TaskCanceledException : A task was canceled.

   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 623
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 546
----- Inner Stack Trace -----
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 623
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 546
----- Inner Stack Trace -----

----- Inner Stack Trace -----
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) in /_/src/libraries/Common/src/System/Threading/Tasks/TaskCompletionSourceWithCancellation.cs:line 21
   at System.Net.Http.HttpConnectionPool.GetHttp2ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 789
   at System.Net.Http.HttpConnectionPool.TrySendUsingHttp2Async(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 954
   at System.Net.Http.HttpConnectionPool.DetermineVersionAndSendAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1002
   at System.Net.Http.HttpConnectionPool.SendAndProcessAltSvcAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1019
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1038
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:line 30
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 527

@karelz karelz removed the untriaged New issue has not been triaged by the area owner label Jul 20, 2021
@karelz karelz added this to the 6.0.0 milestone Jul 20, 2021
@karelz karelz added test-run-core Test failures in .NET Core test runs disabled-test The test is disabled in source code against the issue labels Jul 20, 2021
@karelz
Copy link
Member

karelz commented Jul 21, 2021 •
edited
Loading

Test disabled in PR #56041 on 7/21

Recent regression around 7/15

@alnikola alnikola self-assigned this Jul 28, 2021
alnikola added a commit that referenced this issue Jul 30, 2021
@alnikola
Check _abortException before checking _shutdown flag (#56552)
facbdce 
In case of an error aborting the connection, there is a race between a thread creating new `Http2Stream` to send a request and the thread looping in `ProcessIncomingFrames` that sets _shutdown flag and `_abortException`. If the request thread first sees `_shutdown == true`, then it won't see the `_abortException` even if it's set, so the request will be retried when it shouldn't.

This PR adds `_abortException` check just before the `_shutdown == true` check to make sure an abort exception is observed.

Fixes #1581 
Fixes #56138
Fixes #56026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@alnikola alnikola

Labels
area-System.Net.Http disabled-test The test is disabled in source code against the issue test-run-core Test failures in .NET Core test runs
Projects
None yet
Milestone
6.0.0
6 participants
@stephentoub @karelz @danmoseley @geoffkizer @Dotnet-GitSync-Bot @alnikola