H/3 stress #55098

ManickaP · 2021-07-02T20:17:32Z

Spin off #54762

updates docker images
fixes some build warnings in stress apps
add H/3 support for HttpStress
renames Quic event source

Fixes #40389

ghost · 2021-07-02T20:17:38Z

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Spin off #54762

updates docker images
fixes some build warnings in stress apps
add H/3 support for HttpStress
renames Quic event source

Author:	ManickaP
Assignees:	-
Labels:	`area-System.Net.Http`
Milestone:	-

ManickaP · 2021-07-06T17:40:52Z

The stress tests have run. So the pipelines work, docker files are OK, etc.
There's tons of error in H/2 (Windows and Linux) for content sending requests (PUT, POST etc.) looking like this:

System.Exception: Expected status code OK, got InternalServerError
client_1  |    at HttpStress.ClientOperations.ValidateStatusCode(HttpResponseMessage m, HttpStatusCode expectedStatus) in /app/ClientOperations.cs:line 487
client_1  |    at HttpStress.ClientOperations.<>c.<<get_Operations>b__1_9>d.MoveNext() in /app/ClientOperations.cs:line 407
client_1  | --- End of stack trace from previous location ---
client_1  |    at HttpStress.StressClient.<>c__DisplayClass17_0.<<StartCore>g__RunWorker|0>d.MoveNext() in /app/StressClient.cs:line 204

This might be a recent change in kestrel or our client that breaks it since I'm updating here to 6.0 nightly.

@antonfirsov Do you know anything about it?

cc: @CarnaViire

ManickaP · 2021-07-07T13:15:44Z

So the error reproduces locally without docker. It happened sometime between 6.0 preview 4 and preview 6. I tested it firstly on preview 4 - no error, downloaded preview 6 and retested and it happens.

ManickaP · 2021-07-07T13:23:53Z

H/2 issue filed: #55261
I guess we can get this in and fix the issue separately.

wfurt · 2021-07-07T16:24:11Z

src/libraries/System.Net.Http/tests/StressTests/HttpStress/Dockerfile

@@ -1,4 +1,4 @@
-ARG SDK_BASE_IMAGE=mcr.microsoft.com/dotnet/nightly/sdk:5.0-buster-slim
+ARG SDK_BASE_IMAGE=mcr.microsoft.com/dotnet/nightly/sdk:6.0-bullseye-slim


Do we need this? Seems like buster is still latest stable...

No we don't. I didn't realize it's "testing" and not "stable". I just took the newest without checking. I'll revert it.

We'll need the bullseye after all. Seems like buster doesn't have current 6.0 preview SDK...

wfurt · 2021-07-07T16:25:42Z

src/libraries/System.Net.Http/tests/StressTests/HttpStress/NuGet.config

+<configuration>
+  <packageSources>
+    <!-- Add public nuget feed. -->
+    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />


Isn't this default? I don't mind to be explicit but I'm wondering if we need it in case we do not need custom feeds

Not if run from runtime repo, ours explicitly removes nuget.org: https://github.com/dotnet/runtime/blob/main/NuGet.config
So it all works in docker, where it's isolated, but when you ran it locally from S.N.Http/tests/... it failed due to one dependency coming from the public feed.

wfurt

generally looks ok to me.

ManickaP · 2021-07-07T19:31:51Z

eng/pipelines/libraries/stress/http.yml

@@ -47,6 +47,14 @@ jobs:
    name: buildStress
    displayName: Build HttpStress

+  - bash: |


We'll see, but on Linux we should be now able to run stress tests.

CarnaViire · 2021-07-07T20:53:35Z

src/libraries/System.Net.Http/tests/StressTests/HttpStress/Dockerfile

@@ -7,6 +7,18 @@ RUN echo "DOTNET_VERSION="$DOTNET_VERSION
 WORKDIR /app
 COPY . .

+# Pulling the msquic Debian package from msquic-ci public pipeline and from a hardcoded build.
+# Note that this is a temporary solution until we have properly published Linux packages.
+# Also note that in order to update to a newer msquic build, you have update this link.


Does this link already contain changes needed for #55291? If not, can we update it?

Yes it does contain them.

ManickaP · 2021-07-08T14:20:14Z

/azp run runtime-libraries stress-http

ManickaP · 2021-07-08T14:20:20Z

/azp run runtime-libraries stress-ssl

azure-pipelines · 2021-07-08T14:20:31Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2021-07-08T14:20:33Z

Azure Pipelines successfully started running 1 pipeline(s).

ManickaP · 2021-07-08T15:07:14Z

Stress tests on Linux are running H/3 🥳

ManickaP · 2021-07-08T17:30:28Z

Seeing 10s thousands of System.Exception: Expected status code OK, got InternalServerError in H/2 (#55261)
I'll handle this particular error before merging and put an ActiveIssue note in the code.

ManickaP · 2021-07-08T17:52:19Z

/azp run runtime-libraries stress-http

azure-pipelines · 2021-07-08T17:52:32Z

Azure Pipelines successfully started running 1 pipeline(s).

ManickaP · 2021-07-08T18:43:10Z

/azp run runtime-libraries stress-http

azure-pipelines · 2021-07-08T18:43:20Z

Azure Pipelines successfully started running 1 pipeline(s).

ManickaP · 2021-07-09T07:39:54Z

Looks good now: https://dev.azure.com/dnceng/public/_build/results?buildId=1227406&view=results
1 error on H 1.1 Win and several errors on H/3 GET Aborted, after the suppression of #55261

CI errors are unrelated, merging.

ManickaP added 5 commits July 2, 2021 17:58

Fixed event source name

ef146d3

Updated docker files, updated stress test to H/3

1d83c9e

SSL stress updated as well

67af0a8

Fixed docker and compilation warnings

6dbf212

H/3 fixes

23ceced

dotnet-issue-labeler bot added the area-System.Net.Http label Jul 2, 2021

Fixed nano-server version and nuget reference

26aac78

dotnet deleted a comment from azure-pipelines bot Jul 6, 2021

ManickaP marked this pull request as ready for review July 6, 2021 17:43

ManickaP requested review from antonfirsov, CarnaViire and wfurt July 6, 2021 17:43

CarnaViire mentioned this pull request Jul 7, 2021

HTTP/3 stress tests (requires libmsquic.so) #54762

Closed

ManickaP mentioned this pull request Jul 7, 2021

H/2 stress many failure in content posting methods #55261

Closed

wfurt reviewed Jul 7, 2021

View reviewed changes

wfurt approved these changes Jul 7, 2021

View reviewed changes

Debian stable version revert: bullseye --> buster

fd22b00

dotnet deleted a comment from azure-pipelines bot Jul 7, 2021

Use msquic package in Linux stress container

ca9ab3b

ManickaP force-pushed the mapichov/h3_stress branch from b057c44 to ca9ab3b Compare July 7, 2021 19:30

ManickaP commented Jul 7, 2021

View reviewed changes

CarnaViire reviewed Jul 7, 2021

View reviewed changes

ManickaP added 2 commits July 8, 2021 10:55

Bullseye is back, needs 6.0 SDK current preview

4051367

Fixed pulling images in docker compose

30769dd

dotnet deleted a comment from azure-pipelines bot Jul 8, 2021

Disabled problematic error in H/2

84f63a4

ManickaP force-pushed the mapichov/h3_stress branch from d2ef2ad to 84f63a4 Compare July 8, 2021 18:42

ManickaP merged commit d431d6a into dotnet:main Jul 9, 2021

ManickaP deleted the mapichov/h3_stress branch July 9, 2021 07:41

karelz added this to the 6.0.0 milestone Jul 15, 2021

ManickaP mentioned this pull request Jul 20, 2021

[QUIC] Remove AppContext switch from S.N.Quic #56027

Merged

ghost locked as resolved and limited conversation to collaborators Aug 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

H/3 stress #55098

H/3 stress #55098

ManickaP commented Jul 2, 2021 •

edited

Loading

ghost commented Jul 2, 2021

ManickaP commented Jul 6, 2021 •

edited

Loading

ManickaP commented Jul 7, 2021

ManickaP commented Jul 7, 2021

wfurt Jul 7, 2021

ManickaP Jul 7, 2021

ManickaP Jul 8, 2021

wfurt Jul 7, 2021

ManickaP Jul 7, 2021

wfurt left a comment

ManickaP Jul 7, 2021

CarnaViire Jul 7, 2021

ManickaP Jul 8, 2021

ManickaP commented Jul 8, 2021

ManickaP commented Jul 8, 2021

azure-pipelines bot commented Jul 8, 2021

azure-pipelines bot commented Jul 8, 2021

ManickaP commented Jul 8, 2021

ManickaP commented Jul 8, 2021

ManickaP commented Jul 8, 2021

azure-pipelines bot commented Jul 8, 2021

ManickaP commented Jul 8, 2021

azure-pipelines bot commented Jul 8, 2021

ManickaP commented Jul 9, 2021 •

edited

Loading

		@@ -1,4 +1,4 @@
		ARG SDK_BASE_IMAGE=mcr.microsoft.com/dotnet/nightly/sdk:5.0-buster-slim
		ARG SDK_BASE_IMAGE=mcr.microsoft.com/dotnet/nightly/sdk:6.0-bullseye-slim

H/3 stress #55098

H/3 stress #55098

Conversation

ManickaP commented Jul 2, 2021 • edited Loading

ghost commented Jul 2, 2021

ManickaP commented Jul 6, 2021 • edited Loading

ManickaP commented Jul 7, 2021

ManickaP commented Jul 7, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wfurt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ManickaP commented Jul 8, 2021

ManickaP commented Jul 8, 2021

azure-pipelines bot commented Jul 8, 2021

azure-pipelines bot commented Jul 8, 2021

ManickaP commented Jul 8, 2021

ManickaP commented Jul 8, 2021

ManickaP commented Jul 8, 2021

azure-pipelines bot commented Jul 8, 2021

ManickaP commented Jul 8, 2021

azure-pipelines bot commented Jul 8, 2021

ManickaP commented Jul 9, 2021 • edited Loading

ManickaP commented Jul 2, 2021 •

edited

Loading

ManickaP commented Jul 6, 2021 •

edited

Loading

ManickaP commented Jul 9, 2021 •

edited

Loading