CI failure in System.Net.Http.Functional.Tests - Operation timeout #75611
Labels
Milestone
Comments
ghost
added
the
|
Tagging subscribers to this area: @dotnet/ncl Issue DetailsSeen in this Please help determine if this needs a fix backported to 7.0. BTW - I found two issues that looked similar, but they are not quite the same, which is why I opened this one. The issues are:
DetailsQueue: Libraries Test Run release coreclr Linux x64 Debug
|
|
Also hit in this other release/7.0 backport PR: #75603 Queue: Libraries Test Run release coreclr Linux x64 Debug Callstack |
|
Seen again in an unrelated Queue: Libraries Test Run release coreclr Linux x64 Debug Callstack |
|
New hit in this
Callstack |
|
Another hit in a different
Callstack |
|
@rzikm can you please check frequency on 7.0? Is it worth changing there? |
|
I see it affected 7 PRs in the last 30 days, a total of 62 tests have failed with a timeout. cluster('engsrvprod.kusto.windows.net').database('engineeringdata').AzureDevOpsTests
| where TestName startswith "System.Net.Http"
and Outcome == 'Failed' or Outcome == 'PassedOnRerun'
| extend i = indexof_regex(TestName, @"[^.]+$")
| extend Method = substring(TestName, i)
| extend Type = substring(TestName, 0, i - 1)
| project-away i, TestName
| where Message has "TimeoutException"
| distinct JobId, WorkItemId, Message, StackTrace, Method, Type, Arguments, Outcome
| join kind=inner (cluster('engsrvprod.kusto.windows.net').database('engineeringdata').Jobs
| where Finished > now(-30d)
| extend TargetBranch = extractjson("$.['System.PullRequest.TargetBranch']", Properties)
| where (TargetBranch == 'release/7.0')
| where Type startswith "test/functional/cli/" and not(Properties contains "runtime-staging")
| summarize arg_max(Finished, Properties, Type, Branch, Source, Started, QueueName, Attempt, Build, TargetBranch) by JobId
| project-rename JobType = Type) on JobId
| extend PropertiesJson = parse_json(Properties)
| extend OS = replace_regex(tostring(PropertiesJson.operatingSystem), @'\((.*)\).*|([^\(].*)', @'\1\2')
| extend Runtime = iif(PropertiesJson.runtimeFlavor == "mono", "Mono", iif(PropertiesJson.DefinitionName contains "coreclr", "CoreCLR", ""))
| extend Architecture = PropertiesJson.architecture
| extend DefinitionName = PropertiesJson.DefinitionName
| project-away JobId
| extend JobId = extractjson("$.['System.JobId']", Properties)
| join kind=inner (cluster('engsrvprod.kusto.windows.net').database('engineeringdata').WorkItems
| project-keep ConsoleUri, WorkItemId) on WorkItemId
| project-away PropertiesJson
| project-reorder Started, Type, Method, Arguments, StackTrace, ConsoleUri, OS, Architecture, DefinitionName, Branch, TargetBranch, Attempt, Build, *
; |
|
What I'm seeing is:
This might be environmental. |
|
If I remember correctly, With RHEL.9 out, do we still intend to support RedHat 7 with .NET 8.0 or do we plan to stick to the "two major versions back" @richlander? |
|
Triage: if we intend to support RH 7, we should investigate and fix this, if not, this could be closed when the queues get updated. |
2 tasks
|
No recent hits on CI, closing. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Seen in this
release/7.0backport PR: #75373Please help determine if this needs a fix backported to 7.0.
BTW - I found two issues that looked similar, but they are not quite the same, which is why I opened this one. The issues are:
Details
Queue: Libraries Test Run release coreclr Linux x64 Debug
Job results: https://dev.azure.com/dnceng-public/public/_build/results?buildId=13080&view=logs&j=b827109d-f627-5869-7deb-edacb438680e&t=85e93d53-1ed9-516b-1635-3169c07ac90d&l=104
Log file: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-75373-merge-9a35e1b62af84c0492/System.Net.Http.Functional.Tests.Attempt.3/3/console.b9bce8f9.log?helixlogtype=result
Since logs get deleted, here is an output example:
Callstack
The text was updated successfully, but these errors were encountered: