Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nuget restore error caused by 503s #11239

Closed
2 tasks
ulisesh opened this issue Oct 11, 2022 · 5 comments
Closed
2 tasks

Nuget restore error caused by 503s #11239

ulisesh opened this issue Oct 11, 2022 · 5 comments

Comments

@ulisesh
Copy link
Contributor

ulisesh commented Oct 11, 2022

Build

https://dev.azure.com/dnceng-public/public/_build/results?buildId=48584

Build leg reported

Build_Unix_Debug

Pull Request

dotnet/roslyn#64624

Action required for the engineering services team

To triage this issue (First Responder / @dotnet/dnceng):

  • Open the failing build above and investigate
  • Add a comment explaining your findings

If this is an issue that is causing build breaks across multiple builds and would get benefit from being listed on the build analysis check, follow the next steps:

  1. Add the label "Known Build Error"
  2. Edit this issue and add an error string in the Json below that can help us match this issue with future build breaks. You should use the known issues documentation
{
   "ErrorMessage" : "Response status code does not indicate success: 503 (Service Unavailable)",
   "BuildRetry": false
}

Additional information about the issue reported

No response

Report

Build Definition Step Name Console log Pull Request
58486 dotnet/roslyn Build and Test Log dotnet/roslyn#63092
58351 dotnet/runtime Build product Log dotnet/runtime#77169
58208 dotnet/runtime Docker NanoServer Log
58173 dotnet/sdk Windows_NT_FullFramework Build_Debug Log dotnet/sdk#28679
58163 dotnet/sdk Windows_NT_FullFramework Build_Debug Log dotnet/sdk#28678
58153 dotnet/sdk Build Log dotnet/sdk#28677
58120 dotnet/runtime Libraries Build OSX arm64 Release Log
58110 dotnet/runtime Build iOSSimulator x64 Release AllSubsets_Mono Log dotnet/runtime#77260
58086 dotnet/runtime Build product Log dotnet/runtime#77220
58065 dotnet/roslyn Build and Test Log
57954 dotnet/installer Build Log dotnet/installer#14679
57977 dotnet/runtime Build windows x64 debug NativeAOT Log dotnet/runtime#76483
57466 dotnet/runtime coreclr Pri0 Runtime Tests Run Linux arm checked Log dotnet/runtime#77196
57887 dotnet/roslyn Build and Test Log
57877 dotnet/roslyn Build and Test Log dotnet/roslyn#64263
57869 dotnet/runtime Build Linux arm64 Release NativeAOT Log dotnet/runtime#62689
57830 dotnet/sdk Build Log
57749 dotnet/runtime Build and generate native prerequisites Log dotnet/runtime#72627
57730 dotnet/sdk Build Log dotnet/sdk#28520
57729 dotnet/sdk Build Log dotnet/sdk#28519
57717 dotnet/roslyn Build and Test Log dotnet/roslyn#64843
57651 dotnet/runtime Build windows x64 checked NativeAOT Log dotnet/runtime#74963
57616 dotnet/runtime CoreCLR Product Build Linux_musl arm checked Log
57597 dotnet/sdk Build Log
57542 dotnet/sdk Build Log dotnet/sdk#28520
57541 dotnet/sdk Build Log dotnet/sdk#28519
52883 dotnet/sdk Windows_NT Build_Release Log dotnet/sdk#27566
57266 dotnet/aspnetcore Build shared fx Log dotnet/aspnetcore#44516
57248 dotnet/runtime Build Browser wasm Release AllSubsets_Mono_RuntimeTests Log dotnet/runtime#77194
57213 dotnet/roslyn Build and Test Log dotnet/roslyn#64192
57099 dotnet/runtime coreclr Pri1 Runtime Tests Run OSX arm64 checked Log dotnet/runtime#77176
57126 dotnet/runtime Build product Log dotnet/runtime#76874
56999 dotnet/runtime Libraries Build OSX arm64 Release Log
56886 dotnet/runtime mono minijit Pri0 Runtime Tests Run OSX x64 release Log dotnet/runtime#77169
56804 dotnet/sdk Windows_NT_FullFramework Build_Debug Log
56771 dotnet/roslyn Build and Test Log
56768 dotnet/runtime Build OSX x64 release Runtime_Release Log dotnet/runtime#77197
56756 dotnet/sdk Build Log dotnet/sdk#28520
56755 dotnet/sdk Build Log dotnet/sdk#28519
56688 dotnet/sdk Build Log dotnet/sdk#28519
56615 dotnet/roslyn Build and Test Log
56606 dotnet/roslyn Build and Test Log dotnet/roslyn#64802
56533 dotnet/sdk Build Log dotnet/sdk#28519
56485 dotnet/roslyn Build and Test Log
56185 dotnet/roslyn Correctness_Build_Artifacts Log dotnet/roslyn#64616
56143 dotnet/sdk Build Log
56101 dotnet/runtime Restore and Build Product Log dotnet/runtime#77149
55939 dotnet/runtime coreclr Pri1 Runtime Tests Run windows arm64 checked Log dotnet/runtime#77103
55937 dotnet/sdk Ubuntu_22_04 Build_Release Log dotnet/sdk#28260
55729 dotnet/runtime coreclr Pri1 Runtime Tests Run windows arm checked Log
54686 dotnet/sdk Windows_NT_FullFramework Build_Release Log dotnet/sdk#28620
55652 dotnet/runtime Restore and Build Product Log
55643 dotnet/roslyn Build and Test Log
55639 dotnet/roslyn Build and Test Log
55623 dotnet/runtime Build managed product components and packages Log dotnet/runtime#77149
55624 dotnet/runtime Build product Log dotnet/runtime#77149
55570 dotnet/runtime Build Browser wasm release Browser_wasm_Windows Log dotnet/runtime#77117
55572 dotnet/roslyn Build and Test Log
55555 dotnet/roslyn Build and Test Log
55432 dotnet/aspnetcore Run build.cmd Log dotnet/aspnetcore#44516
55402 dotnet/roslyn Build and Test Log
55243 dotnet/runtime Installer Build and Test coreclr OSX_x64 Release Log dotnet/runtime#76981
55195 dotnet/runtime Build product Log dotnet/runtime#77127
54860 dotnet/runtime Libraries Test Run release coreclr windows x86 Release Log dotnet/runtime#77030
54866 dotnet/sdk Windows_NT Build_Release Log
54834 dotnet/arcade Linux Build_Debug Log
54838 dotnet/roslyn Build and Test Log dotnet/roslyn#64767
54775 dotnet/runtime Build native test components Log dotnet/runtime#69613
54805 dotnet/runtime Restore and Build Product Log dotnet/runtime#77117
54808 dotnet/runtime Build product Log dotnet/runtime#77117
54750 dotnet/runtime mono monointerpreter Pri0 Runtime Tests Run OSX x64 release Log dotnet/runtime#77043
54486 dotnet/runtime coreclr Pri1 Runtime Tests Run OSX arm64 checked Log
54480 dotnet/roslyn Build - Validate determinism Log
54457 dotnet/runtime Build windows x64 checked NativeAOT Log dotnet/runtime#77103
54416 dotnet/runtime Installer Build and Test coreclr OSX_x64 Release Log dotnet/runtime#74157
54421 dotnet/roslyn Build and Test Log
54415 dotnet/roslyn Build and Test Log
52803 dotnet/sdk Windows_NT Build_Release Log dotnet/sdk#28591
54290 dotnet/roslyn Build and Test Log
54122 dotnet/runtime Build managed test components Log
54123 dotnet/runtime Build product Log
54101 dotnet/aspnetcore Run build.sh Log dotnet/aspnetcore#44573
54047 dotnet/runtime Libraries Build Linux_musl x64 Release Log
54022 dotnet/roslyn Build and Test Log
53798 dotnet/runtime Libraries Test Run checked coreclr windows arm64 Release Log
53666 dotnet/sdk Build Log dotnet/sdk#28584
53608 dotnet/runtime Libraries Build Linux x64 Release Log
53606 dotnet/runtime Libraries Build OSX x64 Release Log
53485 dotnet/sdk Build Log dotnet/sdk#28520
53484 dotnet/sdk Build Log dotnet/sdk#28519
53477 dotnet/runtime Build product Log dotnet/runtime#77072
53479 dotnet/runtime Build product Log dotnet/runtime#77072
53478 dotnet/runtime Build product Log dotnet/runtime#77072
53476 dotnet/roslyn Correctness_Rebuild Log dotnet/roslyn#64751
53468 dotnet/aspnetcore Tests: Helix x64 Log dotnet/aspnetcore#44562
53467 dotnet/sdk Windows_NT_FullFramework Build_Release Log dotnet/sdk#28583
53438 dotnet/roslyn Build and Test Log dotnet/roslyn#64750
53406 dotnet/roslyn Build and Test Log
53403 dotnet/roslyn Build and Test Log dotnet/roslyn#64749
53396 dotnet/runtime Build product Log dotnet/runtime#77069
Displaying 100 of 421 results
Build Definition Test Pull Request
58173 dotnet/sdk Microsoft.NET.Build.Tests.dll.11.WorkItemExecution dotnet/sdk#28679
57030 dotnet/performance Partition0.WorkItemExecution
54686 dotnet/sdk Microsoft.NET.Publish.Tests.dll.1.WorkItemExecution dotnet/sdk#28620
53272 dotnet/sdk Microsoft.NET.Build.Tests.dll.10.WorkItemExecution dotnet/sdk#28519
50943 dotnet/sdk Microsoft.NET.Publish.Tests.GivenThatWeWantToPublishAnAotApp.NativeAot_hw_runs_with_cross_PackageReference_PublishAot_is_empty dotnet/sdk#28495
50978 dotnet/installer ItCanCreateAndBuildTemplatesWithDefaultFramework_DisableBuildOnLinuxMusl(templateName: "grpc") dotnet/installer#14746
50964 dotnet/installer Microsoft.NET.Build.Tests.GivenThatWeWantToBuildANetCoreApp.BuildWithTransitiveReferenceToNetCoreAppPackage
50310 dotnet/sdk Microsoft.NET.Build.Tests.dll.4.WorkItemExecution dotnet/sdk#28501
50329 dotnet/performance SDK NUnit 3 Test Project Template Clean Build.WorkItemExecution dotnet/performance#2652
50295 dotnet/sdk Microsoft.NET.Build.Tests.GivenThatWeWantToResolveConflicts.CompileConflictsAreNotRemovedFromRuntimeDepsAssets dotnet/sdk#28517
50043 dotnet/sdk Microsoft.NET.Restore.Tests.GivenThatWeWantToRestoreProjectsUsingNuGetConfigProperties.I_can_restore_a_project_with_implicit_msbuild_nuget_config dotnet/sdk#28512
50067 dotnet/sdk Microsoft.NET.Restore.Tests.dll.WorkItemExecution
50002 dotnet/runtime LibraryImportGenerator.UnitTests.Compiles.ValidateRequireAllowUnsafeBlocksDiagnosticNoTrigger dotnet/runtime#74314
49633 dotnet/sdk Microsoft.NET.Publish.Tests.dll.4.WorkItemExecution
49620 dotnet/sdk Microsoft.NET.Build.Tests.dll.5.WorkItemExecution dotnet/sdk#28505
49671 dotnet/performance WorkItem.WorkItemExecution dotnet/performance#2650
49555 dotnet/runtime LibraryImportGenerator.UnitTests.Compiles.ValidateRequireAllowUnsafeBlocksDiagnosticNoTrigger dotnet/runtime#76361
49247 dotnet/sdk Microsoft.NET.Restore.Tests.dll.WorkItemExecution
49245 dotnet/sdk dotnet.Tests.dll.3.WorkItemExecution
49314 dotnet/arcade-services RolloutScorer.Tests.RolloutScorerTests.RedirectApiTest("same-host-redirect","https://dev.azure.com/dnceng/redirected","Response status code does not indicate success: 401") dotnet/arcade-services#2054
49121 dotnet/sdk dotnet.Tests.dll.3.WorkItemExecution dotnet/sdk#28405
49114 dotnet/sdk Microsoft.NET.Restore.Tests.dll.WorkItemExecution
49141 dotnet/aspnetcore Templates.Tests--net7.0.WorkItemExecution dotnet/aspnetcore#44486
49116 dotnet/sdk Microsoft.NET.Build.Tests.GivenThatWeWantToBuildACrossTargetedLibrary.It_builds_nondesktop_library_successfully_on_all_platforms
49055 dotnet/runtime Microsoft.NET.HostModel.Tests.BundleLegacy.TestNetCoreApp3xApp(minorVersion: 1) dotnet/runtime#75597

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
23 157 446
@ilyas1974
Copy link
Contributor

We are engaged with AzDO to address this issue.

@MattGal
Copy link
Member

MattGal commented Oct 12, 2022

AzDO IcM about the ongoing throttling issues: https://portal.microsofticm.com/imp/v3/incidents/details/340753328/home

@dominoFire
Copy link

Thanks for the update.

@MattGal
Copy link
Member

MattGal commented Oct 13, 2022

Settings have been migrated from the old scale unit and additional cores have been deployed. Per the thread:

"Overnight looked real good compared to the last couple days. Continuing to monitor for today, but feeling good."

@ulisesh
Copy link
Contributor Author

ulisesh commented Oct 20, 2022

As expected, using the new Response status code does not indicate success: 503 (Service Unavailable) as error message makes build analysis match build failures that are not restore issues. I checked many of the latest matches and all of them were unrelated failures. I'll close the issue to stop mismatching failures.

@ulisesh ulisesh closed this as completed Oct 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants