Help teams move servicing builds to -Svc NetCore1ES pool providers

[MattGal]

As a push to relieve some pain from release builds (and, indirectly, main builds) I've opened the following PRs to release branches which are not using "-Svc-" pool providers.

let builds=
BuildTimelineRecord
| where OrganizationName == "dnceng"
| where Type == "Job"
| where ProjectId == "9ee6d478-d288-47f7-aacc-f6e6d082ae6d"
| where WorkerName startswith "NetCore1ESPool-Public"
| distinct BuildId;
Build 
| where OrganizationName == "dnceng"
| where FinishTime > ago(7d)
| where BuildId  in (builds)
| where SourceBranch contains "release"
| summarize arg_max(BuildId,*) by RepositoryId

PRs

Interactive:

• release/jupyter: Update all usages of NetCore* Pool providers to -Svc version interactive#2305

Arcade:

• 6.0: Use svc pool provider in all release/6.0 builds #10572 (pending)
• 3.x: Not needed

AspNetCore:

• release/7.0-rc1 : Move to -Svc pool provider (7.0 RC1) aspnetcore#43528 (Closed in favor of Move to -Svc pool provider (release/7.0) aspnetcore#43529)
• release/7.0 : Move to -Svc pool provider (release/7.0) aspnetcore#43529 (merged)

EFCore:

• release/7.0 : Move to -Svc pool provider in release branches efcore#28873
• release/7.0-rc1 : Move to -Svc pool provider in release branches (7.0 RC1) efcore#28874

dotnet/fsharp :

• release/dev17.4 : Use svc pool to build release/dev17.4 fsharp#13779

Installer:

• release/7.0.1xx-rc1 : not needed
• release/7.0.1xx : Use -svc pool provider in release 7.0.1xx branches installer#14392
• release/6.0.4xx : not needed
• release/6.0.1xx : not needed

dotnet/llvm-project

• dotnet/release/7.0 : Move to -Svc pool provider in release branches llvm-project#255

dotnet/roslyn :

• release/dev17.4 : Move to -Svc pool provider in release branches roslyn#63584
• release/dev17.3 : Move to -Svc pool provider in release branches roslyn#63585
• release/dev17.0-vs-deps : Already done

dotnet/runtime:

• 7.0-rc1: [release/7.0-rc1] Move all build pools to -Svc equivalents runtime#74521

dotnet/sdk

• release/7.0.1xx-rc1 : Move non-eng-common builds to NetCore1ESPool-Svc-* version sdk#27425
• release/7.0.1xx : Move 7.0.1xx (sans eng/common) to -Svc build pools sdk#27426
• release/6.0.4xx : not needed
• release/6.0.3xx : not needed
• release/6.0.1xx : not needed

dotnet/templating

• release/7.0.1xx-rc1 : Move to -Svc pool provider in release branches templating#5154
• release/7.0.1xx : Move to -Svc pool provider in release branches templating#5155
• release/6.0.4xx : already done
• release/6.0.3xx : already done

dotnet/windowsdesktop

• release/7.0-preview7 : Move to -Svc pool provider in release branches windowsdesktop#3123

dotnet/winforms

• release/7.0 : Move to -Svc pool provider in release branches winforms#7659
• release/7.0-rc1 : Move to -Svc pool provider in release branch (7.0-rc1) winforms#7660

dotnet/wpf:

• release/7.0 : Use svc pool for release/7.0 wpf#6947
• release/7.0-rc1 : Move to -Svc pool provider in release branches (7.0 RC1) wpf#6948

Release Note Description

Internal alerting changes - Does not need release notes

[dagood]

As a push to relieve some pain from release builds

I currently happen to be working on a change that will make microsoft/go builds consistently (automatically) use the Svc pool in release branches, but I'm curious what this means. Is it internal pain, or will something be different on the repo side?

(Maybe https://github.com/dotnet/arcade/blob/main/Documentation/AzureDevOps/AzureDevOpsOnboarding.md#agent-queues could be updated to include the "why"?)

[MattGal]

@dagood All I meant by pain there is that the builds trying to go through the "normal" public pool are bottlenecking while the -Svc pool, while having fewer maximum machines, is significantly less busy. Lots of folks trying to get out 7.0 RC1 builds were hitting this.

[dagood]

I see, thanks. 😄 Wasn't aware Svc vs. nonSvc pools actually had different sets of machines--makes sense.

[akoeplinger]

I agree we should use the Svc pool automatically for all release/* branches, maybe we can add some common variable .yml to eng/common so repos can just use that rather than copying the logic around. Same for the public/internal logic.

[MattGal]

Checked in today, dotnet/fsharp#13807 was the only new change to make. All the others I saw were either branches that were being deleted being run scheduled, or repos' Source Build tasks (need arcade updates to change).

[MattGal]

Found some more. New PRs for moving to SVC made today:

dotnet/emsdk#210
dotnet/llvm-project#274
dotnet/msquic#112
https://dnceng.visualstudio.com/internal/_git/dotnet-wpf-int/pullrequest/25664

I've also made sure that the changes are in arcade release/7.0 and all active subscriptions have been triggered.

[MattGal]

Getting slowly better all the time. What remains today:

• AspNetCore has not taken rel/7.0 arcade update due to changed compiler warnings ([release/7.0] Update dependencies from dotnet/arcade aspnetcore#43880)
• FSharp needs to update arcade in the release/dev17.4 branch
• Various small repos still need arcade rel/7.0 updates

[MattGal]

• dotnet/source-build-reference-packages - release/7.0 - Update Arcade with latest .NET 7 Eng channel version  source-build-reference-packages#456
• dotnet-source-build-externals refs/heads/release/7.0 Move to -Svc pools source-build-externals#53
• dotnet-dotnet-monitor release/7.x Move release-branch build over to -Svc pools  dotnet-monitor#2619

[MattGal]

Update:

dnceng-public public dotnet/fsharp release/dev17.4
dnceng-public public dotnet/fsharp release/dev17.5
dnceng-public public dotnet/roslyn release/dev17.4

These branches use arcade main - for free -Svc pool provider usage, recommended to move to release/7.0 - I will start work on making arcade switch automatically on SourceBranch, and reach out to these teams letting them know it'd be nice to move to a release branch of arcade.

dnceng-public public dotnet/templating release/7.0.2xx
dotnet/templating#5476

dnceng-public public dotnet/sdk release/6.0.1xx
dotnet/sdk#28638

dnceng-public public dotnet/sdk release/6.0.3xx
dotnet/sdk#28641

[MattGal]

Today's updates:

dotnet-sdk

Branch: release/6.0.1xx
PR: dotnet/sdk#28744

Branch: release/6.0.3xx
PR: dotnet/sdk#28745

dotnet-dotnet-docker

Branch: internal/release/nightly
PR: https://dnceng.visualstudio.com/internal/_git/dotnet-dotnet-docker/pullrequest/27027

dotnet-roslyn

Branch: release/dev17.2
PR: dotnet/roslyn#64950 (rest will be hit by arcade update)

dotnet-fsharp

Branch: release/dev17.5
PR: dotnet/fsharp#14168 (sent to main)

[MattGal]

Some more today:

• dotnet-interactive refs/heads/release/rename 2031916 Update arcade and use -Svc pools for release/rename branch interactive#2409
• dotnet-windowsdesktop refs/heads/release/7.0 2031787 - Use -Svc build pools  windowsdesktop#3285
• dnceng-public dotnet/Scaffolding refs/heads/release/7.0 - Use "-Svc" build pools in release branches (release/7.0) Scaffolding#2068 (merged)
• dnceng-public dotnet/Scaffolding refs/heads/release/6.0 - Use "-Svc" build pools in release branches (release/6.0) Scaffolding#2067 (merged)

[MattGal]

Yet more:

• dotnet-machinelearning refs/heads/release/2.0 Use -Svc pool providers in release/* branches for billing purposes machinelearning#6434
• dotnet-Scaffolding refs/heads/release/7.0 Use "-Svc" build pools in release branches (release/7.0) Scaffolding#2068 (already merged)
• dotnet-sdk refs/heads/internal/release/6.0.1xx Update Arcade to Use svc providers - release/6.0.1xx branch sdk#28744
• dotnet-templating refs/heads/internal/release/3.1 Use -svc pools for release/ branches templating#5563
• dotnet-toolset refs/heads/internal/release/3.1.4xx Use -Svc pool provider in any release/ branch toolset#5195
• microsoft-reverse-proxy refs/heads/release/latest https://github.com/microsoft/reverse-proxy/pull/1922/files

[MattGal]

I am reducing the number of repos we monitor via https://dev.azure.com/dnceng/internal/_git/dotnet-helix-service/pullrequest/27501?_a=files which should still catch the vast majority of (non-Roslyn) builds that get sent to the wrong place. After reducing the query, we should be green now and covered by the alerts.