Use automatic means to switch to -Svc pool providers

[MattGal]

As part of #10578 , many folks have asked for an automated means to switch over the pool providers to the '-Svc' versions of them when they branch. This would greatly simplify the release process since there are many repos to switch over.

I don't know how this would work since not everyone uses the word "main" for their R&D branch, not everyone uses Arcade build templates, etc. It would be straightforward to add a variables yaml file in arcade's eng/common folder in main and to use it directly in our main arcade-ified repos.

@Chrisboh asked me to file this for discussion this week at triage.

[michellemcdaniel]

We are going to leave this as Needs Triage until next week, at which point we will make a decision on how to fund this work

[chcosta]

I wonder if this relates to https://github.com/orgs/dotnet/projects/86/views/15 and also, have we put any thought into how VMR work may impact the Arcade model. ie, if we're moving away from "Maestro" for dependency flow in repos, we may want to re-evaluate the Arcade eng/common model as well and instead look into Arcade using repository resources

[garath]

@Chrisboh don't forget to add your info here.

[Chrisboh]

I think that if we can do this in an automated fashion we should. We do need to ensure that all the product teams do follow the same process when creating their branches such that this automation is useful.

[MattGal]

PR is merged and Arcade main internal build has completed without issues. I will wait for this build to get promoted, then write some how-to .md stuff in arcade and call it a day.

[MattGal]

Still fighting some fallout from this, hoping #11394 is the last change needed.

[MattGal]

While the changes merged don't hurt anyone, and they make Arcade's (non-PR) builds behave correctly, they are not going to be useful unless I find a mitigation to microsoft/azure-pipelines-yaml#474 . Namely, the values I depend on ( mostly System.PullRequest.TargetBranch) aren't set when template evaluation occurs, and that's when I need it to be set to select the pool provider. I've emailed some folks from that team and depending on their response I may have to abandon this work.

[MattGal]

The changes I made almost work but I didn't realize that System.PullRequest.* variables aren't available at template time. This means providing a general purpose variable that works when the build is a pull request just isn't possible right now.

I ended up creating a feature request to make these variables available to templates here:
https://dev.azure.com/mseng/AzureDevOps/_workitems/edit/2003731

I am putting the issue into tracking/blocked for now, as without the above feature request being implemented there's no good way to provide general purpose switching.

[MattGal]

The code is in Arcade to try to take advantage of automatically switching over to -Svc pool providers, but without the above feature request being completed it isn't actually particularly useful yet. Once the feature is in, this should just "light up" for arcade main and later, if we hit issues then we can address them as FROps issues. As such I'm closing this issue.

[MattGal]

With the advent of the earliest release/8.0 branches starting to clog the alert, I am reopening this issue to have a discussion tomorrow at triage.

The feature request to Azure Devops has had no traction since November and we should proceed as if it is not happening any time soon.

[MattGal]

@akoeplinger may have saved the day with a suggestion, I am trying it in https://github.com/dotnet/arcade/pull/12417/files

[MattGal]

This is going well enough that I've tried to port this to release/7.0 here: #12479

[MattGal]

Checked in on #12345 today. Most repos except for runtime/main are already ported over. dotnet/runtime#82452 should address this.

[MattGal]

This work is done and available. I have changed organizations and will no longer move forward on this; mismatching builds should trigger alerts and be addressed one at a time.