Backport maestro and artifact drop infra improvements from net9.0 (#2…

…4247)

* [ci] Set OfficialBuildId when building asset manifests (#23102)

Context: 812807c

The maestro build promotion step has been failing with:

    PublishArtifactsInManifest.proj(130,5): error : Asset 'D:\a\_work\1\a\3fecf55a-18fb-414d-b980-84c0f56a3856\MergedManifest.xml' already exists with different contents at 'https://dotnetbuilds.blob.core.windows.net/public/assets/manifests/dotnet-maui/9.0.0-ci-dev/MergedManifest.xml'

The manifest version information passed to the `PublishBuildAssets.proj`
build does not contain revision information, causing asset publishing
to fail as it will not overwrite existing assets with the same version.

We should be able to fix this by setting the `OfficialBuildId` property
when building the asset manifests, similar to what is currently being
done when [packing the NuGets][0].

[0]: https://github.com/dotnet/maui/blob/5d3e788f297098417f6c603e5187fb24a37dda63/eng/cake/dotnet.cake#L285

* [ci] Use drop service for SDK insertion artifacts (#23658)

Context: xamarin/yaml-templates@8759ec9

Steps to upload release artifacts to custom blob storage have been
replaced with azure-artifacts-drop (aka.ms/drop).

A new version of nuget-msi-convert has been added that will create a set
of artifact drops for the following shipping artifacts:
  * nugets
  * vs-components
  * vs-packs

The nugets drop contains all shipping packages that should be pushed to
various feeds or NuGet.org.

The components and packs drops are used for VS insertions.

* [ci] Simplify shipping drop metadata names (#24103)

Using the $(System.JobAttempt) variable in the drop metadata artifact
name is problematic. In some cases the drop artifacts created by the
nuget-msi-convert job will be used by a different job, and the job
attempt number will not necessarily match if any jobs are re-ran.

* [ci] Use passwordless auth for darc/maestro (#24220)

Fixes: #23974

Migrates darc/maestro commands to use a passwordless auth flow, as token
based authentication is deprecated and will be removed in the future.

* Update bar manifest build step

* Update bar manifest build working dir

* Use new task name

* Run add build to channel outside sources dir

eng/Version.Details.xml

@@ -149,9 +149,9 @@
     </Dependency>
   </ProductDependencies>
   <ToolsetDependencies>
-    <Dependency Name="Microsoft.DotNet.Build.Tasks.Feed" Version="9.0.0-beta.24421.5">
+    <Dependency Name="Microsoft.DotNet.Build.Tasks.Feed" Version="9.0.0-beta.24408.2">
       <Uri>https://github.com/dotnet/arcade</Uri>
-      <Sha>a3756ed1e8c8a6098ebbff50ba93b1a3fd58e79d</Sha>
+      <Sha>60ae233c3d77f11c5fdb53e570b64d503b13ba59</Sha>
     </Dependency>
     <Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="9.0.0-beta.24421.5">
       <Uri>https://github.com/dotnet/arcade</Uri>

eng/Versions.props

@@ -133,7 +133,7 @@
     <TizenUIExtensionsVersion>0.9.0</TizenUIExtensionsVersion>
     <ExCSSPackageVersion>4.2.3</ExCSSPackageVersion>
     <SystemDrawingCommonPackageVersion>8.0.3</SystemDrawingCommonPackageVersion>
-    <MicrosoftDotNetBuildTasksFeedVersion>9.0.0-beta.24421.5</MicrosoftDotNetBuildTasksFeedVersion>
+    <MicrosoftDotNetBuildTasksFeedVersion>9.0.0-beta.24408.2</MicrosoftDotNetBuildTasksFeedVersion>
   </PropertyGroup>
   <PropertyGroup>
     <MicrosoftNETTestSdkPackageVersion>17.6.0</MicrosoftNETTestSdkPackageVersion>

eng/pipelines/common/sdk-insertion.yml

@@ -57,11 +57,10 @@ jobs:
       buildNumber: $(ReleaseDropPrefix)/nugets
       destinationPath: ${{ parameters.nugetArtifactPath }}
 
-  - task: UseDotNet@2                 # https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/tool/dotnet-core-tool-installer?view=azure-devops
-    displayName: 'Use .NET SDK $(DOTNET_VERSION)'
+  - task: UseDotNet@2
+    displayName: Install .NET 9.x
     inputs:
-      packageType: sdk
-      version: $(DOTNET_VERSION)
+      version: 9.x
       includePreviewVersions: true
 
   - task: AzureCLI@2
@@ -71,11 +70,13 @@ jobs:
       scriptType: ps
       scriptLocation: inlineScript
       inlineScript: >-
-        dotnet build $(Build.SourcesDirectory)\src\Workload\Microsoft.Maui.Sdk\Microsoft.Maui.Sdk.csproj
+        & dotnet build $(Build.SourcesDirectory)\src\Workload\Microsoft.Maui.Sdk\Microsoft.Maui.Sdk.csproj
         -t:PushManifestToBuildAssetRegistry
         -p:OfficialBuildId=$(_BuildOfficalId)
         -p:OutputPath=${{ parameters.nugetArtifactPath }}
         -v:n -bl:$(Build.StagingDirectory)\binlogs\push-bar-manifest.binlog
+    # Execute outside of the source directory to avoid any conflicting global.json file
+      workingDirectory: $(Build.SourcesDirectory)\..
     condition: and(succeeded(), eq('${{ parameters.pushMauiPackagesToMaestro }}', 'true'))
 
   - task: AzureCLI@2
@@ -89,5 +90,7 @@ jobs:
         $arcadeServicesSource = 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json'
         & dotnet tool update microsoft.dotnet.darc --version "$darcVersion" --add-source "$arcadeServicesSource" --tool-path $(Agent.ToolsDirectory)\darc -v n
         & $(Agent.ToolsDirectory)\darc\darc add-build-to-channel --default-channels --id $(BARBuildId) --ci --publishing-infra-version 3 --azdev-pat $(System.AccessToken)
+    # Execute outside of the source directory to avoid any conflicting global.json file
+      workingDirectory: $(Build.SourcesDirectory)\..
     displayName: Add build to default darc channel
     condition: and(succeeded(), eq('${{ parameters.pushMauiPackagesToMaestro }}', 'true'))