Use stages pipeline

[dougbu]

• Convert to arcade YAML templates #11924
• change default-build.yml to use job.yml
  • remove unused parameters e.g. poolName and matrix
  • use enableMicrobuild, enablePublishBuildArtifacts etc. to simplify YAML scripts
• change ci.yml to use stages and post-build.yml
  • add use of publish-build-assets.yml and post-build.yml

[dougbu]

This seems to be getting close. But, job names may still be messed up i.e. job.yml seems to use displayName instead of name parameter.

Internal build: https://dev.azure.com/dnceng/internal/_build/results?buildId=305555&view=results

[dougbu]

/cc @mmitche @riarenas

[wtgodbe]

Changes look reasonable, and the internal build seems good other than failing to publish to BAR:

Artifact AssetManifests not found for build 305555. Please ensure you have published artifacts in any previous phases of the current build.

Maybe due to _DotNetValidationArtifactsCategory not matching _DotNetArtifactsCategory?

[dougbu]

Maybe due to _DotNetValidationArtifactsCategory not matching _DotNetArtifactsCategory?

I think this was due to using distinct artifact names for the various jobs pushing manifests.

[dougbu]

New internal build is https://dev.azure.com/dnceng/internal/_build/results?buildId=306146

[dougbu]

A test failed in https://dev.azure.com/dnceng/internal/_build/results?buildId=306146 Will need to wait a bit before we see how this is really working.

[dougbu]

@riarenas @JohnTortugo @mmitche this is changing our artifact layout significantly. I suspect I could merge all of the *_Packages_Signed artifacts with the Windows ones to create a single PackageArtifacts artifact (or perhaps duplicate the two) as I had to do w/ AssetManifests. But, am I just peeling an onion with multiple additional layers? (Less of a concern that I'm breaking our existing release pipeline of course -- that will mostly deal with blobs that don't go into the centralized release.)

Bottom line, does the separation in https://dev.azure.com/dnceng/internal/_build/results?buildId=306146&view=artifacts raise additional red flags? I'd like to finish this in the next iteration.

[dougbu]

For example, all of the *_Installers packages have files with the same name as https://dev.azure.com/dnceng/_apis/resources/Containers/2153204/Linux_arm64_Installers?itemPath=Linux_arm64_Installers%2FRelease%2Faspnetcore_base_runtime.version The one that "wins" probably doesn't matter but artifact publication may hit errors. (If we need to merge those artifacts into BlobArtifacts too.)

[dougbu]

Separately we have a Microsoft.Web.Xdt.Extensions.pdb file in its own Windows_Symbols artifact. Does that need to be moved somewhere to ensure it's uploaded correctly?

The particular file isn't part of a *.symbols.nupkg because the assembly is used exclusively in the AzDO and they get everything through a MSI in our Windows_Installers artifact.

[dougbu]

(@JunTaoLuo on a mostly-unrelated note: Why don't we have a aspnetcore-targeting-pack for macOS?)

[dougbu]

More for @riarenas @JohnTortugo @mmitche: There may be a simpler fix here that I'm missing. What exactly are we doing differently in the Extensions repo?

See e.g. https://dev.azure.com/dnceng/internal/_build/results?buildId=305842&view=artifacts That list includes BlobArtifacts and PackageArtifacts but I don't see where those get published, just where they're consumed. (Our pipeline only explicitly creates artifacts named "artifacts", NpmArtifacts, PdbArtifacts and a few logging odds and ends.)

[dougbu]

Might be getting closer to a resolution here. Found our other repos are passing some or all of the following when in official (aka internal) builds on Windows:

        - ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}:
          # DotNet-Blob-Feed provides: dotnetfeed-storage-access-key-1
          # Publish-Build-Assets provides: MaestroAccessToken, BotAccount-dotnet-maestro-bot-PAT
          - group: DotNet-Blob-Feed
          - group: Publish-Build-Assets
          - name: _OfficialBuildArgs
            value: /p:DotNetSignType=$(_SignType)
                   /p:TeamName=$(_TeamName)
                   /p:DotNetPublishBlobFeedKey=$(dotnetfeed-storage-access-key-1)
                   /p:DotNetPublishBlobFeedUrl=https://dotnetfeed.blob.core.windows.net/aspnet-extensions/index.json
                   /p:DotNetPublishToBlobFeed=$(_DotNetPublishToBlobFeed)
                   /p:DotNetPublishUsingPipelines=$(_PublishUsingPipelines)
                   /p:DotNetArtifactsCategory=$(_DotNetArtifactsCategory)
                   /p:OfficialBuildId=$(BUILD.BUILDNUMBER)

1. This isn't in this PR. To save me the extra builds, is the above enough to get things going?
2. Are the same properties needed when building our Linux and macOS runtime packages and installers?
3. Can any of the above be left out e.g. isn't $(DotNetPublishBlobFeedUrl) derived from $(DotNetArtifactsCategory) (props in the MSBuild sense here 😸)?

---

Ignore a bit of the above cut 'n paste. We're currently passing /pSignType=$(_SignType) /p:TeamName=$(_TeamName) into official builds.

[dougbu]

Threw everything at it in latest commit. New internal build: https://dev.azure.com/dnceng/internal/_build/results?buildId=306939&view=results

[mmitche]

@dougbu I think the errors are coming because the arcade publishing tasks are not running, and thus the manifest is not being generated.

[dougbu]

@JohnTortugo @riarenas could you please make recommendations based on internal build https://dev.azure.com/dnceng/internal/_build/results?buildId=307941 ? I think I'm setting the publish-related properties correctly but

• Some Linux x64 build failed with a ##[error]Bash exited with code '127'. (command not found?) message and not much more information (but some Linux builds worked)
• Other Linux builds (perhaps those using a container?) are failing in Publish.proj due to missing directories
• Code sign jobs are failing to publish, likely because the signed .nupkg files aren't placed in the expected location. Can that expected location be overridden? If not, I'll change what our signing project does

[riarenas]

@JohnTortugo the errors trying to publish to azure devops build artifacts seems similar than what CoreCLR was facing at some point. Do you remember what the solution was in that case?

[dougbu]

Made a number of changes and rebased. Latest internal build is: https://dev.azure.com/dnceng/internal/_build/results?buildId=308884

[joeloff]

Oh, for those, you need to just update them to *.js;*;

[dougbu]

So, the following should work?

apphost.exe;; Exclude the apphost because this is expected to be code-signed by customers after the SDK modifies it.
.js;; We do not sign JavaScript files.
.binlog;; MSBuild binary logs are not signed though they are sometimes placed where validation thinks they should be.
WixUIWixca|WixDepCA;; We do not sign WiX content in our installers.

(I don't see how the apphost.exe exclusion could work given the check passes a fairly-random filename into Exclusions.IsMatch(...).)

[dougbu]

Disabled signing validation for now. Filed #13864 to cover re-enabling these checks.

[dougbu]

Internal build https://dev.azure.com/dnceng/internal/_build/results?buildId=347936 got all the way to the 'Publish Assets' step in '.NET Tools - Validation Publishing' but failed there:

MSBuild auto-detection: using msbuild version '16.200.19.32702' from 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\bin'.
  Accepted https://pkgs.dev.azure.com/dnceng/9ee6d478-d288-47f7-aacc-f6e6d082ae6d/_packaging/d1622942-d16f-48e5-bc83-96f4539e7601/nuget/v2/ 6098ms
Your package was pushed.
  Pushing 0 packages.
D:\a\1\s\.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.19458.2\tools\SdkTasks\PublishArtifactsInManifest.proj(65,5): error : Sequence contains more than one element
D:\a\1\s\.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.19458.2\tools\SdkTasks\PublishArtifactsInManifest.proj(65,5): error :    at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
D:\a\1\s\.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.19458.2\tools\SdkTasks\PublishArtifactsInManifest.proj(65,5): error :    at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source)
D:\a\1\s\.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.19458.2\tools\SdkTasks\PublishArtifactsInManifest.proj(65,5): error :    at Microsoft.DotNet.Build.Tasks.Feed.PublishArtifactsInManifest.PublishBlobsToAzureStorageNugetFeedAsync(List`1 blobsToPublish, IMaestroApi client, Build buildInformation, FeedConfig feedConfig)
D:\a\1\s\.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.19458.2\tools\SdkTasks\PublishArtifactsInManifest.proj(65,5): error :    at Microsoft.DotNet.Build.Tasks.Feed.PublishArtifactsInManifest.HandleBlobPublishingAsync(IMaestroApi client, Build buildInformation) in /_/src/Microsoft.DotNet.Build.Tasks.Feed/src/PublishArtifactsInManifest.cs:line 497
D:\a\1\s\.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.19458.2\tools\SdkTasks\PublishArtifactsInManifest.proj(65,5): error :    at Microsoft.DotNet.Build.Tasks.Feed.PublishArtifactsInManifest.ExecuteAsync() in /_/src/Microsoft.DotNet.Build.Tasks.Feed/src/PublishArtifactsInManifest.cs:line 199
##[error].packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.19458.2\tools\SdkTasks\PublishArtifactsInManifest.proj(65,5): error : Sequence contains more than one element
   at System.Linq.ThrowHelper.ThrowMoreThanOneElementException()
   at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source)
   at Microsoft.DotNet.Build.Tasks.Feed.PublishArtifactsInManifest.PublishBlobsToAzureStorageNugetFeedAsync(List`1 blobsToPublish, IMaestroApi client, Build buildInformation, FeedConfig feedConfig)
   at Microsoft.DotNet.Build.Tasks.Feed.PublishArtifactsInManifest.HandleBlobPublishingAsync(IMaestroApi client, Build buildInformation) in /_/src/Microsoft.DotNet.Build.Tasks.Feed/src/PublishArtifactsInManifest.cs:line 497
   at Microsoft.DotNet.Build.Tasks.Feed.PublishArtifactsInManifest.ExecuteAsync() in /_/src/Microsoft.DotNet.Build.Tasks.Feed/src/PublishArtifactsInManifest.cs:line 199

Build FAILED.

Exactly what does that exception mean? For example is this failure a statement that we need to remove duplication between our manifest files (in contrast to conversations we've had @riarenas @JohnTortugo) ☹️

[JohnTortugo]

I think the problem here was that same blobs (or same blob name) was described in two different manifests. I'll take a closer look in a few mins.

[dougbu]

Thought we were pretty sure the duplication between (say) aspnetcore-Linux_x64.xml and aspnetcore-Linux_x64-signed.xml was fine?

I do see some differences in the <Build /> attributes in those files. Does any of the following matter?

• InitialAssetsLocation is '/_apis/build/builds//artifacts' in aspnetcore-Linux_x64.xml but 'https://dev.azure.com/dnceng/internal/_apis/build/builds/347936/artifacts' in aspnetcore-Linux_x64-signed.xml
• AzureDevOpsProject is empty in aspnetcore-Linux_x64.xml but 'internal' in aspnetcore-Linux_x64-signed.xml
• AzureDevOpsBuildId is empty in aspnetcore-Linux_x64.xml but '347936' in aspnetcore-Linux_x64-signed.xml
• AzureDevOpsBuildDefinitionId is empty in aspnetcore-Linux_x64.xml but '21' in aspnetcore-Linux_x64-signed.xml
• AzureDevOpsAccount is empty in aspnetcore-Linux_x64.xml but 'dncent' in aspnetcore-Linux_x64-signed.xml

(I don't see why there are empty values for any of the above. Thought such values would be set consistently across all build steps.)

[dougbu]

https://dev.azure.com/dnceng/internal/_build/results?buildId=348634&view=results succeeded with the previous commit 🍾 🚀 🍾 Change in last commit won't impact the build itself (just the BAR). So, we're ready to merge once PR validation completes…