Official build fails due to missing windows_arm job

[sbomer]

Official builds have started failing with:

workloadsbuild depends on unknown job installer__coreclr__windows_arm_Release_.

See for example https://dev.azure.com/dnceng/internal/_build/results?buildId=2178029&view=results. I believe caused by #85947. @BruceForstall

Probably we need to at least remove the arm dependencies from

runtime/eng/pipelines/runtime-official.yml

Line 527 in 11e22ce

- installer__coreclr__windows_arm_Release_

and

runtime/eng/pipelines/mono/templates/workloads-build.yml

Line 75 in 11e22ce

IntermediateArtifacts/windows_arm/Shipping/Microsoft.NETCore.App.Runtime.win-arm*.nupkg

[sbomer]

windows_arm was added to the workloads build in https://github.com/dotnet/runtime/pull/68981/files. Probably they should also be removed from the WorkloadManifest.json.in.

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Official builds have started failing with:

workloadsbuild depends on unknown job installer__coreclr__windows_arm_Release_.

See for example https://dev.azure.com/dnceng/internal/_build/results?buildId=2178029&view=results. I believe caused by #85947. @BruceForstall

Probably we need to at least remove the arm dependencies from

runtime/eng/pipelines/runtime-official.yml

Line 527 in 11e22ce

- installer__coreclr__windows_arm_Release_

and

runtime/eng/pipelines/mono/templates/workloads-build.yml

Line 75 in 11e22ce

IntermediateArtifacts/windows_arm/Shipping/Microsoft.NETCore.App.Runtime.win-arm*.nupkg

Author:	sbomer
Assignees:	BruceForstall
Labels:	area-Infrastructure-coreclr, untriaged, in-pr, needs-area-label
Milestone:	-

[BruceForstall]

Looks like the official build is failing in the Build crossdac publishing step:

D:\a\_work\1\s\.packages\microsoft.dotnet.build.tasks.packaging\8.0.0-beta.23259.5\build\Packaging.targets(1264,5): error : Error when creating nuget lib package from D:\a\_work\1\s\artifacts\packages\Release\specs\runtime.linux-musl-arm.Microsoft.CrossOsDiag.Private.CoreCLR.nuspec. NuGet.Packaging.Core.PackagingException: File not found: 'D:\a\_work\1\s\artifacts\CoreCLRCrossDacArtifacts\CoreCLRCrossDacArtifacts\linux_musl.arm.Release\x86\mscordaccore.pdb'. [D:\a\_work\1\s\src\coreclr\.nuget\Microsoft.CrossOsDiag.Private.CoreCLR\Microsoft.CrossOsDiag.Private.CoreCLR.pkgproj]
D:\a\_work\1\s\.packages\microsoft.dotnet.build.tasks.packaging\8.0.0-beta.23259.5\build\Packaging.targets(1264,5): error :    at NuGet.Packaging.PackageBuilder.AddFiles(String basePath, String source, String destination, String exclude) [D:\a\_work\1\s\src\coreclr\.nuget\Microsoft.CrossOsDiag.Private.CoreCLR\Microsoft.CrossOsDiag.Private.CoreCLR.pkgproj]
D:\a\_work\1\s\.packages\microsoft.dotnet.build.tasks.packaging\8.0.0-beta.23259.5\build\Packaging.targets(1264,5): error :    at NuGet.Packaging.PackageBuilder.PopulateFiles(String basePath, IEnumerable`1 files) [D:\a\_work\1\s\src\coreclr\.nuget\Microsoft.CrossOsDiag.Private.CoreCLR\Microsoft.CrossOsDiag.Private.CoreCLR.pkgproj]
D:\a\_work\1\s\.packages\microsoft.dotnet.build.tasks.packaging\8.0.0-beta.23259.5\build\Packaging.targets(1264,5): error :    at Microsoft.DotNet.Build.Tasks.Packaging.NuGetPack.Pack(String nuspecPath, String nupkgPath, Manifest manifest, Boolean packSymbols) in /_/src/Microsoft.DotNet.Build.Tasks.Packaging/src/NuGetPack.cs:line 261 [D:\a\_work\1\s\src\coreclr\.nuget\Microsoft.CrossOsDiag.Private.CoreCLR\Microsoft.CrossOsDiag.Private.CoreCLR.pkgproj]
D:\a\_work\1\s\.packages\microsoft.dotnet.build.tasks.packaging\8.0.0-beta.23259.5\build\Packaging.targets(1264,5): error : Error when creating nuget lib package from D:\a\_work\1\s\artifacts\packages\Release\specs\runtime.linux-arm.Microsoft.CrossOsDiag.Private.CoreCLR.nuspec. NuGet.Packaging.Core.PackagingException: File not found: 'D:\a\_work\1\s\artifacts\CoreCLRCrossDacArtifacts\CoreCLRCrossDacArtifacts\linux.arm.Release\x86\mscordaccore.pdb'. [D:\a\_work\1\s\src\coreclr\.nuget\Microsoft.CrossOsDiag.Private.CoreCLR\Microsoft.CrossOsDiag.Private.CoreCLR.pkgproj]
D:\a\_work\1\s\.packages\microsoft.dotnet.build.tasks.packaging\8.0.0-beta.23259.5\build\Packaging.targets(1264,5): error :    at NuGet.Packaging.PackageBuilder.AddFiles(String basePath, String source, String destination, String exclude) [D:\a\_work\1\s\src\coreclr\.nuget\Microsoft.CrossOsDiag.Private.CoreCLR\Microsoft.CrossOsDiag.Private.CoreCLR.pkgproj]
D:\a\_work\1\s\.packages\microsoft.dotnet.build.tasks.packaging\8.0.0-beta.23259.5\build\Packaging.targets(1264,5): error :    at NuGet.Packaging.PackageBuilder.PopulateFiles(String basePath, IEnumerable`1 files) [D:\a\_work\1\s\src\coreclr\.nuget\Microsoft.CrossOsDiag.Private.CoreCLR\Microsoft.CrossOsDiag.Private.CoreCLR.pkgproj]
D:\a\_work\1\s\.packages\microsoft.dotnet.build.tasks.packaging\8.0.0-beta.23259.5\build\Packaging.targets(1264,5): error :    at Microsoft.DotNet.Build.Tasks.Packaging.NuGetPack.Pack(String nuspecPath, String nupkgPath, Manifest manifest, Boolean packSymbols) in /_/src/Microsoft.DotNet.Build.Tasks.Packaging/src/NuGetPack.cs:line 261 [D:\a\_work\1\s\src\coreclr\.nuget\Microsoft.CrossOsDiag.Private.CoreCLR\Microsoft.CrossOsDiag.Private.CoreCLR.pkgproj]
##[error].packages\microsoft.dotnet.build.tasks.packaging\8.0.0-beta.23259.5\build\Packaging.targets(1264,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Error when creating nuget lib package from D:\a\_work\1\s\artifacts\packages\Release\specs\runtime.linux-musl-arm.Microsoft.CrossOsDiag.Private.CoreCLR.nuspec. NuGet.Packaging.Core.PackagingException: File not found: 'D:\a\_work\1\s\artifacts\CoreCLRCrossDacArtifacts\CoreCLRCrossDacArtifacts\linux_musl.arm.Release\x86\mscordaccore.pdb'.

@sbomer @jkotas @jkoritzinsky Anyone know how this works? @jkotas Maybe the linux-arm cross-dac build was inadvertently removed?

[jkoritzinsky]

I think we built the Linux arm crossdac components on the Windows arm leg. We can either add the commands to just build it to the crossdac job directly or bring back the build jobs just for the cross-dac builds. Either option should work.

[BruceForstall]

@noahfalk @tommcdon @mikem8361 Is the linux arm crossdac owned by you? We've been removing Windows arm (builds and code) and it appears linux-arm crossdac builds were a casualty. How do we bring it back without bringing back Windows arm builds?

[mikem8361]

@hoyosjs would know how.

[hoyosjs]

The linux crossbuild was done from the arm leg as the crosscomponents built the x86 -> arm DAC and DBI. Maybe we could fold that into the x86 leg (Do we have a windows jit targetting linux-arm? Is it built there?).

[BruceForstall]

(Do we have a windows jit targetting linux-arm? Is it built there?)

The win-x86 build creates clrjit_universal_arm_x86.dll, which is a JIT that runs on win-x86 and targets any arm32 OS.

The win-x64 build creates clrjit_universal_arm_x64.dll, which is a JIT that runs on win-x64 and targets any arm32 OS.

(Note these are intended to be used when running crossgen2)

[hoyosjs]

We don't support win-arm targeting linux-arm, so folding the arm crossdac to the linux build is completely OK to me.

[BruceForstall]

Where and how is that build done? What needs to change?

[ericstj]

@hoyosjs is the expert, but it might be here:

runtime/eng/pipelines/coreclr/templates/crossdac-build.yml

Lines 11 to 15 in 3bd3832

	- script: $(Build.SourcesDirectory)/src/coreclr/build-runtime$(scriptExt) $(buildConfig) ${{ parameters.archType }} -hostarch ${{ parameters.hostArchType }} -ci -os linux -cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1" -ninja $(officialBuildIdArg) -component crosscomponents
	displayName: Build Cross OS Linux DAC for Windows
	- script: $(Build.SourcesDirectory)/src/coreclr/build-runtime$(scriptExt) $(buildConfig) ${{ parameters.archType }} -hostarch ${{ parameters.hostArchType }} -ci -os alpine -cmakeargs "-DCLR_CROSS_COMPONENTS_BUILD=1" -ninja $(officialBuildIdArg) -component crosscomponents
	displayName: Build Cross OS Linux-musl DAC for Windows

Looks like that's invoked once for the same archType as the outer build. Maybe we need to add an additional call for whatever arch we want to use to add the arm build to? Probably lower on down needs a an update to the copy step in that case too.

Or maybe it's easier to just do it in the caller:

runtime/eng/pipelines/coreclr/templates/build-job.yml

Lines 279 to 288 in 3bd3832

	- template: /eng/pipelines/coreclr/templates/crossdac-build.yml
	parameters:
	archType: ${{ parameters.archType }}
	osGroup: ${{ parameters.osGroup }}
	osSubgroup: ${{ parameters.osSubgroup }}
	isOfficialBuild: ${{ parameters.signBinaries }}
	${{ if eq(parameters.archType, 'arm') }}:
	hostArchType: x86
	${{ else }}:
	hostArchType: x64

[BruceForstall]

I'm inclined to revert #85947, #86074, #86065 to fix the build.

My goal was to stop running tests in the CI on win-arm; I don't want to spend time pulling all the threads to completely remove win-arm support across all repos.

[jkotas]

#86138 should fix the official build

[BruceForstall]

Hopefully there aren't upstream repos (installer? sdk?) that still expect to consume win-arm bits from runtime.

[jkotas]

We got a green official build an hour ago

[ericstj]

Hopefully there aren't upstream repos (installer? sdk?) that still expect to consume win-arm bits from runtime.

That's probably a good thing to check before bringing this change back. You can also trigger an official build for this change by pushing it to AzDo and scheduling there.