Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport maestro and artifact drop infra improvements from net9.0 #24247

Merged
merged 8 commits into from
Aug 20, 2024

Conversation

pjcollins
Copy link
Member

@pjcollins pjcollins commented Aug 14, 2024

Backports the following maestro and artifact storage changes,
removing dependencies on bosstoragemirror and adding support
for passwordless maestro auth.

5b6db39
f88260e
e4486e6
b9d711a

Context: 812807c

The maestro build promotion step has been failing with:

    PublishArtifactsInManifest.proj(130,5): error : Asset 'D:\a\_work\1\a\3fecf55a-18fb-414d-b980-84c0f56a3856\MergedManifest.xml' already exists with different contents at 'https://dotnetbuilds.blob.core.windows.net/public/assets/manifests/dotnet-maui/9.0.0-ci-dev/MergedManifest.xml'

The manifest version information passed to the `PublishBuildAssets.proj`
build does not contain revision information, causing asset publishing
to fail as it will not overwrite existing assets with the same version.

We should be able to fix this by setting the `OfficialBuildId` property
when building the asset manifests, similar to what is currently being
done when [packing the NuGets][0].

[0]: https://github.com/dotnet/maui/blob/5d3e788f297098417f6c603e5187fb24a37dda63/eng/cake/dotnet.cake#L285
Context: xamarin/yaml-templates@8759ec9

Steps to upload release artifacts to custom blob storage have been
replaced with azure-artifacts-drop (aka.ms/drop).

A new version of nuget-msi-convert has been added that will create a set
of artifact drops for the following shipping artifacts:
  * nugets
  * vs-components
  * vs-packs

The nugets drop contains all shipping packages that should be pushed to
various feeds or NuGet.org.

The components and packs drops are used for VS insertions.
Using the $(System.JobAttempt) variable in the drop metadata artifact
name is problematic. In some cases the drop artifacts created by the
nuget-msi-convert job will be used by a different job, and the job
attempt number will not necessarily match if any jobs are re-ran.
Fixes: #23974

Migrates darc/maestro commands to use a passwordless auth flow, as token
based authentication is deprecated and will be removed in the future.
@pjcollins pjcollins requested a review from a team as a code owner August 14, 2024 19:38
@pjcollins pjcollins added the area-infrastructure CI, Maestro / Coherency, upstream dependencies/versions label Aug 14, 2024
@pjcollins pjcollins changed the title [release/8.x] Backport maestro and artifact drop infra improvements Backport maestro and artifact drop infra improvements from net9.0 Aug 15, 2024
@rmarinho rmarinho removed the request for review from StephaneDelcroix August 19, 2024 08:52
@rmarinho
Copy link
Member

C:\Windows\system32\cmd.exe /D /S /C ""C:\Program Files\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" account set --subscription 68672ab8-de0c-40f1-8d1b-ffb20bd62c0f"
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\a\_work\_temp\azureclitaskscript1724063377626.ps1'"
The command could not be loaded, possibly because:
  * You intended to execute a .NET application:
      The application 'build' does not exist.
  * You intended to execute a .NET SDK command:
      No .NET SDKs were found.

Download a .NET SDK:
https://aka.ms/dotnet/download

@rmarinho rmarinho merged commit b57d664 into main Aug 20, 2024
99 checks passed
@rmarinho rmarinho deleted the dev/pjc/rel8-artifacts-drop branch August 20, 2024 15:17
@rmarinho
Copy link
Member

/backport to release/8.0.1xx-sr8

Copy link
Contributor

Started backporting to release/8.0.1xx-sr8: https://github.com/dotnet/maui/actions/runs/10508590445

mattleibow added a commit that referenced this pull request Aug 22, 2024
Redth pushed a commit that referenced this pull request Aug 22, 2024
…4247)

* [ci] Set OfficialBuildId when building asset manifests (#23102)

Context: 812807c

The maestro build promotion step has been failing with:

    PublishArtifactsInManifest.proj(130,5): error : Asset 'D:\a\_work\1\a\3fecf55a-18fb-414d-b980-84c0f56a3856\MergedManifest.xml' already exists with different contents at 'https://dotnetbuilds.blob.core.windows.net/public/assets/manifests/dotnet-maui/9.0.0-ci-dev/MergedManifest.xml'

The manifest version information passed to the `PublishBuildAssets.proj`
build does not contain revision information, causing asset publishing
to fail as it will not overwrite existing assets with the same version.

We should be able to fix this by setting the `OfficialBuildId` property
when building the asset manifests, similar to what is currently being
done when [packing the NuGets][0].

[0]: https://github.com/dotnet/maui/blob/5d3e788f297098417f6c603e5187fb24a37dda63/eng/cake/dotnet.cake#L285

* [ci] Use drop service for SDK insertion artifacts (#23658)

Context: xamarin/yaml-templates@8759ec9

Steps to upload release artifacts to custom blob storage have been
replaced with azure-artifacts-drop (aka.ms/drop).

A new version of nuget-msi-convert has been added that will create a set
of artifact drops for the following shipping artifacts:
  * nugets
  * vs-components
  * vs-packs

The nugets drop contains all shipping packages that should be pushed to
various feeds or NuGet.org.

The components and packs drops are used for VS insertions.

* [ci] Simplify shipping drop metadata names (#24103)

Using the $(System.JobAttempt) variable in the drop metadata artifact
name is problematic. In some cases the drop artifacts created by the
nuget-msi-convert job will be used by a different job, and the job
attempt number will not necessarily match if any jobs are re-ran.

* [ci] Use passwordless auth for darc/maestro (#24220)

Fixes: #23974

Migrates darc/maestro commands to use a passwordless auth flow, as token
based authentication is deprecated and will be removed in the future.

* Update bar manifest build step

* Update bar manifest build working dir

* Use new task name

* Run add build to channel outside sources dir
@samhouts samhouts added fixed-in-net9.0-nightly This may be available in a nightly release! fixed-in-net8.0-nightly This may be available in a nightly release! labels Aug 27, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Oct 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-infrastructure CI, Maestro / Coherency, upstream dependencies/versions fixed-in-net8.0-nightly This may be available in a nightly release! fixed-in-net9.0-nightly This may be available in a nightly release!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants