Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Fix maestro publishing for stable packages #9118

Merged
merged 4 commits into from
Jul 18, 2024
Merged

Conversation

pjcollins
Copy link
Member

@pjcollins pjcollins commented Jul 17, 2024

We've seen the build promotion pipeline fail when trying to publish
stable package versions:

error : Package 'Microsoft.iOS.Ref.net8.0_17.5' has stable version '17.5.8001' but is targeted at a non-isolated feed 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8/nuget/v3/index.json'

This is because we were not declaring these packages as stable when
building the build asset registry manifest.

Fix this by passing the $(IsStableBuild) property to the build asset
manifest creation task. This property needs to be updated manually when
switching to stable package versioning (see commit 4ea5dbb).

When $(IsStableBuild) is set to true, packages will be pushed to an
isolated feed during publishing, such as:

  Package Microsoft.Android.Ref.34@34.0.125 (Shipping) should go to https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-android-b8317b6f/nuget/v3/index.json (Isolated, Public)

@pjcollins pjcollins requested a review from jonpryor as a code owner July 17, 2024 20:27
@pjcollins pjcollins requested review from jonathanpeppers and removed request for jonpryor July 17, 2024 20:28
@pjcollins
Copy link
Member Author

pjcollins commented Jul 17, 2024

Tested this by rebuilding the manifest for some .NET 8 stable packs with $(IsStableBuild)=true, and manually trying to add that build to the "General Testing" channel: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=9898256&view=logs&j=ba23343f-f710-5af9-782d-5bd26b102304&t=74531eb2-9b39-5603-839e-94e3ba212b65

This is only relevant for recent .NET 9 publishing changes, and does not need to be backported.

@@ -37,6 +37,7 @@
-->
<AndroidPackVersion>35.0.0</AndroidPackVersion>
<AndroidPackVersionSuffix>preview.7</AndroidPackVersionSuffix>
<IsStableBuild>false</IsStableBuild>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was comparing what we have on the release/8.0.2xx branch:

<!-- NuGet package version numbers. See Documentation/guides/OneDotNet.md.
Rules:
* Major/Minor match Android stable API level, such as 30.0 for API 30.
* Bump first digit of the patch version for feature releases (and reset the first two digits to 0)
-->
<AndroidPackVersion>34.0.1</AndroidPackVersion>
<AndroidPackVersionPatchIndex>$(AndroidPackVersion.LastIndexOf("."))</AndroidPackVersionPatchIndex>
<AndroidPackVersionMajorMinor>$(AndroidPackVersion.Substring(0,$(AndroidPackVersionPatchIndex)))</AndroidPackVersionMajorMinor>
<AndroidPackVersionPatch>$(AndroidPackVersion.Substring($([MSBuild]::Add($(AndroidPackVersionPatchIndex), 1))))</AndroidPackVersionPatch>
<AndroidPackVersionSuffix>rtm</AndroidPackVersionSuffix>

Could this check $(AndroidPackVersionSuffix) == 'rtm' to determine true or false?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds reasonable to me, I think it would be best to not introduce more properties that need to be manually tweaked when stablizing.

I was looking at AndroidPackVersionSuffix and I think it's not used anywhere when moving to stable package versions? I don't think that's an issue, we'll just have to make sure to still update it to rtm each time even though it won't be referenced by anything?

Copy link
Member

@jonathanpeppers jonathanpeppers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's just leave $(AndroidPackVersionSuffix) as-is, if it wasn't doing anything.

@pjcollins pjcollins merged commit 55c2dd8 into main Jul 18, 2024
54 of 58 checks passed
@pjcollins pjcollins deleted the dev/pjc/stablepub branch July 18, 2024 19:17
grendello added a commit that referenced this pull request Jul 23, 2024
* main: (23 commits)
  Localized file check-in by OneLocBuild Task (#9129)
  [ci] Disable CodeQL on CI/PR pipelines (#9128)
  Refine 16k page alignment support (#9075)
  [build] fix `ConfigureLocalWorkload` target (#9124)
  Bump to NDK r27 (#9020)
  [ci] Use drop service for SDK insertion artifacts  (#9116)
  Fix up all mapping paths (#9121)
  [ci] Fix maestro publishing for stable packages (#9118)
  Bump to dotnet/sdk@2f14fea98b 9.0.100-preview.7.24367.21 (#9108)
  Missing androidx.window.[extensions|sidecar] warnings (#9085)
  [ci] Use sign-artifacts template for macOS signing (#9091)
  [ci] Use DotNetCoreCLI to sign macOS files (#9102)
  [ci] Disable CodeQL on macOS, Linux, non-main jobs (#9111)
  [tests] re-enable `JavaAbstractMethodTest` (#9097)
  [Microsoft.Android.Sdk.ILLink] preserve types with `IJniNameProviderAttribute` (#9099)
  [Mono.Android] Data sharing and Close() overrides (#9103)
  [AndroidManifest] Add `Android.App.PropertyAttribute` (#9016)
  [Mono.Android] Add support for AndroidMessageHandler ClientCertificates (#8961)
  [Mono.Android] Bind and enumify API-35 (#9043)
  Bump to dotnet/java-interop@7a058c0e (#9066)
  ...
pjcollins added a commit that referenced this pull request Aug 12, 2024
We've seen the build promotion pipeline fail when trying to publish
stable package versions:

    error : Package 'Microsoft.iOS.Ref.net8.0_17.5' has stable version '17.5.8001' but is targeted at a non-isolated feed 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8/nuget/v3/index.json'

This is because we were not declaring these packages as stable when
building the build asset registry manifest.

Fix this by passing the `$(IsStableBuild)` property to the build asset
manifest creation task. This property needs to be updated manually when
switching to stable package versioning (see commit 4ea5dbb).

When `$(IsStableBuild)` is set to true, packages will be pushed to an
isolated feed during publishing, such as:

      Package Microsoft.Android.Ref.34@34.0.125 (Shipping) should go to https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-android-b8317b6f/nuget/v3/index.json (Isolated, Public)
pjcollins added a commit that referenced this pull request Aug 16, 2024
…ts (#9195)

* [ci] Improve maestro artifact publishing (#8945)

Context: https://github.com/dotnet/arcade/blob/efc3da96e5ac110513e92ebd9ef87c73f44d8540/Documentation/DependencyFlowOnboardingWithoutArcade.md

The steps used to publish build asset information to maestro have been
updated.

With the new `PushToAzureDevOpsArtifacts` task the build pipeline should
now create all of the artifacts required for maestro artifact publishing.
The `add-build-to-channel` darc command will now trigger a
[Build Promotion Pipeline][0] that pushes build assets to the feed that
corresponds to the maestro channel that is being updated.  We should
no longer need to push assets to various NuGet feeds in a separate step.

[0]: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=9577012&view=logs&j=ba23343f-f710-5af9-782d-5bd26b102304&t=c7a8693b-2f9c-5ea8-c909-cde9405ac2e1&l=238

* [ci] Use long version for maestro publishing (#8964)

Commit bbac9fe ran into some issues when attempting to publish to maestro:

    error : Asset 'D:\a\_work\1\a\7dc04dfe-406a-4fa3-aea0-199acc2763fa\MergedManifest.xml' already exists with different contents at assets/manifests/xamarin-xamarin-android/34.99.0-dev/MergedManifest.xml

We should be able to fix this by using the long package version which
optionally includes pre-release labeling and commit distance info.

* [ci] Use drop service for SDK insertion artifacts  (#9116)

Context: xamarin/yaml-templates@8759ec9
Context: xamarin/sdk-insertions#149

Steps to upload release artifacts to custom blob storage have been
replaced with azure-artifacts-drop (aka.ms/drop).

A new version of nuget-msi-convert has been added that will create a set
of artifact drops for the following shipping artifacts:
  * nugets
  * vs-components
  * vs-packs

The nugets drop contains all shipping packages that should be pushed to
various feeds or NuGet.org.

The components and packs drops are used for VS insertions.

* [ci] Fix maestro publishing for stable packages (#9118)

We've seen the build promotion pipeline fail when trying to publish
stable package versions:

    error : Package 'Microsoft.iOS.Ref.net8.0_17.5' has stable version '17.5.8001' but is targeted at a non-isolated feed 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8/nuget/v3/index.json'

This is because we were not declaring these packages as stable when
building the build asset registry manifest.

Fix this by passing the `$(IsStableBuild)` property to the build asset
manifest creation task. This property needs to be updated manually when
switching to stable package versioning (see commit 4ea5dbb).

When `$(IsStableBuild)` is set to true, packages will be pushed to an
isolated feed during publishing, such as:

      Package Microsoft.Android.Ref.34@34.0.125 (Shipping) should go to https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-android-b8317b6f/nuget/v3/index.json (Isolated, Public)

* [ci] Use passwordless auth for darc/maestro (#9182)

Fixes: #9164

Migrates darc/maestro commands to use a passwordless auth flow, as token
based authentication is deprecated and will be removed in the future.

* Use net8.0 version of Microsoft.DotNet.SharedFramework.Sdk

* [ci] Simplify shipping drop metadata names (#9181)

Using the $(System.JobAttempt) variable in the drop metadata artifact
name is problematic. In some cases the drop artifacts created by the
nuget-msi-convert job will be used by a different job, and the job
attempt number will not necessarily match if any jobs are re-ran.
@github-actions github-actions bot locked and limited conversation to collaborators Aug 19, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants