Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Publish dotnet/runtime using standard Arcade Publish tooling #111934

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open

Publish dotnet/runtime using standard Arcade Publish tooling #111934

wants to merge 29 commits into from
+277 −674

Conversation

jkoritzinsky
Copy link
Member

@jkoritzinsky jkoritzinsky commented Jan 28, 2025 •
edited
Loading

Publish the assets from dotnet/runtime using the standard Arcade publish tooling. This tooling doesn't allow any duplicate packages, so change our publishing logic to the following:

  • In a given build, publish all packages that were produced in the current build that contain that job's target RID.
  • For some target RIDs, also publish an additional subset of packages (specified in Publishing.props).
  • For some jobs (specified in runtime-official), publish all packages produced.
  • In the VMR, mark all "packages not to be published" as "Vertical" visibility so they're still available upstack.
  • Outside the VMR, skip publishing all "packages not to be published" packages.

Blocked on MicroBuild fixes for Linux and Mac: dotnet/source-build#4793

Verified in https://dev.azure.com/dnceng/internal/_build/results?buildId=2629649&view=results that all of the same assets that we expect to produce in a runtime official build are produced on this branch.

Fixes #102672
Fixes #1719

@jkoritzinsky
Remove prepare-signed-artifacts and publish from each leg directly.
0364ee2
@jkoritzinsky
Publish in a separate stage to make dependency tracking easy
7e284a4
@jkoritzinsky
Set _SignType so we can pass -sign
d2c9cf8
@jkoritzinsky
Remove old validation target
3338d4d
@jkoritzinsky
!drop remove -sign so we can figure out publishing first
01bca60
@jkoritzinsky
Remove InitialTargets
88ee36c
@jkoritzinsky
Only generate product version files once. Fix asset manifest extensio…
d2fa244 
…n. Don't copy workloads packages as they're already in the right format now.
@jkoritzinsky
Completely restructure how we publish to be more explicit that specif…
f93b7e4 
…ic jobs publish specific packages
@jkoritzinsky
Move "non-RID-specific" special packages logic into Publishing.props
8ee9ac1
@jkoritzinsky
Make sure we build the list of skipped packages after we add the RID-…
4455301 
…specific packages
@jkoritzinsky
Arcade now handles TargetOS/TargetArchitecture for asset manifest nam…
e36e87e 
…es. Use that when possible for unique names and produce our own names when necessary.
@jkoritzinsky
Extract handling hostOS and hostArch into another props file
747bffe
@jkoritzinsky
Fix import and adjust condition for SB
fbe3515
@jkoritzinsky
Fix asset manifest name uniqueness to ensue we don't overwrite asset …
6bbbc4c 
…manifests.
@jkoritzinsky
Set AssetManifestFilePath again
c1068eb
@jkoritzinsky
Fix PublishAllPackages condition and make sure to publish the VS redi…
9e61e42 
…st packages
@jkoritzinsky
Explicitly look up packages as we add zips and other blob artifacts t…
05fe23b 
…o the Artifacts items in Signing.props
@jkoritzinsky
Search in the right folder.
d8b032a
@jkoritzinsky
Look up zips in VSSetup for workloads
2ac88a1
@jkoritzinsky
Add missing semicolon
552bb1d
@jkoritzinsky
Include VSSetup artifacts separately
9247027
@jkoritzinsky
Re-enable signing (depends on Microbuild signing on Linux/Mac)
1eb6a85
@jkoritzinsky
Merge branch 'main' of github.com:dotnet/runtime into standard-arcade…
72af6e0 
…-publish
@jkoritzinsky jkoritzinsky added blocked Issue/PR is blocked on something - see comments area-Infrastructure labels Jan 28, 2025
@build-analysis build-analysis bot mentioned this pull request Jan 29, 2025
Open
ViktorHofer
ViktorHofer reviewed Jan 29, 2025
View reviewed changes
eng/Signing.props Outdated Show resolved Hide resolved
ViktorHofer
ViktorHofer reviewed Jan 29, 2025
View reviewed changes
eng/Publishing.props Outdated Show resolved Hide resolved
ViktorHofer
ViktorHofer reviewed Jan 29, 2025
View reviewed changes
eng/Publishing.props Outdated Show resolved Hide resolved
ViktorHofer
ViktorHofer reviewed Jan 29, 2025
View reviewed changes
eng/Publishing.props Outdated Show resolved Hide resolved
ViktorHofer
ViktorHofer reviewed Jan 29, 2025
View reviewed changes
eng/Publishing.props Outdated Show resolved Hide resolved
This was referenced Jan 29, 2025
Open
Open
@jkoritzinsky jkoritzinsky requested a review from mmitche January 29, 2025 22:55
mmitche
mmitche approved these changes Jan 30, 2025
View reviewed changes
Copy link
Member

@mmitche mmitche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a fantastic change

@build-analysis build-analysis bot mentioned this pull request Jan 30, 2025
Open
ViktorHofer
ViktorHofer reviewed Jan 30, 2025
View reviewed changes
eng/Publishing.props Outdated Show resolved Hide resolved
@jkoritzinsky
In a VMR-style build, produce the product version file in either the …
220a7dc 
…win-x64 leg (for UB) or in all legs (for SB)
ViktorHofer
ViktorHofer reviewed Jan 30, 2025
View reviewed changes
eng/Publishing.props Outdated Show resolved Hide resolved
@jkoritzinsky
Copy link
Member Author

I've verified that signing works for the Mac assets as expected, so we're good to go on that front (once the parallelization issues are fixed)

This was referenced Feb 1, 2025
Open
Open
jkoritzinsky and others added 2 commits February 1, 2025 15:17
@jkoritzinsky
Merge branch 'main' into standard-arcade-publish
19d97f8
@jkoritzinsky @ViktorHofer
Update eng/Publishing.props
aeb2f75 
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
@build-analysis build-analysis bot mentioned this pull request Feb 2, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mmitche mmitche mmitche approved these changes

@ViktorHofer ViktorHofer ViktorHofer approved these changes

@akoeplinger akoeplinger Awaiting requested review from akoeplinger

Assignees

@jkoritzinsky jkoritzinsky

Labels
area-Infrastructure blocked Issue/PR is blocked on something - see comments
Projects
Runtime Infra
Status: No status
Milestone
No milestone
3 participants
@jkoritzinsky @ViktorHofer @mmitche