Skip to content

Move package publishing out of Roslyn official build #80009

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
+99 −70
Draft

Move package publishing out of Roslyn official build #80009

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
d7a2238
Only publish to the VS feed for our official build
dibarbet Aug 21, 2025
3e63087
Publish to vs-impl for language server nuget packages
dibarbet Sep 9, 2025
fb7e019
Do not publish packages via arcade from Roslyn's official build
dibarbet Sep 10, 2025
5d5a50a
Update package versions and nuget doc and
dibarbet Sep 12, 2025
8fe2ae1
update boostrap
dibarbet Sep 16, 2025
4f1c3b4
update docs based on feedback
dibarbet Sep 16, 2025
1e4ebe4
feedback
dibarbet Sep 17, 2025
1d4e693
split version
dibarbet Sep 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions azure-pipelines-official.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@ extends:
# Authenticate with service connections to be able to publish packages to external nuget feeds.
- task: NuGetAuthenticate@1
inputs:
nuGetServiceConnections: azure-public/vs-impl, azure-public/vssdk, devdiv/engineering, devdiv/dotnet-core-internal-tooling
nuGetServiceConnections: azure-public/vs-impl, azure-public/vssdk, devdiv/engineering, devdiv/dotnet-core-internal-tooling, DevDiv - VS package feed

# Needed for SBOM tool
- task: UseDotNet@2
Expand Down Expand Up @@ -403,7 +403,7 @@ extends:
componentPassword: $(dn-bot-dnceng-build-e-code-full-release-e-packaging-r)
componentBuildProjectName: internal
sourceBranch: "$(ComponentBranchName)"
publishDataURI: "https://dev.azure.com/dnceng/internal/_apis/git/repositories/dotnet-roslyn/items?path=eng/config/PublishData.json&api-version=6.0"
publishDataURI: "https://dev.azure.com/dnceng/internal/_apis/git/repositories/dotnet-roslyn/items?path=eng/config/PublishData.json&version=$(ComponentBranchName)&api-version=6.0"
publishDataAccessToken: "$(System.AccessToken)"
dropPath: '$(Pipeline.Workspace)\VSSetup'
cherryPick: ${{ parameters.VisualStudioCherryPickSHA }}
Expand Down
2 changes: 1 addition & 1 deletion azure-pipelines-pr-validation.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ extends:
vsBranchName: ${{ parameters.VisualStudioBranchName }}
titlePrefix: ${{ parameters.OptionalTitlePrefix }}
sourceBranch: $(ComponentBranchName)
publishDataURI: "https://raw.githubusercontent.com/dotnet/roslyn/main/eng/config/PublishData.json"
publishDataURI: "https://raw.githubusercontent.com/dotnet/roslyn/$(ComponentBranchName)/eng/config/PublishData.json"
queueSpeedometerValidation: true
dropPath: '$(Pipeline.Workspace)\VSSetup'
retainInsertedBuild: false
Expand Down
35 changes: 35 additions & 0 deletions docs/wiki/NuGet-packages.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,40 @@ This package is primarily intended as a method for rapidly shipping hotfixes to
Note: this package is intended as a replacement for Microsoft.Net.Compilers (which is a Windows-only package) and Microsoft.NETCore.Compilers. Those packages are now deprecated and will be deleted in the future.

## Versioning
Starting on Roslyn versions `5.0` and above, Roslyn NuGet packages are shipped with the .NET SDK instead of VS. New Roslyn packages will be published on every SDK preview and GA release.
Copy link
Member

@jjonescz jjonescz Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The title of the PR says "move package publishing out of Roslyn official builds" but as far as I understand, we already publish Roslyn in the VMR today (resulting in .1xx versions). We also publish it from Roslyn official builds (.x versions). So the PR is more about just removing the latter, right?

Copy link
Member Author

@dibarbet dibarbet Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally yes - and adjusting the version slightly to ensure VMR can produce non-conflicting stable packages.

Copy link
Member

@jasonmalinowski jasonmalinowski Sep 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In practice we've had challenges when we consume new VS/editor packages, since those sometimes aren't available until a VS release, and those can take a few days to get posted. Does moving this to SDK releases make anything more challenging there?

Copy link
Member Author

@dibarbet dibarbet Sep 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and no. We still need to publish manually because we may have to update for new VS dependencies. If VS releases don't align with the SDK and release slower the SDK there is the possibility of us having to wait for a VS release to publish packages. Can happen today for prereleases for new major versions


Prior to this change, a Roslyn package was matched exactly to a VS version, and did not exactly match SDK versions. Now, Roslyn packages will exactly align with an SDK version, but will no longer exactly match a VS version. While generally the SDK releases ship alongside new VS versions, there may be drift in the exact commit of Roslyn when compared to VS (especially for prereleases).
Copy link
Member

@JoeRobich JoeRobich Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Historically this seems to be ok as new feature band SDK releases corresponded with VS minor version releases (see chart). If the VS ship schedule accelerates, we may find ourselves needing to ship new release packages between when new feature band SDKs are released.

Copy link
Member Author

@dibarbet dibarbet Sep 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is possible that will be necessary, but unless the SDK feature band schedule slows down dramatically, I don't necessarily think we need to do that.

The main issue would be if we add or break an API in a new version of VS, to use it there you have to wait for new GA packages. But breaking changes at least should be rare, and new APIs are not that frequent plus someone needs to need the API (and not be referencing old versions of Roslyn to keep things working in old VS/SDKs).

But yes - that could definitely mean we need to GA faster, but I can't work against an unknown.

Copy link
Member

@jaredpar jaredpar Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marcpopMSFT this is an interesting case to think about with the SDK ship strategy. Don't think it changes plans but it is something to keep in mind.

The main issue would be if we add or break an API in a new version of VS, to use it there you have to wait for new GA packages.

If we add a new API then yes it could delay a 3rd party extension taking advantage of that in VS. How often does that happen today though? Usually at least for analyzers there is a bit of a delay in new API adoption because if you adopted it immediately then you would break in 90+% of SDK that didn't have it.

Copy link
Member Author

@dibarbet dibarbet Sep 18, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a table of how often our public API files have changed. It only includes our changes, and not say a VS dependency API upadating requiring our package to update, but hopefully at least gives an idea.

https://gist.github.com/dibarbet/f836c80e03935b65c230440e11104c9d

Most frequent are changes to the compiler public APIs. VS/Workspaces/Features/Editor features do not change frequently.

And agreed, generally targeting a lower version of Roslyn is ideal so you can work in older VS and SDK versions anyway.

Copy link
Member Author

@dibarbet dibarbet Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filtered out anything that hasn't changed this year and also external access, leaves this set

File Path Last Updated Updates Last 3 Months Updates Last 6 Months Avg Updates Per Month
src/CodeStyle/Core/Analyzers/PublicAPI.Unshipped.txt 2025-06-17 0 2 0.31
src/Compilers/Core/Portable/PublicAPI.Shipped.txt 2025-02-20 0 0 0.46
src/Compilers/Core/Portable/PublicAPI.Unshipped.txt 2025-09-17 9 19 9.89
src/Compilers/CSharp/Portable/PublicAPI.Shipped.txt 2025-07-24 1 1 0.34
src/Compilers/CSharp/Portable/PublicAPI.Unshipped.txt 2025-08-21 3 9 4.99
src/Compilers/VisualBasic/Portable/PublicAPI.Shipped.txt 2025-02-20 0 0 0.25
src/Compilers/VisualBasic/Portable/PublicAPI.Unshipped.txt 2025-02-20 0 0 0.93
src/Features/Core/Portable/PublicAPI.Unshipped.txt 2025-04-02 0 1 0.55
src/RoslynAnalyzers/Microsoft.CodeAnalysis.AnalyzerUtilities/PublicAPI.Shipped.txt 2025-02-07 0 0 0
src/RoslynAnalyzers/Microsoft.CodeAnalysis.AnalyzerUtilities/PublicAPI.Unshipped.txt 2025-03-14 0 0 2
src/Workspaces/Core/Portable/PublicAPI.Shipped.txt 2025-02-20 0 0 0.28
src/Workspaces/Core/Portable/PublicAPI.Unshipped.txt 2025-09-10 3 6 2.75
src/Workspaces/MSBuild/BuildHost/PublicAPI.Shipped.txt 2025-01-20 0 0 0
src/Workspaces/MSBuild/BuildHost/PublicAPI.Unshipped.txt 2025-01-20 0 0 0
src/Workspaces/MSBuild/Core/PublicAPI.Shipped.txt 2025-01-20 0 0 0
src/Workspaces/MSBuild/Core/PublicAPI.Unshipped.txt 2025-01-20 0 0 0


### Version Schema
Copy link
Member Author

@dibarbet dibarbet Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL at these version changes. The versions will mostly be the same for stable packages. Preview packages will have build numbers. I've tried to explain the logic below but let me know if it needs more info.

Generally Roslyn package versions are in the form A.B.C-D. With the new publishing mode this will be represented as
* A = Associated with the current VS Major version (`5` = VS 18, same as before).
* B = The current VS Minor version (same as before)
* C = Our own patch version if we need to service the package for any reason, generally `0`.
* D = The VS prerelease version `.` VMR build version (if any).

We still use VS versions in our version scheme as Roslyn currently branches for releases with VS in mind.
Copy link
Member

@333fred 333fred Sep 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm worried about this, given that we don't do the shipping any more. Since SDK ship dates do not line up with VS, I'm concerned that some 5.X.Y version will be shipped as release, but then VS 18.X will release at a later date and contain a breaking change from when the supposed corresponding Roslyn package was released.

Copy link
Member Author

@dibarbet dibarbet Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned that some 5.X.Y version will be shipped as release, but then VS 18.X will release at a later date and contain a breaking change from when the supposed corresponding Roslyn package was released.

Yeah it is definitely possible to have a breaking change ship in a VS releases that isn't present in the SDK we released before (since the SDK ships a bit sooner than VS). But a couple considerations here

  1. The reverse is true today - VS and our packages could ship a breaking change that is not compatible with the most recent SDK. But I don't think we have hit that recently.
  2. There is a limited amount of time where we can ship a breaking change that will not be picked up in the SDK. For example we're snapping for VS 18.0 next week on Monday, at the same time as the SDK for RC2. There's very little time to get a change into VS that wouldn't be in the SDK. Any other breaking change would need to go into the Roslyn release branch which should have extra scrutiny.
  3. As long as we haven't shipped a binary breaking change, VS will still load the VS versions of the packages (like what happens today)

We definitely should be careful about making breaking changes very late into the SDK release though, as it isn't impossible. Need to do our best to make sure VS either won't break or has the change.

Copy link
Member

@333fred 333fred Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reverse is true today - VS and our packages could ship a breaking change that is not compatible with the most recent SDK. But I don't think we have hit that recently.

Sure, but we're not versioned according to SDK releases. We're versioned according to VS releases. IMO, it's a worse break in this direction.

Copy link
Member Author

@dibarbet dibarbet Sep 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but we're not versioned according to SDK releases. We're versioned according to VS releases

This is true - I'd prefer if our package version was tied to the SDK version when it comes out of the VMR. But a couple of thoughts
a. This is something we can always change later if we decide that aligning our version number to the SDK version is important.
b. If we are to do something, I'd prefer to investigate that with the .NET 11 release as it'll give us more time to figure out how we can fit SDK versioning into our VS branch structure (which is also kind of unknown, see below). For 10 GA, we keep a similar version format and VS compatibility story.

IMO, it's a worse break in this direction.

Think it depends on exactly what the break is, but I could see that. But to hit this
a) we have to actually manage to insert a breaking change (/ new API) into VS but not the SDK (as above)
b) be using the tip of Roslyn and not referencing older versions for older VS/SDK compatibility
c) we are unable to service the package in the SDK to resolve the breaking change.

I will say if the VS ship schedule changes to ship faster (and therefore not align with SDK releases), we may have to revisit this somewhat in order to ensure GA packages are available that contain recent enough VS changes. I'm not entirely convinced that would be necessary though (see below). If we do though, it should be possible to do out of the VMR (if we accept minor commit drift).


What this actually looks like:

| SDK Version | 11.0.100-preview.4 | 11.0.100-rc.2 | 11.0.100 | 11.0.200 |
|---|---|---|---|---|
| Roslyn version | 5.6.0-2.25465.10 | 5.8.0-3.25501.82 | 5.9.0 | 5.10.0 |

SDK GA releases align with VS Minor versions. So going from `11.0.100` to `11.0.200` means a new VS Minor version, hence incrementing the Roslyn package minor version.
Copy link
Member

@jasonmalinowski jasonmalinowski Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to sanity check: this is an absolutely guarantee, not just something that has happened to work in practice?

Copy link
Member Author

@dibarbet dibarbet Sep 19, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from what I have been told, yes, feature bands intentionally line up with VS versions. though I cannot say what the future holds


However, SDK prereleases do not always correspond to new VS Minor or VS preview versions. For example, while `11.0.100-preview.4` might produce Roslyn `5.6.0-2.25465.101`, the subsequent `11.0.100-preview.5` could produce Roslyn `5.6.0-2.25503.114` without any change to the VS minor or preview version. In these cases, we need to include the VMR build number to differentiate between packages. This typically occurs with early SDK versions or during extended VS preview cycles (such as preview 1 for a new VS major version).

Additionally, very early in the SDK preview cycle, Roslyn main is consumed by the new SDK preview and the prior SDK feature band. The SDK preview releases will publish the prerelease packages for a specific version, and the feature band release will end up publishing the GA packages for that version.

| SDK Version | Roslyn Package version |
|---|---|
| 11.0.100-preview1 | 5.6.0-2.25465.10 |
| 10.0.200 | 5.6.0 |

By the later previews for .NET 11, the feature bands for 10 will be locked to an older branch of Roslyn associated with an older VS minor version, hence the GA packages for the next SDK will always have a higher minor version than any GA packages for the prior feature band release. See also https://learn.microsoft.com/en-us/dotnet/core/porting/versioning-sdk-msbuild-vs#supported-net-versions

### Past Versions

Below are the versions of the language available in the NuGet packages. Remember to set a specific language version (or "latest") if you want to use one that is newer than "default" (ie. latest major version).

- Versions `1.x` mean C# 6.0 and VB 14 (Visual Studio 2015 and updates). For instance, `1.3.2` corresponds to the most recent update (update 3) of Visual Studio 2015.
Expand Down Expand Up @@ -52,6 +86,7 @@ Below are the versions of the language available in the NuGet packages. Remember
- Version `4.11` includes C# 12.0 (Visual Studio 2022 version 17.11, .NET 8)
- Version `4.12` includes C# 13.0 (Visual Studio 2022 version 17.12, .NET 9)
- Version `4.13` includes C# 13.0 (Visual Studio 2022 version 17.13, .NET 9)
- Version `4.14` includes C# 13.0 (Visual Studio 2022 version 17.14, .NET 9)

See the [history of C# language features](https://github.com/dotnet/csharplang/blob/main/Language-Version-History.md) for more details.

Expand Down
9 changes: 5 additions & 4 deletions eng/Publishing.props
View file
Open in desktop
Copy link
Member

@jjonescz jjonescz Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like there are failures in the linked VMR builds, is that expected?

Copy link
Member Author

@dibarbet dibarbet Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah the errors are unrelated - the windows job passed and created the correct artifacts

Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,16 @@
</ItemGroup>

<!--
During PR Validation we only need to publish symbols with Arcade,
since our packages are published separately to the CoreXT feed.
In the Roslyn official build, we only want to publish symbols with Arcade.
Packages for VS are published separately to the CoreXT feed.
Roslyn official packages are produced by the VMR build.
Also, do not publish any symbols packages in source-only builds.
-->
<Target Name="_ResolvePublishRoslynNuGetPackages"
DependsOnTargets="BeforePublish"
BeforeTargets="PublishToAzureDevOpsArtifacts"
Condition=" '$(PreReleaseVersionLabel)' == 'pr-validation' or '$(DotNetBuildSourceOnly)' == 'true' ">
Condition=" '$(DotNetBuildFromVMR)' != 'true' or '$(DotNetBuildSourceOnly)' == 'true' ">

<ItemGroup>
<!-- Determine all NuGet packages being published -->
Expand All @@ -37,7 +38,7 @@
Condition="$([System.String]::Copy(%(FullPath)).Contains('.Symbols.'))" />
</ItemGroup>

<ItemGroup Condition=" '$(PreReleaseVersionLabel)' == 'pr-validation' ">
<ItemGroup Condition=" '$(DotNetBuildFromVMR)' != 'true' ">
Copy link
Member Author

@dibarbet dibarbet Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disables darc publishing of packages, but keeps publishing symbols.

<!-- Remove all NuGet packages from being published -->
<ItemsToPushToBlobFeed Remove="@(_NuGetPackagesToPush)" />
</ItemGroup>
Expand Down
13 changes: 11 additions & 2 deletions eng/Versions.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,26 @@
Roslyn version
-->
<PropertyGroup>
<!-- All packages produced by the Roslyn official build should be suffixed with '-vs' to distinguish them from VMR produced packages -->
<_RoslynPackageSuffix Condition="'$(DotNetBuildFromVMR)' != 'true'">-vs</_RoslynPackageSuffix>
Comment on lines +9 to +10
Copy link
Member

@jasonmalinowski jasonmalinowski Sep 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And the -vs versions are still what are being inserted into VS? Do we have any cases where other packages are produced in the VS tree, and will end up with dependencies depending on these VS versions?

Copy link
Member Author

@dibarbet dibarbet Sep 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And the -vs versions are still what are being inserted into VS?

Yes

Do we have any cases where other packages are produced in the VS tree, and will end up with dependencies depending on these VS versions?

Not public packages as far as I have been able to tell. If they are, they would switch to reference the VMR produced version.

Copy link
Member

@jasonmalinowski jasonmalinowski Sep 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not public packages as far as I have been able to tell. If they are, they would switch to reference the VMR produced version.

Is that something we somehow need to add validation to ensure? Because I imagine we'll discover that only after some -vs package got published and folks come to us wondering where it is.


<MajorVersion>5</MajorVersion>
<MinorVersion>0</MinorVersion>
<PatchVersion>0</PatchVersion>
<PreReleaseVersionLabel>2</PreReleaseVersionLabel>
<_RoslynPreReleaseVersion>2</_RoslynPreReleaseVersion>
<PreReleaseVersionLabel>$(_RoslynPreReleaseVersion)$(_RoslynPackageSuffix)</PreReleaseVersionLabel>
Comment on lines +9 to +16
Copy link
Member

@jasonmalinowski jasonmalinowski Sep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to adjust the naming and comments here to make it clear this is just the version being suffixed, not the package itself.

<VersionPrefix>$(MajorVersion).$(MinorVersion).$(PatchVersion)</VersionPrefix>
<!--
By default the assembly version in official builds is "$(MajorVersion).$(MinorVersion).0.0".
Keep the setting conditional. The toolset sets the assembly version to 42.42.42.42 if not set explicitly.
-->
<AssemblyVersion Condition="'$(OfficialBuild)' == 'true' or '$(DotNetUseShippingVersions)' == 'true'">$(MajorVersion).$(MinorVersion).0.0</AssemblyVersion>

<!--
Tell the VMR to produce non-final packages for prereleases. This ensures the build number is appended to the package version,
which we require to differentiate between prereleases when they span a single VS preview.
-->
<SuppressFinalPackageVersion Condition="'$(DotNetFinalVersionKind)' == 'prerelease'">true</SuppressFinalPackageVersion>
</PropertyGroup>
<!--
The maintenance-packages dependency versions need to be conditionally selected:
Expand Down Expand Up @@ -115,7 +125,6 @@
<UsingToolVSSDK Condition="$([MSBuild]::IsOSPlatform('Windows'))">true</UsingToolVSSDK>
<UsingToolPdbConverter>false</UsingToolPdbConverter>
<UsingToolSymbolUploader>true</UsingToolSymbolUploader>
<UsingToolNuGetRepack>true</UsingToolNuGetRepack>
Copy link
Member Author

@dibarbet dibarbet Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was creating the 3 copies of all our packages in our official build. we don't need it now that vmr is producig them.

<!-- Avoid attempting IBC on platforms where ngen won't work, or the IBC tooling won't work -->
<UsingToolIbcOptimization Condition="$([MSBuild]::IsOSPlatform('Windows'))">true</UsingToolIbcOptimization>
<UsingToolVisualStudioIbcTraining Condition="$([MSBuild]::IsOSPlatform('Windows'))">true</UsingToolVisualStudioIbcTraining>
Expand Down
17 changes: 9 additions & 8 deletions eng/build-utils.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ $ErrorActionPreference="Stop"

$VSSetupDir = Join-Path $ArtifactsDir "VSSetup\$configuration"
$PackagesDir = Join-Path $ArtifactsDir "packages\$configuration"
$PublishDataUrl = "https://raw.githubusercontent.com/dotnet/roslyn/main/eng/config/PublishData.json"

$binaryLog = if (Test-Path variable:binaryLog) { $binaryLog } else { $false }
$nodeReuse = if (Test-Path variable:nodeReuse) { $nodeReuse } else { $false }
Expand All @@ -22,19 +21,21 @@ function GetProjectOutputBinary([string]$fileName, [string]$projectName = "", [s
return Join-Path $ArtifactsDir "bin\$projectName\$configuration\$tfm\$ridDir$publishDir$fileName"
}

function GetPublishData() {
function GetPublishData([string]$branchName) {
if (Test-Path variable:global:_PublishData) {
return $global:_PublishData
}

Write-Host "Downloading $PublishDataUrl"
$content = (Invoke-WebRequest -Uri $PublishDataUrl -UseBasicParsing).Content
$publishDataFile = Join-Path $PSScriptRoot "config\PublishData.json"

Write-Host "Reading $publishDataFile"
$content = Get-Content -Path $publishDataFile -Raw

return $global:_PublishData = ConvertFrom-Json $content
}

function GetBranchPublishData([string]$branchName) {
$data = GetPublishData
$data = GetPublishData $branchName

if (Get-Member -InputObject $data.branches -Name $branchName) {
return $data.branches.$branchName
Expand All @@ -48,10 +49,10 @@ function GetFeedPublishData() {
return $data.feeds
}

function GetPackagesPublishData([string]$packageFeeds) {
function GetPackageFeedOverrideData() {
$data = GetPublishData
if (Get-Member -InputObject $data.packages -Name $packageFeeds) {
return $data.packages.$packageFeeds
if (Get-Member -InputObject $data.packageFeedOverride) {
return $data.packageFeedOverride
} else {
return $null
}
Expand Down
30 changes: 21 additions & 9 deletions eng/config/PublishData.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,21 @@
"feeds": {
"vssdk": "https://pkgs.dev.azure.com/azure-public/vside/_packaging/vssdk/nuget/v3/index.json",
"vs-impl": "https://pkgs.dev.azure.com/azure-public/vside/_packaging/vs-impl/nuget/v3/index.json",
"vs": "https://pkgs.dev.azure.com/DevDiv/_packaging/VS/nuget/v3/index.json"
"vs": "https://devdiv.pkgs.visualstudio.com/_packaging/VS/nuget/v3/index.json"
},
"packageFeedOverride": {
"Microsoft.CodeAnalysis.LanguageServer.alpine-arm64": "vs-impl",
Copy link
Member Author

@dibarbet dibarbet Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to be able to directly insert the language server into vscode from the official build, so we keep publishing these to vs-impl (can't easily pull from the devdiv/VS feed).

"Microsoft.CodeAnalysis.LanguageServer.alpine-x64": "vs-impl",
"Microsoft.CodeAnalysis.LanguageServer.linux-arm64": "vs-impl",
"Microsoft.CodeAnalysis.LanguageServer.linux-x64": "vs-impl",
"Microsoft.CodeAnalysis.LanguageServer.linux-musl-x64": "vs-impl",
"Microsoft.CodeAnalysis.LanguageServer.linux-musl-arm64": "vs-impl",
"Microsoft.CodeAnalysis.LanguageServer.neutral": "vs-impl",
"Microsoft.CodeAnalysis.LanguageServer.osx-arm64": "vs-impl",
"Microsoft.CodeAnalysis.LanguageServer.osx-x64": "vs-impl",
"Microsoft.CodeAnalysis.LanguageServer.win-arm64": "vs-impl",
"Microsoft.CodeAnalysis.LanguageServer.win-x64": "vs-impl",
"Microsoft.CodeAnalysis.LanguageServer.win-x86": "vs-impl"
},
"packages": {
Copy link
Member Author

@dibarbet dibarbet Sep 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

servicing branches still rely on this existing in main. Post this change, I plan on backporting the part of this change to pull from the current branch, and we can delete this from main.

Copy link
Member Author

@dibarbet dibarbet Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extracting out the branch specific publish data to #80308 so its easier to backport.

"default": {
Expand Down Expand Up @@ -188,22 +202,20 @@
"insertionCreateDraftPR": false
},
"community": {
"nugetKind": [
"Shipping",
"NonShipping"
],
"vsBranch": "main",
"insertionCreateDraftPR": true,
"insertionTitlePrefix": "[Validation]"
},
"main": {
"nugetKind": [
"Shipping",
"NonShipping"
],
"vsBranch": "main",
"insertionCreateDraftPR": false,
"insertionTitlePrefix": "[dNext]"
},
"dev/dibarbet/vs_packages": {
"vsBranch": "main",
"insertionCreateDraftPR": true,
"insertionTitlePrefix": "[Validation]",
"feed": "vs"
}
}
}
2 changes: 1 addition & 1 deletion eng/pipelines/build-bootstrap.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ steps:
- task: PublishBuildArtifacts@1
displayName: Publish Artifact Packages
inputs:
PathtoPublish: '$(Build.SourcesDirectory)\artifacts\packages\Release\PreRelease'
PathtoPublish: '$(Build.SourcesDirectory)\artifacts\packages\Release'
ArtifactName: 'Bootstrap Packages - ${{parameters.toolset}}'
publishLocation: Container

Expand Down
57 changes: 14 additions & 43 deletions eng/publish-assets.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ function Get-PublishKey([string]$uploadUrl) {
switch ($url.Host) {
"api.nuget.org" { return $nugetApiKey }
# For publishing to azure, the API key can be any non-empty string as authentication is done in the pipeline.
"devdiv.pkgs.visualstudio.com" { return "DevDivAzureArtifacts" }
"pkgs.dev.azure.com" { return "AzureArtifacts"}
default { throw "Cannot determine publish key for $uploadUrl" }
}
Expand All @@ -38,27 +39,14 @@ function Publish-Nuget($publishData, [string]$packageDir) {
# Retrieve the feed name to source mapping.
$feedData = GetFeedPublishData

# Let packageFeeds default to the default set of feeds
$packageFeeds = "default"
if ($publishData.PSobject.Properties.Name -contains "packageFeeds") {
$packageFeeds = $publishData.packageFeeds
}

# If the configured packageFeeds is arcade, then skip publishing here. Arcade will handle publishing packages to their feeds.
if ($packageFeeds.equals("arcade") -and -not $prValidation) {
Write-Host " Skipping publishing for all packages as they will be published by arcade"
continue
}
# The default feed we publish to for our official build is the VS feed.
$feedName = "vs"

# Let packageFeeds default to the default set of feeds
$packageFeeds = "default"
if ($publishData.PSobject.Properties.Name -contains "packageFeeds") {
$packageFeeds = $publishData.packageFeeds
if ($publishData.PSobject.Properties.Name -contains "feed") {
$feedName = $publishData.feed
}

# Each branch stores the name of the package to feed map it should use.
# Retrieve the correct map for this particular branch.
$packagesData = GetPackagesPublishData $packageFeeds
$packageOverrideData = GetPackageFeedOverrideData

foreach ($package in Get-ChildItem *.nupkg) {
Write-Host ""
Expand All @@ -80,23 +68,18 @@ function Publish-Nuget($publishData, [string]$packageDir) {
continue
}

# Lookup the feed name from the packages map using the package name without the version or extension.
if (-not (Get-Member -InputObject $packagesData -Name $nupkgWithoutVersion)) {
throw "$nupkg has no configured feed (looked for $nupkgWithoutVersion)"
# Check if the specific package has a feed override.
if (Get-Member -InputObject $packageOverrideData -Name $nupkgWithoutVersion) {
$feedName = $packageOverrideData.$nupkgWithoutVersion
Write-Host "Using package feed override $feedName for $nupkg"
}

$feedName = $packagesData.$nupkgWithoutVersion

# If we're doing PR validation, we always publish to the VS feed as it runs in the DevDiv AzDo instance
# and does not have permissions to publish anywhere else.
if ($prValidation) {
$feedName = "vs"
}

# If the configured feed is arcade, then skip publishing here. Arcade will handle publishing to their feeds.
if ($feedName.equals("arcade")) {
Write-Host "Skipping publishing for $nupkg as it is published by arcade"
continue
}

# Use the feed name to get the source to upload the package to.
if (-not (Get-Member -InputObject $feedData -Name $feedName)) {
throw "$feedName has no configured source feed"
Expand All @@ -116,23 +99,11 @@ function Publish-Nuget($publishData, [string]$packageDir) {
}
}

# Do basic verification on the values provided in the publish configuration
function Test-Entry($publishData, [switch]$isBranch) {
if ($isBranch) {
foreach ($nugetKind in $publishData.nugetKind) {
if ($nugetKind -ne "PerBuildPreRelease" -and $nugetKind -ne "Shipping" -and $nugetKind -ne "NonShipping") {
throw "Branches are only allowed to publish Shipping, NonShipping, or PerBuildPreRelease"
}
}
}
}

# Publish a given entry: branch or release.
function Publish-Entry($publishData, [switch]$isBranch) {
Test-Entry $publishData -isBranch:$isBranch

# First publish the NuGet packages to the specified feeds
foreach ($nugetKind in $publishData.nugetKind) {
$nugetKinds = @('Shipping', 'NonShipping')
foreach ($nugetKind in $nugetKinds) {
Publish-NuGet $publishData (Join-Path $PackagesDir $nugetKind)
}

Expand Down