Abstractions Package - Pipeline Changes #3628

paulmedynski · 2025-09-23T12:53:40Z

Overview

As part of the Azure split work, a new Abstractions package is being created that contains types shared between MDS and its future extensions (Azure will be the first). This PR contains the YAML pipeline changes for CI and Official builds.

This supercedes the original PRs #3471 and #3567.

This depends on PR #3626.

Issues

The first step towards #1108.

Testing

Manual inspection of all artifacts produced by the pipelines.

paulmedynski · 2025-09-25T10:31:46Z

/azp run

azure-pipelines · 2025-09-25T10:31:51Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

codecov · 2025-09-25T14:01:28Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (feat/azure-split@c96942d). Learn more about missing BASE report.

Additional details and impacted files

@@                 Coverage Diff                 @@
##             feat/azure-split    #3628   +/-   ##
===================================================
  Coverage                    ?   90.82%           
===================================================
  Files                       ?        6           
  Lines                       ?      316           
  Branches                    ?        0           
===================================================
  Hits                        ?      287           
  Misses                      ?       29           
  Partials                    ?        0

Flag	Coverage Δ
addons	`90.82% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

paulmedynski

Commentary for reviewers, and some actions for myself.

paulmedynski · 2025-09-25T19:14:45Z

eng/pipelines/common/templates/jobs/build-signed-package-job.yml

  variables:
  - template: ../../../libraries/variables.yml@self
  - ${{ if parameters.isPreview }}:
-    - name: NugetPackageVersion


The identifier NugetPackageVersion is ambiguous, so I replaced it with a name specific to the NuGet package (i.e. Abstractions, MDS, SqlServer, AKV, etc).

I also eschewed using $(variables) deep in nested templates, since it's difficult to know where they are defined and/or set. I added template parameters to replace a lot of them. Some deep $(variable) use remains though.

Thank you! The variable use at such a deep nesting level was one of my biggest complaints about how the pipelines were written before.

paulmedynski · 2025-09-25T19:18:53Z

eng/pipelines/common/templates/jobs/ci-build-nugets-job.yml

      configuration: ${{ parameters.configuration }}
      operatingSystem: Windows
      build: all
+      abstractionsPackageVersion: ${{parameters.abstractionsPackageVersion}}


I didn't bother making these Abstractions package version parameters conditional on Project vs Package reference type. IMO those two CI pipelines are almost different enough that sharing templates may be more trouble than it's worth. We will do better with the fancy new pipelines.

paulmedynski · 2025-09-25T19:19:51Z

eng/pipelines/common/templates/jobs/ci-build-nugets-job.yml


  - template: ../steps/generate-nuget-package-step.yml@self
    parameters:
-      NugetPackageVersion: $(NugetPackageVersion)


MDS and AKV package versions are now independent in CI builds.

paulmedynski · 2025-09-25T19:20:29Z

eng/pipelines/common/templates/jobs/ci-build-nugets-job.yml


-  - task: PublishBuildArtifacts@1
-    displayName: 'Publish Artifact: Artifacts'
+  - task: PublishPipelineArtifact@1


PublishPipelineArtifact is the preferred way to publish things.

paulmedynski · 2025-09-25T19:21:21Z

eng/pipelines/common/templates/jobs/ci-run-tests-job.yml

 # See the LICENSE file in the project root for more information.                #
 #################################################################################
 parameters:
+  - name: abstractionsArtifactName


I tried to alphabetically order some template parameters, but decided it wasn't worth it given how many files I have touched. So you will see a few like this that have been re-ordered before I gave up.

paulmedynski · 2025-09-25T19:34:27Z

eng/pipelines/libraries/ci-build-variables.yml

  - name: SQLTarget
    value: 'localhost'
-  - name: NugetPackageVersion
-    value: $(Major).$(Minor)$(Patch)-pull.1$(buildnumber)


I don't know why we were prepending 1 to the build number here.

paulmedynski · 2025-09-25T19:36:57Z

eng/pipelines/libraries/common-variables.yml

-  - name: AssemblyFileVersion
-    value: '$(Major).$(Minor)$(Patch).$(Build.BuildNumber)'
+  - name: mdsAssemblyFileVersion
+    value: $(Major).$(Minor).$(Patch).$(assemblyBuildNumber)


Note the tweak here. We used to concatenate the Minor and Patch numbers together because $(Build.BuildNumber) already has a . in it, and C# assembly file versions must not have more than 4 parts. Now we strip the second part (the attempt # ?) from the build number, since we can always find the correct build from just the main build number.

paulmedynski · 2025-09-25T19:38:16Z

eng/pipelines/steps/compound-build-akv-step.yml

          msbuildArguments: >-
              -t:BuildAkv
              -p:AssemblyFileVersion=${{ parameters.assemblyFileVersion }}
-              -p:NugetPackageVersion=${{ parameters.mdsPackageVersion }}


@paulmedynski Verify that this change is correct - it may not be...

paulmedynski · 2025-09-25T19:38:39Z

eng/pipelines/steps/roslyn-analyzers-akv-step.yml

              $(REPO_ROOT)/build.proj
              -t:BuildAkv
              -p:Configuration=${{ parameters.buildConfiguration }}
-              -p:NugetPackageVersion=${{ parameters.mdsPackageVersion }}


@paulmedynski Same here - is this change correct?

paulmedynski · 2025-09-25T19:41:00Z

eng/pipelines/dotnet-sqlclient-ci-core.yml

            # ways to detect if they were present) won't run consistently across forks and non-forks.
            ${{ if eq(variables['system.pullRequest.isFork'], 'False') }}:
              AADPasswordConnectionString: $(AAD_PASSWORD_CONN_STR)
-            AADServicePrincipalId: $(AADServicePrincipalId)


Re-arranged to avoid multiple identical if-statements.

paulmedynski · 2025-09-26T18:38:29Z

/azp run

azure-pipelines · 2025-09-26T18:38:36Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

paulmedynski · 2025-09-29T13:00:09Z

/azp run

azure-pipelines · 2025-09-29T13:00:27Z

Azure Pipelines successfully started running 1 pipeline(s).

paulmedynski · 2025-09-29T13:01:23Z

/azp run

azure-pipelines · 2025-09-29T13:01:40Z

Azure Pipelines successfully started running 2 pipeline(s).

paulmedynski

Commentary for reviewers.

paulmedynski · 2025-09-30T10:59:18Z

eng/pipelines/common/templates/jobs/ci-build-nugets-job.yml

+        targetPath: $(packagePath)
+    # Note that packages/ will have been created by the above step, which is a
+    # pre-requisite for configuring NuGet.
+    - template: ../steps/ci-prebuild-step.yml@self


It turns out that CI-SqlClient-Package runs weren't building the driver in Package mode. They were building the driver in Project mode, and then building the tests in Package mode. I fixed that to use Package mode everywhere. This helps find problems with Package mode before they hit dotnet-sqlclient-official. You will see referenceType parameters added here and in downstream templates.

paulmedynski · 2025-09-30T10:59:58Z

eng/pipelines/common/templates/jobs/ci-build-nugets-job.yml

+      referenceType: ${{ parameters.referenceType }}
      operatingSystem: Windows
-      build: all
+      build: MDS


We now build/package MDS separately from AKV, since AKV needs an MDS package in orderr to build in Package mode.

paulmedynski · 2025-09-30T11:01:17Z

eng/pipelines/common/templates/jobs/build-signed-package-job.yml

    parameters:
      analyzeType: all

+  - template: ../steps/build-all-configurations-signed-dlls-step.yml@self


The analysis step clobbers the versioned DLLs from the build step, so I re-ordered those steps.

@paulmedynski - Verify that the Roslyn analysis results aren't clobbered by the build.

paulmedynski · 2025-09-30T11:02:32Z

eng/pipelines/common/templates/jobs/validate-signed-package-job.yml

+          Exit -1
+      }
+
+      Get-ChildItem *.dll -Path $(extractedNugetPath)  -Recurse | ForEach-Object VersionInfo | Format-List


This prints a detailed list of each DLL and its version information.

paulmedynski · 2025-09-30T11:03:30Z

eng/pipelines/common/templates/steps/ci-project-build-step.yml

      packageType: 'sdk'
      version: '9.0'

- template: ./ensure-dotnet-version.yml@self


Nothing here uses .NET 8 runtime.

paulmedynski · 2025-09-30T11:04:09Z

eng/pipelines/common/templates/steps/copy-dlls-for-test-step.yml


-    Get-ChildItem -recurse "$software\*.dll"
-    Get-ChildItem -recurse "$symbols\*.pdb"
+    Get-ChildItem -recurse "$software\*.dll" | ForEach-Object VersionInfo | Format-List


Full version information for all DLLs we build.

paulmedynski · 2025-09-30T12:53:16Z

build.proj

    <RemoveDir Directories='$([System.IO.Directory]::GetDirectories(".", "bin", SearchOption.AllDirectories))' />
    <RemoveDir Directories='$([System.IO.Directory]::GetDirectories(".", ".nuget", SearchOption.AllDirectories))' />
    <RemoveDir Directories='$([System.IO.Directory]::GetDirectories(".", "obj", SearchOption.AllDirectories))' />
-    <RemoveDir Directories='$([System.IO.Directory]::GetDirectories(".", "packages", SearchOption.AllDirectories))' />


CI runs that use the MSBuild@1 task perform an implicit clean, which would delete the pipeline artifacts we download. So I removed packages/ from clean.

- Fixed expansion of split expression in version numbers. - Removed addition of PDBs from <AllowedOutputExtensionsInPackageBuildOutputFolder>. - Removed MDS package ref dependency on Abstractions until pipelines are ready. - Renamed AbstractionsPackage to Abstractions in targets. - Updated BUILDGUIDE based on project ref behaviour. - Added feature branches to CI pipeline triggers. - Added missing/incomplete paths to the trigger. - Added dev/* branches to the CI triggers so PRs that target other dev/ branches can run CI. - Added missing MdsPackageVersion property to signed build pipeline job. - Commented-out failing NuGet tool installer task. - Re-ordered Guardian analysis step _before_ build step to avoid clobbering versioned DLLs. - Added MDS package version to AKV build/package steps. - Restored AKV nuspec ReferenceType property for AKV Official builds. - Explicitly building tooling before analysis. - Fixing validation steps to match XML props files. - Added PR automation triggers, and documented other pipeline sections. - Added ReferenceType throughout the MDS/AKV CI build steps. - Added NuGet.config update to main CI build step for Package reference mode. - Swapped Abstractions download and NuGet.config update to ensure packages/ exists before attempting to re-configure NuGet. - Clean target no longer removes packages/ - Uncommented package refs to Abstractions. - Added separate MDS and AKV project builds to support Package mode. - Added $ReferenceType$ property to MDS .nuspec like it is for AKV.

mdaigle

Overall, things look good to me. I think it would be helpful to have a walkthrough of a pipeline run to show us the different build artifacts, their version numbers, etc. so that we can see how it all fits together in the end.

mdaigle · 2025-09-30T23:47:32Z

eng/pipelines/common/templates/jobs/build-signed-package-job.yml

  variables:
  - template: ../../../libraries/variables.yml@self
  - ${{ if parameters.isPreview }}:
-    - name: NugetPackageVersion


Thank you! The variable use at such a deep nesting level was one of my biggest complaints about how the pipelines were written before.

paulmedynski added this to the 7.0-preview2 milestone Sep 23, 2025

This was referenced Sep 23, 2025

Abstractions Package - C# project changes #3626

Merged

Create Abstractions package #3567

Closed

priyankatiwari08 self-assigned this Sep 23, 2025

paulmedynski force-pushed the dev/paul/azure-split/abstractions-code branch from 96b8487 to 383a354 Compare September 23, 2025 20:42

paulmedynski force-pushed the dev/paul/azure-split/abstractions-pipelines branch from b02a214 to 49ec317 Compare September 24, 2025 13:46

paulmedynski changed the title ~~[DRAFT] Abstractions Package - Pipeline Changes~~ Abstractions Package - Pipeline Changes Sep 25, 2025

paulmedynski marked this pull request as ready for review September 25, 2025 15:29

paulmedynski requested a review from a team as a code owner September 25, 2025 15:29

paulmedynski commented Sep 25, 2025

View reviewed changes

paulmedynski commented Sep 30, 2025

View reviewed changes

Base automatically changed from dev/paul/azure-split/abstractions-code to feat/azure-split September 30, 2025 18:11

paulmedynski force-pushed the dev/paul/azure-split/abstractions-pipelines branch from 93049ff to 75eb084 Compare September 30, 2025 18:17

paulmedynski force-pushed the dev/paul/azure-split/abstractions-pipelines branch from 75eb084 to 181aa0c Compare September 30, 2025 18:23

Fixing botched rebase!

7c14e0e

mdaigle self-assigned this Sep 30, 2025

mdaigle previously approved these changes Oct 1, 2025

View reviewed changes

- Removed obsolete version variables from merge conflict.

dc7ace3

paulmedynski dismissed mdaigle’s stale review via dc7ace3 October 2, 2025 10:36

Adding $ReferenceType$ back to AKV nuspec.

8b80680

Abstractions Package - Pipeline Changes #3628

Are you sure you want to change the base?

Abstractions Package - Pipeline Changes #3628

Uh oh!

Conversation

paulmedynski commented Sep 23, 2025

Overview

Issues

Testing

Uh oh!

paulmedynski commented Sep 25, 2025

Uh oh!

azure-pipelines bot commented Sep 25, 2025

Uh oh!

codecov bot commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

paulmedynski left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paulmedynski Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paulmedynski commented Sep 26, 2025

Uh oh!

azure-pipelines bot commented Sep 26, 2025

Uh oh!

paulmedynski commented Sep 29, 2025

Uh oh!

azure-pipelines bot commented Sep 29, 2025

Uh oh!

paulmedynski commented Sep 29, 2025

Uh oh!

azure-pipelines bot commented Sep 29, 2025

Uh oh!

paulmedynski left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdaigle left a comment

Choose a reason for hiding this comment

codecov bot commented Sep 25, 2025 •

edited

Loading

paulmedynski Sep 25, 2025 •

edited

Loading