FEAT: Integrate mssql_py_core wheel installation into Dev workflow PR validation, official and release pipelines

[saurabh500]

Work Item / Issue Reference

AB#42668

---

Summary

PR Review Guide — Integrate mssql-py-core into mssql-python

Testing: Official build at https://sqlclientdrivers.visualstudio.com/mssql-python/_build?definitionId=2199&_a=summary
PR pipeline works.

---

What this PR does

Ships mssql_py_core (the Rust-native bulk-copy engine) inside every mssql-python wheel. After this PR, import mssql_py_core works out of the box after pip install mssql-python.

Before / After

                        BEFORE                                          AFTER
                        ──────                                          ─────

  ┌─────────────────────────┐                          ┌─────────────────────────┐
  │     mssql-rs repo       │                          │     mssql-rs repo       │
  │  (builds mssql_py_core) │                          │  (builds mssql_py_core) │
  └────────┬────────────────┘                          └────────┬────────────────┘
           │                                                    │
           │ NuGet publish                                      │ NuGet publish
           ▼                                                    ▼
  ┌─────────────────────┐                              ┌─────────────────────┐
  │  Azure Artifacts    │                              │  Azure Artifacts    │
  │  (nupkg with .whl)  │                              │  (nupkg with .whl)  │
  └─────────────────────┘                              └────────┬────────────┘
                                                                │
           (no connection)                              install-mssql-py-core.sh/.ps1
                                                                │ downloads + extracts
                                                                ▼
  ┌─────────────────────────┐                          ┌─────────────────────────┐
  │   mssql-python repo     │                          │   mssql-python repo     │
  │                         │                          │                         │
  │   setup.py bdist_wheel  │                          │   setup.py bdist_wheel  │
  │   └── mssql_python/     │                          │   ├── mssql_python/     │
  │                         │                          │   └── mssql_py_core/  ◄─── NEW
  └────────┬────────────────┘                          └────────┬────────────────┘
           │                                                    │
           ▼                                                    ▼
  ┌─────────────────────┐                              ┌─────────────────────┐
  │  .whl (PyPI)        │                              │  .whl (PyPI)        │
  │  └── mssql_python/  │                              │  ├── mssql_python/  │
  └─────────────────────┘                              │  └── mssql_py_core/ │
                                                       └─────────┬───────────┘
                                                                 │
                                                       ┌─────────▼───────────┐
                                                       │  Release pipelines  │
                                                       │  validate version   │
                                                       │  is NOT dev/alpha   │
                                                       └─────────────────────┘

Key design decisions

Decision	Rationale
mssql_py_core binary is pre-built in mssql-rs, not compiled here	It depends on OpenSSL 3 / glibc 2.34 (manylinux_2_34). Building it inside mssql-python's 2_28 container isn't possible.
mssql-python wheels stay on manylinux_2_28	Broader distro compatibility (RHEL 8, Debian 10+, Ubuntu 20.04+). The pre-built .so from mssql-rs is just byte-copied in.
Version pinned in eng/versions/mssql-py-core.version	Single source of truth. All install scripts read from here. Release pipelines validate it.
Release pipelines reject pre-release versions	Prevents accidentally shipping dev/alpha/beta/rc builds of mssql_py_core to PyPI.

---

Recommended review order

The PR has three areas. Review them in this order:

Area 1 — Install infrastructure (core of this PR)

Read these together — they form a single pipeline:

#	File	What changed
1	eng/versions/mssql-py-core.version	New. Single-line version file (0.1.0-dev.20260222.140833). All scripts read from here.
2	eng/scripts/resolve_nuget_feed.py	New. Fetches NuGet v3 service index, prints PackageBaseAddress URL. Used by the shell script.
3	eng/scripts/extract_wheel.py	New. Extracts mssql_py_core/ entries from a .whl (ZIP), skipping .dist-info and .libs. Used by both PS1 and SH.
4	eng/scripts/install-mssql-py-core.ps1	New. Windows install script: read version → resolve feed → download nupkg → find matching wheel → extract → verify import.
5	eng/scripts/install-mssql-py-core.sh	New. Unix install script: same flow as PS1, plus musl/glibc detection for Alpine.
6	OneBranchPipelines/scripts/validate-release-versions.ps1	New. Reads a .version file, rejects dev|alpha|beta|rc. Called by OneBranch release pipelines.

Area 2 — Pipeline integration

These files wire the install scripts into every pipeline:

#	File	What changed
7	eng/pipelines/steps/install-mssql-py-core.yml	New. Step template with 3 variants: windows, unix, container. Calls PS1 or SH.
8	eng/pipelines/pr-validation-pipeline.yml	Modified. Adds install-mssql-py-core.yml template ref in all 12 test jobs (Windows, macOS, Linux containers ×2, RHEL ×2, Alpine ×2, Azure SQL ×3, codecov).
9	OneBranchPipelines/stages/build-linux-single-stage.yml	Modified. Adds install-mssql-py-core.sh call inside the per-Python-version build loop. Adds curl to package installs. Updates comment about manylinux_2_28 vs 2_34.
10	OneBranchPipelines/stages/build-windows-single-stage.yml	Modified. Adds PowerShell install step before testing.
11	OneBranchPipelines/stages/build-macos-single-stage.yml	Modified. Adds bash install step before testing.
12	OneBranchPipelines/jobs/consolidate-artifacts-job.yml	Modified. checkout: self instead of checkout: none so we can copy the version file into dist/ for traceability.
13	OneBranchPipelines/official-release-pipeline.yml	Modified. Adds version validation step — calls validate-release-versions.ps1 to block pre-release versions before ESRP release.
14	OneBranchPipelines/dummy-release-pipeline.yml	Modified. Same version validation gate as official pipeline.
15	OneBranchPipelines/build-release-package-pipeline.yml	Trivial. Comment fix (manylinux_2_28 → manylinux_2_34).

Area 3 — Build packaging and tests

#	File	What changed
16	setup.py	Modified. Adds validate_mssql_py_core() pre-flight check, includes mssql_py_core in packages and package_data, uses manylinux_2_28 tags.
17	tests/test_019_bulkcopy.py	New. Basic bulkcopy integration test — insert 3 rows via _bulkcopy(), verify round-trip.

Ancillary

#	File	What changed
18	.gitignore	Adds mssql_py_core/ (extracted at dev time, not checked in).

---

Things to look for

1. Version file path consistency — every script resolves eng/versions/mssql-py-core.version relative to repo root. Verify the consolidation job copies it correctly into the artifact.

2. musl/glibc detection in install-mssql-py-core.sh — Alpine uses musl. The script gracefully exits 0 (skip) when no musllinux wheel is available rather than failing the build.

3. setup.py validation — validate_mssql_py_core() runs at wheel-build time. Must not block pip install from PyPI (the check only runs during bdist_wheel).

4. manylinux_2_28 everywhere in mssql-python — setup.py platform tags and build-linux-single-stage.yml container selection both use manylinux_2_28. The mssql_py_core .so is built on manylinux_2_34 in the mssql-rs repo but byte-copied in here.

5. Release pipeline gating — both OneBranch release pipelines (OneBranchPipelines/official-release-pipeline.yml and OneBranchPipelines/dummy-release-pipeline.yml) validate the version file from build artifacts and reject pre-release versions before ESRP submission. The script lives at OneBranchPipelines/scripts/validate-release-versions.ps1.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 100%

🎯 Overall Coverage 76%

📈 Total Lines Covered: 5520 out of 7185 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.hpp: 58.8%
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.row.py: 66.2%
mssql_python.pybind.ddbc_bindings.cpp: 69.4%
mssql_python.pybind.ddbc_bindings.h: 69.7%
mssql_python.pybind.connection.connection.cpp: 75.3%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.cursor.py: 84.7%
mssql_python.__init__.py: 84.9%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[saurabh500]

Devcontainer changes and developer documentation changes are out of scope

[Copilot]

Pull request overview

This PR integrates the mssql_py_core (Rust-based bulk copy engine) into the mssql-python package by downloading pre-built wheels from a NuGet feed and including them in every mssql-python wheel. After this change, import mssql_py_core works out of the box after pip install mssql-python.

Changes:

• Adds installation infrastructure to download and extract mssql_py_core wheels from Azure Artifacts NuGet feed
• Integrates mssql_py_core installation into all build and test pipelines (PR validation, official builds, release pipelines)
• Adds version validation to prevent pre-release versions of mssql_py_core from being shipped to PyPI
• Adds basic bulkcopy integration tests and includes mssql_py_core in the mssql-python wheel packaging

Reviewed changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
eng/versions/mssql-py-core.version	Single-line version file specifying which mssql_py_core version to download
eng/scripts/resolve_nuget_feed.py	Python script to fetch NuGet v3 service index and extract PackageBaseAddress URL
eng/scripts/extract_wheel.py	Python script to extract mssql_py_core files from a wheel into the repository root
eng/scripts/install-mssql-py-core.ps1	Windows PowerShell script orchestrating download, platform detection, and extraction
eng/scripts/install-mssql-py-core.sh	Unix bash script with same functionality as PS1, plus musl/glibc detection for Alpine
eng/scripts/validate-release-versions.ps1	PowerShell script to reject pre-release versions (dev/alpha/beta/rc)
eng/pipelines/steps/install-mssql-py-core.yml	Reusable step template with Windows/Unix/container variants
eng/pipelines/pr-validation-pipeline.yml	Adds mssql_py_core installation to all 12 test jobs
eng/pipelines/build-whl-pipeline.yml	Adds mssql_py_core installation before wheel builds, adds curl dependency
OneBranchPipelines/stages/build-linux-single-stage.yml	Adds install step in per-Python build loop, adds curl, updates comments
OneBranchPipelines/stages/build-windows-single-stage.yml	Adds PowerShell install step before testing
OneBranchPipelines/stages/build-macos-single-stage.yml	Adds bash install step before testing
OneBranchPipelines/jobs/consolidate-artifacts-job.yml	Changes checkout from none to self, copies version file to dist/
eng/pipelines/official-release-pipeline.yml	Downloads version file, validates it's not pre-release before ESRP submission
eng/pipelines/dummy-release-pipeline.yml	Same version validation as official pipeline
OneBranchPipelines/build-release-package-pipeline.yml	Comment update about manylinux version
setup.py	Adds validate_mssql_py_core() check, includes mssql_py_core in packages and package_data
tests/test_019_bulkcopy.py	Basic bulkcopy integration test with 3-row insert and verification
.gitignore	Excludes mssql_py_core/ directory from version control

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[gargsaumya]

Copilot has some good suggestions that we can consider. I don’t have any blocking comments from my end. Approving the PR.

[bewithgaurav]

went through mssql-rs flow as well and added some comments, some queries and improvements - copilot has pointed out some good things as well
none of them fall into the PR blocker category
will wait for the response, and then approve