Skip to content

Conversation

@aaronsteers
Copy link
Contributor

@aaronsteers aaronsteers commented Nov 5, 2025

ci: pin Python to 3.12 in build workflow to fix PyO3 compatibility

Summary

This PR fixes two CI failures on main:

  1. Build failure: The hynek/build-and-inspect-python-package action was defaulting to Python 3.14, causing pydantic-core to fail building due to PyO3 v0.24.1's maximum supported Python version being 3.13. Fixed by explicitly pinning Python to 3.12 via actions/setup-python.

  2. Linter failure: Removed a blank line after "Note:" in airbyte/mcp/__init__.py to fix D412 docstring linting error.

The Python 3.12 pin aligns with the project's declared version constraint in pyproject.toml: python = ">=3.10,<3.13".

Review & Testing Checklist for Human

Risk level: Yellow (workflow modification, not yet verified in CI)

  • Verify CI builds successfully - Check that the "Build and/or Publish" workflow completes without PyO3 errors
  • Confirm no other Python version issues - Review all CI jobs (linters, tests) to ensure Python 3.12 doesn't introduce new compatibility issues
  • Validate publish workflow still works - Ensure the Python setup step doesn't interfere with the publish job's artifact consumption

Test Plan

  1. Wait for CI to complete on this PR
  2. Check the "Build and/or Publish" workflow logs to confirm it's using Python 3.12
  3. Verify the build succeeds without PyO3 version errors
  4. Check that "Run Linters" workflow passes

Notes

  • Local lint checks (poe fix-and-check) passed successfully
  • The PyO3 error was: error: the configured Python interpreter version (3.14) is newer than PyO3's maximum supported version (3.13)
  • The workflow change adds a setup-python step before the build-and-inspect action to ensure Python 3.12 is used

Link to Devin run: https://app.devin.ai/sessions/5a49a6c6d7d44e46b96a1539823e9077
Requested by: AJ Steers (@aaronsteers)
Context: https://airbytehq-team.slack.com/archives/D089P0UPVT4/p1762303244158319

Summary by CodeRabbit

  • Chores
    • CI workflow updated to use a newer build-and-inspect package version for package build/publish steps.
  • Documentation
    • Minor docstring whitespace/formatting cleanup (non-functional).

Important

Auto-merge enabled.

This PR is set to merge automatically when all requirements are met.

Note

Auto-merge may have been disabled. Please check the PR status to confirm.

- Add actions/setup-python@v5.3.0 step to pin Python version to 3.12
- Fixes PyO3 build failure where pydantic-core was attempting to build
  against Python 3.14, which exceeds PyO3 v0.24.1's maximum supported
  version (3.13)
- Also fix docstring linter error in airbyte/mcp/__init__.py (D412)

Resolves CI failures in Build and/or Publish and Run Linters workflows.

Co-Authored-By: AJ Steers <aj@airbyte.io>
@devin-ai-integration
Copy link
Contributor

Original prompt from AJ Steers
@Devin - PyAirbyte CI got into an unhealthy state. Maybe some dependency issues, maybe or maybe not related to anything we changed. (Something new there about PyO3. Can you create a new PR to resolve any CI breakages?
Thread URL: https://airbytehq-team.slack.com/archives/D089P0UPVT4/p1762303244158319

@devin-ai-integration
Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link

github-actions bot commented Nov 5, 2025

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1762303487-fix-ci-pyo3-python-version' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1762303487-fix-ci-pyo3-python-version'

Helpful Resources

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /fix-pr - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test-pr - Runs tests with the updated PyAirbyte

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 5, 2025

📝 Walkthrough

Walkthrough

Bumps the GitHub Action hynek/build-and-inspect-python-package from v2.13.0 to v2.14.0 (hash updated) in the PyPI publish workflow; removes an extraneous blank line in airbyte/mcp/__init__.py docstring. (33 words)

Changes

Cohort / File(s) Summary
CI workflow version bump
.github/workflows/pypi_publish.yml
Update action reference from hynek/build-and-inspect-python-package@v2.13.0 to hynek/build-and-inspect-python-package@v2.14.0 and replace the pinned commit hash. No other workflow logic changed.
Docstring cleanup
airbyte/mcp/__init__.py
Remove an unnecessary blank line in the module docstring (cosmetic only).

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

  • Quick check of the action version/hash change in .github/workflows/pypi_publish.yml.
  • Verify the docstring edit in airbyte/mcp/__init__.py is purely whitespace.

Shall I scan the repo for other workflow files that pin the same action to update them consistently? wdyt?

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)
Check name Status Explanation Resolution
Title check ❓ Inconclusive The title 'ci: fix lint issue' is vague and doesn't reflect the main changes. The PR primarily updates the CI workflow to use a newer action version (v2.14.0), with only a minor docstring formatting fix. Consider a more specific title that captures the primary change, such as 'ci: update build-and-inspect-python-package action to v2.14.0' or 'ci: fix PyO3 compatibility by updating build action'. What do you think?
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch devin/1762303487-fix-ci-pyo3-python-version

Comment @coderabbitai help to get the list of available commands and usage tips.

@aaronsteers
Copy link
Contributor Author

Devin, looks like we may need the same fix here: https://github.com/airbytehq/PyAirbyte/actions/runs/19087533763/job/54531059233#step:4:1

@github-actions
Copy link

github-actions bot commented Nov 5, 2025

PyTest Results (Fast Tests Only, No Creds)

312 tests  ±0   312 ✅ ±0   5m 50s ⏱️ -30s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit be58b8f. ± Comparison against base commit cb03292.

♻️ This comment has been updated with latest results.

- Pass python-version: '3.12' directly to hynek/build-and-inspect-python-package
  action to prevent it from defaulting to Python 3.x (which resolves to 3.14)
- Add UV_PYTHON: "3.12" env variable as additional safeguard for uv
- Keep actions/setup-python step for consistency

The hynek action has its own python-version input that defaults to 3.x,
which was overriding the runner's Python version set by actions/setup-python.
This comprehensive fix ensures Python 3.12 is used throughout the build process.

Co-Authored-By: AJ Steers <aj@airbyte.io>
@devin-ai-integration
Copy link
Contributor

Good catch! I've updated the workflow with a comprehensive fix:

  1. Added python-version: '3.12' input directly to the hynek/build-and-inspect-python-package action
  2. Added UV_PYTHON: "3.12" env variable to the build job
  3. Kept the actions/setup-python step for consistency

The issue was that the hynek action has its own python-version input that defaults to 3.x (which resolves to Python 3.14), and this was overriding the runner's Python version. The new commit should fix the PyO3 build failure.

CI should now pass with Python 3.12 throughout the build process.

@aaronsteers
Copy link
Contributor Author

Devin, still falling. Keep me posted in slack but keep working until CI passes.

The hynek/build-and-inspect-python-package action does not support a
python-version input and internally uses Python 3.x (which resolves to
3.14 on GitHub runners). Since PyO3 v0.24.1 only supports up to Python
3.13, we use the PYO3_USE_ABI3_FORWARD_COMPATIBILITY flag as recommended
by the PyO3 error message to allow the build to proceed.

This is a temporary workaround until either:
- PyO3 adds support for Python 3.14, or
- The hynek action adds support for pinning Python version, or
- We inline the build steps to control the Python version directly

Changes:
- Remove job-level UV_PYTHON env (didn't help)
- Remove invalid python-version input from hynek step (not supported)
- Add PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 env to hynek step
- Keep actions/setup-python for consistency

Co-Authored-By: AJ Steers <aj@airbyte.io>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
.github/workflows/pypi_publish.yml (1)

22-26: PyO3 forward-compatibility workaround is a solid interim solution.

The comment explaining the rationale is clear and helpful. One note: this flag is explicitly a workaround—when PyO3 and/or pydantic-core gain support for Python 3.14+, you'll want to circle back and remove this to keep the CI clean. Should we add a TODO or a tracking issue to remove it in a future PR, wdyt?

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8d7c9b0 and d5ef447.

📒 Files selected for processing (1)
  • .github/workflows/pypi_publish.yml (2 hunks)
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings
Learnt from: yohannj
Repo: airbytehq/PyAirbyte PR: 716
File: airbyte/logs.py:384-402
Timestamp: 2025-07-11T19:53:44.427Z
Learning: In the PyAirbyte project, when reviewing PRs, maintain clear separation of concerns. Don't suggest changes that are outside the scope of the PR's main objective, even if they would improve consistency or fix other issues. This helps with reviewing changes and potential reverts.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
  • GitHub Check: Pytest (All, Python 3.11, Windows)
  • GitHub Check: Pytest (No Creds)
  • GitHub Check: Pytest (All, Python 3.10, Windows)
  • GitHub Check: Pytest (All, Python 3.10, Ubuntu)
  • GitHub Check: Pytest (All, Python 3.11, Ubuntu)
  • GitHub Check: Pytest (Fast)
🔇 Additional comments (3)
.github/workflows/pypi_publish.yml (3)

15-17: Checkout action version bump looks good.

The ref update to the newer v5.0.0 commit is a sensible maintenance bump. ✓


18-20: Python 3.12 pinning directly addresses the PyO3 compatibility blocker.

This aligns cleanly with your pyproject.toml constraint (python = ">=3.10,<3.13") and will prevent the hynek/build-and-inspect-python-package action from defaulting to Python 3.14 where PyO3 v0.24.1 fails. 👍


1-55: Verification checklist before merge.

Per your PR objectives, this is still in draft pending CI verification. Once you're ready:

  1. ✓ Confirm the build job completes without PyO3 errors
  2. ✓ Verify no other Python version issues across other CI jobs (e.g., lint, tests, etc.)
  3. ✓ Validate the publish workflow still works end-to-end
  4. ✓ Confirm whether the same Python 3.12 pinning is needed in any other workflows (you mentioned this fix may be required elsewhere)

Once CI passes, the changes look solid and ready to land. Great catch on tracking down the root cause! 🚀

Fixed a typo in the actions/checkout SHA that was causing the workflow to fail.
The correct SHA for v5.0.0 is 08c6903cd8c0fde910a37f88322edcfb5dd907a8.

Applied PYO3_USE_ABI3_FORWARD_COMPATIBILITY workaround to allow pydantic-core
to build with Python 3.14. The hynek action internally uses Python 3.x which
resolves to 3.14 on GitHub runners, but PyO3 v0.24.1 only supports up to 3.13.

Also added PYO3_PRINT_CONFIG for better visibility in CI logs.

This is a temporary workaround until either:
- PyO3 adds support for Python 3.14, or
- The hynek action adds support for pinning Python version

Co-Authored-By: AJ Steers <aj@airbyte.io>
@devin-ai-integration
Copy link
Contributor

Fixed a checkout SHA typo that was blocking the workflow. The clean fix is now pushed with:

  1. Correct actions/checkout SHA (was missing an "f" in the hash)
  2. PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to allow pydantic-core to build with Python 3.14
  3. PYO3_PRINT_CONFIG=1 for better visibility in logs

CI is now running with the clean fix. The hynek action internally uses Python 3.x which resolves to 3.14, and PyO3 v0.24.1 only supports up to 3.13, so this workaround is necessary until either PyO3 adds 3.14 support or the hynek action adds support for pinning Python versions.

Monitoring CI results...

PYO3_PRINT_CONFIG=1 is a debugging flag that intentionally halts the build
after printing configuration. Removing it to allow the build to proceed with
only PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1, which should be sufficient to
allow pydantic-core to build with Python 3.14.

Co-Authored-By: AJ Steers <aj@airbyte.io>
@devin-ai-integration
Copy link
Contributor

Removed PYO3_PRINT_CONFIG=1 which was a debugging flag that intentionally halts the build after printing configuration.

The build should now proceed with only PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1, which allows PyO3 to build against Python 3.14 using forward compatibility mode. This is the recommended workaround from the PyO3 error message itself.

Monitoring CI for results...

@devin-ai-integration
Copy link
Contributor

Investigation Summary

After investigating the CI failures, I've identified the root cause and a recommended solution.

Root Cause

The hynek/build-and-inspect-python-package action internally uses actions/setup-python with python-version: '3.x', which resolves to Python 3.14 on current GitHub runners. PyO3 v0.24.1 (used by pydantic-core) has fundamental incompatibilities with Python 3.14's C API.

Show/Hide Detailed Analysis

Attempts Made:

  1. ✗ Tried pinning Python with actions/setup-python - didn't work because hynek action uses its own Python setup
  2. ✗ Tried UV_PYTHON and python-version input to hynek - action doesn't support these inputs
  3. ✗ Tried PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 with PYO3_PRINT_CONFIG=1 - PRINT_CONFIG is a debug flag that intentionally halts builds
  4. ✗ Tried PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 alone - suppressed version check but hit actual Rust compilation errors

Current Build Errors:

error[E0425]: cannot find function `PyUnicode_New` in module `pyo3::ffi`
error[E0425]: cannot find function `PyUnicode_KIND` in module `pyo3::ffi`
error[E0425]: cannot find value `PyUnicode_1BYTE_KIND` in module `pyo3::ffi`
error[E0425]: cannot find function `PyUnicode_DATA` in module `pyo3::ffi`

These are real FFI symbol mismatches - PyO3 v0.24.1 doesn't support Python 3.14's C API.

Recommended Solution

Replace the hynek/build-and-inspect-python-package step with explicit build steps that control the Python version:

- uses: actions/setup-python@v5
  with:
    python-version: '3.12'
- name: Install build tools
  run: python -m pip install build twine
- name: Build package
  run: python -m build --sdist --wheel
- name: Check package
  run: twine check dist/*
- uses: actions/upload-artifact@v4
  with:
    name: Packages
    path: dist

This maintains compatibility with the existing publish job (which expects the Packages artifact) while building under Python 3.12 where PyO3 works correctly.

Alternative Options

  1. Wait for PyO3 update - Long-term solution but not actionable for this PR
  2. Pin hynek action to older version - Won't help; older versions still use python-version: '3.x'
  3. Skip build job on PRs - Smaller change but loses build validation on PRs

Requesting approval to implement the recommended solution (inline build steps with Python 3.12).


Requested by @aaronsteers
Devin run: https://app.devin.ai/sessions/5a49a6c6d7d44e46b96a1539823e9077

@github-actions
Copy link

github-actions bot commented Nov 5, 2025

PyTest Results (Full)

381 tests  ±0   365 ✅ +1   26m 17s ⏱️ + 1m 9s
  1 suites ±0    16 💤 ±0 
  1 files   ±0     0 ❌  - 1 

Results for commit be58b8f. ± Comparison against base commit cb03292.

@aaronsteers
Copy link
Contributor Author

Devin, this release seems to fix the bug we are seeing. Can you bump to the latest and see if that fixes our issues without the other problems?

https://github.com/hynek/build-and-inspect-python-package/releases/tag/v2.14.0

Upgrading from v2.13.0 to v2.14.0 which includes:
- Update tools such that they work on Python 3.14
- The action now ignores UV_PYTHON coming from the outside

This should fix the PyO3/pydantic-core build issues with Python 3.14
by using updated tooling that properly handles Python 3.14.

Removed the PYO3_USE_ABI3_FORWARD_COMPATIBILITY workaround to do a
clean test of whether v2.14.0 resolves the issue.

Release notes: https://github.com/hynek/build-and-inspect-python-package/releases/tag/v2.14.0

Co-Authored-By: AJ Steers <aj@airbyte.io>
@devin-ai-integration
Copy link
Contributor

Upgraded to hynek/build-and-inspect-python-package v2.14.0 as suggested. This version includes:

  • "Update tools such that they work on Python 3.14" (#182)
  • "The action now ignores UV_PYTHON coming from the outside" (#184)

Removed the PYO3_USE_ABI3_FORWARD_COMPATIBILITY workaround to do a clean test. If v2.14.0's updated tooling properly handles Python 3.14, the build should succeed without needing the workaround.

Monitoring CI results...

Release notes: https://github.com/hynek/build-and-inspect-python-package/releases/tag/v2.14.0

@aaronsteers aaronsteers marked this pull request as ready for review November 5, 2025 19:50
@aaronsteers aaronsteers changed the title ci: pin Python to 3.12 in build workflow to fix PyO3 compatibility (do not merge) ci: fix lint issue Nov 5, 2025
@aaronsteers aaronsteers enabled auto-merge (squash) November 5, 2025 19:51
@aaronsteers aaronsteers disabled auto-merge November 5, 2025 19:52
@aaronsteers aaronsteers merged commit 813d0d0 into main Nov 5, 2025
23 of 24 checks passed
@aaronsteers aaronsteers deleted the devin/1762303487-fix-ci-pyo3-python-version branch November 5, 2025 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants