error: the configured Python interpreter version (3.13) is newer than PyO3's maximum supported

[jeertmans]

Hello,

Two days ago, my CI suddenly started to fail, and it seems to be caused by the
build command from the maturin action:

...
🐍 Found CPython 3.9, CPython 3.10, CPython 3.11, CPython 3.12, CPython 3.13, PyPy 3.9, PyPy 3.10
...
📦 Built wheel for CPython 3.12 to dist/DiffeRT-0.0.8-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
   Compiling pyo3-build-config v0.20.3
   Compiling pyo3-macros-backend v0.20.3
   Compiling pyo3-ffi v0.20.3
   Compiling pyo3 v0.20.3
error: failed to run custom build command for `pyo3-ffi v0.20.3`

Caused by:
  process didn't exit successfully: `/home/runner/work/DiffeRT/DiffeRT/target/release/build/pyo3-ffi-e115a4a9047a20ae/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG
  cargo:rerun-if-env-changed=PYO3_USE_ABI3_FORWARD_COMPATIBILITY

  --- stderr
  error: the configured Python interpreter version (3.13) is newer than PyO3's maximum supported version (3.12)
  = help: please check if an updated version of PyO3 is available. Current version: 0.20.3
  = help: set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to suppress this check and build anyway using the stable ABI
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
  Caused by: Failed to build a native library through cargo

My CI setup is mostly the same as you provide, see https://github.com/jeertmans/DiffeRT/actions/runs/8039642162/workflow#L16-L43

Any idea how to fix this? I haven't found any similar issue on the web :/

I guess this issue is finding CPython3.13, but should be skipped I guess if unsupported?

[messense]

This is due to PyO3/pyo3#3821, consider commit your Cargo.lock to avoid such breakage.

[jeertmans]

This is due to PyO3/pyo3#3821, consider commit your Cargo.lock to avoid such breakage.

Should I also pin to a specific version? Because just committing Cargo.lock didn't fix: jeertmans/DiffeRT#56 (or directly https://github.com/jeertmans/DiffeRT/actions/runs/8062290415/job/22021733511?pr=56).

[messense]

Should I also pin to a specific version?

Yeah, pin to pyo3 <= 0.20.2, generating the lock file now will update to 0.20.3 which contains the version check.

Generally speaking, you should commit Cargo.lock for reproducibility.

[jeertmans]

Should I also pin to a specific version?

Yeah, pin to pyo3 <= 0.20.2, generating the lock file now will update to 0.20.3 which contains the version check.

Thanks, it now works!

Generally speaking, you should commit Cargo.lock for reproducibility.

I generally do, but I thought it was not so important for this project, so I removed it to "clean a bit the repo", but I get it serves some purposes.

[davidhewitt]

@jeertmans as a late follow-up here, the point of the PyO3 change is to protect people from shipping builds which might be horribly broken due to lack of support.

I would recommend un-pinning PyO3 and instead set the environment variable PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 like the error message suggests.

If the build doesn't succeed with that environment variable set, then either:

• PyO3 has a bug or some functionality missing with abi3, please report that as a PyO3 issue
• The fact you have a successful build at the moment suggests that your 3.13 alpha build is at risk of being broken as soon as a newer alpha comes along.

[davidhewitt]

@messense maybe in maturin-action we should try to be smart about whether to find 3.13 alpha releases? It is probably not a good idea to be uploading alpha wheels to PyPI unless they are using abi3 only...

[jeertmans]

@jeertmans as a late follow-up here, the point of the PyO3 change is to protect people from shipping builds which might be horribly broken due to lack of support.

I would recommend un-pinning PyO3 and instead set the environment variable PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 like the error message suggests.

If the build doesn't succeed with that environment variable set, then either:

• PyO3 has a bug or some functionality missing with abi3, please report that as a PyO3 issue
• The fact you have a successful build at the moment suggests that your 3.13 alpha build is at risk of being broken as soon as a newer alpha comes along.

@davidhewitt thanks for the clarification! Looks like this also fixes the previous issues, see jeertmans/DiffeRT#59. I would tend to prefer this fix over the above one, as I do not need to restrict the pyo3 version with this one :-)

[victorteokw]

What should we update to the CI yaml?

[davidhewitt]

I believe that maturin 1.5 no longer detects 3.13, so maybe start by retrying?

[victorteokw]

Thanks @davidhewitt

[jeertmans]

I believe that maturin 1.5 no longer detects 3.13, so maybe start by retrying?

Didn’t see it was released! It now works without the environ variable trick: jeertmans/DiffeRT#67.

Thank you!

[victorteokw]

This problem still exists. It tries to find Python 3.13 only on this system like before.

[davidhewitt]

@victorteokw can you share job logs?

[victorteokw]

Hi @davidhewitt, sorry about this. I forgot to upgrade Maturin. Maturin 1.5 and pyo3 0.20.3 works fine.