test failure with CPython 3.12b3: err::tests::fetching_panic_exception_resumes_unwind

[decathorpe]

I'm trying to update the pyo3 packages on Fedora Linux to the latest version (v0.19.1), but I'm getting a weird test failure with CPython 3.12b3:

---- err::tests::fetching_panic_exception_resumes_unwind stdout ----
--- PyO3 is resuming a panic after fetching a PanicException from Python. ---
Python stack trace below:
note: panic did not contain expected string
      panic message: `"Unwrapped panic from Python code"`,
 expected substring: `"new panic"`

failures:
    err::tests::fetching_panic_exception_resumes_unwind

c.f. https://github.com/PyO3/pyo3/blob/v0.19.1/src/err/mod.rs#L855-L868

I'm not sure how or why this is happening, but it looks like PanicException message is no longer extracted successfully here:
https://github.com/PyO3/pyo3/blob/v0.19.1/src/err/mod.rs#L317-L321

[davidhewitt]

Yes, we haven't got 3.12 passing in CI yet, I think realistically this will come in PyO3 0.20 which might be a month off. Is it helpful for you if we try to expedite and backport onto a 0.19.2 release?

[decathorpe]

That would be very helpful, yes 👍 Let me know if I can help with something.

Note that I also need to support PyO3 v0.15, v0.16, v0.17, and v0.18 on top of Python 3.12 (mostly due to Python packages like cryptography using ancient PyO3 version to be compatible with debian stable or something) ... I'll likely need to do some patching to make that work, would you be interested in getting fixes out for those older branches as well?

[davidhewitt]

Oof. I'll take a quick spin at fixing the 3.12 test suite now. (Might take a few days depending on how much is broken.)

TBH I think it will be a lot of work to backport proper releases for all those versions, as they will have multiple ways that their CI has bitrotted by now. I don't think I'll have capacity to do that, but if there's any information I can offer to help you with patching I will do my best to share it.

cryptography have updated their PyO3 dependency to 0.19 not too long ago, so you may be able to update that first to avoid some patching pain. I'd be interested in hearing what other packages require old PyO3 versions, potentially it would be better for the ecosystem if we collaborate to update those packages' PyO3 dependencies?

[decathorpe]

Oof. I'll take a quick spin at fixing the 3.12 test suite now. (Might take a few days depending on how much is broken.)

Thanks, no hurry. :)

TBH I think it will be a lot of work to backport proper releases for all those versions, as they will have multiple ways that their CI has bitrotted by now. I don't think I'll have capacity to do that, but if there's any information I can offer to help you with patching I will do my best to share it.

I'll take a look how much work it would be to fix things on older branches - if it's doable, I'd of course not keep those patches for myself but submit PRs for them.

cryptography have updated their PyO3 dependency to 0.19 not too long ago, so you may be able to update that first to avoid some patching pain. I'd be interested in hearing what other packages require old PyO3 versions, potentially it would be better for the ecosystem if we collaborate to update those packages' PyO3 dependencies?

That's good to know! Sadly cryptography is known to subtly break things with every release, so it's not as easy as just updating it, since that will break a dozen other things. :( But yes, in general, it would be great if I didn't have to support 5 different versions of pyo3.

Right now, Packages in Fedora Linux that depend on different versions of pyo3 are:

• pyo3 v0.15: fapolicy-analyzer, bcrypt (looks like they bumped to pyo3 v0.19 in the dev branch recently), cryptography (also bumped to v0.19 recently)
• pyo3 v0.16: y-py
• pyo3 v0.17: matrix-synapse
• pyo3 v0.18: orjson (recently bumped to pyo3 v0.19)

[alex]

@decathorpe FWIW if you're interested in discussing things we can be doing to reduce the maintenance burden with pyca/cryptography, @reaperhulk and I are happy to discuss and share the different constraints we operate under (and yes, the fact that many of our users are on distros with hilarious out of date software, and they wish to use this week's cryptography with last millenium's OS is, hilariously one of them).

[davidhewitt]

Looks like #3306 is part of the story for 3.12b3. There is still an occasional refcounting failure in some tests which is probably due to the immortal objects changes (probably our FFI definitions need updating).

[decathorpe]

Can confirm that this was fixed with pyo3 v0.19.2. Thanks again!