Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3-sys: port Py_3_12 changes from pyo3-ffi #295

Merged
merged 4 commits into from
Jul 2, 2024

Conversation

quark-zju
Copy link
Contributor

@quark-zju quark-zju commented May 19, 2024

I looked at all Py_3_12 changes in pyo3-ffi/src/cpython and ported them back. Functions, structs, and fields in the code object not already defined in python3-sys are ignored. For PyUnicode_READY, I kept a dummy version so the cpython crate does not need change.

Fixes #294 and facebook/sapling#895

I looked at all Py_3_12 changes in pyo3-ffi/src/cpython and ported them
back. Functions, structs, and fields in the code object not already
defined in python3-sys are ignored. For PyUnicode_READY, I kept a dummy
version so the cpython crate does not need change.
@quark-zju quark-zju force-pushed the py312 branch 2 times, most recently from f487687 to 57c9fdc Compare July 1, 2024 17:25
    Symbols missing in /opt/hostedtoolcache/Python/3.12.4/x64/lib/libpython3.12.so:
    PyCode_New
    PyCode_NewWithPosOnlyArgs
    PyUnicode_AsUnicode
    PyUnicode_AsUnicodeAndSize
    PyUnicode_FromUnicode
    _Py_GetAllocatedBlocks
    _Py_PackageContext

Python 3.12/3.11 renamed/deprecated some APIs. Update code to skip them.
This affects `linux (ubuntu-latest, 3.12, nightly)` CI.
@quark-zju quark-zju force-pushed the py312 branch 4 times, most recently from d735056 to 3a7f288 Compare July 1, 2024 17:52
Drop Python 2.7, as it is no longer available:

    Warning: The support for python 2.7 will be removed on June 19. Related issue: actions/setup-python#672
    Version 2.7 was not found in the local cache
    Error: The version '2.7' with architecture 'x64' was not found for Ubuntu 22.04.
    The list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json

Add Python 3.12 as its support was just added.

Fork macOS steps from Linux due to the ARM arch change:

- Rust 1.41 does not have a `aarch64-apple-darwin` version. The first Rust
  version that supports `aarch64-apple-darwin` is 1.49.0.
- Python 3.7 to 3.10 do not have a macOS ARM version in CI.
- Rust < 1.54 will fail on CI. See also See rust-lang/rust#107252:

    error: linking with `cc` failed: exit status: 1
    |
    = note: "cc" "-arch" "arm64" "-L" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.0.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.1.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.10.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.11.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.12.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.13.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.14.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.15.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.2.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.3.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.4.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.5.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.6.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.7.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.8.rcgu.o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.build_script_build.f402nxks-cgu.9.rcgu.o" "-o" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/build/memchr-30eba211f26bbb03/build_script_build-30eba211f26bbb03.1dr67yq0qjg3o93v.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/Users/runner/work/rust-cpython/rust-cpython/target/debug/deps" "-L" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd-dd5aba3397285ad0.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-cb187f1a5b379d1a.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libobject-d263574460ac0b7b.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-0a9a23101265d9a7.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgimli-e6457880b831342b.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect-9590d4ced205b108.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle-a1f4d95a4a2cc2a3.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown-dbf1072a4ef077a4.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-7ff002204e919c46.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunwind-a2049ddc023c32b9.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-02cf13a7707ef825.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liblibc-e628929d2a4d215f.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liballoc-1f245a76196b996b.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_core-49a44db0ac13b7f4.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcore-a04810d031c48fa4.rlib" "/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-7cd54448073dfb2c.rlib" "-lSystem" "-lresolv" "-lc" "-lm" "-liconv"
    = note: ld: multiple errors: archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd-dd5aba3397285ad0.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpanic_unwind-cb187f1a5b379d1a.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libobject-d263574460ac0b7b.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libaddr2line-0a9a23101265d9a7.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libgimli-e6457880b831342b.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libstd_detect-9590d4ced205b108.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_demangle-a1f4d95a4a2cc2a3.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libhashbrown-dbf1072a4ef077a4.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_alloc-7ff002204e919c46.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libunwind-a2049ddc023c32b9.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcfg_if-02cf13a7707ef825.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liblibc-e628929d2a4d215f.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/liballoc-1f245a76196b996b.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc_std_workspace_core-49a44db0ac13b7f4.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcore-a04810d031c48fa4.rlib'; archive member 'lib.rmeta' not a mach-o file in '/Users/runner/.rustup/toolchains/1.53.0-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libcompiler_builtins-7cd54448073dfb2c.rlib'
            clang: error: linker command failed with exit code 1 (use -v to see invocation)
@markbt markbt merged commit 5489b23 into dgrunwald:master Jul 2, 2024
36 checks passed
@gracinet
Copy link
Collaborator

gracinet commented Jul 2, 2024

Thank you so much for doing this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Python 3.12 support
3 participants