Merge pull request #2126 from DSPOM2/main

move ffi module to separate crate

Architecture.md

@@ -19,7 +19,7 @@ Since implementing `PyClass` requires lots of boilerplate, we have a proc-macro
 To summarize, there are six main parts to the PyO3 codebase.
 
 1. [Low-level bindings of Python/C API.](#1-low-level-bindings-of-python-capi)
-   - [`src/ffi`]
+   - [`pyo3-ffi`] and [`src/ffi`]
 2. [Bindings to Python objects.](#2-bindings-to-python-objects)
    - [`src/instance.rs`] and [`src/types`]
 3. [`PyClass` and related functionalities.](#3-pyclass-and-related-functionalities)
@@ -34,7 +34,7 @@ To summarize, there are six main parts to the PyO3 codebase.
 
 ## 1. Low-level bindings of Python/C API
 
-[`src/ffi`] contains wrappers of [Python/C API].
+[`pyo3-ffi`] contains wrappers of [Python/C API].
 
 We aim to provide straight-forward Rust wrappers resembling the file structure of
 [`cpython/Include`](https://github.com/python/cpython/tree/v3.9.2/Include).
@@ -43,7 +43,7 @@ However, we still lack some APIs and are continuously updating the module to mat
 the file contents upstream in CPython.
 The tracking issue is [#1289](https://github.com/PyO3/pyo3/issues/1289), and contribution is welcome.
 
-In the [`src/ffi`] module, there is lots of conditional compilation such as `#[cfg(Py_LIMITED_API)]`,
+In the [`pyo3-ffi`] crate, there is lots of conditional compilation such as `#[cfg(Py_LIMITED_API)]`,
 `#[cfg(Py_37)]`, and `#[cfg(PyPy)]`.
 `Py_LIMITED_API` corresponds to `#define Py_LIMITED_API` macro in Python/C API.
 With `Py_LIMITED_API`, we can build a Python-version-agnostic binary called an
@@ -208,6 +208,7 @@ Some of the functionality of `pyo3-build-config`:
 [`pyo3-macros`]: https://github.com/PyO3/pyo3/tree/main/pyo3-macros
 [`pyo3-macros-backend`]: https://github.com/PyO3/pyo3/tree/main/pyo3-macros-backend
 [`pyo3-build-config`]: https://github.com/PyO3/pyo3/tree/main/pyo3-build-config
+[`pyo3-ffi`]: https://github.com/PyO3/pyo3/tree/main/pyo3-ffi
 
 <!-- Directories -->
 

CHANGELOG.md

@@ -16,6 +16,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `pyo3-build-config` no longer enables the `resolve-config` feature by default. [#2008](https://github.com/PyO3/pyo3/pull/2008)
 - Update `inventory` optional dependency to 0.2. [#2019](https://github.com/PyO3/pyo3/pull/2019)
 - Drop `paste` dependency. [#2081](https://github.com/PyO3/pyo3/pull/2081)
+- The bindings found `pyo3::ffi` are now a re-export of the new `pyo3-ffi` crate. [#2126](https://github.com/PyO3/pyo3/pull/2126)
 
 ### Added
 
@@ -50,6 +51,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - `#[pyclass]` type object creation [#2076](https://github.com/PyO3/pyo3/pull/2076) [#2081](https://github.com/PyO3/pyo3/pull/2081)
 - `__ipow__` now supports modulo argument on Python 3.8+. [#2083](https://github.com/PyO3/pyo3/pull/2083)
 - `pyo3-macros-backend` is now compiled with PyO3 cfgs to enable different magic method definitions based on version. [#2083](https://github.com/PyO3/pyo3/pull/2083)
+- `_PyCFunctionFast` now correctly reflects the signature defined in the [Python docs](https://docs.python.org/3/c-api/structures.html#c._PyCFunctionFast). [#2126](https://github.com/PyO3/pyo3/pull/2126)
+- `PyDateTimeAPI` and `PyDateTime_TimeZone_UTC` are are now unsafe functions instead of statics. [#2126](https://github.com/PyO3/pyo3/pull/2126)
+- `PyDateTimeAPI` does not implicitly call `PyDateTime_IMPORT` anymore to reflect the original Python API more closely. Before the first call to `PyDateTime_IMPORT` a null pointer is returned. Therefore before calling any of the following FFI functions `PyDateTime_IMPORT` must be called to avoid undefined behaviour: [#2126](https://github.com/PyO3/pyo3/pull/2126)
+  - `PyDateTime_TimeZone_UTC`
+  - `PyDate_Check`
+  - `PyDate_CheckExact`
+  - `PyDateTime_Check`
+  - `PyDateTime_CheckExact`
+  - `PyTime_Check`
+  - `PyTime_CheckExact`
+  - `PyDelta_Check`
+  - `PyDelta_CheckExact`
+  - `PyTZInfo_Check`
+  - `PyTZInfo_CheckExact`
+  - `PyDateTime_FromTimestamp`
+  - `PyDate_FromTimestamp`
 
 ### Removed
 

Cargo.toml

@@ -19,6 +19,9 @@ cfg-if = "1.0"
 libc = "0.2.62"
 parking_lot = "0.11.0"
 
+# ffi bindings to the python interpreter, split into a seperate crate so they can be used independently
+pyo3-ffi = { path = "pyo3-ffi", version = "=0.15.1" }
+
 # support crates for macros feature
 pyo3-macros = { path = "pyo3-macros", version = "=0.15.1", optional = true }
 indoc = { version = "1.0.3", optional = true }
@@ -60,16 +63,16 @@ multiple-pymethods = ["inventory", "pyo3-macros/multiple-pymethods"]
 # Use this feature when building an extension module.
 # It tells the linker to keep the python symbols unresolved,
 # so that the module can also be used with statically linked python interpreters.
-extension-module = []
+extension-module = ["pyo3-ffi/extension-module"]
 
 # Use the Python limited API. See https://www.python.org/dev/peps/pep-0384/ for more.
-abi3 = ["pyo3-build-config/abi3"]
+abi3 = ["pyo3-build-config/abi3", "pyo3-ffi/abi3"]
 
 # With abi3, we can manually set the minimum Python version.
-abi3-py37 = ["abi3-py38", "pyo3-build-config/abi3-py37"]
-abi3-py38 = ["abi3-py39", "pyo3-build-config/abi3-py38"]
-abi3-py39 = ["abi3-py310", "pyo3-build-config/abi3-py39"]
-abi3-py310 = ["abi3", "pyo3-build-config/abi3-py310"]
+abi3-py37 = ["abi3-py38", "pyo3-build-config/abi3-py37", "pyo3-ffi/abi3-py37"]
+abi3-py38 = ["abi3-py39", "pyo3-build-config/abi3-py38", "pyo3-ffi/abi3-py38"]
+abi3-py39 = ["abi3-py310", "pyo3-build-config/abi3-py39", "pyo3-ffi/abi3-py39"]
+abi3-py310 = ["abi3", "pyo3-build-config/abi3-py310", "pyo3-ffi/abi3-py310"]
 
 # Changes `Python::with_gil` and `Python::acquire_gil` to automatically initialize the
 # Python interpreter if needed.
@@ -126,6 +129,7 @@ harness = false
 
 [workspace]
 members = [
+    "pyo3-ffi",
     "pyo3-macros",
     "pyo3-macros-backend",
     "pytests/pyo3-benchmarks",

build.rs

@@ -1,48 +1,7 @@
 use std::{env, process::Command};
 
-use pyo3_build_config::{
-    bail, ensure,
-    pyo3_build_script_impl::{
-        cargo_env_var, env_var, errors::Result, resolve_interpreter_config, InterpreterConfig,
-        PythonVersion,
-    },
-};
-
-/// Minimum Python version PyO3 supports.
-const MINIMUM_SUPPORTED_VERSION: PythonVersion = PythonVersion { major: 3, minor: 6 };
-
-fn ensure_python_version(interpreter_config: &InterpreterConfig) -> Result<()> {
-    ensure!(
-        interpreter_config.version >= MINIMUM_SUPPORTED_VERSION,
-        "the configured Python interpreter version ({}) is lower than PyO3's minimum supported version ({})",
-        interpreter_config.version,
-        MINIMUM_SUPPORTED_VERSION,
-    );
-
-    Ok(())
-}
-
-fn ensure_target_pointer_width(interpreter_config: &InterpreterConfig) -> Result<()> {
-    if let Some(pointer_width) = interpreter_config.pointer_width {
-        // Try to check whether the target architecture matches the python library
-        let rust_target = match cargo_env_var("CARGO_CFG_TARGET_POINTER_WIDTH")
-            .unwrap()
-            .as_str()
-        {
-            "64" => 64,
-            "32" => 32,
-            x => bail!("unexpected Rust target pointer width: {}", x),
-        };
-
-        ensure!(
-            rust_target == pointer_width,
-            "your Rust target architecture ({}-bit) does not match your python interpreter ({}-bit)",
-            rust_target,
-            pointer_width
-        );
-    }
-    Ok(())
-}
+use pyo3_build_config::pyo3_build_script_impl::{cargo_env_var, errors::Result};
+use pyo3_build_config::{bail, InterpreterConfig};
 
 fn ensure_auto_initialize_ok(interpreter_config: &InterpreterConfig) -> Result<()> {
     if cargo_env_var("CARGO_FEATURE_AUTO_INITIALIZE").is_some() {
@@ -84,36 +43,6 @@ fn rustc_minor_version() -> Option<u32> {
     pieces.next()?.parse().ok()
 }
 
-fn emit_link_config(interpreter_config: &InterpreterConfig) -> Result<()> {
-    let target_os = cargo_env_var("CARGO_CFG_TARGET_OS").unwrap();
-    let is_extension_module = cargo_env_var("CARGO_FEATURE_EXTENSION_MODULE").is_some();
-    if target_os == "windows" || target_os == "android" || !is_extension_module {
-        // windows and android - always link
-        // other systems - only link if not extension module
-        println!(
-            "cargo:rustc-link-lib={link_model}{alias}{lib_name}",
-            link_model = if interpreter_config.shared {
-                ""
-            } else {
-                "static="
-            },
-            alias = if target_os == "windows" {
-                "pythonXY:"
-            } else {
-                ""
-            },
-            lib_name = interpreter_config.lib_name.as_ref().ok_or(
-                "attempted to link to Python shared library but config does not contain lib_name"
-            )?,
-        );
-        if let Some(lib_dir) = &interpreter_config.lib_dir {
-            println!("cargo:rustc-link-search=native={}", lib_dir);
-        }
-    }
-
-    Ok(())
-}
-
 /// Prepares the PyO3 crate for compilation.
 ///
 /// This loads the config from pyo3-build-config and then makes some additional checks to improve UX
@@ -122,23 +51,12 @@ fn emit_link_config(interpreter_config: &InterpreterConfig) -> Result<()> {
 /// Emits the cargo configuration based on this config as well as a few checks of the Rust compiler
 /// version to enable features which aren't supported on MSRV.
 fn configure_pyo3() -> Result<()> {
-    let interpreter_config = resolve_interpreter_config()?;
-
-    if env_var("PYO3_PRINT_CONFIG").map_or(false, |os_str| os_str == "1") {
-        print_config_and_exit(&interpreter_config);
-    }
-
-    ensure_python_version(&interpreter_config)?;
-    ensure_target_pointer_width(&interpreter_config)?;
-    ensure_auto_initialize_ok(&interpreter_config)?;
-
-    if !interpreter_config.suppress_build_script_link_lines {
-        emit_link_config(&interpreter_config)?;
-    }
+    let interpreter_config = pyo3_build_config::get();
 
     interpreter_config.emit_pyo3_cfgs();
 
     let rustc_minor_version = rustc_minor_version().unwrap_or(0);
+    ensure_auto_initialize_ok(interpreter_config)?;
 
     // Enable use of const generics on Rust 1.51 and greater
     if rustc_minor_version >= 51 {
@@ -150,22 +68,9 @@ fn configure_pyo3() -> Result<()> {
         println!("cargo:rustc-cfg=addr_of");
     }
 
-    // Extra lines come last, to support last write wins.
-    for line in &interpreter_config.extra_build_script_lines {
-        println!("{}", line);
-    }
-
     Ok(())
 }
 
-fn print_config_and_exit(config: &InterpreterConfig) {
-    println!("\n-- PYO3_PRINT_CONFIG=1 is set, printing configuration and halting compile --");
-    config
-        .to_writer(&mut std::io::stdout())
-        .expect("failed to print config to stdout");
-    std::process::exit(101);
-}
-
 fn main() {
     if let Err(e) = configure_pyo3() {
         eprintln!("error: {}", e.report());

pyo3-ffi/Cargo.toml

@@ -0,0 +1,39 @@
+[package]
+name = "pyo3-ffi"
+version = "0.15.1"
+description = "Python-API bindings for the PyO3 ecosystem"
+authors = ["PyO3 Project and Contributors <https://github.com/PyO3>"]
+keywords = ["pyo3", "python", "cpython", "ffi"]
+homepage = "https://github.com/pyo3/pyo3"
+repository = "https://github.com/pyo3/pyo3"
+categories = ["api-bindings", "development-tools::ffi"]
+license = "Apache-2.0"
+edition = "2018"
+
+[dependencies]
+libc = "0.2.62"
+
+[features]
+
+default = []
+
+# Use this feature when building an extension module.
+# It tells the linker to keep the python symbols unresolved,
+# so that the module can also be used with statically linked python interpreters.
+extension-module = []
+
+# Use the Python limited API. See https://www.python.org/dev/peps/pep-0384/ for more.
+abi3 = ["pyo3-build-config/abi3"]
+
+# With abi3, we can manually set the minimum Python version.
+abi3-py37 = ["abi3-py38", "pyo3-build-config/abi3-py37"]
+abi3-py38 = ["abi3-py39", "pyo3-build-config/abi3-py38"]
+abi3-py39 = ["abi3-py310", "pyo3-build-config/abi3-py39"]
+abi3-py310 = ["abi3", "pyo3-build-config/abi3-py310"]
+
+
+
+[build-dependencies]
+pyo3-build-config = { path = "../pyo3-build-config", version = "0.15.1", features = ["resolve-config"] }
+
+