Merge branch 'main' into expose-cross-compiling

.github/workflows/ci.yml

@@ -211,6 +211,10 @@ jobs:
       - name: Build (all additive features)
         run: cargo build --lib --tests --no-default-features --features "${{ steps.settings.outputs.all_additive_features }}"
 
+      - if: ${{ startsWith(matrix.python-version, 'pypy') }}
+        name: Build PyPy (abi3-py36)
+        run: cargo build --lib --tests --no-default-features --features "abi3-py36 ${{ steps.settings.outputs.all_additive_features }}"
+
       # Run tests (except on PyPy, because no embedding API).
       - if: ${{ !startsWith(matrix.python-version, 'pypy') }}
         name: Test
@@ -266,7 +270,11 @@ jobs:
 
   coverage:
     needs: [fmt]
-    runs-on: ubuntu-latest
+    name: coverage-${{ matrix.os }}
+    strategy:
+      matrix:
+        os: ["windows", "macos", "ubuntu"]
+    runs-on: ${{ matrix.os }}-latest
     steps:
       - uses: actions/checkout@v2
       - uses: actions/setup-python@v2
@@ -280,12 +288,19 @@ jobs:
             target
           key: coverage-cargo-${{ hashFiles('**/Cargo.toml') }}
         continue-on-error: true
+      - uses: actions-rs/toolchain@v1
+        with:
+          toolchain: nightly
+          override: true
+          profile: minimal
+          components: llvm-tools-preview
       - name: install cargo-llvm-cov
+        shell: bash
         run: |
-          wget https://github.com/taiki-e/cargo-llvm-cov/releases/download/v${CARGO_LLVM_COV_VERSION}/cargo-llvm-cov-x86_64-unknown-linux-gnu.tar.gz -qO- | tar -xzvf -
-          mv cargo-llvm-cov ~/.cargo/bin
+          host=$(rustc -Vv | grep host | sed 's/host: //')
+          curl -fsSL https://github.com/taiki-e/cargo-llvm-cov/releases/download/v${CARGO_LLVM_COV_VERSION}/cargo-llvm-cov-"$host".tar.gz | tar xzf - -C ~/.cargo/bin
         env:
-          CARGO_LLVM_COV_VERSION: 0.1.10
+          CARGO_LLVM_COV_VERSION: 0.1.11
       - uses: actions-rs/toolchain@v1
         with:
           toolchain: nightly
@@ -299,8 +314,10 @@ jobs:
           cargo llvm-cov --package $ALL_PACKAGES --no-report --features $(make list_all_additive_features)
           cargo llvm-cov --package $ALL_PACKAGES --no-report --features abi3 $(make list_all_additive_features)
           cargo llvm-cov --package $ALL_PACKAGES --no-run --lcov --output-path coverage.lcov
+        shell: bash
         env:
           ALL_PACKAGES: pyo3 pyo3-build-config pyo3-macros-backend pyo3-macros
       - uses: codecov/codecov-action@v2
         with:
           file: coverage.lcov
+          name: ${{ matrix.os }}

Architecture.md

@@ -28,8 +28,9 @@ To summarize, there are six main parts to the PyO3 codebase.
    - [`src/class`]
 5. [Procedural macros to simplify usage for users.](#5-procedural-macros-to-simplify-usage-for-users)
    - [`src/derive_utils.rs`], [`pyo3-macros`] and [`pyo3-macros-backend`]
-6. [`build.rs`](#6-buildrs)
+6. [`build.rs` and `pyo3-build-config`](#6-buildrs-and-pyo3-build-config)
    - [`build.rs`](https://github.com/PyO3/pyo3/tree/main/build.rs)
+   - [`pyo3-build-config`]
 
 ## 1. Low-level bindings of Python/C API
 
@@ -50,7 +51,7 @@ With `Py_LIMITED_API`, we can build a Python-version-agnostic binary called an
 `Py_37` means that the API is available from Python >= 3.7.
 There are also `Py_38`, `Py_39`, and so on.
 `PyPy` means that the API definition is for PyPy.
-Those flags are set in [`build.rs`](#6-buildrs).
+Those flags are set in [`build.rs`](#6-buildrs-and-pyo3-build-config).
 
 ## 2. Bindings to Python objects
 
@@ -166,25 +167,37 @@ some internal tricks for making `#[pyproto]` flexible.
 [`src/derive_utils.rs`] contains some utilities used in code generated by these proc-macros,
 such as parsing function arguments.
 
-## 6. `build.rs`
+## 6. `build.rs` and `pyo3-build-config`
 
-PyO3's [`build.rs`](https://github.com/PyO3/pyo3/tree/main/build.rs) is relatively long
-(about 900 lines) to support multiple architectures, interpreters, and usages.
-Below is a non-exhaustive list of its functionality:
+PyO3 supports a wide range of OSes, interpreters and use cases. The correct environment must be
+detected at build time in order to set up relevant conditional compilation correctly. This logic
+is captured in the [`pyo3-build-config`] crate, which is a `build-dependency` of `pyo3` and
+`pyo3-macros`, and can also be used by downstream users in the same way.
 
-- Cross-compiling support.
-  - If `TARGET` architecture and `HOST` architecture differ, we find cross compile information
-    from environment variables (`PYO3_CROSS_LIB_DIR` and `PYO3_CROSS_PYTHON_VERSION`) or system files.
+In [`pyo3-build-config`]'s `build.rs` the build environment is detected and inlined into the crate
+as a "config file". This works in all cases except for cross-compiling, where it is necessary to
+capture this from the `pyo3` `build.rs` to get some extra environment variables that Cargo doesn't
+set for build dependencies.
+
+The `pyo3` `build.rs` also runs some safety checks such as ensuring the Python version detected is
+actually supported.
+
+Some of the functionality of `pyo3-build-config`:
 - Find the interpreter for build and detect the Python version.
-  - We have to set some version flags like `Py_37`.
-  - If the interpreter is PyPy, we set `PyPy`.
-  - If `PYO3_NO_PYTHON` environment variable is set then the interpreter detection is bypassed
+  - We have to set some version flags like `#[cfg(Py_3_7)]`.
+  - If the interpreter is PyPy, we set `#[cfg(PyPy)`.
+  - If the `PYO3_CONFIG_FILE` environment variable is set then that file's contents will be used
+    instead of any detected configuration.
+  - If the `PYO3_NO_PYTHON` environment variable is set then the interpreter detection is bypassed
     entirely and only abi3 extensions can be built.
 - Check if we are building a Python extension.
   - If we are building an extension (e.g., Python library installable by `pip`),
     we don't link `libpython`.
     Currently we use the `extension-module` feature for this purpose. This may change in the future.
     See [#1123](https://github.com/PyO3/pyo3/pull/1123).
+- Cross-compiling configuration
+  - If `TARGET` architecture and `HOST` architecture differ, we find cross compile information
+    from environment variables (`PYO3_CROSS_LIB_DIR` and `PYO3_CROSS_PYTHON_VERSION`) or system files.
 
 <!-- External Links -->
 
@@ -194,6 +207,7 @@ Below is a non-exhaustive list of its functionality:
 
 [`pyo3-macros`]: https://github.com/PyO3/pyo3/tree/main/pyo3-macros
 [`pyo3-macros-backend`]: https://github.com/PyO3/pyo3/tree/main/pyo3-macros-backend
+[`pyo3-build-config`]: https://github.com/PyO3/pyo3/tree/main/pyo3-build-config
 
 <!-- Directories -->
 

CHANGELOG.md

@@ -6,6 +6,25 @@ PyO3 versions, please see the [migration guide](https://pyo3.rs/latest/migration
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+
+- Add implementations for `Py::as_ref()` and `Py::into_ref()` for `Py<PySequence>`, `Py<PyIterator>` and `Py<PyMapping>`. [#1682](https://github.com/PyO3/pyo3/pull/1682)
+- Add `PyTraceback` type to represent and format Python tracebacks. [#1977](https://github.com/PyO3/pyo3/pull/1977)
+
+### Changed
+
+- `#[classattr]` constants with a known magic method name (which is lowercase) no longer trigger lint warnings expecting constants to be uppercase. [#1969](https://github.com/PyO3/pyo3/pull/1969)
+
+### Fixed
+
+- Fix creating `#[classattr]` by functions with the name of a known magic method. [#1969](https://github.com/PyO3/pyo3/pull/1969)
+- Fix use of `catch_unwind` in `allow_threads` which can cause fatal crashes. [#1989](https://github.com/PyO3/pyo3/pull/1989)
+- Fix build failure on PyPy when abi3 features are activated. [#1991](https://github.com/PyO3/pyo3/pull/1991)
+- Fix mingw platform detection. [#1993](https://github.com/PyO3/pyo3/pull/1993)
+- Fix panic in `__get__` implementation when accessing descriptor on type object. [#1997](https://github.com/PyO3/pyo3/pull/1997)
+
 ## [0.15.0] - 2021-11-03
 
 ### Packaging

guide/pyo3_version.py

@@ -18,11 +18,13 @@
 
 if PYO3_VERSION_TAG == "main":
     PYO3_DOCS_URL = "https://pyo3.rs/main/doc"
+    PYO3_DOCS_VERSION = "latest"
     PYO3_CRATE_VERSION = 'git = "https://github.com/pyo3/pyo3"'
 else:
     # v0.13.2 -> 0.13.2
     version = PYO3_VERSION_TAG.lstrip("v")
     PYO3_DOCS_URL = f"https://docs.rs/pyo3/{version}"
+    PYO3_DOCS_VERSION = version
     PYO3_CRATE_VERSION = f'version = "{version}"'
 
 
@@ -35,6 +37,7 @@ def replace_section_content(section):
         section["Chapter"]["content"]
         .replace("{{#PYO3_VERSION_TAG}}", PYO3_VERSION_TAG)
         .replace("{{#PYO3_DOCS_URL}}", PYO3_DOCS_URL)
+        .replace("{{#PYO3_DOCS_VERSION}}", PYO3_DOCS_VERSION)
         .replace("{{#PYO3_CRATE_VERSION}}", PYO3_CRATE_VERSION)
     )
 

guide/src/SUMMARY.md

@@ -31,4 +31,5 @@
 [Appendix A: Migration Guide](migration.md)
 [Appendix B: PyO3 and rust-cpython](rust_cpython.md)
 [Appendix C: Trait bounds](trait_bounds.md)
+[Appendix D: Python typing hints](python_typing_hints.md)
 [CHANGELOG](changelog.md)

guide/src/building_and_distribution.md

@@ -44,7 +44,11 @@ Caused by:
   build_flags=WITH_THREAD
 ```
 
-> Note: if you save the output config to a file, it is possible to manually override the contents and feed it back into PyO3 using the `PYO3_CONFIG_FILE` env var. For now, this is an advanced feature that should not be needed for most users. The format of the config file and its contents are deliberately unstable and undocumented. If you have a production use-case for this config file, please file an issue and help us stabilize it!
+### Advanced: config files
+
+If you save the above output config from `PYO3_PRINT_CONFIG` to a file, it is possible to manually override the contents and feed it back into PyO3 using the `PYO3_CONFIG_FILE` env var.
+
+If your build environment is unusual enough that PyO3's regular configuration detection doesn't work, using a config file like this will give you the flexibility to make PyO3 work for you. To see the full set of options supported, see the documentation for the [`InterpreterConfig` struct](https://docs.rs/pyo3-build-config/{{#PYO3_DOCS_VERSION}}/pyo3_build_config/struct.InterpreterConfig.html).
 
 ## Building Python extension modules
 

guide/src/class.md

@@ -696,6 +696,112 @@ num=44
 num=-1
 ```
 
+## Making class method signatures available to Python
+
+The [`#[pyo3(text_signature = "...")]`](./function.md#text_signature) option for `#[pyfunction]` also works for classes and methods:
+
+```rust
+# #![allow(dead_code)]
+use pyo3::prelude::*;
+use pyo3::types::PyType;
+
+// it works even if the item is not documented:
+#[pyclass]
+#[pyo3(text_signature = "(c, d, /)")]
+struct MyClass {}
+
+#[pymethods]
+impl MyClass {
+    // the signature for the constructor is attached
+    // to the struct definition instead.
+    #[new]
+    fn new(c: i32, d: &str) -> Self {
+        Self {}
+    }
+    // the self argument should be written $self
+    #[pyo3(text_signature = "($self, e, f)")]
+    fn my_method(&self, e: i32, f: i32) -> i32 {
+        e + f
+    }
+    #[classmethod]
+    #[pyo3(text_signature = "(cls, e, f)")]
+    fn my_class_method(cls: &PyType, e: i32, f: i32) -> i32 {
+        e + f
+    }
+    #[staticmethod]
+    #[pyo3(text_signature = "(e, f)")]
+    fn my_static_method(e: i32, f: i32) -> i32 {
+        e + f
+    }
+}
+#
+# fn main() -> PyResult<()> {
+#     Python::with_gil(|py| {
+#         let inspect = PyModule::import(py, "inspect")?.getattr("signature")?;
+#         let module = PyModule::new(py, "my_module")?;
+#         module.add_class::<MyClass>()?;
+#         let class = module.getattr("MyClass")?;
+#
+#         if cfg!(not(Py_LIMITED_API)) || py.version_info() >= (3, 10)  {
+#             let doc: String = class.getattr("__doc__")?.extract()?;
+#             assert_eq!(doc, "");
+#
+#             let sig: String = inspect
+#                 .call1((class,))?
+#                 .call_method0("__str__")?
+#                 .extract()?;
+#             assert_eq!(sig, "(c, d, /)");
+#         } else {
+#             let doc: String = class.getattr("__doc__")?.extract()?;
+#             assert_eq!(doc, "");
+#
+#             inspect.call1((class,)).expect_err("`text_signature` on classes is not compatible with compilation in `abi3` mode until Python 3.10 or greater");
+#          }
+#
+#         {
+#             let method = class.getattr("my_method")?;
+#
+#             assert!(method.getattr("__doc__")?.is_none());
+#
+#             let sig: String = inspect
+#                 .call1((method,))?
+#                 .call_method0("__str__")?
+#                 .extract()?;
+#             assert_eq!(sig, "(self, /, e, f)");
+#         }
+#
+#         {
+#             let method = class.getattr("my_class_method")?;
+#
+#             assert!(method.getattr("__doc__")?.is_none());
+#
+#             let sig: String = inspect
+#                 .call1((method,))?
+#                 .call_method0("__str__")?
+#                 .extract()?;
+#             assert_eq!(sig, "(cls, e, f)");
+#         }
+#
+#         {
+#             let method = class.getattr("my_static_method")?;
+#
+#             assert!(method.getattr("__doc__")?.is_none());
+#
+#             let sig: String = inspect
+#                 .call1((method,))?
+#                 .call_method0("__str__")?
+#                 .extract()?;
+#             assert_eq!(sig, "(e, f)");
+#         }
+#
+#         Ok(())
+#     })
+# }
+```
+
+Note that `text_signature` on classes is not compatible with compilation in
+`abi3` mode until Python 3.10 or greater.
+
 ## Implementation details
 
 The `#[pyclass]` macros rely on a lot of conditional code generation: each `#[pyclass]` can optionally have a `#[pymethods]` block as well as several different possible `#[pyproto]` trait implementations.

guide/src/class/protocols.md

@@ -47,6 +47,25 @@ given signatures should be interpreted as follows:
   - `__str__(<self>) -> object (str)`
   - `__repr__(<self>) -> object (str)`
   - `__hash__(<self>) -> isize`
+    <details>
+    <summary>Disabling Python's default hash</summary>
+
+    By default, all `#[pyclass]` types have a default hash implementation from Python. Types which should not be hashable can override this by setting `__hash__` to `None`. This is the same mechanism as for a pure-Python class. This is done like so:
+
+    ```rust
+    # use pyo3::prelude::*;
+    #
+    #[pyclass]
+    struct NotHashable { }
+
+    #[pymethods]
+    impl NotHashable {
+        #[classattr]
+        const __hash__: Option<PyObject> = None;
+    }
+    ```
+    </details>
+
   - `__richcmp__(<self>, object, pyo3::basic::CompareOp) -> object`
   - `__getattr__(<self>, object) -> object`
   - `__setattr__(<self>, object, object) -> ()`
@@ -79,7 +98,7 @@ impl PyCounter {
     fn __new__(wraps: Py<PyAny>) -> Self {
         PyCounter { count: 0, wraps }
     }
-
+    #[args(args="*", kwargs="**")]
     fn __call__(
         &mut self,
         py: Python,
@@ -140,6 +159,24 @@ TODO; see [#1884](https://github.com/PyO3/pyo3/issues/1884)
 
   - `__len__(<self>) -> usize`
   - `__contains__(<self>, object) -> bool`
+    <details>
+    <summary>Disabling Python's default contains</summary>
+
+    By default, all `#[pyclass]` types with an `__iter__` method support a default implementation of the `in` operator. Types which do not want this can override this by setting `__contains__` to `None`. This is the same mechanism as for a pure-Python class. This is done like so:
+
+    ```rust
+    # use pyo3::prelude::*;
+    #
+    #[pyclass]
+    struct NoContains { }
+
+    #[pymethods]
+    impl NoContains {
+        #[classattr]
+        const __contains__: Option<PyObject> = None;
+    }
+    ```
+    </details>
   - `__getitem__(<self>, object) -> object`
   - `__setitem__(<self>, object, object) -> ()`
   - `__delitem__(<self>, object) -> ()`

guide/src/conversions/traits.md

@@ -17,7 +17,7 @@ variety of Rust types, which you can check out in the implementor list of
 [`FromPyObject`].
 
 [`FromPyObject`] is also implemented for your own Rust types wrapped as Python
-objects (see [the chapter about classes](class.md)).  There, in order to both be
+objects (see [the chapter about classes](../class.md)).  There, in order to both be
 able to operate on mutable references *and* satisfy Rust's rules of non-aliasing
 mutable references, you have to extract the PyO3 reference wrappers [`PyRef`]
 and [`PyRefMut`].  They work like the reference wrappers of