Skip to content

Commit

Permalink
inspect: gate behind experimental-inspect feature
Browse files Browse the repository at this point in the history
  • Loading branch information
davidhewitt committed Jan 15, 2023
1 parent ed0f338 commit 20ca3be
Show file tree
Hide file tree
Showing 15 changed files with 81 additions and 13 deletions.
5 changes: 5 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,10 @@ pyo3-build-config = { path = "pyo3-build-config", version = "0.17.3", features =
[features]
default = ["macros"]

# Enables pyo3::inspect module and additional type information on FromPyObject
# and IntoPy traits
experimental-inspect = []

# Enables macros: #[pyclass], #[pymodule], #[pyfunction] etc.
macros = ["pyo3-macros", "indoc", "unindent"]

Expand Down Expand Up @@ -105,6 +109,7 @@ full = [
"indexmap",
"eyre",
"anyhow",
"experimental-inspect",
]

[[bench]]
Expand Down
6 changes: 6 additions & 0 deletions guide/src/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,12 @@ If you do not enable this feature, you should call `pyo3::prepare_freethreaded_p

## Advanced Features

### `experimental-inspect`

This feature adds the `pyo3::inspect` module, as well as `IntoPy::type_output` and `FromPyObject::type_input` APIs to produce Python type "annotations" for Rust types.

This is a first step towards adding first-class support for generating type annotations automatically in PyO3, however work is needed to finish this off. All feedback and offers of help welcome on [issue #2454](https://github.com/PyO3/pyo3/issues/2454).

### `macros`

This feature enables a dependency on the `pyo3-macros` crate, which provides the procedural macros portion of PyO3's API:
Expand Down
2 changes: 2 additions & 0 deletions guide/src/python_typing_hints.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ PyO3 provides an easy to use interface to code native Python libraries in Rust.

Currently the best solution for the problem is to manually maintain `*.pyi` files and ship them along with the package.

There is a sketch of a roadmap towards completing [the `experimental-inspect` feature](./features.md#experimental-inspect) which may eventually lead to automatic type annotations generated by PyO3. This needs more testing and implementation, please see [issue #2454](https://github.com/PyO3/pyo3/issues/2454).

## Introduction to `pyi` files

`pyi` files (an abbreviation for `Python Interface`) are called "stub files" in most of the documentation related to them. A very good definition of what it is can be found in [old MyPy documentation](https://github.com/python/mypy/wiki/Creating-Stubs-For-Python-Modules):
Expand Down
3 changes: 3 additions & 0 deletions src/conversion.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

//! Defines conversions between Rust and Python types.
use crate::err::{self, PyDowncastError, PyResult};
#[cfg(feature = "experimental-inspect")]
use crate::inspect::types::TypeInfo;
use crate::pyclass::boolean_struct::False;
use crate::type_object::PyTypeInfo;
Expand Down Expand Up @@ -253,6 +254,7 @@ pub trait IntoPy<T>: Sized {
///
/// For most types, the return value for this method will be identical to that of [`FromPyObject::type_input`].
/// It may be different for some types, such as `Dict`, to allow duck-typing: functions return `Dict` but take `Mapping` as argument.
#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::Any
}
Expand Down Expand Up @@ -309,6 +311,7 @@ pub trait FromPyObject<'source>: Sized {
///
/// For most types, the return value for this method will be identical to that of [`IntoPy::type_output`].
/// It may be different for some types, such as `Dict`, to allow duck-typing: functions return `Dict` but take `Mapping` as argument.
#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
TypeInfo::Any
}
Expand Down
7 changes: 6 additions & 1 deletion src/conversions/std/map.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
use std::{cmp, collections, hash};

#[cfg(feature = "experimental-inspect")]
use crate::inspect::types::TypeInfo;
use crate::{
inspect::types::TypeInfo,
types::{IntoPyDict, PyDict},
FromPyObject, IntoPy, PyAny, PyErr, PyObject, Python, ToPyObject,
};
Expand Down Expand Up @@ -40,6 +41,7 @@ where
IntoPyDict::into_py_dict(iter, py).into()
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::dict_of(K::type_output(), V::type_output())
}
Expand All @@ -57,6 +59,7 @@ where
IntoPyDict::into_py_dict(iter, py).into()
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::dict_of(K::type_output(), V::type_output())
}
Expand All @@ -77,6 +80,7 @@ where
Ok(ret)
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
TypeInfo::mapping_of(K::type_input(), V::type_input())
}
Expand All @@ -96,6 +100,7 @@ where
Ok(ret)
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
TypeInfo::mapping_of(K::type_input(), V::type_input())
}
Expand Down
17 changes: 15 additions & 2 deletions src/conversions/std/num.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
#[cfg(feature = "experimental-inspect")]
use crate::inspect::types::TypeInfo;
use crate::{
exceptions, ffi, inspect::types::TypeInfo, AsPyPointer, FromPyObject, IntoPy, PyAny, PyErr,
PyObject, PyResult, Python, ToPyObject,
exceptions, ffi, AsPyPointer, FromPyObject, IntoPy, PyAny, PyErr, PyObject, PyResult, Python,
ToPyObject,
};
use std::convert::TryFrom;
use std::num::{
Expand All @@ -22,6 +24,7 @@ macro_rules! int_fits_larger_int {
(self as $larger_type).into_py(py)
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
<$larger_type>::type_output()
}
Expand All @@ -34,6 +37,7 @@ macro_rules! int_fits_larger_int {
.map_err(|e| exceptions::PyOverflowError::new_err(e.to_string()))
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
<$larger_type>::type_input()
}
Expand All @@ -55,6 +59,7 @@ macro_rules! int_convert_u64_or_i64 {
unsafe { PyObject::from_owned_ptr(py, $pylong_from_ll_or_ull(self)) }
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::builtin("int")
}
Expand All @@ -74,6 +79,7 @@ macro_rules! int_convert_u64_or_i64 {
}
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
Self::type_output()
}
Expand All @@ -93,6 +99,7 @@ macro_rules! int_fits_c_long {
unsafe { PyObject::from_owned_ptr(py, ffi::PyLong_FromLong(self as c_long)) }
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::builtin("int")
}
Expand All @@ -115,6 +122,7 @@ macro_rules! int_fits_c_long {
.map_err(|e| exceptions::PyOverflowError::new_err(e.to_string()))
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
Self::type_output()
}
Expand Down Expand Up @@ -183,6 +191,7 @@ mod fast_128bit_int_conversion {
}
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::builtin("int")
}
Expand All @@ -209,6 +218,7 @@ mod fast_128bit_int_conversion {
}
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
Self::type_output()
}
Expand Down Expand Up @@ -255,6 +265,7 @@ mod slow_128bit_int_conversion {
}
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::builtin("int")
}
Expand All @@ -278,6 +289,7 @@ mod slow_128bit_int_conversion {
}
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
Self::type_output()
}
Expand Down Expand Up @@ -324,6 +336,7 @@ macro_rules! nonzero_int_impl {
.map_err(|_| exceptions::PyValueError::new_err("invalid zero value"))
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
<$primitive_type>::type_input()
}
Expand Down
10 changes: 8 additions & 2 deletions src/conversions/std/set.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
use std::{cmp, collections, hash};

#[cfg(feature = "experimental-inspect")]
use crate::inspect::types::TypeInfo;
use crate::{
inspect::types::TypeInfo, types::set::new_from_iter, types::PySet, FromPyObject, IntoPy, PyAny,
PyObject, PyResult, Python, ToPyObject,
types::set::new_from_iter, types::PySet, FromPyObject, IntoPy, PyAny, PyObject, PyResult,
Python, ToPyObject,
};

impl<T, S> ToPyObject for collections::HashSet<T, S>
Expand Down Expand Up @@ -39,6 +41,7 @@ where
.into()
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::set_of(K::type_output())
}
Expand All @@ -54,6 +57,7 @@ where
set.iter().map(K::extract).collect()
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
TypeInfo::set_of(K::type_input())
}
Expand All @@ -69,6 +73,7 @@ where
.into()
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::set_of(K::type_output())
}
Expand All @@ -83,6 +88,7 @@ where
set.iter().map(K::extract).collect()
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
TypeInfo::set_of(K::type_input())
}
Expand Down
9 changes: 5 additions & 4 deletions src/conversions/std/slice.rs
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
use crate::{
inspect::types::TypeInfo, types::PyBytes, FromPyObject, IntoPy, PyAny, PyObject, PyResult,
Python, ToPyObject,
};
#[cfg(feature = "experimental-inspect")]
use crate::inspect::types::TypeInfo;
use crate::{types::PyBytes, FromPyObject, IntoPy, PyAny, PyObject, PyResult, Python, ToPyObject};

impl<'a> IntoPy<PyObject> for &'a [u8] {
fn into_py(self, py: Python<'_>) -> PyObject {
PyBytes::new(py, self).to_object(py)
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::builtin("bytes")
}
Expand All @@ -18,6 +18,7 @@ impl<'a> FromPyObject<'a> for &'a [u8] {
Ok(obj.downcast::<PyBytes>()?.as_bytes())
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
Self::type_output()
}
Expand Down
14 changes: 12 additions & 2 deletions src/conversions/std/string.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
use std::borrow::Cow;

#[cfg(feature = "experimental-inspect")]
use crate::inspect::types::TypeInfo;
use crate::{
inspect::types::TypeInfo, types::PyString, FromPyObject, IntoPy, Py, PyAny, PyObject, PyResult,
Python, ToPyObject,
types::PyString, FromPyObject, IntoPy, Py, PyAny, PyObject, PyResult, Python, ToPyObject,
};

/// Converts a Rust `str` to a Python object.
Expand All @@ -20,6 +21,7 @@ impl<'a> IntoPy<PyObject> for &'a str {
PyString::new(py, self).into()
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
<String>::type_output()
}
Expand All @@ -31,6 +33,7 @@ impl<'a> IntoPy<Py<PyString>> for &'a str {
PyString::new(py, self).into()
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
<String>::type_output()
}
Expand All @@ -51,6 +54,7 @@ impl IntoPy<PyObject> for Cow<'_, str> {
self.to_object(py)
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
<String>::type_output()
}
Expand All @@ -77,6 +81,7 @@ impl IntoPy<PyObject> for char {
PyString::new(py, self.encode_utf8(&mut bytes)).into()
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
<String>::type_output()
}
Expand All @@ -87,6 +92,7 @@ impl IntoPy<PyObject> for String {
PyString::new(py, &self).into()
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::builtin("str")
}
Expand All @@ -98,6 +104,7 @@ impl<'a> IntoPy<PyObject> for &'a String {
PyString::new(py, self).into()
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
<String>::type_output()
}
Expand All @@ -110,6 +117,7 @@ impl<'source> FromPyObject<'source> for &'source str {
ob.downcast::<PyString>()?.to_str()
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
<String>::type_input()
}
Expand All @@ -122,6 +130,7 @@ impl FromPyObject<'_> for String {
obj.downcast::<PyString>()?.to_str().map(ToOwned::to_owned)
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
Self::type_output()
}
Expand All @@ -140,6 +149,7 @@ impl FromPyObject<'_> for char {
}
}

#[cfg(feature = "experimental-inspect")]
fn type_input() -> TypeInfo {
<String>::type_input()
}
Expand Down
2 changes: 2 additions & 0 deletions src/conversions/std/vec.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#[cfg(feature = "experimental-inspect")]
use crate::inspect::types::TypeInfo;
use crate::types::list::new_from_iter;
use crate::{IntoPy, PyObject, Python, ToPyObject};
Expand Down Expand Up @@ -32,6 +33,7 @@ where
list.into()
}

#[cfg(feature = "experimental-inspect")]
fn type_output() -> TypeInfo {
TypeInfo::list_of(T::type_output())
}
Expand Down
1 change: 1 addition & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -445,6 +445,7 @@ mod macros;
#[cfg(all(test, feature = "macros"))]
mod test_hygiene;

#[cfg(feature = "experimental-inspect")]
pub mod inspect;

/// Test readme and user guide
Expand Down
Loading

0 comments on commit 20ca3be

Please sign in to comment.