Fix IntoPyCallbackOutput paper cuts #2326

mejrs · 2022-04-23T21:04:56Z

Changes impl<T, const N: usize> IntoPy<PyObject> for [T; N] to use IntoPy for it elements instead. Fixes IntoPyCallbackOutput<_> is not implimented for arrays of pyclass #2323
Adds a new trait to result-wrap function outputs which runs before IntoPyCallbackOutput , which (generally) generates a better compile error.
For example, consider this:

struct Blah;

#[pyfunction]
fn blah() -> Blah{
    Blah
}

Current error:

error[E0277]: the trait bound `Blah: IntoPyCallbackOutput<_>` is not satisfied
   --> src\lib.rs:5:1
    |
5   | #[pyfunction]
    | ^^^^^^^^^^^^^ the trait `IntoPyCallbackOutput<_>` is not implemented for `Blah`
    |
note: required by a bound in `pyo3::callback::convert`
   --> C:\Users\bruno\rust\pyo3\src\callback.rs:182:8
    |
182 |     T: IntoPyCallbackOutput<U>,
    |        ^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `pyo3::callback::convert`
    = note: this error originates in the attribute macro `pyfunction` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0277`.

Error with this PR:

error[E0599]: the method `into_result` exists for struct `Blah`, but its trait bounds were not satisfied
   --> src\lib.rs:5:1
    |
3   | struct Blah;
    | ------------
    | |
    | method `into_result` not found for this
    | doesn't satisfy `Blah: IntoPy<Py<PyAny>>`
    | doesn't satisfy `Blah: IntoResult<Result<Blah, PyErr>>`
4   |
5   | #[pyfunction]
    | ^^^^^^^^^^^^^ method cannot be called on `Blah` due to unsatisfied trait bounds
    |
    = note: the following trait bounds were not satisfied:
            `Blah: IntoPy<Py<PyAny>>`
            which is required by `Blah: IntoResult<Result<Blah, PyErr>>`
            `&Blah: IntoPy<Py<PyAny>>`
            which is required by `&Blah: IntoResult<Result<&Blah, PyErr>>`
            `&mut Blah: IntoPy<Py<PyAny>>`
            which is required by `&mut Blah: IntoResult<Result<&mut Blah, PyErr>>`
note: the following trait must be implemented
   --> C:\Users\bruno\rust\pyo3\src\conversion.rs:182:1
    |
182 | / pub trait IntoPy<T>: Sized {
183 | |     /// Performs the conversion.
184 | |     fn into_py(self, py: Python<'_>) -> T;
185 | | }
    | |_^
    = note: this error originates in the attribute macro `pyfunction` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0599`.

Todo:

Changelog
Tests
Some ui tests

davidhewitt

Thanks!

First commit message - "Make arrays of pyclasses easier to sue" - oops :)

src/callback.rs

src/conversions/array.rs

davidhewitt

Really nice! Just a few final thoughts, and I think it would be better to change from IntoPyResult<R> -> IntoPyResult<T>. After that, I think this is good to merge 👍

src/callback.rs

src/conversions/array.rs

tests/ui/invalid_result_conversion.stderr

mejrs · 2022-05-06T22:57:13Z

pyo3-macros-backend/src/method.rs

+            if false {
+                use _pyo3::impl_::ghost::IntoPyResult;
+                ret.into_py_result();
+            }


I prefer this approach over what I did before. This won't actually call into_py_result if an user accidentally writes it for their type.

mejrs · 2022-05-06T23:00:07Z

src/impl_/ghost.rs

+/// If it does nothing, was it ever really there? 👻
+///
+/// This is code that is just type checked to e.g. create better compile errors,
+/// but that never affects anything at runtime,
+use crate::{IntoPy, PyErr, PyObject};
+
+#[allow(clippy::wrong_self_convention)]
+pub trait IntoPyResult<T> {
+    fn into_py_result(&mut self) {}
+}
+
+impl<T> IntoPyResult<T> for T where T: IntoPy<PyObject> {}
+
+impl<T, E> IntoPyResult<T> for Result<T, E>
+where
+    T: IntoPy<PyObject>,
+    E: Into<PyErr>,
+{
+}


I prefer this approach of code that doesn't actually do anything 😉

mejrs · 2022-05-06T23:02:31Z

tests/ui/missing_intopy.stderr

+error[E0277]: the trait bound `Blah: IntoPyCallbackOutput<_>` is not satisfied
+   --> tests/ui/missing_intopy.rs:3:1
+    |
+3   | #[pyo3::pyfunction]
+    | ^^^^^^^^^^^^^^^^^^^ the trait `IntoPyCallbackOutput<_>` is not implemented for `Blah`
+    |
+note: required by a bound in `pyo3::callback::convert`
+   --> src/callback.rs
+    |
+    |     T: IntoPyCallbackOutput<U>,
+    |        ^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `pyo3::callback::convert`
+    = note: this error originates in the attribute macro `pyo3::pyfunction` (in Nightly builds, run with -Z macro-backtrace for more info)


It's unfortunate this IntoPyCallbackOutput part is back - but overall I think it's worth it. The first part of the error is very very good.

davidhewitt

Sneaky! Nicely done 👍

davidhewitt · 2022-05-08T06:54:31Z

src/impl_/ghost.rs

+/// but that never affects anything at runtime,
+use crate::{IntoPy, PyErr, PyObject};
+
+#[allow(clippy::wrong_self_convention)]


Is this lint actually necessary? I wonder if it's a hangover from experimentation.

Yes, that hits the methods called into_* usually take self by value lint. The reason it takes &mut self is to prevent a cascade like this:

which is required by `Blah: IntoResult<Result<Blah, PyErr>>` which is required by `Blah: IntoPyResult<Result<Blah, PyErr>>` `&Blah: IntoPy<Py<PyAny>>` which is required by `&Blah: IntoResult<Result<&Blah, PyErr>>` which is required by `&Blah: IntoPyResult<Result<&Blah, PyErr>>` `&mut Blah: IntoPy<Py<PyAny>>` which is required by `&mut Blah: IntoResult<Result<&mut Blah, PyErr>>` which is required by `&mut Blah: IntoPyResult<Result<&mut Blah, PyErr>>`

Ah I see! Do you think we should name the method differently then? E.g. into_py_result_hint or assert_into_py_result?

davidhewitt

Thanks, LGTM! To squash out the various interim commits I'm gonna squash-merge this one 👍

mejrs mentioned this pull request Apr 23, 2022

IntoPyCallbackOutput<_> is not implimented for arrays of pyclass #2323

Closed

davidhewitt reviewed Apr 24, 2022

View reviewed changes

src/callback.rs Outdated Show resolved Hide resolved

src/callback.rs Outdated Show resolved Hide resolved

src/callback.rs Outdated Show resolved Hide resolved

src/callback.rs Outdated Show resolved Hide resolved

src/conversions/array.rs Show resolved Hide resolved

mejrs marked this pull request as draft April 28, 2022 21:48

mejrs force-pushed the callback branch 2 times, most recently from 7e54611 to 7477842 Compare May 1, 2022 09:07

mejrs added 2 commits May 1, 2022 11:22

Implement IntoPy for arrays of IntoPy

bcb1762

Improve IntoPyCallbackOutput compile error

139cae1

mejrs force-pushed the callback branch from a6d6640 to 139cae1 Compare May 1, 2022 09:23

Merge branch 'main' into callback

7af2326

mejrs force-pushed the callback branch from c151a40 to e4645a5 Compare May 1, 2022 14:31

Add tests

924cb28

mejrs force-pushed the callback branch from e4645a5 to 924cb28 Compare May 1, 2022 14:39

mejrs added 2 commits May 1, 2022 21:53

Rename helper trait

f348c17

Add to changelog and migration

acae555

mejrs marked this pull request as ready for review May 1, 2022 20:49

davidhewitt reviewed May 3, 2022

View reviewed changes

mejrs force-pushed the callback branch from 299e1e5 to 66c58ef Compare May 6, 2022 17:38

A different approach

63f67f1

mejrs force-pushed the callback branch from 66c58ef to 63f67f1 Compare May 6, 2022 21:33

Ignore ghost in coverage

44f3bc7

mejrs commented May 6, 2022

View reviewed changes

davidhewitt approved these changes May 8, 2022

View reviewed changes

Change method name

ab0bf9e

davidhewitt approved these changes May 9, 2022

View reviewed changes

davidhewitt merged commit c57e509 into PyO3:main May 9, 2022

davidhewitt mentioned this pull request Jul 3, 2022

Inline the layout-related methods as these basically change types and could benefit from cold branch removal. PyO3/rust-numpy#332

Merged

mejrs mentioned this pull request Aug 23, 2022

pyproto: remove deprecated feature #2587

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix IntoPyCallbackOutput paper cuts #2326

Fix IntoPyCallbackOutput paper cuts #2326

mejrs commented Apr 23, 2022 •

edited

Loading

davidhewitt left a comment

davidhewitt left a comment

mejrs May 6, 2022

mejrs May 6, 2022

mejrs May 6, 2022

davidhewitt left a comment

davidhewitt May 8, 2022

mejrs May 8, 2022 •

edited

Loading

davidhewitt May 9, 2022

mejrs May 9, 2022

davidhewitt left a comment

Fix IntoPyCallbackOutput paper cuts #2326

Fix IntoPyCallbackOutput paper cuts #2326

Conversation

mejrs commented Apr 23, 2022 • edited Loading

davidhewitt left a comment

Choose a reason for hiding this comment

davidhewitt left a comment

Choose a reason for hiding this comment

mejrs May 6, 2022

Choose a reason for hiding this comment

mejrs May 6, 2022

Choose a reason for hiding this comment

mejrs May 6, 2022

Choose a reason for hiding this comment

davidhewitt left a comment

Choose a reason for hiding this comment

davidhewitt May 8, 2022

Choose a reason for hiding this comment

mejrs May 8, 2022 • edited Loading

Choose a reason for hiding this comment

davidhewitt May 9, 2022

Choose a reason for hiding this comment

mejrs May 9, 2022

Choose a reason for hiding this comment

davidhewitt left a comment

Choose a reason for hiding this comment

mejrs commented Apr 23, 2022 •

edited

Loading

mejrs May 8, 2022 •

edited

Loading