Is it possible to use from_py_with when deriving both FromPyObject and IntoPyObject?

[baxtersa]

Hi all. I've been working with FromPyObject for values passed as arguments from python code calling into Rust, but am having trouble implementing the IntoPyObject direction. We have a custom from_py_with function because one of our fields is a generic type with a const parameter, so we manually convert from a Vec to a fixed-size array. When adding the derivation for IntoPyObject, we get compiler errors on the pyo3(...) attribute saying error: expected 'attribute' or 'item'. Without derive(IntoPyObject), the rest of this code compiles just fine.

Here is a simplified reproduction of what I'm seeing

fn custom_fn(obj: &pyo3::Bound<pyo3::PyAny>) -> pyo3::PyResult<OtherType> {
    Ok(OtherType {
        other_field: 0
    })
}

pub struct OtherType {
    pub other_field: u32,
}

// No compilation error if I remove `pyo3::IntoPyObject` here
#[derive(pyo3::FromPyObject, pyo3::IntoPyObject)]
pub struct MinimalRepro {
    #[pyo3(from_py_with = "custom_fn")]
    pub field: OtherType,
}

and the compiler error we receive

error: expected `attribute` or `item`
   --> src/repro.rs:163:12
    |
163 |     #[pyo3(from_py_with = "custom_fn")]
    |            ^^^^^^^^^^^^

Is it possible to use from_py_with when also deriving IntoPyObject? I have seen that something like into_py_with is in-progress too, so maybe this is just a current limitation?

I'm using pyo3=0.23.4.

Thanks for the assistance.

[Icxolu]

I agree this should be supported and would consider this a bug on PyO3s side. The problem here is that derive macros declare the same
#[pyo3] attribute and thus get passed the same set of options. However they both only accept a subset of all options and reject everything else, i.e. IntoPyObject does not know what to do with from_py_with. So this problem is present for all attributes that are not common between these two.

Currently I can see two ways to solve this:

• We could just accept unknown/all pyo3 options. I think this is what serde does. This would solve the error, but the experience around single derive and meaningless options is a bit worse.
• We could have different top-level attributes for the derives. This way each macro would only get the options intended for it. It however also has its ergonomics issues: Having to name/remember multiple option entry points, less consistency across PyO3s macros, having to specify common options multiple times, ....

This is scratching the same issue David pointed out in #4850 (comment) (even though that one resolved itself in a different way) from a different angle.

I tend to think that the first options is probably acceptable. We could still reject unknown/misspelled/incompatible options and give appropriate errors, but would have to allow some no-op options on each derive.

---

As a workaround you could create a wrapper type around that field and implement FromPyObject on it manually instead of using from_py_with.

[baxtersa]

Thanks for the response. That workaround with a manual impl for FromPyObject should at least unblock us for now. I'll keep an eye on the related ongoing work with IntoPyObject derive macro attributes 👍