[QUESTION] How to bind a pointer of an incomplete type?

[xpmemeda]

class A;

A* myfunc();

I just want to use function myfunc, but an error appeared when binding it.

incomplete type of A.

[bstaletic]

Wrap it in a lambda and return a py::capsule.

[xpmemeda]

Thank you, it works. But how about this?

void myfunc2(A* p);

If i want to get a pointer by myfunc and use it in myfunc2, type conversion seems troublesome especially when I have many such functions.

[YannickJadoul]

@xpmemeda I believe pybind11 needs a full type, and not just a forward declaration. At any rate, if you return A *, you also need to expose class_<A>, so you need a full type. Why can't you use the full type?

A type-checked alternative would be to write some kind of wrapper struct A_wrapper { A *ptr; } and expose that to pybind11.

[bstaletic]

If i want to get a pointer by myfunc and use it in myfunc2, type conversion seems troublesome especially when I have many such functions.

Right, the "obvious" solution is something like:

 [](py::capsule a_cap) { myfunct2(a_cap.get_pointer()); }

Both, me and @YannickJadoul, have previously made some ugly template metaprogramming tricks to automatically "replace" arguments of type X with type Y, which would be a start for your "whenever you see A*, use py::capsule". In my case, I was replacing T&& with T& (don't ask), but that's still a reference to T. You have a bigger problem at hand if you want to go that way.

I was also thinking about implementing a custom caster for A* <-> py::capsule, but that also would require a full definition of A. At least I'm pretty sure of it.

In short, pybind11 doesn't have a convenient way of dealing with FILE*-like APIs.

Why can't you use the full type?

One reason would be an API that uses (something like) FILE* - a library with a C wrapper that uses opaque pointers (C meaning of opaque, not pybind meaning of opaque).

[YannickJadoul]

@bstaletic That's the Python equivalent of void *, though. Not really type safe?

[bstaletic]

It's not any less type safe than the actual C API. You can always reinterpret_cast<WrongType*>. Is py::capsule really worse than that?

But sure, a wrapper type that holds the opaque pointer could be a better solution.

[YannickJadoul]

Is py::capsule really worse than that?

Yes, because pybind11 will not/cannot type check when passing a Python capsule, whereas it can if you expose a new type.

Depends a bit on the target audience, of course. If it's just personal, that's fine, and you can (probably/maybe/hopefully) trust yourself. If it's meant to be used by other Python users, I'm more sceptical. Python users shouldn't be able to cause segfaults.

[xpmemeda]

@xpmemeda I believe pybind11 needs a full type, and not just a forward declaration. At any rate, if you return A *, you also need to expose class_<A>, so you need a full type. Why can't you use the full type?

A type-checked alternative would be to write some kind of wrapper struct A_wrapper { A *ptr; } and expose that to pybind11.

Thanks for your advice. I can reach my goal through struct A_wrapper {A* ptr;} like below.

PYBIND11_MODULE(xxx, m){
py::class_<Awrapper>(m, "A_wrapper");
m.def("myfunc", [](){return A_wrapper(myfunc());});
m.def("myfunc2", [](A_wrapper a){myfunc2(a.ptr);});

Is there a way to set an automatic conversion from A_wrapper to A_wrapper.ptr when calling myfunc2 in python code? I will always use A_wrapper.ptr.

[YannickJadoul]

Yes, I can see two options:

1. Write a templated function that takes a callable thing and wraps it in a new lambda. Similar to e.g. some solution I recently wrote on Gitter (and inspired by my own code here; watch out, this is released as GPLv3, though):

template <typename T>
struct wrapped {
    using type = T;
    static T &&convert(T &&value) { return value; }
};

struct wrapped<A *> {
    using type = A_wrapper;
    static int *convert(A_wrapper &value) { return value.ptr; }
};

template <typename Return, typename... Args>
auto wrap(Return (*f)(Args...)) {
        return [f](typename wrapped<Args>::type &&... args) { return f(wrapped<Args>::convert(args)...); };
}

Then you could

m.def("myfunc2", wrap(myfunc2));

Disclaimer: code not tested, only works for function pointers currently, doesn't do the return value, etc. But it's meant to demonstrate the idea.
Also beware of who's taking ownership, if you return A_wrapper. Should that thing delete its ptr in its destructor? Tricky things to take into account!

2. Write a custom caster: https://pybind11.readthedocs.io/en/stable/advanced/cast/custom.html