Segfault when dealing with references to objects with multiple base classes / virtual inheritance #4303
-
|
Hey together,
I'm currently experiencing trouble with segfaults in Pybind11 and I think that I have traced the issue back to a Python-binded function that returns a reference to an object in a container, where the object has virtual inheritance in its class hierarchy. Destroying such a handle in Python is only possible as long as the original object still exists. The error is reproducible on recent dev versions of Pybind11 (I've used fcb5554), so I think this is unrelated to similar older issues with multiple inheritance that are already fixed. This example reproduces the issue: cmake_minimum_required(VERSION 3.12.0)
project(pybind11_test)
find_package(pybind11 2.9.1 REQUIRED)
pybind11_add_module(virtual_inheritance virtual_inheritance.cpp)#include <pybind11/pybind11.h>
#include <iostream>
namespace py = pybind11;
struct A
{
int a = 0;
};
struct B0 : public virtual A
{
int b0 = 1;
};
struct B1 : public virtual A
{
int b1 = 2;
};
struct C
: public B0
, public B1
{
int c = 3;
C()
{}
};
/*
* A single value of C does not trigger a segfault on my machine,
* so we fill a vector until it goes wrong.
*/
class C_Container
{
// Bug in reproducer: Need to initialize the vector to the right amount
// in order to avoid invalidating the returned references
std::vector<C> vec; // = std::vector<C>(10000);
public:
C &reference()
{
auto & ref = vec.emplace_back();
return ref;
}
};
PYBIND11_MODULE(virtual_inheritance, m)
{
py::class_<A>(m, "A").def_readwrite("a", &A::a);
py::class_<B0, A>(m, "B0", py::multiple_inheritance())
.def_readwrite("b0", &B0::b0);
py::class_<B1, A>(m, "B1", py::multiple_inheritance())
.def_readwrite("b1", &B1::b1);
py::class_<C, B0, B1>(m, "C").def_readwrite("c", &C::c);
py::class_<C_Container>(m, "C_Container")
.def(py::init<>())
.def(
"reference",
[](C_Container &cont) -> C & { return cont.reference(); },
py::return_value_policy::reference_internal);
}#!/usr/bin/env python3
import build.virtual_inheritance as vi
def main():
cont = vi.C_Container()
for i in range(10000):
print("Iteration", i)
ref = cont.reference()
print(ref.a)
print(ref.b0)
print(ref.b1)
print(ref.c)
main()On my machine, the segfault triggers at iteration 2048, so I suppose that the boundary of memory paging is necessary to trigger it. The backtrace of the segfault is: And the troublesome line in the code is inside Alternatively, the following Python script makes the error trigger after iteration 9999 (with the same backtrace): #!/usr/bin/env python3
import build.virtual_inheritance as vi
def main():
cont = vi.C_Container()
reflist = []
for i in range(10000):
print("Iteration", i)
reflist.append(cont.reference())
print(reflist[-1].a)
print(reflist[-1].b0)
print(reflist[-1].b1)
print(reflist[-1].c)
main()For this one, I have an idea what might be the problem:
This does unfortunately not explain the segfault of the first Python script. Using |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
|
THE REPRODUCER HAS A BUG, I WILL NEED TO SEE IF I CAN FIX IT |
Beta Was this translation helpful? Give feedback.
-
|
Actually, the bug in the reproducer pointed me to the issue in my original use case. The bug in the reproducer is that the vector gets reallocated, thus invalidating the old references. They are used no longer, but deregistering the references will still fail. |
Beta Was this translation helpful? Give feedback.
Actually, the bug in the reproducer pointed me to the issue in my original use case.
Having a Python reference / reference_internal to a C++ object that no longer exists is undefined behavior in Pybind11. For this, the reference needs not even be used, as the deregistering routines of Pybind11 for objects with multiple inheritance assume that the object still exists. So, the lifetime of such references must be strictly not large than that of the referenced C++ object.
The bug in the reproducer is that the vector gets reallocated, thus invalidating the old references. They are used no longer, but deregistering the references will still fail.