Skip to content

Double-nested gil_scoped_release in a pythread causes fatal interpreter error #1276

Open
@KyleFromKitware

Description

@KyleFromKitware

Assume the following scenario:

  1. Create a new pythread
  2. Call a C++ function from within that pythread
  3. That C++ function calls some Python
  4. That Python calls some MORE C++

Here is the minimum working example:

#include <pybind11/pybind11.h>
#include <pybind11/embed.h>

#include <iostream>
#include <thread>

static void thread()
{
  pybind11::gil_scoped_acquire acquire;
  {
    // Pretend this block is nested inside Python
    pybind11::gil_scoped_release release;
    std::cout << "Calling C++ code" << std::endl;
  }
}

static void init_threads()
{
  // NOTE: This is a workaround for #1273
  if (!PyEval_ThreadsInitialized())
  {
    {
      pybind11::gil_scoped_acquire acquire;
    }
    PyEval_SaveThread();
  }
}

static const char thread_code[] =
"import threading\n"
"import testmod\n"
"t = threading.Thread(target=testmod.func)\n"
"t.start()\n"
"t.join()\n";

PYBIND11_EMBEDDED_MODULE(testmod, m)
{
  m.def("func", thread, pybind11::call_guard<pybind11::gil_scoped_release>());
}

int main()
{
  pybind11::scoped_interpreter interp;
  init_threads();

  {
    pybind11::gil_scoped_acquire acquire;
    pybind11::exec(thread_code);
  }

  return 0;
}

When the inner nested gil_scoped_release is destructed, it causes the Python interpreter to throw a fatal error:

Calling C++ code
Fatal Python error: Invalid thread state for this thread
Aborted (core dumped)

because there are two different thread states for the same thread, one created by the pythread and one created by pybind.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions