"signal only works in main thread" in main thread

[RichardWarfield]

BPO	38904
Nosy	@vstinner, @ericsnowcurrently

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-11-24.06:06:20.843>
labels = ['type-bug', '3.8']
title = '"signal only works in main thread" in main thread'
updated_at = <Date 2019-12-17.17:14:27.640>
user = 'https://bugs.python.org/RichardWarfield'

bugs.python.org fields:

activity = <Date 2019-12-17.17:14:27.640>
actor = 'Richard Warfield'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2019-11-24.06:06:20.843>
creator = 'Richard Warfield'
dependencies = []
files = []
hgrepos = []
issue_num = 38904
keywords = []
message_count = 6.0
messages = ['357390', '357391', '358363', '358466', '358558', '358559']
nosy_count = 3.0
nosy_names = ['vstinner', 'eric.snow', 'Richard Warfield']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue38904'
versions = ['Python 3.8']

[RichardWarfield]

I have an application (https://github.com/litxio/ptghci) using embedded Python, which needs to set a signal handler (and use the prompt-toolkit library which itself sets signal handlers).

My call to signal.signal is guarded by a check that we're running in the main thread:

        if threading.current_thread() is threading.main_thread():
            print (threading.current_thread().name)
            signal.signal(signal.SIGINT, int_handler)

And the above indeed prints "MainThread". But this raises an exception:

File "app.py", line 45, in __init__
signal.signal(signal.SIGINT, int_handler)
File "/usr/lib/python3.8/signal.py", line 47, in signal
handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
ValueError: signal only works in main thread

This seems like something that should not happen.

Now, I tried to generate a simple replicating example but thus far haven't been able to do so -- simply calling signal.signal from PyRun_SimpleString doesn't do the trick, even within a pthreads thread.

[RichardWarfield]

I should mention, this behavior is new in 3.8.0. It did not occur in 3.7.x.

[ericsnowcurrently]

Before 3.8, the "signal" module checked against the thread in which the module was initially loaded, treating that thread as the "main" thread. That same was true (and still is) for the "threading" module. The problem for both modules is that the Python runtime may have actually been initialized in a different thread, which is the actual "main" thread.

Since Python 3.8 we store the ID of the thread where the runtime is initialized and use that in the check the "signal" module does. However, the "threading" module still uses the ID of the thread where it is first imported. So your check against "threading.main_thread()" must be in code that isn't running in the same thread where you ran Py_Initialize(). It probably used to work because you imported "signal" and "threading" for the first time in the same thread.

So what next?

First, I've created bpo-39042 to address the current different meanings of "main thread". That should resolve the discrepancy between the signal and threading modules.

Second, what can we do to help embedders make sure they are running their code in the main thread (e.g. when setting signals)? Is there any C-API we could add which would have helped you here?

[RichardWarfield]

Thanks for looking into this. Changing the behavior of the
"threading" module to be consistent with the runtime and "signal" module
would be sufficient, at least for my particular case. If the "if
threading.current_thread() is threading.main_thread()" guard worked as
expected it would be clear how to resolve the problem.

On Sat, Dec 14, 2019 at 6:25 AM Eric Snow <report@bugs.python.org> wrote:

Eric Snow <ericsnowcurrently@gmail.com> added the comment:

Before 3.8, the "signal" module checked against the thread in which the
module was initially loaded, treating that thread as the "main" thread.
That same was true (and still is) for the "threading" module. The problem
for both modules is that the Python runtime may have actually been
initialized in a different thread, which is the actual "main" thread.

Since Python 3.8 we store the ID of the thread where the runtime is
initialized and use that in the check the "signal" module does. However,
the "threading" module still uses the ID of the thread where it is first
imported. So your check against "threading.main_thread()" must be in code
that isn't running in the same thread where you ran Py_Initialize(). It
probably used to work because you imported "signal" and "threading" for the
first time in the same thread.

So what next?

First, I've created bpo-39042 to address the current different meanings
of "main thread". That should resolve the discrepancy between the signal
and threading modules.

Second, what can we do to help embedders make sure they are running their
code in the main thread (e.g. when setting signals)? Is there any C-API we
could add which would have helped you here?

----------
nosy: +eric.snow, vstinner

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue38904\>

---

[ericsnowcurrently]

So resolving bpo-39042 would be enough, particularly if we backported
the change to 3.8?

[RichardWarfield]

I think so, yes.

On Wed, Dec 18, 2019 at 1:10 AM Eric Snow <report@bugs.python.org> wrote:

Eric Snow <ericsnowcurrently@gmail.com> added the comment:

So resolving bpo-39042 would be enough, particularly if we backported
the change to 3.8?

----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue38904\>

---

[wangxudong-cq]

I have the same issue

Invoke Exception: signal only works in main thread of the main interpreter

version:both 3.8 and 3.10,
OS:just for 'Ubuntu 20.04.3 LTS ' not 'windows11'

[wangxudong-cq]

Name: invoke
Version: 1.7.0
Summary: Pythonic task execution
Home-page: https://pyinvoke.org
Author: Jeff Forcier
Author-email: jeff@bitprophet.org
License: BSD
Location: /home/willjiang/.local/lib/python3.10/site-packages
Requires:
Required-by: fabric

how to solve?

[ericsnowcurrently]

@pablogsal this is the issue (along with gh-83223) I was talking about.