multiprocessing: ApplyResult.get() hangs if the pool is terminated

[vstinner]

BPO	35478
Nosy	@pitrou, @vstinner, @applio, @pablogsal
PRs	bpo-35479: Optimize multiprocessing.Pool.join() #11136 WIP: bpo-35478: pool terminated #11139

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-09-21.22:26:36.594>
created_at = <Date 2018-12-13.00:30:51.742>
labels = ['3.8', 'library']
title = 'multiprocessing: ApplyResult.get() hangs if the pool is terminated'
updated_at = <Date 2021-09-21.22:26:36.593>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-09-21.22:26:36.593>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-21.22:26:36.594>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2018-12-13.00:30:51.742>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 35478
keywords = ['patch']
message_count = 3.0
messages = ['331724', '331793', '336940']
nosy_count = 4.0
nosy_names = ['pitrou', 'vstinner', 'davin', 'pablogsal']
pr_nums = ['11136', '11139']
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue35478'
versions = ['Python 3.8']

[vstinner]

The following code hangs:
---

import multiprocessing, time
pool = multiprocessing.Pool(1)
result = pool.apply_async(time.sleep, (1.0,))
pool.terminate()
result.get()

---

pool.terminate() terminates workers before time.sleep(1.0) completes, but the pool doesn't mark result as completed with an error.

Would it be possible to mark all pending tasks as failed? For example, "raise" a RuntimeError("pool terminated before task completed").

[vstinner]

Attached PR 11139 sets RuntimeError("Pool terminated") error in pending results if the pool is terminated.

[vstinner]

Pablo: since you worked on multiprocessing recently, did you see this bug? I'm not sure about my PR 11139...

If someone else wants to work on a fix, ignore my PR ;-)

[mdavis-xyz]

@vstinner this bug was closed as completed in 2021, however it seems the bug is still present in the latest version of Python (3.13.0 and also 3.14.0a1). I think this issue should be re-openned.

[vstinner]

Please open a new issue with a reproducer.