[3.12] gh-114440: Close writer pipe in multiprocessing.Queue, not concurrent.futures #114489
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ot concurrent.futures This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
vstinner
approved these changes
Jan 23, 2024
There was a problem hiding this comment.
LGTM.
If I recall correctly, @serhiy-storchaka had doubts about this backport to stable branches, and wanted extra changes. But his PR to make extra changes didn't land. And apparently, then we forgot to finish these backports.
Misc/NEWS.d/next/Library/2023-09-25-02-11-14.gh-issue-114440.b2TrqG.rst
Outdated
Show resolved
Hide resolved
|
!buildbot AMD64 Windows11 Refleaks 3.12 |
|
The regex 'AMD64 Windows11 Refleaks 3.12' did not match any buildbot builder. Is the requested builder in the list of stable builders? |
|
!buildbot AMD64 Windows11 Refleaks |
|
🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit 62af0b7 🤖 The command will test the builders whose names match following regular expression: The builders matched are:
|
Should this change be backported to 3.11? |
naveen521kk
pushed a commit
to naveen521kk/cpython
that referenced
this pull request
Feb 19, 2024
…ot concurrent.futures (pythonGH-114489) This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
naveen521kk
pushed a commit
to naveen521kk/cpython
that referenced
this pull request
Feb 19, 2024
…ot concurrent.futures (pythonGH-114489) This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
naveen521kk
pushed a commit
to naveen521kk/cpython
that referenced
this pull request
Feb 19, 2024
…ot concurrent.futures (pythonGH-114489) This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
naveen521kk
pushed a commit
to naveen521kk/cpython
that referenced
this pull request
Feb 19, 2024
…ot concurrent.futures (pythonGH-114489) This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
naveen521kk
pushed a commit
to naveen521kk/cpython
that referenced
this pull request
Feb 21, 2024
…ot concurrent.futures (pythonGH-114489) This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
naveen521kk
pushed a commit
to naveen521kk/cpython
that referenced
this pull request
Jul 11, 2024
…ot concurrent.futures (pythonGH-114489) This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
naveen521kk
pushed a commit
to naveen521kk/cpython
that referenced
this pull request
Jul 11, 2024
…ot concurrent.futures (pythonGH-114489) This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
naveen521kk
pushed a commit
to naveen521kk/cpython
that referenced
this pull request
Jul 11, 2024
…ot concurrent.futures (pythonGH-114489) This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
naveen521kk
pushed a commit
to msys2-contrib/cpython-mingw
that referenced
this pull request
Aug 5, 2024
…ot concurrent.futures (pythonGH-114489) This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
naveen521kk
pushed a commit
to naveen521kk/cpython
that referenced
this pull request
Sep 4, 2024
…ot concurrent.futures (pythonGH-114489) This was left out of the 3.12 backport for three related issues: - pythongh-107219 (which adds `self.call_queue._writer.close()` to `_ExecutorManagerThread` in `concurrent.futures`) - pythongh-109370 (which changes this to be only called on Windows) - pythongh-109047 (which moves the call to `multiprocessing.Queue`'s `_terminate_broken`) Without this change, ProcessPoolExecutor sometimes hangs on Windows when a worker process is terminated. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This was left out of the 3.12 backport for three related issues:
self.call_queue._writer.close()to_ExecutorManagerThreadinconcurrent.futures)multiprocessing.Queue's_terminate_broken)Without this change,
ProcessPoolExecutorsometimes hangs on Windows when a worker process is terminated.Co-authored-by: Victor Stinner vstinner@python.org
Co-authored-by: Serhiy Storchaka storchaka@gmail.com