Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiprocessing.Event.wait race condition when timeout is not none on Mac OS #95826

Closed
mcclurem opened this issue Aug 9, 2022 · 1 comment
Closed
Labels
3.9 only security fixes topic-multiprocessing type-bug An unexpected behavior, bug, or error

Comments

@mcclurem
Copy link

mcclurem commented Aug 9, 2022

Current Python.org docs for multiprocessing.Event link directly to threading.Event
The documents for threading.Event.wait read (emphasis mine)

This method returns True if and only if the internal flag has been set to true, either before the wait call or after the wait starts, so it will always return True except if a timeout is given and the operation times out.

It appears that a race condition exists in the multiprocessing implementation but not in the threading implementation.
If we implement a child process that just does a set/clear on an event over and over (as a heartbeat), then in the parent process do a wait() on that event (with extremely long timeout), we would expect to never see a timeout.

I've attached the proof of concept below showing how both threading.Event and multiprocessing.Event yield different results.

Possible cousin bug: #85772

Observed on:

  • Mac OS 12.3 Monterey
  • Apple M1 Max MacBook Pro
  • Homebrew python: Python 3.9.13 (main, May 24 2022, 21:13:54) [Clang 13.0.0 (clang-1300.0.29.30)]
#!/usr/bin/env python3

import multiprocessing
import threading

class SimpleRepro:
    def __init__(self, use_thread=False):
        if use_thread:
            self.heartbeat_event = threading.Event()
            self.shutdown_event = threading.Event()
            self.child_proc = threading.Thread(target=self.child_process)
        else:
            self.heartbeat_event = multiprocessing.Event()
            self.shutdown_event = multiprocessing.Event()
            self.child_proc = multiprocessing.Process(target=self.child_process, daemon=True)
        self.child_proc.start()

    def child_process(self):
        while True:
            if self.shutdown_event.is_set():
                return
            self.heartbeat_event.set()
            self.heartbeat_event.clear()

    def test_heartbeat(self):
        any_failures=False
        for i in range(10000):
            success = self.heartbeat_event.wait(100)
            if not success:
                print(f"Failed at iteration {i}")
                any_failures = True
        self.shutdown_event.set()
        if not any_failures:
            print("Successfully tested 10000 times without failure")


if __name__ == '__main__':
    print("Testing with multiprocessing.Event")
    foo = SimpleRepro(use_thread=False)
    foo.test_heartbeat()
    print("Testing with threading.Event")
    foo = SimpleRepro(use_thread=True)
    foo.test_heartbeat()
@mdboom mdboom added type-bug An unexpected behavior, bug, or error topic-multiprocessing 3.9 only security fixes labels Aug 9, 2022
ivarref added a commit to ivarref/cpython that referenced this issue Nov 13, 2024
…it()-ing. Raise an exception if that is the case. Fix race condition in multiprocessing.Event.wait() as described in python#95826
@gpshead
Copy link
Member

gpshead commented Nov 14, 2024

Duplicate of #126434 & #85772. lets follow up on those.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.9 only security fixes topic-multiprocessing type-bug An unexpected behavior, bug, or error
Projects
Development

No branches or pull requests

3 participants