Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On Windows machines with >= 32 cores (more than 64 logical processors) execution fails #132

Closed
ssproessig opened this issue Oct 3, 2023 · 1 comment · Fixed by #159
Closed

Comments

@ssproessig
Copy link

Shown as

(venv) my_dir>bump-pydantic --version      
bump-pydantic version: 0.7.0

(venv) my_dir>bump-pydantic .         
[13:44:38] Start bump-pydantic.                                                                                                                                                                                                                                                                                                                                        main.py:61
           Found 10 files to process                                                                                                                                                                                                                                                                                                                                   main.py:76
Exception in thread Thread-3 (_handle_workers):
Traceback (most recent call last):
  File "C:\_\py311\Lib\threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "C:\_\py311\Lib\threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
  File "C:\_\py311\Lib\multiprocessing\pool.py", line 522, in _handle_workers
    cls._wait_for_updates(current_sentinels, change_notifier)
  File "C:\_\py311\Lib\multiprocessing\pool.py", line 502, in _wait_for_updates
    wait(sentinels, timeout=timeout)
  File "C:\_\py311\Lib\multiprocessing\connection.py", line 878, in wait
    ready_handles = _exhaustive_wait(waithandle_to_obj.keys(), timeout)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\_\py311\Lib\multiprocessing\connection.py", line 810, in _exhaustive_wait
    res = _winapi.WaitForMultipleObjects(L, False, timeout)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: need at most 63 handles, got a sequence of length 66
Executing codemods... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━━  80% 0:00:01Process SpawnPoolWorker-64:

Afterwards it freezes without any processing. Even Ctrl+C doesn't work.

Googling for it suggests that this is a common problem with multiprocessing on Windows.

ssproessig added a commit to ssproessig/bump-pydantic that referenced this issue Oct 3, 2023
ssproessig added a commit to ssproessig/bump-pydantic that referenced this issue Oct 3, 2023
ssproessig added a commit to ssproessig/bump-pydantic that referenced this issue Oct 3, 2023
@Kludex
Copy link
Member

Kludex commented Mar 29, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants