Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed potential issues in examples/show_progress.py #829

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

DavidSingh3
Copy link

Because the examples/show_progress.py file monkey patches modules using gevent.monkey.patch_all(), implementing it like this in a real project can lead to weird errors:

Traceback (most recent call last):
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/multiprocessing/managers.py", line 189, in accepter
    c = self.listener.accept()
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/multiprocessing/connection.py", line 463, in accept
    c = self._listener.accept()
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/multiprocessing/connection.py", line 609, in accept
    s, self._last_accepted = self._socket.accept()
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/site-packages/gevent/_socket3.py", line 212, in accept
    self._wait(self._read_event)
  File "src/gevent/_hub_primitives.py", line 317, in gevent._gevent_c_hub_primitives.wait_on_socket
  File "src/gevent/_hub_primitives.py", line 322, in gevent._gevent_c_hub_primitives.wait_on_socket
  File "src/gevent/_hub_primitives.py", line 304, in gevent._gevent_c_hub_primitives._primitive_wait
  File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
  File "src/gevent/_hub_primitives.py", line 46, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
  File "src/gevent/_hub_primitives.py", line 55, in gevent._gevent_c_hub_primitives.WaitOperationsGreenlet.wait
  File "src/gevent/_waiter.py", line 154, in gevent._gevent_c_waiter.Waiter.get
  File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
greenlet.error: cannot switch to a different thread
python-BaseException
Traceback (most recent call last):
  File "/home/david/.config/JetBrains/PyCharm2024.1/scratches/scratch_37.py", line 9, in <module>
    print(manager.list())
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/multiprocessing/managers.py", line 723, in temp
    token, exp = self._create(typeid, *args, **kwds)
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/multiprocessing/managers.py", line 606, in _create
    conn = self._Client(self._address, authkey=self._authkey)
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/multiprocessing/connection.py", line 508, in Client
    answer_challenge(c, authkey)
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/multiprocessing/connection.py", line 752, in answer_challenge
    message = connection.recv_bytes(256)         # reject large message
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes
    buf = self._recv(4)
  File "/home/david/ffmpeg-python/.venv/lib/python3.10/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
BlockingIOError: [Errno 11] Resource temporarily unavailable
python-BaseException

Process finished with exit code 1

The above error occurs when the project includes something like this codeblock after the patching has been done:

from multiprocessing import Manager

with Manager() as manager:
    print(manager.list())

After some trial and error, I found that it's only necessary to patch the selectors module instead of running patch_all().
I implemented this in a context manager that restores the original selectors module afterwards, to avoid any other potential issues further down in a project when this module has been patched for no reason.

I also moved the main logic to a separate function, so that this issue can be tested more easily with a scratch file like this:

from multiprocessing import Manager

from examples import show_progress

if __name__ == "__main__":
    show_progress.main()

    with Manager() as manager:
        print(manager.list())

This resolves the error I mentioned, and the progress bar still works correctly.

Thanks for providing this repository, it has been nice using ffmpeg video generation in my project while having the progress bar instead of the default output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants