Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exec_run: filedescriptor out of range in select() (python3) #2278

Closed
kub00n opened this issue Mar 18, 2019 · 0 comments · Fixed by #2865 · May be fixed by #2279
Closed

exec_run: filedescriptor out of range in select() (python3) #2278

kub00n opened this issue Mar 18, 2019 · 0 comments · Fixed by #2865 · May be fixed by #2279

Comments

@kub00n
Copy link

kub00n commented Mar 18, 2019

The following code works with python2, but throws error on python3:

import docker

files = [open('/tmp/test_' + str(i), 'w+b') for i in range(0, 1024)]
client = docker.from_env()
cnt = client.containers.run("ubuntu:18.04", command="sleep 10", detach=True, auto_remove=True)
cnt.exec_run("ls -ls")

Output:

    cnt.exec_run("ls -ls")
  File "/usr/lib/python3/dist-packages/docker/models/containers.py", line 185, in exec_run
    resp['Id'], detach=detach, tty=tty, stream=stream, socket=socket
  File "/usr/lib/python3/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/docker/api/exec_api.py", line 165, in exec_start
    return self._read_from_socket(res, stream, tty)
  File "/usr/lib/python3/dist-packages/docker/api/client.py", line 377, in _read_from_socket
    return six.binary_type().join(gen)
  File "/usr/lib/python3/dist-packages/docker/utils/socket.py", line 75, in frames_iter
    n = next_frame_size(socket)
  File "/usr/lib/python3/dist-packages/docker/utils/socket.py", line 62, in next_frame_size
    data = read_exactly(socket, 8)
  File "/usr/lib/python3/dist-packages/docker/utils/socket.py", line 47, in read_exactly
    next_data = read(socket, n - len(data))
  File "/usr/lib/python3/dist-packages/docker/utils/socket.py", line 27, in read
    select.select([socket], [], [])
ValueError: filedescriptor out of range in select()

if socket has fd > 1024, select will fail
Stack Overflow suggests using poll() in place of select(), as it's not bound by 1024 fd limit.

Environment :
ubuntu: 18.04
docker: 18.09.3
docker API: 1.39
python: 3.6.7
docker-py: 3.7.0

bieron added a commit to bieron/docker-py that referenced this issue Mar 18, 2019
Fixes docker#2278
because poll does not have select's builtin limit of 1024 descriptors
bieron added a commit to bieron/docker-py that referenced this issue Mar 18, 2019
Fixes docker#2278
because poll does not have select's builtin limit of 1024 descriptors

Signed-off-by: Jan Bieron <jbieron+github@gmail.com>
bieron added a commit to bieron/docker-py that referenced this issue Apr 4, 2019
Fixes docker#2278
because poll does not have select's builtin limit of 1024 descriptors

Signed-off-by: Jan Bieron <jbieron+github@gmail.com>
I-question-this added a commit to I-question-this/docker-py that referenced this issue Jul 22, 2021
Fixes docker#2278, which was originally addressed in docker#2279, but was not
properly merged. Additionally it did not address the problem
of poll not existing on Windows. This patch falls back on the
more limited select method if host system is Windows.

Signed-off-by: Tyler Westland <tylerofthewest@gmail.com>
milas pushed a commit that referenced this issue Apr 21, 2023
Fixes #2278, which was originally addressed in #2279, but was not
properly merged. Additionally it did not address the problem
of poll not existing on Windows. This patch falls back on the
more limited select method if host system is Windows.

Signed-off-by: Tyler Westland <tylerofthewest@gmail.com>
felixfontein added a commit to felixfontein/community.docker that referenced this issue May 10, 2023
…er-py#2865)

Fixes docker/docker-py#2278, which was originally addressed in docker/docker-py#2279, but was not
properly merged. Additionally it did not address the problem
of poll not existing on Windows. This patch falls back on the
more limited select method if host system is Windows.

Cherry-picked from docker/docker-py@a02ba74

Co-authored-by: Tyler Westland <tylerofthewest@gmail.com>
felixfontein added a commit to ansible-collections/community.docker that referenced this issue May 20, 2023
* socket: fix for errors on pipe close in Windows (docker/docker-py#3099)

Need to return data, not size. By returning an empty
string, EOF will be detected properly since `len()`
will be `0`.

Fixes docker/docker-py#3098.

Cherry-picked from docker/docker-py@f846232

Co-authored-by: Milas Bowman <milas.bowman@docker.com>

* socket: use poll() instead of select() except on Windows (docker/docker-py#2865)

Fixes docker/docker-py#2278, which was originally addressed in docker/docker-py#2279, but was not
properly merged. Additionally it did not address the problem
of poll not existing on Windows. This patch falls back on the
more limited select method if host system is Windows.

Cherry-picked from docker/docker-py@a02ba74

Co-authored-by: Tyler Westland <tylerofthewest@gmail.com>

* api: respect timeouts on Windows named pipes (docker/docker-py#3112)

Cherry-picked from docker/docker-py@9cadad0

Co-authored-by: Imogen <59090860+ImogenBits@users.noreply.github.com>

* Add URL to changelog.

* api: avoid socket timeouts when executing commands (docker/docker-py#3125)

Only listen to read events when polling a socket in order
to avoid incorrectly trying to read from a socket that is
not actually ready.

Cherry-picked from docker/docker-py@c5e582c

Co-authored-by: Loïc Leyendecker <loic.leyendecker@gmail.com>

---------

Co-authored-by: Milas Bowman <milas.bowman@docker.com>
Co-authored-by: Tyler Westland <tylerofthewest@gmail.com>
Co-authored-by: Imogen <59090860+ImogenBits@users.noreply.github.com>
Co-authored-by: Loïc Leyendecker <loic.leyendecker@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant