Error while trying to use unix socket, when aiohttp>=3.10

[aljazmedic]

Long story short

• Expected behaviour: Communicating via unix socket should work.
• Actual behaviour: Error. Its traceback:

Traceback (most recent call last):
  File "/tmp/aiodocker_fail/example.py", line 12, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/tmp/aiodocker_fail/example.py", line 8, in main
    await docker.version()
  File "/tmp/aiodocker_fail/venv/lib/python3.11/site-packages/aiodocker/docker.py", line 190, in version
    data = await self._query_json("version")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/aiodocker_fail/venv/lib/python3.11/site-packages/aiodocker/docker.py", line 319, in _query_json
    async with self._query(
  File "/usr/lib/python3.11/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/aiodocker_fail/venv/lib/python3.11/site-packages/aiodocker/docker.py", line 231, in _query
    yield await self._do_query(
          ^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/aiodocker_fail/venv/lib/python3.11/site-packages/aiodocker/docker.py", line 257, in _do_query
    await self._check_version()
  File "/tmp/aiodocker_fail/venv/lib/python3.11/site-packages/aiodocker/docker.py", line 209, in _check_version
    ver = await self._query_json("version", versioned_api=False)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/aiodocker_fail/venv/lib/python3.11/site-packages/aiodocker/docker.py", line 319, in _query_json
    async with self._query(
  File "/usr/lib/python3.11/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/aiodocker_fail/venv/lib/python3.11/site-packages/aiodocker/docker.py", line 231, in _query
    yield await self._do_query(
          ^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/aiodocker_fail/venv/lib/python3.11/site-packages/aiodocker/docker.py", line 267, in _do_query
    response = await self.session.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/aiodocker_fail/venv/lib/python3.11/site-packages/aiohttp/client.py", line 510, in _request
    raise NonHttpUrlClientError(url)
aiohttp.client_exceptions.NonHttpUrlClientError: unix://localhost/version
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f5ede16bf10>

How to reproduce

In fresh virtual environment run pip install aiodocker, the following script fails:

#!/usr/bin/env python3
from aiodocker import Docker
import asyncio

async def main():
    docker = Docker()
    await docker.version()

if __name__ == "__main__":
    asyncio.run(main())

Your environment

Just the latest version of aiodocker with combination of 3.10 aiohttp.

My complete requirements.txt of the environment:

aiodocker==0.22.2
aiohappyeyeballs==2.3.4
aiohttp==3.10.0
aiosignal==1.3.1
attrs==24.1.0
frozenlist==1.4.1
idna==3.7
multidict==6.0.5
yarl==1.9.4

Current fix for me is the usage of aiohttp==3.9.0

[bdraco]

Likely similar to #8482

[Artucuno]

+1

[Dreamsorcerer]

I actually didn't realise the client can make unix socket requests.

Looking at the connectors, there is UnixConnector and also NamedPipeConnector. It is also possible to create custom connectors...

Maybe we need the connector to define what a valid URL looks like and then the client should use that information?

[bdraco]

That sounds like a better design to me. The downside is it might technically be a breaking change for anyone subclassing the BaseConnector as they would have to add this now.

[Dreamsorcerer]

Maybe allow anything in the BaseConnector for 3.10, and have it abstract in v4.

[ualex73]

Same problem here with monitor_docker component using aiodocker. The aiohttp 3.10.x broke the unix socket and the tcp socket.

Following worked in aiohttp 3.9.5:
tcp://IP:2375
unix://localhost

Both are broken now, for the same reason. The tcp can be fixed by replacing it with http.

[ualex73]

@Dreamsorcerer

UnixConnector

It looks like the aiodocker is already doing that, I see in the code:

connector = aiohttp.UnixConnector(docker_host[UNIX_PRE_LEN:])
and also a pipe one:
connector = aiohttp.NamedPipeConnector(docker_host[WIN_PRE_LEN:].replace("/", "\"))

[Dreamsorcerer]

It looks like the aiodocker is already doing that, I see in the code:

We're aware of that. The point is that currently the client just tries a whitelist of protocols, but really the whitelist should depend which connector is being used (http:// won't work if UnixConnector is used, for example).

I guess tcp:// should be added to the list? I'm not aware of this being a recognised URL protocol though, so not sure why that is in use..

[bdraco]

I don't think tcp:// is needed since that is always expected to have a higher level http,https,ws,or wss.

[Dreamsorcerer]

I don't think tcp:// is needed since that is always expected to have a higher level http,https,ws,or wss.

The linked issue suggests there are multiple people doing this in homeassistant though? ualex73/monitor_docker#157 (comment)

[bdraco]

I guess we do need it. I couldn't find anything in the aiodocker code that uses it though but probably because its passed in