Python 3.12 support blocked by aiohttp

[martin-traverse]

Is your feature request related to a problem? Please describe.

To use async IO with the azure-core library there is a dependency on aiohttp. The aiohttp project currently does not publish wheels for Python 3.12. Although the async transport is an optional dependency of azure-core, it is a hard dependency of the Azure fsspec implementation, adlfs. We would like to support Python 3.12 on Azure and cannot do so until this issue is resolved. This will also be impacting other projects that require async transport.

The lead maintainers of the aiohttp project are saying they need help to make the necessary updates, and that there are significant testing gaps which they are concerned about and wish to fix before a 3.12-compatible version can be released. To drive the point home, they also published this image in a recent thread on the subject:

Describe the solution you'd like

I think this is a broader issues than simply getting the 3.12 package for aiohttp out of the door. I can see two potential solutions:

• MS can contribute development effort to aiohttp, to resolve this specific issue and help maintain the package going forward as and when issues or feature requirements arise
• I see there is an alternate async transport in development using HTTPX, with sample code in the azure-core-experimental library and also in a separate module called "corehttp". If the HTTPX transport were completed and made available in azure-core that should remove the dependency on aiohttp for projects requiring async, including the Azure implementation of fsspec.

Describe alternatives you've considered

The Apache Arrow project is also working on an Azure implementation of their FileSystem interface, for projects using Apache Arrow these Python libraries may not be needed at all once that is available. This might help my particular project eventually, but probably doesn't help most people who want to use Python on Azure.

Additional context

Issue thread for aiohttp: aio-libs/multidict#887
Note in adlfs about requirement of aiohttp: fsspec/adlfs#452
Azure core docs on HTTP transport: CLIENT_LIBRARY_DEVELOPER#transport
Azure core experimental library: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/core/azure-core-experimental
Azure corehttp library: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/core/corehttp

[xiangyan99]

Thanks for reaching out.

It seems to me aiohttp has released 3.12 support. https://pypi.org/project/aiohttp/#files

Did I miss anything?

[github-actions]

Hi @martin-traverse. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[martin-traverse]

There is a thing called "multidict", a library which is part of the AIO project and sits near the bottom of the stack. The aiohttp package has a hard dependency on multidict, and multidict is not available for Python 3.12.

https://github.com/aio-libs/multidict
https://pypi.org/project/multidict/#files

Running pip install aiohttp, or pip install adlfs or even pip install azure-core[aio] on Python 3.12 will trigger a build for multidict, which fails if you don't have the C build chain installed and configured.

[kristapratico]

@kashifkhan