aiohttp VS requests proxy from environment variables difference

[MRigal]

Long story short

I know the problem was already shortly discussed in the issues #529 and #102 but I would like to present it again as I believe the discussion was not satisfying for people who would assume aiohttp would work as similar as possible to the requests library

Expected behaviour

I expect aiohttp to work as similar as possible to requests. This is not the case for proxies from environment variables

Actual behaviour

proxy info from environment variables is not used.

Steps to reproduce

The following code will work differently on a machine with proxy and without proxy. The aiohttp code will just raise TimeoutError permanently...

import aiohttp
import asyncio
import requests

async def main():
    params = {'language': 'DE', 'address': 'Yorckstr. 1, Berlin'}
    url = 'https://maps.googleapis.com/maps/api/geocode/json'
    timeout = 5.0
    # The first synchronous request works the same with or without proxy
    with requests.Session() as session:
        location = session.get(url=url, params=params, timeout=timeout)
        print(location)
    # The asynchronous aiohttp request works properly without proxy 
    # but times out on a machine requiring a proxy and having it in an environment variable
    async with aiohttp.ClientSession() as session:
        location = await session.get(url=url, params=params, timeout=timeout)
        print(location)

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

Your environment

As well OSX as Linux

Suggestions

1. Similarly to requests (https://github.com/kennethreitz/requests/blob/master/requests/compat.py#L57), we could import the getproxies() (https://docs.python.org/3/library/urllib.request.html#urllib.request.getproxies) function from urllib.requests. Although people expressed concerns about it, it would be similar behaviour
2. If not, it should be stated explicitely in the docs that the behaviour towards environment variables is different from requests!
3. We could create a session_from_env() method. But I would then also tend to update the _request() method to be able to take proxy_from_env=False as kwargs, in order to give enough flexibility to the user.

I'm willing to work on these ones, just tell me where the road goes!

[fafhrd91]

I don't have strong opinion on this. I am fine with add new class method to ClientSession, something like from_env. It just needs to be documented.

[fafhrd91]

thanks again, fixed by #1825

[MRigal]

You're welcome! We should just keep the remark from @kxepal in mind, to make it True by default for a possible 3.0 release. Although in that case we should also read the bypass_proxy variable from env too! :)

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.