aiohttp.ClientSession.get context manager gives AttributeError: __aexit__

[bradwood]

Long story short

It appears as if the context manager on aiohttp.ClientSession.get is failing for some reason.
It works in an isolated test, but when i run it here I get this. I tried the get() without headers too to see if that was upsetting it -- no difference, yet the session looks okay. Timeout is set to 60 sec and it fails well before that.

Expected behaviour

I expect __aexit__() to be there and the async with to work as normal.

Actual behaviour

pytest output:

        LOGGER.debug(f'Fetch({self}) called started.')
        to_ = ClientTimeout(total=timeout)
        async with ClientSession(timeout=to_) as session:
            LOGGER.debug(f'Client session created: {session}. About to fetch url={self._url}')
            byte_start = 0
            byte_stop = range_size - 1 # as we count from 0
            with open(self._full_path, 'wb') as file_desc:  #TODO: use aiofile
                while True:
                    req_header = {"Range": f'bytes={byte_start}-{byte_stop}'}
>                   async with session.get(self._url, headers=req_header) as resp:
E                   AttributeError: __aexit__

src/pyskyq/xmltvlisting.py:183: AttributeError

Log:

------------------------------------------------------------ Captured log call ------------------------------------------------------------
epg.py                      66 DEBUG    Initialised EPG object using SkyQ box=test_load_channel_list_fake_host
epg.py                      93 DEBUG    Fetching channel list from http://test_load_channel_list_fake_host:9006/as/services
xmltvlisting.py             76 DEBUG    XMLTVListing initialised: <List: url='http://foo.com/feed/6715', path='.epg_data', filename='ea51e77b9fdede19528d599f50182d37edcdbc082b06358146041fe446f6a855.xml'>
selector_events.py          64 DEBUG    Using selector: KqueueSelector
asyncthread.py             128 DEBUG    Event loop still running in thread: pyskyq.asyncthread
selector_events.py          64 DEBUG    Using selector: KqueueSelector
epg.py                     293 DEBUG    Listing = <List: url='http://foo.com/feed/6715', path='.epg_data', filename='ea51e77b9fdede19528d599f50182d37edcdbc082b06358146041fe446f6a855.xml'>
xmltvlisting.py            174 DEBUG    Fetch(<List: url='http://foo.com/feed/6715', path='.epg_data', filename='ea51e77b9fdede19528d599f50182d37edcdbc082b06358146041fe446f6a855.xml'>) called started.
xmltvlisting.py            177 DEBUG    Client session created: <aiohttp.client.ClientSession object at 0x104e79a90>. About to fetch url=http://foo.com/feed/6715

Steps to reproduce

As illustrated above.

Your environment

MacOS mojave
aiohttp==3.4.4
Python 3.7.0

[aio-libs-bot]

GitMate.io thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are #2154 (Question: Do I need to use a context manager with all ClientSession HTTP methods?), #548 (Implement timeout context manager), #2362 (Drop sync context manager supports), #2867 (aiohttp.ClientSession reconnect ), and #3119 (Using client sessions without a context manager).

[asvetlov]

I'm sorry but I cannot figure out how the problem can appear.
Any help with a reproducible scenario can help a lot.
The session and response objects have no attribute magic, __aexit__ should always exist.

[bradwood]

I'm trying to isolate it -- problem still happens if timeout is left as default in ClientTimeout...

Trying to make it reproducible is not easy as it tends to work as expected when I reduce the calling code around it.

I'm trying to reason about it. If session.get() is misbehaving in some way, could it __aenter__ and then crash before __axit__ ?

[asvetlov]

__aexit__ existence is checked by Python before __aenter__ call.
Maybe you use mocking? Like mocking session.get() or session._request()?

[bradwood]

Yes! This is a mocked connection... I am mocking get(). I got this stack dump, which might shed some light:

[2018-10-16 14:45:10,409] ERROR:asyncio:Exception in callback Cron.set_result(<_GatheringFu...dError(None)]>)
handle: <Handle Cron.set_result(<_GatheringFu...dError(None)]>)>
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/Users/brad/.virtualenvs/pyskyq-4vSEKDfZ/src/aiocron/aiocron/__init__.py", line 96, in set_result
    raise result
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/coroutines.py", line 132, in coro
    res = yield from await_meth()
  File "/Users/brad/Code/pyskyq/src/pyskyq/epg.py", line 296, in _download_and_apply_XMLTVListing
    await listing.fetch()  # do the download
  File "/Users/brad/Code/pyskyq/src/pyskyq/xmltvlisting.py", line 183, in fetch
    async with session.get(self._url, headers=req_header) as resp:
  File "/Users/brad/.virtualenvs/pyskyq-4vSEKDfZ/lib/python3.7/site-packages/aiohttp/client.py", line 855, in __aenter__
    self._resp = await self._coro
  File "/Users/brad/.virtualenvs/pyskyq-4vSEKDfZ/lib/python3.7/site-packages/aiohttp/client.py", line 391, in _request
    await resp.start(conn)
  File "/Users/brad/.virtualenvs/pyskyq-4vSEKDfZ/lib/python3.7/site-packages/aiohttp/client_reqrep.py", line 757, in start
    message, payload = await self._protocol.read()
  File "/Users/brad/.virtualenvs/pyskyq-4vSEKDfZ/lib/python3.7/site-packages/aiohttp/streams.py", line 543, in read
    await self._waiter
aiohttp.client_exceptions.ServerDisconnectedError: None

[bradwood]

I think I found the problem @asvetlov -- my mock is behaving badly... so unlikely an issue with the library... You put me on the right thrack, thanks!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new