"InvalidStateError: invalid state" when data is chunked

[rq4w7z]

I am use simple aiohttp (3.0.2) server for Telegram bot:

API_TOKEN = 'XXXXxxXXXXxxxxxx-XXXXXXX_xxxxXXXXXx'
bot = telebot.TeleBot(API_TOKEN)
app = web.Application()
async def handle(request):
    if request.match_info.get('token') == bot.token:
        request_body_dict = await request.json()
        update = telebot.types.Update.de_json(request_body_dict)
        bot.process_new_updates([update])
        return web.Response()
    else:
        return web.Response(status=403)

app.router.add_post('/{token}/', handle)
bot.remove_webhook()
bot.set_webhook(url=WEBHOOK_URL_BASE+WEBHOOK_URL_PATH,
                    certificate=open(WEBHOOK_SSL_CERT, 'r'))
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
context.load_cert_chain(WEBHOOK_SSL_CERT, WEBHOOK_SSL_PRIV)
web.run_app(
        app,
        host=WEBHOOK_LISTEN,
        port=WEBHOOK_PORT,
        ssl_context=context)

When message send from Telegram server I receive this error:

ERROR:asyncio:Exception in callback _SelectorSocketTransport._read_ready()
handle: <Handle _SelectorSocketTransport._read_ready()>
Traceback (most recent call last):
File "/usr/lib64/python3.6/asyncio/events.py", line 127, in _run
self._callback(*self._args)
File "/usr/lib64/python3.6/asyncio/selector_events.py", line 731, in _read_ready
self._protocol.data_received(data)
File "/usr/lib64/python3.6/asyncio/sslproto.py", line 516, in data_received
self._app_protocol.data_received(chunk)
File "/home/telegram/bwb_env/lib64/python3.6/site-packages/aiohttp/web_protocol.py", line 252, in data_received
self._waiter.set_result(None)
asyncio.base_futures.InvalidStateError: invalid state

I get the response completely, but further after a number of responses there are big delays and denial of service

When I use curl for make test request I did not receive this error

curl -X POST -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'User-Agent:' -H 'Accept-Encoding: gzip, deflate' -H 'Accept:' -H 'Host: 111.111.111.111' -k -d '{"update_id":285112997, "message":{"message_id":802,"from":{"id":111111111,"is_bot":false,"first_name":"X","last_name":"X","username":"XXXXXXXXXXX","language_code":"uk-UA"},"chat":{"id":111111111,"first_name":"X","last_name":"X","username":"XXXXXXXXXXX","type":"private"},"date":1519540489,"text":"\u0440 \u0440\u0440\u0440\u0440\u0440\u0440\u0440\u0440\u0440"}}' https://111.111.111.111:8443/222222222:XXXXxxXXXXxxxxxx-XXXXXXX_xxxxXXXXXx/

I am writing server for example

from config import *
import asyncio
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
context.load_cert_chain(WEBHOOK_SSL_CERT, WEBHOOK_SSL_PRIV)

class Serv(asyncio.Protocol):
    def connection_made(self, transport):
        self.transport = transport
        self.peername = transport.get_extra_info("peername")
        print("connection_made: {}".format(self.peername))

    def data_received(self, data):
        txid = data.decode()
        print("data_received: {}".format(txid))

    def eof_received(self):
        print('eof_received')

    def connection_lost(self, ex):
        print("connection_lost: {}".format(self.peername))

loop = asyncio.get_event_loop()
coro = loop.create_server(Serv, host='0.0.0.0', port=8443, ssl=context)
asyncio.ensure_future(coro)
loop.run_forever()

This request from curl:

connection_made: ('121.121.121.121', 60939)
data_received: POST /222222222:XXXXxxXXXXxxxxxx-XXXXXXX_xxxxXXXXXx/ HTTP/1.1
Host: 111.111.111.111
Content-Type: application/json
Connection: keep-alive
Accept-Encoding: gzip, deflate
Content-Length: 362

{"update_id":285112997,
"message":{"message_id":802,"from":{"id":111111111,"is_bot":false,"first_name":"X","last_name":"X","username":"XXXXXXXXXXX","language_code":"uk-UA"},"chat":{"id":111111111,"first_name":"X","last_name":"X","username":"XXXXXXXXXXX","type":"private"},"date":1519540489,"text":"\u0440 \u0440\u0440\u0440\u0440\u0440\u0440\u0440\u0440\u0440"}}

connection_lost: ('121.121.121.121', 60939)

This from Telegram server:

connection_made: ('149.154.167.231', 33538)
data_received: POST /222222222:XXXXxxXXXXxxxxxx-XXXXXXX_xxxxXXXXXx/ HTTP/1.1
Host: 111.111.111.111
Content-Type: application/json
Content-Length: 362
Connection: keep-alive
Accept-Encoding: gzip, deflate

data_received: {"update_id":285113004,
"message":{"message_id":822,"from":{"id":111111111,"is_bot":false,"first_name":"X","last_name":"X","username":"XXXXXXXXXXX","language_code":"uk-UA"},"chat":{"id":111111111,"first_name":"X","last_name":"X","username":"XXXXXXXXXXX","type":"private"},"date":1519544321,"text":"\u0440 \u0440\u0440\u0440\u0440\u0440\u0440\u0440\u0440\u0440"}}

As see from Telegram response, have two calls data_received
In aiohttp==2.0.7 does not have this error.

[asvetlov]

Thanks for report

[asvetlov]

I think we should add a check for self._waiter.done() before self._waiter.set_result() call

[asvetlov]

Fixed by #2775
Wait for aiohttp 3.0.5 release

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new