Unable to cancel Server.serve_forever() if a reader is blocked while reading (3.12 regression)

[paravoid]

Bug report

Bug description:

Consider this code, slightly simplifying the documentation's TCPServer example code:

import asyncio

async def handle_echo(reader, writer):
    print("Reading")
    data = await reader.read(100)
    message = data.decode()
    print(f"Received '{message!r}'")

    print("Closing the connection")
    writer.close()
    await writer.wait_closed()

async def main():
    server = await asyncio.start_server(handle_echo, "127.0.0.1", 8888)
    print("Serving forever...")
    async with server:
        try:
            await server.serve_forever()  
        except asyncio.CancelledError:
            print("Cancelled by Ctrl+C")
            server.close()

asyncio.run(main())

(My code is closer to a while True: await reader.readline(); ..., but the above probably suffices as a demonstration)

Running this results in a Serving forever, and hitting Ctrl+C, results in a Cancelled by Ctrl+C, and a normal exit.

However, if in another window we nc 127.0.0.1 8888, and leave the connection open, Ctrl+C (SIGINT) does nothing, and a second Ctrl+C is required to terminate. (This however breaks out of the asyncio loop by raising KeyboardInterrupt() , as documented).

So basically clients can prevent the server from cleanly exiting by just keeping their connection open.

This is a regression: this fails with 3.12.5 and 3.13.0-rc1 but works with 3.11.9.

This is because (TTBOMK) of this code in base_events.py:

        try:
            await self._serving_forever_fut
        except exceptions.CancelledError:
            try:
                self.close()
                await self.wait_closed()
            finally:
                raise
        finally:
            self._serving_forever_fut = None

I believe this to be related to the wait_closed() changes, 5d09d11, 2655369 etc. (Cc @gvanrossum). Related issues #104344 and #113538.

Per @gvanrossum in #113538 (comment): "In 3.10 and before, server.wait_closed() was a no-op, unless you called it before server.close(), in a task (asyncio.create_task(server.wait_closed())). The unclosed connection was just getting abandoned."

CancelledError() is caught here, which spawns wait_closed() before re-raising the exception. In 3.12+, wait_closed()... actually waits for the connection to close, as intended. However, while this prevents the reader task from being abandoned, it does not allow neither the callers of reader.read() or serve_forever() to catch CancelledError() and clean up (such as actually closing the connection, potentially after e.g. a signaling to the client a server close through whatever protocol is implemented here).

Basically no user code is executed until the client across the network drops the connection.

As far as I know, it's currently impossible to handle SIGINTs cleanly with clients blocked in a read() without messing with deep asyncio/selector internals, which seems like a pretty serious limitation? Have I missed something?

CPython versions tested on:

3.11, 3.12, 3.13

Operating systems tested on:

Linux

Linked PRs

• gh-123720: When closing an asyncio server, stop the handlers #124689

[gvanrossum]

I will try to look at this when I'm at the core sprint, Sept 23-27.

[gvanrossum]

@paravoid The above code doesn't work; it crashes on line 6. I guess you didn't actually run it before filing the bug? Also I don't know what nc does with these parameters, could you please add a client that triggers the issue so I can understand, or at least explain what this nc invocation does?

Finally, your code differs (apart from the crash) from the original code in several ways:

• The handler never writes back to the writer
• It has a try/except around serve_forever()

Which of these is significant?

[paravoid]

@gvanrossum Thanks for taking a look! I did run the code but tried simplifying it after the fact (it's all in the GitHub history). Fixed now, it was a s/line/data/. Sorry about that!

nc just connects and waits for input. There is no need to do anything on the socket. Instead of nc you can also do: python3 -c "import socket, time; s = socket.socket(socket.AF_INET, socket.SOCK_STREAM); s.connect(('localhost', 8888)); time.sleep(100)" in another window.

Finally, your code differs (apart from the crash) from the original code in several ways:
* The handler never writes back to the writer
* It has a try/except around serve_forever()

Which of these is significant?

None of the two are significant. The write has been removed for simplicity (it doesn't matter). The try/except is there to make it easier to see that error handling does work when a client is not connected.