Redesign shutdown process #7718

Dreamsorcerer · 2023-10-17T17:48:06Z

This contains a few changes to rework the shutdown process to be a bit clearer and better match user expectations.

To summarise, I'm proposing the shutdown process should look like this now:

Stop each site listening for new connections.
Close idle keep-alive connections (and set active ones to close upon completion).
Call on_shutdown signal (should be used to close websocket connections).
Wait for running tasks to complete (with timeout).
Close any remaining connections (with timeout again).
Call on_cleanup signal.
Cancel any remaining tasks and wait on them.

Docs demo preview: https://aiohttp--7718.org.readthedocs.build/en/7718/web_advanced.html#graceful-shutdown

for more information, see https://pre-commit.ci

tests/test_run_app.py

aiohttp/web_runner.py

docs/web_advanced.rst

webknjaz · 2023-10-25T22:56:37Z

Cancel any remaining tasks and wait on them.

Does this pose a deadlock threat?

Dreamsorcerer · 2023-10-26T16:27:58Z

Cancel any remaining tasks and wait on them.

Does this pose a deadlock threat?

I think there is the possibility that a task could stop the program exiting, but at that point the rest of shutdown has already happened, so it'll be as safe as it can be for someone to kill the program. This step is unchanged from the 3.8 release.

Dreamsorcerer · 2023-10-26T16:34:41Z

Cancel any remaining tasks and wait on them.

Does this pose a deadlock threat?

I think there is the possibility that a task could stop the program exiting, but at that point the rest of shutdown has already happened, so it'll be as safe as it can be for someone to kill the program. This step is unchanged from the 3.8 release.

In fact, I just remembered that this code is copied from asyncio.run(), so it'll be consistent with asyncio generally.

Co-authored-by: Sviatoslav Sydorenko <sviat@redhat.com>

codecov · 2023-10-27T00:15:14Z

Codecov Report

Merging #7718 (4d21a28) into master (d8936aa) will increase coverage by 0.02%.
Report is 5 commits behind head on master.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #7718      +/-   ##
==========================================
+ Coverage   97.36%   97.38%   +0.02%     
==========================================
  Files         106      106              
  Lines       31718    31798      +80     
  Branches     3633     3645      +12     
==========================================
+ Hits        30881    30967      +86     
+ Misses        633      627       -6     
  Partials      204      204

Flag	Coverage Δ
CI-GHA	`97.30% <97.93%> (+0.01%)`	⬆️
OS-Linux	`96.97% <96.90%> (+0.02%)`	⬆️
OS-Windows	`95.44% <95.87%> (+0.02%)`	⬆️
OS-macOS	`96.64% <96.90%> (+0.03%)`	⬆️
Py-3.10.11	`95.37% <93.81%> (+0.01%)`	⬆️
Py-3.10.13	`96.82% <94.84%> (+0.01%)`	⬆️
Py-3.11.6	`96.48% <90.72%> (+<0.01%)`	⬆️
Py-3.8.10	`95.33% <93.81%> (+0.01%)`	⬆️
Py-3.8.18	`96.75% <94.84%> (+0.01%)`	⬆️
Py-3.9.13	`95.33% <95.87%> (+0.02%)`	⬆️
Py-3.9.18	`96.79% <96.90%> (+0.03%)`	⬆️
Py-pypy7.3.11	`96.19% <94.84%> (+0.01%)`	⬆️
VM-macos	`96.64% <96.90%> (+0.03%)`	⬆️
VM-ubuntu	`96.97% <96.90%> (+0.02%)`	⬆️
VM-windows	`95.44% <95.87%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
aiohttp/web.py	`99.27% <100.00%> (+0.11%)`	⬆️
aiohttp/web_runner.py	`97.75% <100.00%> (-0.08%)`	⬇️
aiohttp/web_server.py	`95.34% <100.00%> (+0.34%)`	⬆️
aiohttp/worker.py	`91.86% <ø> (ø)`
tests/test_run_app.py	`98.46% <100.00%> (+0.20%)`	⬆️
tests/test_worker.py	`98.95% <100.00%> (+0.01%)`	⬆️

... and 5 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Dreamsorcerer · 2023-11-02T22:46:54Z

I'm merging this, if there's any feedback, I can update it before the final release (maybe in a week or so).

patchback · 2023-11-02T22:48:26Z

Backport to 3.9: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 107dd24 on top of patchback/backports/3.9/107dd2445720cf354c84c3a271691f14e2e2875e/pr-7718

Backporting merged PR #7718 into master

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/aio-libs/aiohttp.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.9/107dd2445720cf354c84c3a271691f14e2e2875e/pr-7718 upstream/3.9

Now, cherry-pick PR Redesign shutdown process #7718 contents into that branch:
```
$ git cherry-pick -x 107dd2445720cf354c84c3a271691f14e2e2875e
```
If it'll yell at you with something like fatal: Commit 107dd2445720cf354c84c3a271691f14e2e2875e is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x 107dd2445720cf354c84c3a271691f14e2e2875e
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Redesign shutdown process #7718 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.9/107dd2445720cf354c84c3a271691f14e2e2875e/pr-7718

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

This contains a few changes to rework the shutdown process to be a bit clearer and better match user expectations. To summarise, I'm proposing the shutdown process should look like this now: 1. Stop each site listening for new connections. 2. Close idle keep-alive connections (and set active ones to close upon completion). 3. Call on_shutdown signal (should be used to close websocket connections). 4. Wait for running tasks to complete (with timeout). 5. Close any remaining connections (with timeout again). 6. Call on_cleanup signal. 7. Cancel any remaining tasks and wait on them. Docs demo preview: https://aiohttp--7718.org.readthedocs.build/en/7718/web_advanced.html#graceful-shutdown --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sviatoslav Sydorenko <sviat@redhat.com> (cherry picked from commit 107dd24)

bdraco · 2023-11-02T23:38:25Z

I'm merging this, if there's any feedback, I can update it before the final release (maybe in a week or so).

Sorry for the late feedback, I wasn't shut if this was ready to test or not.

I tested this with Home Assistant, and shutdown is delayed if we don't patch out wait.

Home Assistant has long running tasks that are created after aiohttp starts up, and are expected to be torn down after aiohttp is shutdown. Unfortunately we can't change that design to shutdown aiohttp later.

Patching out wait it probably fine for Home Assistant. It might be nice to have a way to disable the shutdown logic a bit more cleanly though.

webknjaz · 2023-11-02T23:44:21Z

CHANGES/7718.feature

@@ -0,0 +1,3 @@
+Fixed keep-alive connections stopping a graceful shutdown.


@Dreamsorcerer “fixed” implies a “bugfix” changelog note type, but the filename says “feature”. If it's both, it might be worth splitting into two fragments.

webknjaz · 2023-11-02T23:46:16Z

Home Assistant has long running tasks that are created after aiohttp starts up, and are expected to be torn down after aiohttp is shutdown. Unfortunately we can't change that design to shutdown aiohttp later.

Maybe, you could shield them from being cancelled, doing an unshielded cancellation later, manually?

webknjaz · 2023-11-03T00:10:27Z

Home Assistant has long running tasks that are created after aiohttp starts up, and are expected to be torn down after aiohttp is shutdown. Unfortunately we can't change that design to shutdown aiohttp later.

What if there was an interface to augment the new starting_tasks structure with your tasks? As I understand, you track what's yours, right?

webknjaz · 2023-11-03T00:11:48Z

@Dreamsorcerer I'm not very familiar with the shutdown internals, so I can't contribute an involved feedback. Just want to say that I don't see anything obviously wrong, and it seems like a good idea.

webknjaz · 2023-11-03T00:12:40Z

Unfortunately we can't change that design to shutdown aiohttp later.

@bdraco could you explain why?

bdraco · 2023-11-03T00:19:35Z

Unfortunately we can't change that design to shutdown aiohttp later.

@bdraco could you explain why?

Home Assistant has its own tasks that are created with our helper that we can track, but there are also ~1600 dependencies that can start their own tasks so we don't have a direct way to manage them.

While most of them will listen for a stop event and cancel the tasks, there are many custom integrations (and libraries) that don't and we can't wait for them to shutdown because we would end up waiting forever in many cases.

To give some better context, there are likely quite a few integrations that have a task running a retry loop when a device is offline that keep trying to reconnect forever until Home Assistant shuts down the event loop from under them.

webknjaz · 2023-11-03T01:35:44Z

@bdraco have you tried exploring having a separate event loop for those?

Dreamsorcerer · 2023-11-03T01:40:55Z

@bdraco We already discussed this in the testing issue. My understanding is that HA is using the runners directly, so in this PR I've moved the wait logic into run_app(). This means it should have no impact on HA anymore, and there isn't anything there to patch out. For users using run_app(), this is the entry point of the event loop and we expect to control the lifecycle of the event loop.

Dreamsorcerer · 2023-11-03T01:42:38Z

(This is also now highlighted in the docs section linked at the top of the PR)

bdraco · 2023-11-03T01:53:47Z

@bdraco We already discussed this in the testing issue. My understanding is that HA is using the runners directly, so in this PR I've moved the wait logic into run_app(). This means it should have no impact on HA anymore, and there isn't anything there to patch out. For users using run_app(), this is the entry point of the event loop and we expect to control the lifecycle of the event loop.

I agree that I shouldn't be seeing a problem anymore, but I can reproduce still. I will double check that I haven't screwed something up with my testing.

bdraco · 2023-11-03T02:07:36Z

TLDR: This PR solves the issue, there is an unrelated regression on my Home Assistant integration branch from trying to test too many changes at once that resulted in a bad test.

The problem goes away when I use Home Assistant stable + this PR. I must have a regression my Home Assistant integration branch that is blocking shutdown. I will bisect and find the bad commit but its certainly unrelated.

I didn't notice since I was running 3.9 prior to this PR which was always shutting down slowly (I didn't want to forget to make sure this was sorted) since it wasn't patched and I assumed the slow shutdown was still from that.

Thanks for getting this sorted. Sorry about the bad test.

bdraco · 2023-11-03T02:20:12Z

I found the bad commit, its definitely unrelated. Sorry again for the side trip. The good news is that 3.9 is working well in production 👍

[![Mend Renovate logo banner](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [aiohttp](https://togithub.com/aio-libs/aiohttp) | `==3.8.6` -> `==3.9.0` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/aiohttp/3.9.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/aiohttp/3.9.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/aiohttp/3.8.6/3.9.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/aiohttp/3.8.6/3.9.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>aio-libs/aiohttp (aiohttp)</summary> ### [`v3.9.0`](https://togithub.com/aio-libs/aiohttp/blob/HEAD/CHANGES.rst#390-2023-11-18) [Compare Source](https://togithub.com/aio-libs/aiohttp/compare/v3.8.6...v3.9.0) \================== ## Features - Introduced `AppKey` for static typing support of `Application` storage. See https://docs.aiohttp.org/en/stable/web_advanced.html#application-s-config `#5864 <https://github.com/aio-libs/aiohttp/issues/5864>`\_ - Added a graceful shutdown period which allows pending tasks to complete before the application's cleanup is called. The period can be adjusted with the `shutdown_timeout` parameter. -- by :user:`Dreamsorcerer`. See https://docs.aiohttp.org/en/latest/web_advanced.html#graceful-shutdown `#7188 <https://github.com/aio-libs/aiohttp/issues/7188>`\_ - Added `handler_cancellation <https://docs.aiohttp.org/en/stable/web_advanced.html#web-handler-cancellation>`\_ parameter to cancel web handler on client disconnection. -- by :user:`mosquito` This (optionally) reintroduces a feature removed in a previous release. Recommended for those looking for an extra level of protection against denial-of-service attacks. `#7056 <https://github.com/aio-libs/aiohttp/issues/7056>`\_ - Added support for setting response header parameters `max_line_size` and `max_field_size`. `#2304 <https://github.com/aio-libs/aiohttp/issues/2304>`\_ - Added `auto_decompress` parameter to `ClientSession.request` to override `ClientSession._auto_decompress`. -- by :user:`Daste745` `#3751 <https://github.com/aio-libs/aiohttp/issues/3751>`\_ - Changed `raise_for_status` to allow a coroutine. `#3892 <https://github.com/aio-libs/aiohttp/issues/3892>`\_ - Added client brotli compression support (optional with runtime check). `#5219 <https://github.com/aio-libs/aiohttp/issues/5219>`\_ - Added `client_max_size` to `BaseRequest.clone()` to allow overriding the request body size. -- :user:`anesabml`. `#5704 <https://github.com/aio-libs/aiohttp/issues/5704>`\_ - Added a middleware type alias `aiohttp.typedefs.Middleware`. `#5898 <https://github.com/aio-libs/aiohttp/issues/5898>`\_ - Exported `HTTPMove` which can be used to catch any redirection request that has a location -- :user:`dreamsorcerer`. `#6594 <https://github.com/aio-libs/aiohttp/issues/6594>`\_ - Changed the `path` parameter in `web.run_app()` to accept a `pathlib.Path` object. `#6839 <https://github.com/aio-libs/aiohttp/issues/6839>`\_ - Performance: Skipped filtering `CookieJar` when the jar is empty or all cookies have expired. `#7819 <https://github.com/aio-libs/aiohttp/issues/7819>`\_ - Performance: Only check origin if insecure scheme and there are origins to treat as secure, in `CookieJar.filter_cookies()`. `#7821 <https://github.com/aio-libs/aiohttp/issues/7821>`\_ - Performance: Used timestamp instead of `datetime` to achieve faster cookie expiration in `CookieJar`. `#7824 <https://github.com/aio-libs/aiohttp/issues/7824>`\_ - Added support for passing a custom server name parameter to HTTPS connection. `#7114 <https://github.com/aio-libs/aiohttp/issues/7114>`\_ - Added support for using Basic Auth credentials from :file:`.netrc` file when making HTTP requests with the :py:class:`~aiohttp.ClientSession` `trust_env` argument is set to `True`. -- by :user:`yuvipanda`. `#7131 <https://github.com/aio-libs/aiohttp/issues/7131>`\_ - Turned access log into no-op when the logger is disabled. `#7240 <https://github.com/aio-libs/aiohttp/issues/7240>`\_ - Added typing information to `RawResponseMessage`. -- by :user:`Gobot1234` `#7365 <https://github.com/aio-libs/aiohttp/issues/7365>`\_ - Removed `async-timeout` for Python 3.11+ (replaced with `asyncio.timeout()` on newer releases). `#7502 <https://github.com/aio-libs/aiohttp/issues/7502>`\_ - Added support for `brotlicffi` as an alternative to `brotli` (fixing Brotli support on PyPy). `#7611 <https://github.com/aio-libs/aiohttp/issues/7611>`\_ - Added `WebSocketResponse.get_extra_info()` to access a protocol transport's extra info. `#7078 <https://github.com/aio-libs/aiohttp/issues/7078>`\_ - Allow `link` argument to be set to None/empty in HTTP 451 exception. `#7689 <https://github.com/aio-libs/aiohttp/issues/7689>`\_ ## Bugfixes - Implemented stripping the trailing dots from fully-qualified domain names in `Host` headers and TLS context when acting as an HTTP client. This allows the client to connect to URLs with FQDN host name like `https://example.com./`. \-- by :user:`martin-sucha`. `#3636 <https://github.com/aio-libs/aiohttp/issues/3636>`\_ - Fixed client timeout not working when incoming data is always available without waiting. -- by :user:`Dreamsorcerer`. `#5854 <https://github.com/aio-libs/aiohttp/issues/5854>`\_ - Fixed `readuntil` to work with a delimiter of more than one character. `#6701 <https://github.com/aio-libs/aiohttp/issues/6701>`\_ - Added `__repr__` to `EmptyStreamReader` to avoid `AttributeError`. `#6916 <https://github.com/aio-libs/aiohttp/issues/6916>`\_ - Fixed bug when using `TCPConnector` with `ttl_dns_cache=0`. `#7014 <https://github.com/aio-libs/aiohttp/issues/7014>`\_ - Fixed response returned from expect handler being thrown away. -- by :user:`Dreamsorcerer` `#7025 <https://github.com/aio-libs/aiohttp/issues/7025>`\_ - Avoided raising `UnicodeDecodeError` in multipart and in HTTP headers parsing. `#7044 <https://github.com/aio-libs/aiohttp/issues/7044>`\_ - Changed `sock_read` timeout to start after writing has finished, avoiding read timeouts caused by an unfinished write. -- by :user:`dtrifiro` `#7149 <https://github.com/aio-libs/aiohttp/issues/7149>`\_ - Fixed missing query in tracing method URLs when using `yarl` 1.9+. `#7259 <https://github.com/aio-libs/aiohttp/issues/7259>`\_ - Changed max 32-bit timestamp to an aware datetime object, for consistency with the non-32-bit one, and to avoid a `DeprecationWarning` on Python 3.12. `#7302 <https://github.com/aio-libs/aiohttp/issues/7302>`\_ - Fixed `EmptyStreamReader.iter_chunks()` never ending. -- by :user:`mind1m` `#7616 <https://github.com/aio-libs/aiohttp/issues/7616>`\_ - Fixed a rare `RuntimeError: await wasn't used with future` exception. -- by :user:`stalkerg` `#7785 <https://github.com/aio-libs/aiohttp/issues/7785>`\_ - Fixed issue with insufficient HTTP method and version validation. `#7700 <https://github.com/aio-libs/aiohttp/issues/7700>`\_ - Added check to validate that absolute URIs have schemes. `#7712 <https://github.com/aio-libs/aiohttp/issues/7712>`\_ - Fixed unhandled exception when Python HTTP parser encounters unpaired Unicode surrogates. `#7715 <https://github.com/aio-libs/aiohttp/issues/7715>`\_ - Updated parser to disallow invalid characters in header field names and stop accepting LF as a request line separator. `#7719 <https://github.com/aio-libs/aiohttp/issues/7719>`\_ - Fixed Python HTTP parser not treating 204/304/1xx as an empty body. `#7755 <https://github.com/aio-libs/aiohttp/issues/7755>`\_ - Ensure empty body response for 1xx/204/304 per RFC 9112 sec 6.3. `#7756 <https://github.com/aio-libs/aiohttp/issues/7756>`\_ - Fixed an issue when a client request is closed before completing a chunked payload. -- by :user:`Dreamsorcerer` `#7764 <https://github.com/aio-libs/aiohttp/issues/7764>`\_ - Edge Case Handling for ResponseParser for missing reason value. `#7776 <https://github.com/aio-libs/aiohttp/issues/7776>`\_ - Fixed `ClientWebSocketResponse.close_code` being erroneously set to `None` when there are concurrent async tasks receiving data and closing the connection. `#7306 <https://github.com/aio-libs/aiohttp/issues/7306>`\_ - Added HTTP method validation. `#6533 <https://github.com/aio-libs/aiohttp/issues/6533>`\_ - Fixed arbitrary sequence types being allowed to inject values via version parameter. -- by :user:`Dreamsorcerer` `#7835 <https://github.com/aio-libs/aiohttp/issues/7835>`\_ - Performance: Fixed increase in latency with small messages from websocket compression changes. `#7797 <https://github.com/aio-libs/aiohttp/issues/7797>`\_ ## Improved Documentation - Fixed the `ClientResponse.release`'s type in the doc. Changed from `comethod` to `method`. `#5836 <https://github.com/aio-libs/aiohttp/issues/5836>`\_ - Added information on behavior of base_url parameter in `ClientSession`. `#6647 <https://github.com/aio-libs/aiohttp/issues/6647>`\_ - Fixed `ClientResponseError` docs. `#6700 <https://github.com/aio-libs/aiohttp/issues/6700>`\_ - Updated Redis code examples to follow the latest API. `#6907 <https://github.com/aio-libs/aiohttp/issues/6907>`\_ - Added a note about possibly needing to update headers when using `on_response_prepare`. -- by :user:`Dreamsorcerer` `#7283 <https://github.com/aio-libs/aiohttp/issues/7283>`\_ - Completed `trust_env` parameter description to honor `wss_proxy`, `ws_proxy` or `no_proxy` env. `#7325 <https://github.com/aio-libs/aiohttp/issues/7325>`\_ - Expanded SSL documentation with more examples (e.g. how to use certifi). -- by :user:`Dreamsorcerer` `#7334 <https://github.com/aio-libs/aiohttp/issues/7334>`\_ - Fix, update, and improve client exceptions documentation. `#7733 <https://github.com/aio-libs/aiohttp/issues/7733>`\_ ## Deprecations and Removals - Added `shutdown_timeout` parameter to `BaseRunner`, while deprecating `shutdown_timeout` parameter from `BaseSite`. -- by :user:`Dreamsorcerer` `#7718 <https://github.com/aio-libs/aiohttp/issues/7718>`\_ - Dropped Python 3.6 support. `#6378 <https://github.com/aio-libs/aiohttp/issues/6378>`\_ - Dropped Python 3.7 support. -- by :user:`Dreamsorcerer` `#7336 <https://github.com/aio-libs/aiohttp/issues/7336>`\_ - Removed support for abandoned `tokio` event loop. -- by :user:`Dreamsorcerer` `#7281 <https://github.com/aio-libs/aiohttp/issues/7281>`\_ ## Misc - Made `print` argument in `run_app()` optional. `#3690 <https://github.com/aio-libs/aiohttp/issues/3690>`\_ - Improved performance of `ceil_timeout` in some cases. `#6316 <https://github.com/aio-libs/aiohttp/issues/6316>`\_ - Changed importing Gunicorn to happen on-demand, decreasing import time by ~53%. -- :user:`Dreamsorcerer` `#6591 <https://github.com/aio-libs/aiohttp/issues/6591>`\_ - Improved import time by replacing `http.server` with `http.HTTPStatus`. `#6903 <https://github.com/aio-libs/aiohttp/issues/6903>`\_ - Fixed annotation of `ssl` parameter to disallow `True`. -- by :user:`Dreamsorcerer`. `#7335 <https://github.com/aio-libs/aiohttp/issues/7335>`\_ *** </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/allenporter/pyrainbird).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Dreamsorcerer and others added 9 commits October 14, 2023 19:08

Fix graceful shutdown with keep-alive connections

09174e2

Fix type error

ca1b88d

Remove redundant code

f838406

Redesign shutdown process.

d25f703

Update web_app.py

501b7d3

Update web_runner.py

540a076

Add websocket test and fixes

b48ee70

[pre-commit.ci] auto fixes from pre-commit.com hooks

1c6fb9d

for more information, see https://pre-commit.ci

Remove import

9a85bcc

github-advanced-security bot found potential problems Oct 17, 2023

View reviewed changes

tests/test_run_app.py Fixed Show fixed Hide fixed

tests/test_run_app.py Fixed Show fixed Hide fixed

Dreamsorcerer mentioned this pull request Oct 19, 2023

** 3.9 Beta testing ** #7675

Closed

8 tasks

Dreamsorcerer added 2 commits October 22, 2023 16:42

Move task waiting to run_app()

824c608

Revert pre_shutdown signal

d875703

github-advanced-security bot found potential problems Oct 23, 2023

View reviewed changes

tests/test_run_app.py Dismissed Show dismissed Hide dismissed

tests/test_run_app.py Dismissed Show dismissed Hide dismissed

Dreamsorcerer added 3 commits October 23, 2023 20:45

Update docs

af9d07a

Fix tests

be5886e

Fix type errors.

5750fd1

Dreamsorcerer commented Oct 23, 2023

View reviewed changes

aiohttp/web_runner.py Show resolved Hide resolved

Dreamsorcerer added 3 commits October 23, 2023 21:49

Fix

7ec7189

Fix

ef205d9

Create 7718.feature

2af21df

psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Oct 23, 2023

Dreamsorcerer marked this pull request as ready for review October 23, 2023 20:59

Dreamsorcerer requested review from webknjaz and asvetlov as code owners October 23, 2023 20:59

Dreamsorcerer mentioned this pull request Oct 23, 2023

Fix graceful shutdown with keep-alive connections #7699

Closed

Dreamsorcerer added 3 commits October 23, 2023 23:51

Merge branch 'master' into rework-shutdown

7a64217

Fix

f6be076

Remove incorrect wait_closed()

0a6151a

Dreamsorcerer commented Oct 24, 2023

View reviewed changes

aiohttp/web_runner.py Show resolved Hide resolved

Dreamsorcerer mentioned this pull request Oct 25, 2023

Fix warning displayed by default #7677

Merged

webknjaz reviewed Oct 25, 2023

View reviewed changes

docs/web_advanced.rst Outdated Show resolved Hide resolved

webknjaz requested a review from mjpieters October 25, 2023 22:56

Update docs/web_advanced.rst

4d21a28

Co-authored-by: Sviatoslav Sydorenko <sviat@redhat.com>

Dreamsorcerer merged commit 107dd24 into master Nov 2, 2023
29 of 34 checks passed

Dreamsorcerer deleted the rework-shutdown branch November 2, 2023 22:47

Dreamsorcerer added the backport-3.9 label Nov 2, 2023

webknjaz reviewed Nov 2, 2023

View reviewed changes

cdce8p mentioned this pull request Dec 18, 2023

Set aiohttp shutdown_timeout on AppRunner instead of TCPSite home-assistant/core#105973

Merged

20 tasks

ralphm mentioned this pull request Mar 7, 2024

wrong server shutdown order? #8179

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Redesign shutdown process #7718

Redesign shutdown process #7718

Dreamsorcerer commented Oct 17, 2023 •

edited by webknjaz

Loading

webknjaz commented Oct 25, 2023

Dreamsorcerer commented Oct 26, 2023

Dreamsorcerer commented Oct 26, 2023

codecov bot commented Oct 27, 2023

Dreamsorcerer commented Nov 2, 2023

patchback bot commented Nov 2, 2023 •

edited

Loading

bdraco commented Nov 2, 2023

webknjaz Nov 2, 2023

webknjaz commented Nov 2, 2023

webknjaz commented Nov 3, 2023

webknjaz commented Nov 3, 2023

webknjaz commented Nov 3, 2023

bdraco commented Nov 3, 2023 •

edited

Loading

webknjaz commented Nov 3, 2023

Dreamsorcerer commented Nov 3, 2023

Dreamsorcerer commented Nov 3, 2023 •

edited

Loading

bdraco commented Nov 3, 2023

bdraco commented Nov 3, 2023 •

edited

Loading

bdraco commented Nov 3, 2023 •

edited

Loading

		@@ -0,0 +1,3 @@
		Fixed keep-alive connections stopping a graceful shutdown.

Redesign shutdown process #7718

Redesign shutdown process #7718

Conversation

Dreamsorcerer commented Oct 17, 2023 • edited by webknjaz Loading

webknjaz commented Oct 25, 2023

Dreamsorcerer commented Oct 26, 2023

Dreamsorcerer commented Oct 26, 2023

codecov bot commented Oct 27, 2023

Codecov Report

Dreamsorcerer commented Nov 2, 2023

patchback bot commented Nov 2, 2023 • edited Loading

Backport to 3.9: 💔 cherry-picking failed — conflicts found

Backporting merged PR #7718 into master

bdraco commented Nov 2, 2023

webknjaz Nov 2, 2023

Choose a reason for hiding this comment

webknjaz commented Nov 2, 2023

webknjaz commented Nov 3, 2023

webknjaz commented Nov 3, 2023

webknjaz commented Nov 3, 2023

bdraco commented Nov 3, 2023 • edited Loading

webknjaz commented Nov 3, 2023

Dreamsorcerer commented Nov 3, 2023

Dreamsorcerer commented Nov 3, 2023 • edited Loading

bdraco commented Nov 3, 2023

bdraco commented Nov 3, 2023 • edited Loading

bdraco commented Nov 3, 2023 • edited Loading

Dreamsorcerer commented Oct 17, 2023 •

edited by webknjaz

Loading

patchback bot commented Nov 2, 2023 •

edited

Loading

bdraco commented Nov 3, 2023 •

edited

Loading

Dreamsorcerer commented Nov 3, 2023 •

edited

Loading

bdraco commented Nov 3, 2023 •

edited

Loading

bdraco commented Nov 3, 2023 •

edited

Loading