Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing information about ASP.NET Core worker process recycling requirements and setup options on GNU/Linux servers #42897

Open
progmars opened this issue Jul 25, 2022 · 1 comment
Assignees
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions Docs This issue tracks updating documentation net8_docathon This issue is part of the documentation push in Fall 2023.
Milestone

Comments

@progmars
Copy link

As many developers and web server administrators know, IIS has a built-in feature for application pool recycling. Here is a refresher for those who might have forgotten about this.

Recycling means that the worker process that handles requests for that application pool is terminated and a new one is started. This is generally done to avoid unstable states that can lead to application crashes, hangs, or memory leaks.
By default IIS uses the overlapped recycle method, which keeps the old process up until the current requests are finished processing (or a set timeout elapses) while the new process handles new requests. This ensures service continuity so that you usually do not notice a recycle.

When planning to deploy my ASP.NET Core web apps on Linux servers, I was worried about this mechanism on Linux. I already have one customer complaining that Kestrel process on their server seems to be consuming more and more memory, leading to swapping, while there seem to be no such issues on IIS since the web app gets recycled preemptively.

When setting my ASP.NET Core app on Linux, I followed this article:
https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/linux-nginx?view=aspnetcore-6.0

It describes the recommended Kestrel recovery mechanism using Restart=always and RestartSec=10, which is great. However, it has no mention of recycling requirements and does not provide "the officially recommended setup", or at least some reasonable starting point to achieve something similar to IIS pool recycling.
I also could not find any information on the internet regarding this. I don't think that Kestrel on Linux is "magical" and can keep the same process alive for ages without accumulating any "debris" that would benefit from preemptive recycling, so I assume we need something similar to IIS app pool recycling. Correct me if I'm wrong here and Kestrel can and should be left alone running for months without recycling and fears of memory debris.

It would be great for someone experienced to look into this and improve that article or write a new article to clear people's doubts regarding worker process recycle management, so that we have more confidence recommending ASP.NET Core web apps for customers who want to host on Linux servers.
Is it at all possible to achieve similar zero-downtime background recycle on Linux with Nginx and Kestrel the same way as it works on IIS? Is it necessary at all? I wouldn't want to reinvent a wheel here, so it would be great to have something time-tested and officially recommended, following best practices and playing nicely with general Kestrel lifecycle mechanics, which we, mere mortal developers, might not know enough to be able to build a reliable solution from scratch.

@adityamandaleeka
Copy link
Member

@progmars There's some info here in a prior similar discussion: #40147

To answer your other question, in general, it's not necessary to recycle Kestrel processes. There are some cases where you may want to (for instance, if you have a surge in load which causes Kestrel to allocate more in its memory pool, that memory isn't returned when the load drops back down -- we have an issue to address this in the future #27394). And of course, it's not uncommon to see people addressing memory leaks in their own server-side code by restarting servers periodically.

We should improve the docs to explain this and show the recommended pattern for doing it on Linux for folks who need/want to.

@adityamandaleeka adityamandaleeka added the Docs This issue tracks updating documentation label Jul 25, 2022
@adityamandaleeka adityamandaleeka added this to the .NET 8 Planning milestone Jul 25, 2022
@adityamandaleeka adityamandaleeka added the net8_docathon This issue is part of the documentation push in Fall 2023. label Aug 25, 2023
@amcasey amcasey added area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions and removed area-runtime labels Aug 25, 2023
@amcasey amcasey modified the milestones: .NET 8 Planning, Backlog Feb 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions Docs This issue tracks updating documentation net8_docathon This issue is part of the documentation push in Fall 2023.
Projects
None yet
Development

No branches or pull requests

5 participants