How to handle globally OutOfMemory exception in large project

[plachor]

Hi, I'm working on core components that are used by other developers to create a working application. Within my core components there is a global exception handling strategy were we handle domain and unhandled exceptions. Yet I do not fully control how my components will be used by end-developer. So end-developer can handle exceptions that I would wish him not to be able too. For instance OutOfMemory exception.

We were excited once netcore3.1 started to handle correctly memory limits in containerized environments. We stopped seeing containers being killed by OOM killer without being able to log anything or given any chance to perform graceful shutdown.

But after several months we see that in our case for OOM's best thing we can do is to perform a restart to ensure container is healthy. This is also suggested by documentation. However in my case I do not fully control how my components are being used. Additionally there are many entry points like request pipeline, background services, asynchronous message bus were such exception can occur. For background services we might need to wait for #43637 to be able to handle it correctly.

I'm hoping to find a single place where I could express that OutOfMemory is fatal exception that it cannot be suppressed by end developer using try-catch block. Or other way around, looking for notification that it occurred and was handled somewhere so I can log it and perform graceful shutdown.

Are there any options for me, or am I forced to ensure that each try-catch is not by any chance handling OutOfMemory exception?

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[danmoseley]

Adding VM label as such a hook would be in the runtime itself.

[SimonCropp]

@danmoseley would any hook theoretically be possible? wouldnt calling it likely require memory to be allocated?

[danmoseley]

It's not my area. @janvorli

[KalleOlaviNiemitalo]

Could the component add a handler for AppDomain.FirstChanceException and detect OutOfMemoryException that way? The documentation specifically suggests using this for "infrastructure-related exceptions such as out-of-memory".