Unhandled exception can only be unwound once in .NET 10 rc1

[gregg-miskelly]

Description

Before .NET 10 rc1, if you stop on an unhandled exception, you could continuously unwind that exception and attempt to fix the problem. In the rc1 build, unwinding works only once.

Reproduction Steps

1. Create a C# program with the following code
2. In Visual Studio, open the exception settings window (Debug->Windows->Exception Settings)
3. Click the 'Common Language Runtime Exceptions' category twice so that it shows you the empty box (so that the debugger will not stop when an exception is thrown)
4. Run the program under the debugger, which should stop when the null reference exception is unhandled.
5. Open the call stack window, right click on the Program.Main frame, and invoke 'Unwind Exception to This Frame'.
6. Continue again

internal class Program
{
    private static void Main(string[] args)
    {
        Console.WriteLine(NullString.Length);
    }

    private static string NullString = null!;
}

Expected behavior

The debugger should break in exactly the same spot as it did the first time

Actual behavior

The debugger breaks with the following stack and the exception cannot be unwound a second time.

[System.NullReferenceException unhandled]
[Exception] CsUnhandledException.dll!Program.Main(string[] args) Line 5
[Managed to Native Transition]
System.Private.CoreLib.dll!System.Environment.FailFast(System.Runtime.CompilerServices.StackCrawlMarkHandle mark, string message, System.Runtime.CompilerServices.ObjectHandleOnStack exception, string errorMessage)
System.Private.CoreLib.dll!System.Environment.FailFast(ref System.Threading.StackCrawlMark mark, string message, System.Exception exception, string errorMessage)
System.Private.CoreLib.dll!System.Environment.FailFast(string message)
System.Private.CoreLib.dll!System.AppContext.OnUnhandledException(object e)
[Native to Managed Transition]
CsUnhandledException.dll!Program.Main(string[] args) Line 5

Regression?

Regression from .NET 10 preview 7

Known Workarounds

Restart the target app

Configuration

I tested .NET 10 rc1 on Windows. The problem reproduces for both x86 and x64. I didn't try any other configurations.

Other information

No response

[janvorli]

@AaronRobinsonMSFT this looks like it probably stems from your change in the recursion detection in the OnUnhandledException. Guessing by the FailFast called from there.

[janvorli]

There is a pending PR with a fix to a bug in that code, but I don't think it would fix this problem.

[AaronRobinsonMSFT]

@AaronRobinsonMSFT this looks like it probably stems from your change in the recursion detection in the OnUnhandledException. Guessing by the FailFast called from there.

Hmmm. Perhaps the FailFast should only be done if a debugger isn't attached?

There is a pending PR with a fix to a bug in that code, but I don't think it would fix this problem.

What PR? Found it, #119734.

[jkotas]

Hmmm. Perhaps the FailFast should only be done if a debugger isn't attached?

It is one of the options. Alternatively, we can delete the single-shot block and allow the unhandled exception handlers to be called on multiple threads or recursively like before. This is second issue we have run into with single-shot block, so chances are that there are more problems that we do not know about. Deleting the single-shot block and going to previous behavior may be better.

I am not sure how well this debugger flow works end-to-end in practice. It is not unusual for libraries to do uninitialization in the UnhandledException handler (e.g. including our own System.Console on Unix). If the debugger allows the app to resume after UnhandledException handlers have been called, it is unlikely to go very far since it is going to hit one of the uninitialized libraries.