Debugger does not halt on uncaught exceptions in async trio code when testing with pytest-trio

[andersea]

This bug is similar to microsoft/ptvsd#2027, but only when debugging test functions with pytest and pytest-trio.

Environment data

• debugpy version: 1.0.0b12
• OS and version: Windows 10 Pro 1909
• Python version: 3.7.3
• Using VS Code or Visual Studio: vscode

Actual behavior

When running an async test function in debug mode using pytest, if code raises an exception, the exception is dumped to the debug console and the program halts.

Expected behavior

With the setting 'Uncaught exceptions' enabled, the debugger should halt at soon as it encounters user code. See microsoft/ptvsd#2027 for a similar discussion.

Steps to reproduce:

1. With poetry installed, create a new test project with poetry new test-project
2. cd into project directory and add the following packages

poetry add trio
poetry add --dev pytest-trio

3. Add pytest trio fixtures by adding the file pytest.ini with the following contents:

[pytest]
trio_mode = true

4. In the project module folder edit the file <project_name>\__init__.py and add this code:

class MyTestClass:
    pass

async def thrower():
    my_test_class = MyTestClass()
    print(my_test_class.nonexistant_attribute)

5. In the test folder edit the file tests\test_<project_name>.py paste this code:

from <project_name> import thrower

async def test_thrower():
    await thrower()

6. Add the venv created by poetry, configure pytest in vscode and run test discovery.
7. Run the test function test_thrower in debug mode.

[fabioz]

This isn't really a bug, rather, the mechanic of unhandled exceptions expects an unhandled exception and in this case the exception is handled (i.e.: it doesn't make python exit here).

We actually added support for user-handled exceptions as a part of #275.

Still, it was hidden in the UI due to an issue (which should be solved now), but it was not added back because it's not as useful until we actually walk the stack to show the place where the exception originated and not the current execution stack frame (but this may require some work on how to actually present it as VSCode doesn't seem to have a way to say that the exception was originated in some place but current execution is in another -- see: #275 (comment) and the comment just below for details, so, we probably need a way to format the exception to make that clear).

@int19h @karthiknadig do you think that should this be tackled now?

As a note, it's possible to make pytest itself actually crash in that case so that the exception is really unhandled (see: #275 (comment)) -- note: I haven't actually tried it, it's possible that it doesn't work due to the async.

[int19h]

The reason why we don't expose it in VSCode yet is because there was some concern about the UX being somewhat confusing (due to lack of clarity between "unhandled" and "user-unhandled"). @luabud, do we want to do an A/B test for this?

[luabud]

@int19h love the idea of doing an A/B test, yes!

[andersea]

The UI is a bit confusing as it is, when it comes to debugging exceptions. Right now, it is possible to enable Raised Exceptions and Uncaught Exceptions at the same time, which does not make sense. If Raised Exceptions is enabled, any exception anywhere, should stop execution, so whether Uncaught Exceptions is enabled or not is irrelevant.

[int19h]

"Raised" and "Uncaught" will report at different points - "Raised" reports as soon as exception occurs, but "Uncaught" only reports when it bubbles out of user code, after all the stack has been unwound. So the app state might be very different, and sometimes you want to debug that difference.

But yes, a good example of how it's already confusing!

[fabioz]

This should be fixed by the User Uncaught Exceptions mode:

Git reference: 3d472df

Note: when released this will require opting-in by setting "DEBUGPY_EXCEPTION_FILTER_USER_UNHANDLED" = "1" in the environment.