Debugger incorrectly Highlights every assertion error that has occurred in a debugger if the test case a whole fails

[Serhiy1]

Describe the bug
The Codebase that I work on has a lot of Polling, waiting for certain conditions to become true. This is mainly achieved through the use of keywords Run Keyword And Ignore Error and Wait Until Keyword Succeeds

This polling makes it extremely difficult to use the debugger. A whole load of noise will be introduced if the test case I'm working on fails on an assertion that happens after the polling.

To Reproduce
To help explain the pain I have made this small reproducible

*** Test Cases ***
Debugger Example
    ${_counter}=    Convert To Integer    1
    Set Suite Variable    ${COUNTER}    ${_counter}

    ${data}    ${status}=    Run Keyword And Ignore Error    Keyword Will Fail
    ${data}=    Wait Until Keyword Succeeds    5    1s    Poll Keyword

    Fail

*** Keywords ***
Keyword Will Fail
    [Documentation]    Example of failing keyword
    Fail

Poll Keyword
    [Documentation]    Example of polling keyword
    IF    ${COUNTER} == 5
        Return From Keyword    pass
    ELSE
        Set Suite Variable    ${COUNTER}    ${COUNTER+1}
        Fail
    END

This is the output of the debugger when run:

Expected behavior

Ideally, I would prefer that the debugger ignored all the assertion errors that were captured and handled and only highlighted the line that caused the test to fail

Additionally, when Running debugging the test, with the break on FAIL option enabled, This code will break on every single fail, which makes the feature not very helpful in this context.

[fabioz]

I agree, that really seems annoying... it seems we need to have something akin to the handled / unhandled failures/errors (right now the feature is really stop on any handled exception and not just on the unhandled ones).

[Serhiy1]

I think this issue will also apply to the new TRY .... EXCEPT Syntax that will be introduced in RF 5.0.

[fabioz]

Yes, I was taking a look at the RF source code and it doesn't really provide any API to know whether a given error is the one that'd make the test fail (the main problem is that the debugger needs to stop at the Fail call and not later on when the test fails because that'd be too late to inspect variables as the state may have already changed).

What do you think about having something as a way to automatically ignore failures inside some keywords?

That way it'd ignore failures inside of keywords such as Run Keyword and Ignore Error and Wait Until Keyword Succeeds -- note: you would still be able to extend that for framework-specific calls as needed.

[fabioz]

For the TRY..EXCEPT it should be presumably simpler because the model should have the information on when it would need to stop (unlike keywords where this information is not available).

[Serhiy1]

What do you think about having something as a way to automatically ignore failures inside some keywords?

Yeah, I don't mind that. Personally, I don't see any reason to see the failures in these keywords, Unless they timeout.

Though I can't speak for everyone, and they want the behaviour to be togglable.