Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid raising debuggee not finished error if assertions failed #923

Merged
merged 1 commit into from
Mar 14, 2023

Conversation

st0012
Copy link
Member

@st0012 st0012 commented Mar 9, 2023

When assertions failed, we should let the AssertionFailedError surface and not calling flunk so we can get correct failure messages.

Assume we have this test that fails the assertion:

      run_protocol_scenario PROGRAM do
        req_add_breakpoint 5
        req_continue
        # should be 3 instead of 5
        assert_repl_result({value: '5', type: 'Integer'}, '1+2') 
        req_terminate_debuggee
      end

Before

The failure message is completely overridden by this flunk call and becomes misleading.

  -------------------
  | Failure Message |
  -------------------
  
  Expected the debuggee program to finish.
/Users/hung-wulo/src/github.com/ruby/debug/test/support/assertions.rb:97:in `assert_block'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:340:in `ensure in execute_dap_scenario'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:344:in `execute_dap_scenario'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:46:in `block in run_protocol_scenario'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `block in catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:198:in `timeout'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:44:in `run_protocol_scenario'
test/protocol/eval_test.rb:17:in `test_eval_evaluates_arithmetic_expressions'
     14:     RUBY
     15: 
     16:     def test_eval_evaluates_arithmetic_expressions
  => 17:       run_protocol_scenario PROGRAM do
     18:         req_add_breakpoint 5
     19:         req_continue
     20:         assert_repl_result({value: '5', type: 'Integer'}, '1+2') # should be 3
======================================================================================================================================

After

F
==============================================================================================================================================================
Failure: test_eval_evaluates_arithmetic_expressions(DEBUGGER__::EvalTest)
/Users/hung-wulo/src/github.com/ruby/debug/test/support/assertions.rb:97:in `assert_block'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:480:in `assert_eval_result'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:302:in `assert_repl_result'
test/protocol/eval_test.rb:22:in `block in test_eval_evaluates_arithmetic_expressions'
     19:         req_continue
     20:         assert_repl_result({value: '2', type: 'Integer'}, 'a')
     21:         assert_repl_result({value: '4', type: 'Integer'}, 'd')
  => 22:         assert_repl_result({value: '5', type: 'Integer'}, '1+2')
     23:         req_terminate_debuggee
     24:       end
     25:     end
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:336:in `execute_dap_scenario'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:46:in `block in run_protocol_scenario'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:189:in `block in timeout'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `block in catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:36:in `catch'
/opt/rubies/3.2.1/lib/ruby/3.2.0/timeout.rb:198:in `timeout'
/Users/hung-wulo/src/github.com/ruby/debug/test/support/protocol_test_case.rb:44:in `run_protocol_scenario'
test/protocol/eval_test.rb:17:in `test_eval_evaluates_arithmetic_expressions'
-------------------------
| All Protocol Messages |
-------------------------

(protocol messages removed as they're not important here)

--------------------------
| Last Protocol Messages |
--------------------------

(protocol messages removed as they're not important here)

--------------------
| Debuggee Session |
--------------------

> DEBUGGER: Debugger can attach via UNIX domain socket (/var/folders/lf/4xqm_gk10ts83_mxrhsz9qf80000gn/T/ruby-debug-sock-501/ruby-debug-hung-wulo-61875-1)
> DEBUGGER: wait for debugger connection...
> DEBUGGER: Connected.


-------------------
| Failure Message |
-------------------

result:
{
  "type": "response",
  "command": "evaluate",
  "request_seq": 14,
  "success": true,
  "message": "Success",
  "body": {
    "result": "3",
    "type": "Integer",
    "variablesReference": 4,
    "indexedVariables": 0,
    "namedVariables": 1
  },
  "seq": 17
}
<"5"> expected but was
<"3">
==============================================================================================================================================================

@ko1 ko1 requested a review from ono-max March 10, 2023 08:12
@@ -334,9 +334,12 @@ def execute_dap_scenario scenario

attach_to_dap_server
scenario.call
rescue Test::Unit::AssertionFailedError => e
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add this implementation to cdp, too?

@ono-max
Copy link
Member

ono-max commented Mar 12, 2023

I would be happy to write a test for this change in https://github.com/ruby/debug/blob/master/test/support/protocol_test_case_test.rb. I'll leave it up to you whether to write the test in a separate PR or the same PR.

@st0012 st0012 force-pushed the st0012-improve-protocol-test-case branch from 2d91f5d to 0940ccc Compare March 13, 2023 09:17
When assertions failed, we should let the AssertionFailedError surface
and not calling flunk so we can get correct failure messages.
@st0012 st0012 force-pushed the st0012-improve-protocol-test-case branch from 0940ccc to 2154375 Compare March 13, 2023 09:22
@st0012
Copy link
Member Author

st0012 commented Mar 13, 2023

@ono-max Both updated 👍

@st0012 st0012 requested a review from ono-max March 13, 2023 09:29
Copy link
Member

@ono-max ono-max left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thank you.

@ko1 ko1 merged commit c258ee0 into ruby:master Mar 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants