Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the test framework for DAP #417

Merged
merged 11 commits into from
Dec 14, 2021
Merged

Conversation

ono-max
Copy link
Member

@ono-max ono-max commented Nov 29, 2021

This PR adds the following features.

1. The test framework

Failure log

% ruby test/dap/step_test.rb
Tests on local and remote. You can disable remote tests with RUBY_DEBUG_TEST_NO_REMOTE=1.
Loaded suite test/dap/step_test
Started
F
================================================================================================================================================================================================================================================================================
Failure: test_step_works_correctly(DEBUGGER__::StepTest1638676609)
/Users/s15236/workspace/debug/test/support/assertions.rb:79:in `assert_block'
/Users/s15236/workspace/debug/test/support/utils.rb:634:in `block (2 levels) in run_dap_scenario'
/Users/s15236/workspace/debug/test/support/utils.rb:628:in `each'
/Users/s15236/workspace/debug/test/support/utils.rb:628:in `block in run_dap_scenario'
/Users/s15236/workspace/debug/test/support/utils.rb:567:in `each'
/Users/s15236/workspace/debug/test/support/utils.rb:567:in `run_dap_scenario'
test/dap/step_test.rb:21:in `test_step_works_correctly'
     18:     RUBY
     19:
     20:     def test_step_works_correctly
  => 21:       run_dap_scenario PROGRAM do
     22:         [
     23:           *CFG_DAP,
     24:           {
-------------------------
| All Protocol Messages |
-------------------------

V>D {"seq":1,"command":"initialize","arguments":{"clientID":"vscode","clientName":"Visual Studio Code","adapterID":"rdbg","pathFormat":"path","linesStartAt1":true,"columnsStartAt1":true,"supportsVariableType":true,"supportsVariablePaging":true,"supportsRunInTerminalRequest":true,"locale":"en-us","supportsProgressReporting":true,"supportsInvalidatedEvent":true,"supportsMemoryReferences":true},"type":"request"}
V>D {"seq":2,"command":"attach","arguments":{"type":"rdbg","name":"Attach with rdbg","request":"attach","rdbgPath":"/Users/s15236/workspace/debug/exe/rdbg","debugPort":"/var/folders/kv/w1k6nh1x5fl7vx47b2pd005w0000gn/T/ruby-debug-sock-501/ruby-debug-naotto-8845","autoAttach":true,"__sessionId":"141d9c79-3669-43ec-ac1f-e62598c5a65a"},"type":"request"}
V>D {"seq":3,"command":"setFunctionBreakpoints","arguments":{"breakpoints":[]},"type":"request"}
V>D {"seq":4,"command":"setExceptionBreakpoints","arguments":{"filters":[],"filterOptions":[{"filterId":"RuntimeError"}]},"type":"request"}
V>D {"seq":5,"command":"configurationDone","type":"request"}
V<D {"type":"response","command":"initialize","request_seq":1,"success":true,"message":"Success","body":{"supportsConfigurationDoneRequest":true,"supportsFunctionBreakpoints":true,"supportsConditionalBreakpoints":true,"supportTerminateDebuggee":true,"supportsTerminateRequest":true,"exceptionBreakpointFilters":[{"filter":"any","label":"rescue any exception"},{"filter":"RuntimeError","label":"rescue RuntimeError","default":true}],"supportsExceptionFilterOptions":true,"supportsStepBack":true,"supportsEvaluateForHovers":true},"seq":1}
V<D {"type":"event","event":"initialized","seq":2}
V<D {"type":"response","command":"attach","request_seq":2,"success":true,"message":"Success","seq":3}
V<D {"type":"response","command":"setFunctionBreakpoints","request_seq":3,"success":true,"message":"Success","seq":4}
V<D {"type":"response","command":"setExceptionBreakpoints","request_seq":4,"success":true,"message":"Success","body":{"breakpoints":[{"verified":true,"message":"#<DEBUGGER__::CatchBreakpoint:0x00007fba0209a768 @pat=\"RuntimeError\", @key=[:catch, \"RuntimeError\"], @last_exc=nil, @cond=nil, @command=nil, @path=nil, @deleted=false, @tp=#<TracePoint:enabled>>"}]},"seq":5}
V<D {"type":"response","command":"configurationDone","request_seq":5,"success":true,"message":"Success","seq":6}
V<D {"type":"event","event":"stopped","body":{"reason":"pause","threadId":1,"allThreadsStopped":true},"seq":7}

--------------------------
| Last Protocol Messages |
--------------------------

{
  "type": "response",
  "command": "setFunctionBreakpoints",
  "request_seq": 3,
  "success": true,
  "message": "Success",
  "seq": 4
}
{
  "type": "response",
  "command": "setExceptionBreakpoints",
  "request_seq": 4,
  "success": true,
  "message": "Success",
  "body": {
    "breakpoints": [
      {
        "verified": true,
        "message": "#<DEBUGGER__::CatchBreakpoint:0x00007fba0209a768 @pat=\"RuntimeError\", @key=[:catch, \"RuntimeError\"], @last_exc=nil, @cond=nil, @command=nil, @path=nil, @deleted=false, @tp=#<TracePoint:enabled>>"
      }
    ]
  },
  "seq": 5
}
{
  "type": "response",
  "command": "configurationDone",
  "request_seq": 5,
  "success": true,
  "message": "Success",
  "seq": 6
}
{
  "type": "event",
  "event": "stopped",
  "body": {
    "reason": "pause",
    "threadId": 1,
    "allThreadsStopped": true
  },
  "seq": 7
}

--------------------
| Debuggee Session |
--------------------

> EBUGGER: Debugger can attach via UNIX domain socket (/var/folders/5j/z2c9zm7124q81f_py4xmd3scp7w9j7/T/ruby-debug-sock-746464807/ruby-debug-s15236-91848-1)
> DEBUGGER: wait for debugger connection...
> DEBUGGER: Connected.


-------------------
| Failure Message |
-------------------

expected:
{
  "type": "event",
  "event": "stopped",
  "body": {
    "reason": "paus",
    "threadId": 1,
    "allThreadsStopped": true
  }
}

result:
{
  "type": "event",
  "event": "stopped",
  "body": {
    "reason": "pause",
    "threadId": 1,
    "allThreadsStopped": true
  },
  "seq": 7
}
<"paus"> expected but was
<"pause">

diff:
? pause
================================================================================================================================================================================================================================================================================

Finished in 0.377592 seconds.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 tests, 3 assertions, 1 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
0% passed
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2.65 tests/s, 7.95 assertions/s

2. The test generator for DAP

Example of Generated file

# frozen_string_literal: true

require_relative '../support/test_case'

module DEBUGGER__
  
  class FooTest1639301775 < TestCase
    PROGRAM = <<~RUBY
      1| a = 1
      2| b = 2
      3| c = 3
      4| d = 4
      5| e = 5
    RUBY
    
    def test_1639301775
      run_dap_scenario PROGRAM do
        [
          *CFG_DAP,
          {
            seq: 7,
            type: "event",
            event: "stopped",
            body: {
              reason: "pause",
              threadId: 1,
              allThreadsStopped: true
            }
          },
          {
            seq: 6,
            command: "threads",
            type: "request"
          },
          {
            seq: 8,
            type: "response",
            command: "threads",
            request_seq: 6,
            success: true,
            message: "Success",
            body: {
              threads: [
                {
                  id: 1,
                  name: /#1 .*/
                }
              ]
            }
          },
          {
            seq: 7,
            command: "threads",
            type: "request"
          },
          {
            seq: 9,
            type: "response",
            command: "threads",
            request_seq: 7,
            success: true,
            message: "Success",
            body: {
              threads: [
                {
                  id: 1,
                  name: /#1 .*/
                }
              ]
            }
          },
          {
            seq: 8,
            command: "stackTrace",
            arguments: {
              threadId: 1,
              startFrame: 0,
              levels: 20
            },
            type: "request"
          },
          {
            seq: 10,
            type: "response",
            command: "stackTrace",
            request_seq: 8,
            success: true,
            message: "Success",
            body: {
              stackFrames: [
                {
                  name: "<main>",
                  line: 1,
                  column: 1,
                  source: {
                    name: /#{File.basename temp_file_path}/,
                    path: /#{temp_file_path}/,
                    sourceReference: nil
                  },
                  id: 1
                }
              ]
            }
          },
          {
            seq: 9,
            command: "scopes",
            arguments: {
              frameId: 1
            },
            type: "request"
          },
          {
            seq: 11,
            type: "response",
            command: "scopes",
            request_seq: 9,
            success: true,
            message: "Success",
            body: {
              scopes: [
                {
                  name: "Local variables",
                  presentationHint: "locals",
                  namedVariables: /\d+/,
                  indexedVariables: 0,
                  expensive: false,
                  variablesReference: 2
                },
                {
                  name: "Global variables",
                  presentationHint: "globals",
                  variablesReference: 1,
                  namedVariables: /\d+/,
                  indexedVariables: 0,
                  expensive: false
                }
              ]
            }
          },
          {
            seq: 10,
            command: "variables",
            arguments: {
              variablesReference: 2
            },
            type: "request"
          },
          {
            seq: 12,
            type: "response",
            command: "variables",
            request_seq: 10,
            success: true,
            message: "Success",
            body: {
              variables: [
                {
                  name: "%self",
                  value: "main",
                  type: "Object",
                  variablesReference: 3,
                  indexedVariables: 0,
                  namedVariables: /\d+/
                },
                {
                  name: "a",
                  value: "nil",
                  type: "NilClass",
                  variablesReference: 4,
                  indexedVariables: 0,
                  namedVariables: /\d+/
                },
                {
                  name: "b",
                  value: "nil",
                  type: "NilClass",
                  variablesReference: 5,
                  indexedVariables: 0,
                  namedVariables: /\d+/
                },
                {
                  name: "c",
                  value: "nil",
                  type: "NilClass",
                  variablesReference: 6,
                  indexedVariables: 0,
                  namedVariables: /\d+/
                },
                {
                  name: "d",
                  value: "nil",
                  type: "NilClass",
                  variablesReference: 7,
                  indexedVariables: 0,
                  namedVariables: /\d+/
                },
                {
                  name: "e",
                  value: "nil",
                  type: "NilClass",
                  variablesReference: 8,
                  indexedVariables: 0,
                  namedVariables: /\d+/
                }
              ]
            }
          },
          {
            seq: 11,
            command: "continue",
            arguments: {
              threadId: 1
            },
            type: "request"
          },
          {
            seq: 13,
            type: "response",
            command: "continue",
            request_seq: 11,
            success: true,
            message: "Success",
            body: {
              allThreadsContinued: true
            }
          }
        ]
      end
    end
  end
end

3. Some test cases

@ono-max ono-max force-pushed the testframework-for-dap branch 12 times, most recently from 460c702 to dc3c160 Compare December 6, 2021 12:06
@ono-max ono-max changed the title [WIP] Add the test framework for DAP Add the test framework for DAP Dec 6, 2021
@ono-max ono-max force-pushed the testframework-for-dap branch 7 times, most recently from 605571d to b9a2638 Compare December 12, 2021 08:54
@ono-max ono-max marked this pull request as ready for review December 12, 2021 08:54
@ono-max ono-max force-pushed the testframework-for-dap branch 2 times, most recently from 2e4d5ab to 113215f Compare December 13, 2021 00:25
@ko1 ko1 merged commit 25c14c1 into ruby:master Dec 14, 2021
@ono-max ono-max deleted the testframework-for-dap branch January 11, 2022 00:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants