debugging scripts with VS Code

[maxpietsch]

Debugging scripts such as population_template without debugger isn't fun so looked into debugging MRtrix3 scripts with VS Code.

The wrapper around python in VS Code breaks the inspect logic in the mrtrix3 module:

Exception has occurred: AttributeError
module 'runpy' has no attribute 'usage'
  File "/Users/mp/mrtrix3/lib/mrtrix3/app.py", line 125, in _execute
    module.usage(CMDLINE)
  File "/Users/mp/mrtrix3/lib/mrtrix3/__init__.py", line 94, in execute
    app._execute(inspect.getmodule(inspect.stack()[-1][0])) # pylint: disable=protected-access
  File "/Users/mp/mrtrix3/bin/population_template", line 1487, in <module>
    mrtrix3.execute() #pylint: disable=no-member

VS Code adds a lot of frames to the end of the stack:

# VS Code:
[FrameInfo(frame=<frame at 0x7fcd9b828350, file '/Users/mp/mrtrix3/lib/mrtrix3/__init__.py', line 94, code execute>, filename='/Users/mp/mrtrix3/lib/mrtrix3/__init__.py', lineno=94, function='execute', code_context=['  print(inspect.stack())\n'], index=0), FrameInfo(frame=<frame at 0x110b1a040, file '/Users/mp/mrtrix3/bin/population_template', line 1487, code <module>>, filename='/Users/mp/mrtrix3/bin/population_template', lineno=1487, function='<module>', code_context=['mrtrix3.execute() #pylint: disable=no-member\n'], index=0), FrameInfo(frame=<frame at 0x7fcd9af4d7a0, file '/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py', line 87, code _run_code>, filename='/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py', lineno=87, function='_run_code', code_context=['    exec(code, run_globals)\n'], index=0), FrameInfo(frame=<frame at 0x7fcd9af39b20, file '/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py', line 97, code _run_module_code>, filename='/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py', lineno=97, function='_run_module_code', code_context=['        _run_code(code, mod_globals, init_globals,\n'], index=0), FrameInfo(frame=<frame at 0x7fcd9b833f40, file '/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py', line 268, code run_path>, filename='/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py', lineno=268, function='run_path', code_context=['        return _run_module_code(code, init_globals, run_name,\n'], index=0), FrameInfo(frame=<frame at 0x7fcdbb8a0fd0, file '/Users/mp/.vscode/extensions/ms-python.python-2022.4.1/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py', line 285, code run_file>, filename='/Users/mp/.vscode/extensions/ms-python.python-2022.4.1/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py', lineno=285, function='run_file', code_context=['    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))\n'], index=0), FrameInfo(frame=<frame at 0x7fcdaaf32570, file '/Users/mp/.vscode/extensions/ms-python.python-2022.4.1/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py', line 444, code main>, filename='/Users/mp/.vscode/extensions/ms-python.python-2022.4.1/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py', lineno=444, function='main', code_context=['        run()\n'], index=0), FrameInfo(frame=<frame at 0x10f3ec040, file '/Users/mp/.vscode/extensions/ms-python.python-2022.4.1/pythonFiles/lib/python/debugpy/__main__.py', line 45, code <module>>, filename='/Users/mp/.vscode/extensions/ms-python.python-2022.4.1/pythonFiles/lib/python/debugpy/__main__.py', lineno=45, function='<module>', code_context=['    cli.main()\n'], index=0), FrameInfo(frame=<frame at 0x7fcdac018df0, file '/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py', line 87, code _run_code>, filename='/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py', lineno=87, function='_run_code', code_context=['    exec(code, run_globals)\n'], index=0), FrameInfo(frame=<frame at 0x7fcdac01a280, file '/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py', line 197, code _run_module_as_main>, filename='/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py', lineno=197, function='_run_module_as_main', code_context=['    return _run_code(code, main_globals, None,\n'], index=0)]


# terminal:
[FrameInfo(frame=<frame at 0x7f7b677107d0, file '/Users/mp/mrtrix3/lib/mrtrix3/__init__.py', line 94, code execute>, filename='/Users/mp/mrtrix3/lib/mrtrix3/__init__.py', lineno=94, function='execute', code_context=['  print(inspect.stack())\n'], index=0), FrameInfo(frame=<frame at 0x7f7b8002d7a0, file '/Users/mp/mrtrix3/bin/population_template', line 1488, code <module>>, filename='/Users/mp/mrtrix3/bin/population_template', lineno=1488, function='<module>', code_context=['mrtrix3.execute() #pylint: disable=no-member\n'], index=0)]

Replacing the [-1] by [1] in app._execute(inspect.getmodule(inspect.stack()[-1][0])) # pylint: disable=protected-access lets me run scripts in VS Code. Is that sensible or should we look for a specific frame? Couldn't find any issues with this modification but don't fully understand the logic and whether that breaks things elesewhere.

With this, one can use the VS Code debugger which is similar to Matlab's debugger or use pdb pr pdb++ on the terminal.

We have a wiki entry for binaries but couldn't find anything for python scripts. Maybe we could add a section about python and the VS Code debugger:

My launch.json:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "mode": "debug",
            "program": "${file}",
            "cwd": "/Users/mp/mrtrix3/",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args" : ["/path/to/data", "/tmp/out.mif", "-debug"]
        }
    ]
}

[Lestropie]

For specifically the execute() call, it's probably OK? Been a while since I've played with it. There are other areas that inspect.stack() is used that it definitely needs to read from the top of the stack rather than the bottom (e.g. getting code filename and line number for debug calls), but for execute() I'm not sure what else would potentially change the contents of the stack at that point. Would nevertheless need to test across OS's; also slightly worried about running natively on Windows (ie. not in MSYS2) as I've never set that up myself to test it.

[Lestropie]

Closed by #2475.