vscode python debugger is very slow

[zehongs]

Hi ptvsd team,

My code looks like...

    # ptvsd.tracing(False)
    function of file-io and some parsing operations
    # ptvsd.tracing(True)```

By uncommenting ptvsd.tracing(), the script could be 10x faster (from about 50s to 5s). So the vscode debugger speed must have something to do with ptvsd, right?

Can you improve its related performance? Or what should I do? I'm not adding this ptvsd.tracing() everywhere in my code :)

[fabioz]

Well, the first thing would be checking if you have the accelerators modules there. To check, please provide the logs from running the debugger there -- it can be on a print('hello') module (see below for instructions):

1. make sure you're using the latest version (you may have to opt-in -- see: Multiprocessing via debug adapter #1706 (comment) for details).

2. Make a simple run and provide the logs.

i.e.:

• Open VS Code
• Select the command Extensions: Open Extensions Folder
• Locate the Python extension directory, typically of the form ms-python.python-2020..***
• In that directory ensure you do not have any debug*.log files, if you do, please delete them
• Go back into VS Code and modify your launch.json to add the setting "logToFile": true, see below:

"version": "0.2.0",
"configurations": [
    {
        "name": "Python: Current File (Integrated Terminal)",
        "type": "python",
        "request": "launch",
        "program": "${file}",
        "stopOnEntry": true,
        "console": "integratedTerminal",
        "logToFile": true
    },

• Start debugging
• When done, go back into the extension directory and upload the file debug*.log into this GitHub issue.

[zehongs]

Hi fabioz, thanks for your reply.
I just found that whether there is a breakpoint in the python script will affect the speed. And I'm trying to put that breakpoint after the file-io&parsing function.

the log output

0 Starting Session:
{
    "name": "get_get_b5_dbnn_database",
    "type": "python",
    "request": "launch",
    "program": "/home/shenzehong/Code/psd_descriptor/run.py",
    "console": "integratedTerminal",
    "args": [
        "--type",
        "get_b5_dbnn_database"
    ],
    "logToFile": true,
    "pythonPath": "/home/shenzehong/miniconda3/bin/python",
    "cwd": "/home/shenzehong/Code/psd_descriptor",
    "envFile": "/home/shenzehong/Code/psd_descriptor/.env",
    "env": {
        "PYTHONIOENCODING": "UTF-8",
        "PYTHONUNBUFFERED": "1"
    },
    "stopOnEntry": false,
    "showReturnValue": true,
    "internalConsoleOptions": "neverOpen",
    "debugOptions": [
        "ShowReturnValue"
    ],
    "justMyCode": true,
    "workspaceFolder": "/home/shenzehong/Code/psd_descriptor"
}
54 Client --> Adapter:
{
    "command": "initialize",
    "arguments": {
        "clientID": "vscode",
        "clientName": "Visual Studio Code - Insiders",
        "adapterID": "python",
        "pathFormat": "path",
        "linesStartAt1": true,
        "columnsStartAt1": true,
        "supportsVariableType": true,
        "supportsVariablePaging": true,
        "supportsRunInTerminalRequest": true,
        "locale": "zh-cn"
    },
    "type": "request",
    "seq": 1
}
92 Client <-- Adapter:
{
    "seq": 1,
    "type": "event",
    "event": "output",
    "body": {
        "category": "telemetry",
        "output": "ptvsd",
        "data": {
            "packageVersion": "5.0.0a12"
        }
    }
}
94 Client <-- Adapter:
{
    "seq": 2,
    "type": "response",
    "request_seq": 1,
    "success": true,
    "command": "initialize",
    "body": {
        "supportsCompletionsRequest": true,
        "supportsConditionalBreakpoints": true,
        "supportsConfigurationDoneRequest": true,
        "supportsDebuggerProperties": true,
        "supportsDelayedStackTraceLoading": true,
        "supportsEvaluateForHovers": true,
        "supportsExceptionInfoRequest": true,
        "supportsExceptionOptions": true,
        "supportsHitConditionalBreakpoints": true,
        "supportsLogPoints": true,
        "supportsModulesRequest": true,
        "supportsSetExpression": true,
        "supportsSetVariable": true,
        "supportsValueFormattingOptions": true,
        "supportsTerminateDebuggee": true,
        "supportsGotoTargetsRequest": true,
        "exceptionBreakpointFilters": [
            {
                "filter": "raised",
                "label": "Raised Exceptions",
                "default": false
            },
            {
                "filter": "uncaught",
                "label": "Uncaught Exceptions",
                "default": true
            }
        ]
    }
}
152 Client --> Adapter:
{
    "command": "launch",
    "arguments": {
        "name": "get_get_b5_dbnn_database",
        "type": "python",
        "request": "launch",
        "program": "/home/shenzehong/Code/psd_descriptor/run.py",
        "console": "integratedTerminal",
        "args": [
            "--type",
            "get_b5_dbnn_database"
        ],
        "logToFile": true,
        "pythonPath": "/home/shenzehong/miniconda3/bin/python",
        "cwd": "/home/shenzehong/Code/psd_descriptor",
        "envFile": "/home/shenzehong/Code/psd_descriptor/.env",
        "env": {
            "PYTHONIOENCODING": "UTF-8",
            "PYTHONUNBUFFERED": "1"
        },
        "stopOnEntry": false,
        "showReturnValue": true,
        "internalConsoleOptions": "neverOpen",
        "debugOptions": [
            "ShowReturnValue"
        ],
        "justMyCode": true,
        "workspaceFolder": "/home/shenzehong/Code/psd_descriptor",
        "__sessionId": "edd0a65b-216d-40d6-9d49-6aca40e55844"
    },
    "type": "request",
    "seq": 2
}
154 Client <-- Adapter:
{
    "seq": 3,
    "type": "request",
    "command": "runInTerminal",
    "arguments": {
        "kind": "integrated",
        "title": "Python Debug Console",
        "args": [
            "/home/shenzehong/miniconda3/bin/python",
            "/home/shenzehong/.vscode-server-insiders/extensions/ms-python.python-2020.2.63072/pythonFiles/lib/python/new_ptvsd/wheels/ptvsd/launcher",
            "/home/shenzehong/Code/psd_descriptor/run.py",
            "--type",
            "get_b5_dbnn_database"
        ],
        "env": {
            "PTVSD_LAUNCHER_PORT": "34773",
            "PTVSD_LOG_DIR": "/home/shenzehong/.vscode-server-insiders/extensions/ms-python.python-2020.2.63072"
        }
    }
}
273 Client --> Adapter:
{
    "type": "response",
    "seq": 3,
    "command": "runInTerminal",
    "request_seq": 3,
    "success": true,
    "body": {
        "shellProcessId": 37553
    }
}
986 Client <-- Adapter:
{
    "seq": 4,
    "type": "event",
    "event": "initialized"
}
1082 Client --> Adapter:
{
    "command": "setBreakpoints",
    "arguments": {
        "source": {
            "name": "handle_b5_data.py",
            "path": "/home/shenzehong/Code/psd_descriptor/lib/datasets/b5_finetune/handle_b5_data.py"
        },
        "lines": [
            376
        ],
        "breakpoints": [
            {
                "line": 376
            }
        ],
        "sourceModified": false
    },
    "type": "request",
    "seq": 4
}
1096 Client <-- Adapter:
{
    "seq": 5,
    "type": "response",
    "request_seq": 4,
    "success": true,
    "command": "setBreakpoints",
    "body": {
        "breakpoints": [
            {
                "verified": true,
                "id": 0,
                "source": {
                    "name": "handle_b5_data.py",
                    "path": "/home/shenzehong/Code/psd_descriptor/lib/datasets/b5_finetune/handle_b5_data.py"
                },
                "line": 376
            }
        ]
    }
}
1144 Client --> Adapter:
{
    "command": "setExceptionBreakpoints",
    "arguments": {
        "filters": [
            "uncaught"
        ]
    },
    "type": "request",
    "seq": 5
}
1148 Client <-- Adapter:
{
    "seq": 6,
    "type": "response",
    "request_seq": 5,
    "success": true,
    "command": "setExceptionBreakpoints"
}
1213 Client --> Adapter:
{
    "command": "configurationDone",
    "type": "request",
    "seq": 6
}
1217 Client <-- Adapter:
{
    "seq": 7,
    "type": "response",
    "request_seq": 6,
    "success": true,
    "command": "configurationDone"
}
1218 Client <-- Adapter:
{
    "seq": 8,
    "type": "response",
    "request_seq": 2,
    "success": true,
    "command": "launch"
}
1219 Client <-- Adapter:
{
    "seq": 9,
    "type": "event",
    "event": "process",
    "body": {
        "startMethod": "launch",
        "isLocalProcess": true,
        "systemProcessId": 48113,
        "name": "/home/shenzehong/Code/psd_descriptor/run.py",
        "pointerSize": 64
    }
}
1219 Client <-- Adapter:
{
    "seq": 10,
    "type": "event",
    "event": "thread",
    "body": {
        "reason": "started",
        "threadId": 1
    }
}
1282 Client --> Adapter:
{
    "command": "threads",
    "type": "request",
    "seq": 7
}
1290 Client <-- Adapter:
{
    "seq": 11,
    "type": "response",
    "request_seq": 7,
    "success": true,
    "command": "threads",
    "body": {
        "threads": [
            {
                "id": 1,
                "name": "MainThread"
            }
        ]
    }
}
1390 Client --> Adapter:
{
    "command": "threads",
    "type": "request",
    "seq": 8
}
1393 Client <-- Adapter:
{
    "seq": 12,
    "type": "response",
    "request_seq": 8,
    "success": true,
    "command": "threads",
    "body": {
        "threads": [
            {
                "id": 1,
                "name": "MainThread"
            }
        ]
    }
}
1839 Client <-- Adapter:
{
    "seq": 13,
    "type": "event",
    "event": "thread",
    "body": {
        "reason": "started",
        "threadId": 2
    }
}
1989 Client --> Adapter:
{
    "command": "threads",
    "type": "request",
    "seq": 9
}
2002 Client <-- Adapter:
{
    "seq": 14,
    "type": "response",
    "request_seq": 9,
    "success": true,
    "command": "threads",
    "body": {
        "threads": [
            {
                "id": 1,
                "name": "MainThread"
            },
            {
                "id": 2,
                "name": "Thread-6"
            }
        ]
    }
}
9925 Client <-- Adapter:
{
    "seq": 15,
    "type": "event",
    "event": "thread",
    "body": {
        "reason": "exited",
        "threadId": 2
    }
}
9927 Client <-- Adapter:
{
    "seq": 16,
    "type": "event",
    "event": "thread",
    "body": {
        "reason": "started",
        "threadId": 3
    }
}
10084 Client --> Adapter:
{
    "command": "threads",
    "type": "request",
    "seq": 10
}
10328 Client <-- Adapter:
{
    "seq": 17,
    "type": "response",
    "request_seq": 10,
    "success": true,
    "command": "threads",
    "body": {
        "threads": [
            {
                "id": 1,
                "name": "MainThread"
            },
            {
                "id": 3,
                "name": "Thread-7"
            }
        ]
    }
}
23738 Client --> Adapter:
{
    "command": "disconnect",
    "arguments": {
        "restart": false
    },
    "type": "request",
    "seq": 11
}
23781 Client <-- Adapter:
{
    "seq": 18,
    "type": "event",
    "event": "exited",
    "body": {
        "exitCode": 247
    }
}
23782 Client <-- Adapter:
{
    "seq": 19,
    "type": "event",
    "event": "terminated"
}
23783 Client <-- Adapter:
{
    "seq": 20,
    "type": "response",
    "request_seq": 11,
    "success": true,
    "command": "disconnect"
}
23813 Stopping Session
23815 Error:
{}
23816 Exit:
Exit-Code: 0
Signal: none

[fabioz]

Humm, it seems that the info on whether the accelerator modules are being used is not there as I thought it'd be (sorry about that).

There's another way to get it though:

Run (under the debugger) the code below and please provide the output.

from _pydevd_bundle.pydevd_trace_dispatch import USING_CYTHON
from _pydevd_frame_eval.pydevd_frame_eval_main import USING_FRAME_EVAL
print('USING_CYTHON', USING_CYTHON)
print('USING_FRAME_EVAL', USING_FRAME_EVAL)

[fabioz]

Humm, it seems that the accelerator modules are forced to be disabled... I think the reason for that is historical and it should no longer be needed (I created a pull request to fix that: microsoft/debugpy#49).

In the meanwhile, you can locally do that same change (basically commenting the pydevd_constants.CYTHON_SUPPORTED = False in force_pydevd.py as in that pull request -- and after doing that change provide the output from the code in the previous comment).

[zehongs]

cannot import name 'USING_CYTHON' from '_pydevd_bundle.pydevd_trace_dispatch' (/home/shenzehong/.vscode-server-insiders/extensions/ms-python.python-2020.2.63990/pythonFiles/lib/python/old_ptvsd/ptvsd/_vendored/pydevd/_pydevd_bundle/pydevd_trace_dispatch.py)

I suppose you're talking about use_cython, which is NO. And use_frame_eval is None.

And I'm curious why I am still using old_ptvsd. It turns out I can only add these to my User setting

    "python.experiments.optInto": [
        "DebugAdapterFactory - experiment",
        "PtvsdWheels37 - experiment"
    ]

But I'm using ssh-remote, and in remote setting and workspace setting I cannot add such optInto settings.

[zehongs]

[fabioz]

@karthiknadig do you have info on how to opt into the experiments with ssh-remote as @zehongs is doing?

[fabioz]

@zehongs it's really expected that depending on where you have the breakpoint things will run slower... still, I just double checked and there's one information missing in the DAP integration, which makes the debugger slower if there's any breakpoint in the same file and not in a particular function, so, I've created an issue to address that: microsoft/debugpy#51

-- note that this may not be as important if you use Python 3.7+ with the accelerator modules, but it does a big difference on pure-tracing (which would also be the case for PyPy), so, it's high on my priority list to fix that.

[zehongs]

@fabioz Thanks!

[karthiknadig]

@fabioz @zehongs This is currently a limitation, experiments cannot be set for the remote scenario. I will create a issue on the extension to support this.

Created microsoft/vscode-python#10232 to address it.

[int19h]

Since we have separate items tracking the underlying issues, I'm going to go ahead and close this one.