Debugging scripts that use matplotlib interactively on vscode....

[amine-aboufirass]

Environment data

• VS Code version: 1.55.2 user setup
• Extension version (available under the Extensions sidebar): v2021.4.765268190
• OS and version: Windows 10 Version 10.0.19042 Build 19042
• Python version (& distribution if applicable, e.g. Anaconda): python 3.9.2 with conda 4.9.2
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...):
• Relevant/affected Python packages and their versions: matplotlib 3.3.4
• Relevant/affected Python-related VS Code extensions and their versions: None that I can think of
• Value of the python.languageServer setting: I don't think this is related to my issue

Expected behaviour

Would like to debug script that uses matplotlib interactively. By interactively I mean placing a break point and watching my figure update as I actually step through my code. PyCharm has this behavior out of the box and it is extremely useful.

Actual behaviour

Stepping through a script using matplotlib will not generate an interactive figure.

Steps to reproduce:

Create a file called test.py and paste the following code:

import matplotlib.pyplot as plt


figure = plt.figure()
ax = plt.gca()
ax.plot([1,2,3])
ax.set_title("hello")

Place a breakpoint on line 4, create a standard debug configuration and start the debugger. Step over lines 4, 5, and 6. No plot appears.

[fabioz]

This happens because debugpy has a custom main which isn't initializing the matplotlib support -- the support is initialized on pydevd.run where it does:

if INTERACTIVE_MODE_AVAILABLE:
    self.init_matplotlib_support()

i.e.: https://github.com/microsoft/debugpy/blob/main/src/debugpy/_vendored/pydevd/pydevd.py#L2343

and that's skipped when pydevd.settrace is set directly (which is the way that debugpy is currently activating pydevd at https://github.com/microsoft/debugpy/blob/main/src/debugpy/server/api.py#L43).

@awa5114 A simple workaround for now would be calling that method yourself.

i.e.: until this issue is fixed, you can add to the start of your code something as:

try:
    import pydevd
except ImportError:
    pass
else:
    dbg = pydevd.get_global_debugger()
    if dbg is not None:
        dbg.init_matplotlib_support()

[amine-aboufirass]

Yes... it works thank you. Would be nice to not have to copy paste this code in every .py file I work on. Is there a cleaner way? I guess I could paste that code in a separate file and import it, but I was wondering whether I could just have this as a sort of global setting on vscode?

[karthiknadig]

@fabioz Can we add this to launch json setting, pass this through to pydevd?

[amine-aboufirass]

@karthiknadig that would be awesome... if there's anything I can do to help please let me know.

[fabioz]

@fabioz Can we add this to launch json setting, pass this through to pydevd?

@karthiknadig this is actually always activated in pydevd, so, I'm not sure we even need a flag to that (the way that it works is that it autodetects that matplotlib is imported and then calls the related method to update the UI when the debugger is paused)... what do you think?

[amine-aboufirass]

@fabioz its not done by default, thats why I opened this issue...

[fabioz]

@fabioz its not done by default, thats why I opened this issue...

I meant that it's always activated on pydevd runs, but not on debugpy runs, so, my suggestion was to always activate in debugpy too (instead of adding a flag as @karthiknadig had mentioned in @fabioz Can we add this to launch json setting, pass this through to pydevd?)

[amine-aboufirass]

@fabioz Ah I see. It would have to be implemented in the debugpy launcher directly then.

Is there a temporary solution? i.e. a way to eliminate that try except block from my code but somehow ensure the behavior occurs while debugging? Does the launch.json provide options to somehow preprocess the target file?

[fabioz]

Well, some options to always launch that could be doing a custom sitecustomize or locally changing pydevd to do that.

i.e.: run the following in the debugger:

import pydevd
print(pydevd)

Then, open that file and change the function:

def _locked_settrace

at the end of that function (before the comment # Suspend as the last thing after all tracing is in place.)
add: py_db.init_matplotlib_support().

[amine-aboufirass]

Hmmm... I feel a bit uncomfortable modifying source files like that. Was hoping a solution might exist more at the vscode level.

Is there really no other option but modifying the source code? If so can I submit a pull request containing that change such that it can be included in future releases of vscode?

[fabioz]

Hmmm... I feel a bit uncomfortable modifying source files like that. Was hoping a solution might exist more at the vscode level.

Is there really no other option but modifying the source code like that? If so can I submit a pull request containing that change such that it can be included in future releases of vscode?

You can create the custom sitecustomize.py to add that workaround, which I also mentioned (see: https://docs.python.org/3/library/site.html for more details).

I'll actually provide a fix for this before the next release anyways (it's really a one line change, I just have to decide on the better place to have that, so, having the pull request wouldn't really help me here -- although it's probably where I mentioned before I need to check whether this is ok in all possible scenarios)... the workaround is so that you can have it working right now in your computer. You can do it in your own code through a sitecustomize.py, change the debugger directly or just change your code directly where you need it.

I'm not aware of a different solution at the vscode level.

[amine-aboufirass]

@fabioz fair enough. I see that you have already added the relevant code. If I uninstall/reinstall or update the extension, should it now work as expected? At which point does the change get incorporated into the extension itself such that stupid users like me can make use of it :P ?

[amine-aboufirass]

@fabioz sorry to bug you but would it be possible to get a response on the above comment?

[fabioz]

Sorry about the delay... I don't really know when a new vscode-python release including this fix will be done.

@karthiknadig do you have any insight here?

As a note, if you want, you can get the current version in the repo and use it until an official version with this fix is released.

i.e.:

git clone https://github.com/microsoft/debugpy.git
cd debugpy
pwd

Then, in your launch config add to your launch configuration:

"debugAdapterPath": "<pwd path>/src/debugpy/adapter"

[WXZhao7]

Even in v2021.7.1050252941, this bug is stilled at here. 👎

I don't like the code as below.

[karthiknadig]

This is not released yet. We have not published a new version of debugpy. If you want to use debugpy from main branch, then you can do this:

1. Create a folder 'C:\GIT\debugger`.
2. Install to a local directory from git:

python -m pip install -t C:\GIT\debugger  git+https://github.com/microsoft/debugpy.git  

3. Use debugAdapterPath field like this:

        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "debugAdapterPath": "C:\\GIT\\debugger\\debugpy\\adapter"
        },

[WXZhao7]

This is not released yet. We have not published a new version of debugpy. If you want to use debugpy from main branch, then you can do this:

1. Create a folder 'C:\GIT\debugger`.
2. Install to a local directory from git:

python -m pip install -t C:\GIT\debugger  git+https://github.com/microsoft/debugpy.git  

3. Use debugAdapterPath field like this:

        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "debugAdapterPath": "C:\\GIT\\debugger\\debugpy\\adapter"
        },

Thanks! I will try it.

[cactus1549]

This is not released yet. We have not published a new version of debugpy. If you want to use debugpy from main branch, then you can do this:

1. Create a folder 'C:\GIT\debugger`.
2. Install to a local directory from git:

python -m pip install -t C:\GIT\debugger  git+https://github.com/debugpy/debugpy.git  

3. Use debugAdapterPath field like this:

        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "debugAdapterPath": "C:\\GIT\\debugger\\debugpy\\adapter"
        },

Thanks! I will try it.

I tried this (on Windows 10) and I get:
ERROR: Directory 'C:\\GIT\\debugger' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

[karthiknadig]

@cactus1549 Can you share the command you ran? This will happen if -t is not used with pip.

[cactus1549]

@cactus1549 Can you share the command you ran? This will happen if -t is not used with pip.

Hi, I used

python -m pip install C:\GIT\debugger git+https://github.com/debugpy/debugpy.git

I just tried your suggestion and this is what I got:

 python -m pip install -t C:\GIT\debugger git+https://github.com/debugpy/debugpy.git
Collecting git+https://github.com/debugpy/debugpy.git
  Cloning https://github.com/debugpy/debugpy.git to c:\users\sacha\appdata\local\temp\pip-req-build-us39l4ti
  Running command git clone --filter=blob:none --quiet https://github.com/debugpy/debugpy.git 'C:\Users\sacha\AppData\Local\Temp\pip-req-build-us39l4ti'
info: please complete authentication in your browser...
  remote: Repository not found.
  fatal: repository 'https://github.com/debugpy/debugpy.git/' not found
  error: subprocess-exited-with-error

  × git clone --filter=blob:none --quiet https://github.com/debugpy/debugpy.git 'C:\Users\sacha\AppData\Local\Temp\pip-req-build-us39l4ti' did not run successfully.
  │ exit code: 128
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× git clone --filter=blob:none --quiet https://github.com/debugpy/debugpy.git 'C:\Users\sacha\AppData\Local\Temp\pip-req-build-us39l4ti' did not run successfully.
│ exit code: 128
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[karthiknadig]

@cactus1549 Try this looks like there was a mistake in the path:

python -m pip install -t C:\GIT\debugger git+https://github.com/microsoft/debugpy.git

I fixed the path in my original comment.

[cactus1549]

> python -m pip install -t C:\GIT\debugger git+https://github.com/debugpy/debugpy.git

I used python -m pip install -t C:\GIT\debugger git+https://github.com/microsoft/debugpy.git and that seemed to install fine. I also added "debugAdapterPath": "C:\\GIT\\debugger\\debugpy\\adapter" to my launch.json but it seems to have made no difference. If I put a break point in program.py and then in the debug console I type:

plt.plot(somedata)
plt.show()

the matplotlib Figure displays empty, so it's effectively impossible to debug figures interactively.

[fabioz]

@cactus1549 I just picked this thread.

Version 1.5.1 (which is the last one released at Oct/2021) already had the changes, so, it's not surprising your case also isn't working with the current version...

Can you create a new issue regarding your use case?

Please also include the output of running:

import matplotlib
backend = matplotlib.rcParams['backend']
print('backend', backend)
from matplotlib.rcsetup import interactive_bk, non_interactive_bk
print('interactive', backend in interactive_bk)
print('non interactive', backend in non_interactive_bk)

[cactus1549]

Hi @fabioz,
Thank you for your reply. Where should I create the new issue? You speak about 1.5.1 - is it a microsoft/vscode? issue? Sorry (I'm rather new and) it's not clear to me why the issue is in debugpy in the first place.
PS. The output from the code is:

backend QtAgg        
interactive True     
non interactive False

[fabioz]

Thank you for your reply. Where should I create the new issue?

You can create it in this repository (https://github.com/microsoft/debugpy/) which is related to debugger issues in vscode-python.