Unnecessary truncation

[zljubisic]

If you look at the picture:

In debug console, I have created a dictionary with long keys.
If I execute dct directly, keys are truncated with "..." in the middle.
If I print it, all keys are visible (it is funny that both lines are of the same width, isn't it?).
Last print is just to show you how much horizontal space I have.

Please don't truncate something that can fit in the debug console.
Try to wisely use whole horizontal and vertical space as much as you can.

Extension version: 2022.16.0
VS Code version: Code 1.72.1 (129500ee4c8ab7263461ffe327268ba56b9f210d, 2022-10-10T17:22:48.346Z)
OS version: Windows_NT x64 10.0.19042
Modes:
Sandboxed: No
Remote OS version: Linux x64 3.10.0-1127.el7.x86_64
Remote OS version: Linux x64 3.10.0-1127.el7.x86_64

[karthiknadig]

I think there is a 300 character limit on strings. There is a way to increase the limit in the debugger. Moving this to debugpy.

[rchiodo]

I believe this is set here:

https://github.com/microsoft/debugpy/blob/main/src/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py#L63

and then used here

https://github.com/microsoft/debugpy/blob/main/src/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py#L276

Meaning it's currently hardcoded. 'Inner' strings max out at 30 chars. In this case, the 'inner' string is the key for the dictionary.

@int19h, @fabioz does this sound correct? I don't believe there is currently a way to change this.

[int19h]

The problem is that the debugger gets no clue from the client as to the size the output can fit. The best that we can do is to not trim anything if the output is REPL (which is the only bit of context that we do have). But then if you accidentally eval something that has a multi-megabyte repr(), you'll have to wait for it to finish printing, which is much worse.

I don't think we have any knobs exposed to adjust this, unfortunately. You can look at pydevd_safe_repr.py in to see the limits and tweak them in your local install.

A proper fix would require DAP changes - e.g. ValueFormat could get a new property to propagate the output width?

[zljubisic]

@int19h, @fabioz, @rchiodo From my side, vscode debugger is a very unpleasant place and as time goes by, I can see why (there is even a little rhyme in this sentence 😁). I briefly looked in the https://github.com/microsoft/debugpy/blob/main/src/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_safe_repr.py and as I understand I should change values for maxstring_inner and maxother_outer. I can set them to some big values, but how should I set them to not trim anything?

Another question is... why you don't use something like rich's print (from rich import print as rprint)?
So, when I execute dct (dct = {f"a quite very very very long key {cnt}": cnt for cnt in range(5)}) from the picture above I will get:

What is wrong with rich.print()?

But than again...

...you really messed it up.

Anyway, PyCharm has a solution for this problem from the very beginning:

... not beautiful as rich.print() but much better than vscode.

[zljubisic]

Can you please provide an update?

[rchiodo]

You would set the limits in your local pydevd_safe_repr.py to whatever limit you want. Something like say 300

[starball5]

Related Stack Overflow question: Is there any way to get the output in debug mode, not to be truncated?

[zljubisic]

Can you please provide an update for this issue?

[int19h]

This is unlikely to change in the current major debugpy release (1.x). But we're in the process of doing a major rewrite for the debugger, and as part of that, I would like to update this logic and make it more configurable. In principle, this could include customizing which function gets called in lieu of repr() - I don't think we'd want to hardcode rich.print or any other specific implementation that deviates significantly from what regular repr() does, since users might not expect that, but as a customization point, this is fine. Would it adequately address your scenario?

[zljubisic]

Hi @int19h,

I opened this issue regarding problems with regular print and truncation.
Any print, no matter standard or rich should work.
It could be customization I am OK with it, but when I read "...major rewrite for the debugger..." I get chills. :)

Regards.

[int19h]

In your original repro, truncation happens for cases where you do not use print, but rely on the REPL itself to write out the result. This is the case where the "safe repr" logic kicks in, same as in Variables and Watch. If you explicitly do print (or rich.print etc) explicitly, that should not have "..." anywhere - if it does, it's because of str() for that type producing them.

So, the original issue is specifically about "safe repr". We cannot just ditch that for reasons described earlier, but we can make that customizable, and improve the implementation to be saner wrt what gets truncated with ... when it kicks in. With respect to making it auto-configure itself to allow anything up to debug console width, I don't think we can do it because the string has to be generated by the debug adapter, but the debug adapter doesn't know the debug console width. If you look at interface EvaluateArguments in https://microsoft.github.io/debug-adapter-protocol/specification, this is the entirety of what debugpy gets from the DAP client. We kinda sorta know that it's debug console because it sets "context": "repl", but we don't know anything else about what it actually looks like, its dimensions or line-wrapping policy etc.

Similarly, print() itself doesn't know anything about the debug console. It just prints to stdout, and however that is implemented, it can only look to stdout and the associated tty (if any) for clues on how to truncate if it needs to do that. But debug console is not stdout in and of itself; it merely tees stdout and prints everything that was sent there (if "redirectOutput": true). If you're running with "console":"internalConsole", the stdout is not a terminal at all, actually, so I think pretty-printing implementations just assume 80-column width - this is why print does what it does in your screenshot above. In this case, since it's a Pandas dataframe and print just calls its __str__, the following bit from their docs is applicable:

display.width : int
Width of the display in characters. In case python/IPython is running in a terminal this can be set to None and pandas will correctly auto-detect the width. Note that the IPython notebook, IPython qtconsole, or IDLE do not run in a terminal and hence it is not possible to correctly detect the width. [default: 80] [currently: 80]

So even if debugpy could determine the actual width of debug console somehow, the best we could do here is special-case Pandas by detecting when it gets loaded, and if "console": "internalConsole", automatically setting display.width to None. But there's no generic solution that I can see here that would magically work across all implementations of __str__.

[zljubisic]

Hi Pavel (@int19h). You see how it goes.
Due to interface implementation... there's no generic solution...
When foundation is not good, it is hard to support our requirements.
You have huge number of issues regarding debug console.
We should fix it but in some systematic way.
Probably, some components should be rewritten from the very beginning.

[judej]

consider changing the width property in pandas to None.