Change in behavior in how diagnostic squiggle is shown between v1.61 and 1.62+

[DetachHead]

VS Code version

1.62.1

Extension version

2021.11.1422169775

OS type

Windows

OS version

10

Python distribution

python.org

Python version

3.10

Language server

Pylance

Expected behaviour

Actual behaviour

Steps to reproduce

1. use this config in settings.json:

   {
       "python.linting.mypyArgs": [
           "--show-column-numbers"
       ],
       "python.linting.mypyEnabled": true,
       "python.analysis.typeCheckingMode": "off"
   }

2. cause a type error, eg:

   def foo(value: int) -> None:
           ...
   
   
   foo("asasdf")

Logs

> ~\proj\.venv\Scripts\python.exe -m mypy --show-column-numbers ~\proj\Sandpit\foo.py
cwd: ~\proj
> "~\proj\.venv\Scripts\activate.bat && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python c:\Users\user\.vscode\extensions\ms-python.python-2021.11.1422169775\pythonFiles\printEnvVariables.py"
##########Linting Output - flake8##########
##########Linting Output - mypy##########
Sandpit\foo.py:5:5: error: Argument 1 to "foo" has incompatible type "str"; expected "int"
Found 1 error in 1 file (checked 1 source file)
> ~\proj\.venv\Scripts\python.exe ~\.vscode\extensions\ms-python.python-2021.11.1422169775\pythonFiles\testing_tools\unittest_discovery.py . *.py
cwd: ~\proj

Code of Conduct

• I agree to follow this project's Code of Conduct

[vleandersson]

Same issue on MacOS but hovering the code does not show any tooltip at all.
Mypy 0.910
Python 3.8.7
VsCode 1.62.1

Downgrading VsCode to 1.61.2 (Universal) solves the issue

Working version
Version: 1.61.2 (Universal)
Commit: 6cba118
Date: 2021-10-19T15:49:28.381Z
Electron: 13.5.1
Chrome: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Darwin x64 20.6.0

[DetachHead]

hovering the code does not show any tooltip at all.

yeah there's like a 1 pixel hitbox for the tooltip to appear, it's driving me nuts

[saulshanabrook]

This happened to me as well, only started recently. If anyone has any tips on settings that can be changed to work around this, please post! I have to look at the PROBLEMS section of the bottom window to see the mypy errors now.

The linting output looks OK to me, in the OUTPUT section:

##########Linting Output - mypy##########
lineapy/ssa/__init__.py:70:1: error: Module "dis" has no attribute "_unpack_opargs"
lineapy/ssa/__init__.py:101:16: error: Missing positional arguments "kwonlyargcount", "nlocals", "stacksize", "flags", "names", "varnames", "name", "firstlineno", "lnotab" in call to "CodeType"
lineapy/ssa/__init__.py:106:22: error: "CodeData" has no attribute "d"
Found 3 errors in 1 file (checked 1 source file)

[diegovalenzuelaiturra]

Hi, here is a workaround:

By default, vscode sets the the following "python.linting.{pylint,flake8,mypy,pycodestyle}CategorySeverity" values in the defaultSettings.json

{
    // pylint
    "python.linting.pylintCategorySeverity.convention": "Information",
    "python.linting.pylintCategorySeverity.error": "Error",
    "python.linting.pylintCategorySeverity.fatal": "Error",
    "python.linting.pylintCategorySeverity.refactor": "Hint",
    "python.linting.pylintCategorySeverity.warning": "Warning",
    // flake8
    "python.linting.flake8CategorySeverity.E": "Error",
    "python.linting.flake8CategorySeverity.F": "Error",
    "python.linting.flake8CategorySeverity.W": "Warning",
    // mypy
    "python.linting.mypyCategorySeverity.error": "Error",
    "python.linting.mypyCategorySeverity.note": "Information",
    // pycodestyle
    "python.linting.pycodestyleCategorySeverity.E": "Error",
    "python.linting.pycodestyleCategorySeverity.W": "Warning",
}

I've noticed that only the pylint categories marked asrefactor (which happen to have a CategorySeverity set as "Hint") were working okay - So I've set them all to "Hint" and now it works ok (not as colorful as before, but still ok)

Also, if you work with flake8 and plugins that uses error codes which prefixes are different than E, F, W will not be affected and they will not be displayed - So, the second workaround is to include them all (all uppercase letters should be enough)

Since everything would be considered to be a "Hint" then they may not be listed as PROBLEMS.

{
    // pylint
    "python.linting.pylintCategorySeverity.convention": "Hint",
    "python.linting.pylintCategorySeverity.error": "Hint",
    "python.linting.pylintCategorySeverity.fatal": "Hint",
    "python.linting.pylintCategorySeverity.refactor": "Hint",
    "python.linting.pylintCategorySeverity.warning": "Hint",
    // flake8
    "python.linting.flake8CategorySeverity.A": "Hint",
    "python.linting.flake8CategorySeverity.B": "Hint",
    "python.linting.flake8CategorySeverity.C": "Hint",
    "python.linting.flake8CategorySeverity.D": "Hint",
    "python.linting.flake8CategorySeverity.E": "Hint",
    "python.linting.flake8CategorySeverity.F": "Hint",
    "python.linting.flake8CategorySeverity.G": "Hint",
    "python.linting.flake8CategorySeverity.H": "Hint",
    "python.linting.flake8CategorySeverity.I": "Hint",
    "python.linting.flake8CategorySeverity.J": "Hint",
    "python.linting.flake8CategorySeverity.K": "Hint",
    "python.linting.flake8CategorySeverity.L": "Hint",
    "python.linting.flake8CategorySeverity.M": "Hint",
    "python.linting.flake8CategorySeverity.N": "Hint",
    "python.linting.flake8CategorySeverity.O": "Hint",
    "python.linting.flake8CategorySeverity.P": "Hint",
    "python.linting.flake8CategorySeverity.Q": "Hint",
    "python.linting.flake8CategorySeverity.R": "Hint",
    "python.linting.flake8CategorySeverity.S": "Hint",
    "python.linting.flake8CategorySeverity.T": "Hint",
    "python.linting.flake8CategorySeverity.U": "Hint",
    "python.linting.flake8CategorySeverity.V": "Hint",
    "python.linting.flake8CategorySeverity.W": "Hint",
    "python.linting.flake8CategorySeverity.X": "Hint",
    "python.linting.flake8CategorySeverity.Y": "Hint",
    "python.linting.flake8CategorySeverity.Z": "Hint",
    // mypy
    "python.linting.mypyCategorySeverity.error": "Hint",
    "python.linting.mypyCategorySeverity.note": "Hint",
    // pycodestyle
    "python.linting.pycodestyleCategorySeverity.E": "Hint",
    "python.linting.pycodestyleCategorySeverity.W": "Hint",
}

• NOTE: I have only check it works for pylint, flake8, mypy, but I suspect it should also work for pycodestyle
• NOTE: vscode does not have the CategorySeverity configuration available for bandit, or others linters, so this workaround will not work for those 😢

[tlambert03]

thank you @diegovalenzuelaiturra ... very useful info! losing them all in PROBLEMS is rather a bummer, but it's good to know

[cdce8p]

hovering the code does not show any tooltip at all.

yeah there's like a 1 pixel hitbox for the tooltip to appear, it's driving me nuts

This has been addressed in vscode main. I guess it will be included in 1.63.0 #136711

[diegovalenzuelaiturra]

Still not fixed in 1.62.3

Version: 1.62.3 (Universal)
Commit: ccbaa2d27e38e5afa3e5c21c1c7bef4657064247
Date: 2021-11-17T07:59:13.865Z
Electron: 13.5.2
Chrome: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Darwin x64 20.6.0

[cdce8p]

Still not fixed in 1.62.3

As expected. The issue is targeted for November 2021, i.e. 1.63.0 -> see milestone. #136711

[DevilXD]

@cdce8p As per discussion on the issue you've linked, the only thing that was fixed is the mouse-over hitbox of the underlined character - it's not one pixel wide anymore, but rather actually covers the single character that's underlined.

Still, with the hitbox issue fixed, the issue where it underlines only one character (instead of the whole word like before) still persists.

[MalcolmMielle]

This bug is still present in the latest version

[vepain]

Version vscode: 1.64.0 (Ubuntu 20.04)

Date: 2022-02-03T04:23:11.224Z
Electron: 13.5.2
Chromium: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Linux x64 5.13.0-28-generic

• Flake8: bug still here
• Pylint: bug has changed
  • for functions, warnings on docstring (e.g. missing-return-doc & missing-param-doc) underline all the function (from def to end)

[moto-ctrl]

Is it going to be ever fixed? This is super annoying bug, I'm so surprised there's not that much noise around it from what I can see here. It's been going on for 3 months now! The python code with just one character underscored for errors coming from linters (like was shown on the screenshots on the original report) is very hard to work with! @alexdima any chance yourself or someone else from the team can look into it, please?

[karthiknadig]

Sorry for the delayed response on this. somehow this got missed.

I looks at the diagnostic object that we generate. we have not changed how we do this for over three years. The output from mypy looks like this:

##########Linting Output - mypy##########

testcode.py:4:5: error: Argument 1 to "foo" has incompatible type "str"; expected "int"
Found 1 error in 1 file (checked 1 source file)

The diagnostic object for this case looks like this:

{
    "severity":"Error","message":"Argument 1 to \"foo\" has incompatible type \"str\"; expected \"int\"",
    "range":{
        "start"{"line":3,"character":4}, 
        "end": {"line":3,"character":4}
    },
    "source":"mypy",
    "code":"error"
}

In the older version of VS Code this used to put the squiggle under the whole word. Transferring this to VS Code for further investigation.

[karthiknadig]

Repro steps:

1. Create testcode.py with following content:

def foo(value:int):
    pass

foo("abcde")

2. Add following to workspace setting:

    "python.linting.mypyEnabled": true,
    "python.linting.enabled": true

Ensure that you have python extension activated, for this example you will need to install mypy linter. If you have the python.linting.mypyEnabled set to true, you should get the prompt to install it.

Actual:

Expected:

[alexdima]

You're right, I forgot about this change. The editor used to artificially expand zero width diagnostics (empty ranges) on the word next to the diagnostic.

I changed this for #66482 where there were examples where expanding to the word next to the diagnostic is not desirable (e.g. missing semicolon at end of line).

[joyceerhl]

Still seems to repro for me in 1.65-insider:

[alexdima]

@joyceerhl I can reproduce what you are seeing in 1.65-insider. However, I can reproduce the same when using VS Code 1.60 or 1.61:

So @karthiknadig I believe that at least 2 distinct things have changed in the same time-frame:

1. the way vscode handles zero-width errors has changed, but has now been reverted
2. there must be at least one more change, maybe in the way the mypy linter reports errors?

Here are new verification steps that are independent of mypy:

• create a new folder in your $HOME/.vscode-insiders/extensions called e.g. 142810 and then create the following files:
• package.json:

{
    "name": "142810",
    "version": "0.0.1",
    "publisher": "alex",
    "engines": { "vscode": "^1.34.0" },
    "activationEvents": [ "*" ],
    "main": "main.js"
}

• main.js:

const vscode = require('vscode');

exports.activate = function() {
    const collection = vscode.languages.createDiagnosticCollection('142810');
    collection.set(vscode.window.activeTextEditor.document.uri, [
        new vscode.Diagnostic(new vscode.Range(0,5,0,5), 'some error 1'),
        new vscode.Diagnostic(new vscode.Range(1,5,1,5), 'some error 2'),
        new vscode.Diagnostic(new vscode.Range(2,5,2,5), 'some error 3'),
        new vscode.Diagnostic(new vscode.Range(3,5,3,5), 'some error 4'),
        new vscode.Diagnostic(new vscode.Range(4,5,4,5), 'some error 5'),
    ])
};

• file.txt:

something   - in the middle of a word
!@#$%^&     - in the middle of non-word chars
hello
hellos

Please reload insiders with the file.txt opened and verify the following:

• if the diagnostic falls within a word, it is expanded to the word (line1, line4)
• if the diagnostic falls on non-word characters, it is expanded to the width of 1ch (line2)
• if the diagnostic falls at the end of a line, even if it would touch a word, it is expanded to the width of 1ch (line3)
• if the diagnostic falls on an empty line, it is expanded to the width of 1ch (line 5)

[joyceerhl]

Verified based on new steps:

[karthiknadig]

@alexdima This is the diagnostic object we get back after parsing mypy output:

{
    "severity":"Error","message":"Argument 1 to \"foo\" has incompatible type \"str\"; expected \"int\"",
    "range": {
        "start"{"line":3,"character":4}, 
        "end": {"line":3,"character":4}
    },
    "source":"mypy",
    "code":"error"
}

For this code:

def foo(v:int):
    pass

foo("something")

I believe since the diagnostic is falling on " it is treating it as a non-word character, hence not squiggle-ing the entire word. I verified that mypy returns the same bit of information from previous versions.

[alexdima]

@karthiknadig Then this issue was always there, because the current behavior matches that of 1.61. If a zero-width error lands in a non-word character, it is rendered with the width of 1ch. You can also try 1.61 or older versions of VS Code using the download links at the top e.g. https://code.visualstudio.com/updates/v1_61

[DevilXD]

@karthiknadig At this point, VSCode has "fixed" the issue on this side, which ultimately ends up in a breaking change. Now it's time for the Python extension to alter how the MyPy and Flake8 output is parsed, so that the "parsed diagnostic object" you get will actually span the entire argument. Given your example code as test.py:

def foo(v:int):
    pass

foo("something")

..., this is the raw MyPy output (with --show-column-numbers) for it:

test.py:4:5: error: Argument 1 to "foo" has incompatible type "str"; expected "int"

..., which should result in a diagnostic object along the lines of:

{
    "severity":"Error","message":"Argument 1 to \"foo\" has incompatible type \"str\"; expected \"int\"",
    "range": {
        "start"{"line":4,"character":5}, 
        "end": {"line":4,"character":15}
    },
    "source":"mypy",
    "code":"error"
}

Note that I've included the exact range here. Alternatively, the existing solution with the 5th column could be moved 1 character to the right so that it lands on s, and then the resulting 0-width range would expand to either something or "something". Not sure which one of the two it'd do, so it'd need some testing.

Also, I've installed v1.61.2 and downgraded the Python extension solely to see what would happen, and this is the result:

And the same from v1.60.2:

I even tried v1.53.2 (January 2021, from a year ago), but it looks like you can't go too far back in time without everything collapsing (the Python extension didn't even start because Jupyther crashed), so I'll leave it at that.

[karthiknadig]

@DevilXD where is the end part of the range coming from? From the raw output that you have from mypy I see the start 4:5 but not the end range. Without the tool telling us the exact range, how do we know where to stop?

test.py:4:5: error: Argument 1 to "foo" has incompatible type "str"; expected "int"

I don't see how we can take this (without knowing what exact issue that the linter might be talking about) and translate it to range 4:5 and 4:15. What if the linter as complaining about the type of quote (literal character) that was used vs the entire string. If we squiggle the entire string it might be wrong. For the best experience here we need the range from the tool.

An interim solution could be:

• [Option 1]: Change the logic @alexdima quoted here Change in behavior in how diagnostic squiggle is shown between v1.61 and 1.62+ #142810 (comment) to special case ". Include the full range of the quoted string.
• [Option 2]: From the extension, parse the python content and figure out which token marker that the diagnostic lands on. Doing this from the python extension right now is going to be very expensive. We will have to run two separate scripts to get this answer. First to get the raw linting, and again to parse python and get full ranges. We can do this, but this is a massive change and probably not the way we want to go.
• [Option 3]: We do have a prototype extension for that wraps linters into their own extension and does this entirely over LSP. Mypy over Language Server Protocol and VS Code extension python/mypy#12168 . We have lot more freedom there to parse python and send back better ranges. This would be the best interim solution (until tools provide full ranges), but again might take some time to develop. Also, depends on where the ownership of that lands.
• [Option 4]: A way to do this without domain specific info on the range is to get this from the language server. This means that it would be a feature to add on to the LSP.

@alexdima I feel Option 1 might be something we could do now. If that is not the route we want to take, then the next best option is [option 3] for the individual linting extensions to be ready. If you have another recommendation, let me know.

Neiter of the above options provide the best experience, ideally this range should come from the tools themselves. pylint recently added start and end ranges to their lint output. Hopefully, other tools also adopt this.

/cc @luabud @brettcannon

[DevilXD]

where is the end part of the range coming from?

Yes, this is the part where I'm not familiar with how the internals of the extension work, so it'd be "up to figure out". I can see that you've already listed several options, neither of which are trivial. What about the shifting approach then? Still using the 0-width range, but placing it 1 character further to the right, so that it lands on the initial s in something and expands to the whole word automatically? Feels a little bit hacky, but should generally work for most cases I'd think. Only case I can think of where it could not work properly, would probably be an empty string, where it'd underline the second quote only. Still, I think it'd be simple and good enough for now.

[alexdima]

@karthiknadig Adding a special case for " is not an option.

• The logic of expanding a zero-width diagnostic to the word it touches is driven by the word definition for the current language, which is defined via language configuration. Going outside of the configured word definition and giving special meaning to " as the beginning of a string is not something we can do in the core of the editor. There are languages we must support where " is not the beginning of a string, e.g. " can be the beginning of a line comment, etc.
• Even adding something special for " does not solve the underlying problem. I think the first argument could be an expression which results in a string, so the " heuristic would not fly for foo(("a"+"b")).

I suggest that either the extension which can understand the underlying python syntax tries to massage the error or otherwise I suggest that you reach out to the maintainers of mypy and ask them for a special CLI flag or something that would output the errors in a format which contains a range, not just a position.

[moto-ctrl]

I just had a go at 1.65.0-insiders to see what's changed.
I should stress that the issue with underscoring the right element I could see in all linters that I use simultaneously, that is: pylint, flake8 and mypy which is quite apparent when I browse larger codebase.
For our test case I compared results of 1.65 to 1.64.2 and 1.60.2 stable and my observations are as follow:

• mypy seems to be indeed just one char underscore for this error
• flake8 reporting is now fixed - I can see whole words rather than characters underscored (red squiggles on value: , foo call or pass indentation)
• there's still something off comparing to 1.60.2 that seems to be related to pylint
  • the newer vscode versions underscore whole function definition with blue squiggle rather than just def word like before - in this case these are "disallowed foo name" and "missing docstring". It was much easier to read before
  • "unused argument 'value'" yellow squiggle got expanded to value:int rather than just value (again I considered that cleaner in the earlier version - though not something I would fight super hard for)

update: red squiggle from mypy on "abcde" used to point at the first character of the string. now it's somewhere between "(" and "a" characters. just a cosmetic thing but I think it was slightly more readable in the old version

1.65.0-insider:

1.64.2

1.60.2

TL;dr I favour most how the original version 1.60.2 reported issues from linters

[cdce8p]

@moto-ctrl Regarding pylint. We recently added support for error ranges. Those aren't perfect yet, but we (the pylint team) are working on improving them for the next release. Please take a look at pylint-dev/pylint#5466

Like others have mentioned already, this is different to mypy and flake which both only report a position, not a range, which VS Code then has to interpret. What is being discussed here is how that behavior has changed / should be changed going forward.

[moto-ctrl]

ah I see, so the old version of vscode didn't understand ranges therefore even with latest pylint it still just highlights only the first word (def in this case) whereas new vscode parses ranges and the highlight of the whole function comes from the pylint itself - which is being discussed in the issue you linked. Thanks for clarifying that.
I'm going to stand by my earlier statement - old style looks the best :-)

[cdce8p]

ah I see, so the old version of vscode didn't understand ranges therefore even with latest pylint

Correct, we had to add range support to pylint first, after which I contributed a patch to the VSC Python to actually use them.

I'm going to stand by my earlier statement - old style looks the best :-)

We, the pylint team, would appreciate feedback about specific issues with ranges (in the pylint repo). Probably best after the next update though since that will already address a lot of issues.

Another point, I've seen discussions about custom language server extensions for Python linters the VSC Python team is working on. A switch to select positions or ranges for highlighting might be a good suggestion.

[moto-ctrl]

And something else occurred to me - sorry this was perhaps obvious but I'll write it down for benefit of everyone

• blue squiggle that covers all function - comes from pylint and how it reports error ranges. Will be/needs to be fixed in pylint.
• red squiggles from mypy and flake8 - don't report ranges yet, vscode used to handle it by underscoring whole words and that feature has been brought back and is fixed in this ticket
• however in some edge situations - like error pointing special character as in mypy example - this is not currently expanded due to more complex algorithm required for detection
  edit: I shouldn't really call it 'edge case' as it would be everywhere, where string type doesn't match input type - or vice versa. As discussed earlier this is up to vscode and linter maintainers to decide on what to do with it

Here's my example with mypy and flake8 (switched off pylint to get better picture)
I changed foo definition to take string rather than int and can confirm the fix is working pretty much as expected
(value, pass, foo errors come from flake8, 12345 comes from mypy)

1.65-insiders

1.64.2

[DetachHead]

so as this is issue is still not resolved in 1.65, can it be re-opened and transferred back to the vscode-python repo? or should i re-raise it there?

[karthiknadig]

@DetachHead I think this issue is addressed from VS Code.

There are now two issues:

1. On mypy, to provide ranges on the reported items. This will resolve the problem in all scenarios.
2. On vscode-python, to detect and expand the squiggle, for the " case, temporarily only for mypy, until mypy provides ranges.