Support LSP for TypeInferenceProvider

[rowillia]

Having access to type information of a node would be hugely beneficial to refactoring. LibCST offers this to Pyre users via the Pyre Query API, but not to projects using other type checkers and there's unfortunately enough variance where this matters.

The other major type checkers (mypy, pyright) all support LSP, as does Pyre.

[jakkdl]

This seems quite doable (and I'm up for writing PRs to do so), the Pyre Query API is considered legacy code - so replacing that with interfacing with pyre persistent through LSP seems good, and can then extend that support to the other type checkers.

https://github.com/python-lsp/python-lsp-jsonrpc looks like a solid library for interfacing through LSP as used in https://github.com/astral-sh/ruff-lsp/blob/main/tests/client/session.py

[jakkdl]

Okay, after looking into this quite a bit ... it unfortunately seems like it's not really possible with the given LSP features supported by mypy & pyre

1. https://github.com/python-lsp/pylsp-mypy seems to only support triggering a type-error-check-trigger through LSP, and not querying symbols for their type definition.
2. pyre persistent only returns variables when querying for documentSymbols - and I don't see any support for querying for the type of any of them.
3. pyright... might work? But I fail to get any response to my requests (using code that works perfectly with the two above). I'm probably doing something wrong somewhere though

for pyre it seems reasonable to open an issue and ask for improved LSP support.

There are other avenues than LSP one could go for though, a quick look at the json files in .mypy_cache reveals that it might be quite doable to just ... manually parse them - the symbols and types are stored in plaintext.
pyright does not have an on-disk cache though.

[zsol]

AFAIK our best bet is to use a hover lsp request at least for pyre, have you tried that?

@grievejia we've talked about something like this for a while but never really arrived at a nice interface between pyre and LibCST/Fixit

[grievejia]

Yeah I don't think LSP is a good protocol for exposing typing information. As Zsolt mentioned, the best one could do is to ask for hover results, but hover texts are usually tuned towards human consumption (i.e. potentially including a lot of pretty format artifacts and free-formed texts) rather than machine consumptions.

Moreover, what exactly is a good structural format of typing info that gets exposed from type checkers is also a surprisingly non-trivial problem, especially when one takes into account language features like generic classes, overloads, protocols, etc. We do have some long-term plan to tackle that problem from Pyre directly and experiment with various ideas, but at the moment it's unclear what that kind of interface would look like.

[jakkdl]

AFAIK our best bet is to use a hover lsp request at least for pyre, have you tried that?

Looks like this gives some type info for some symbols, but I'm getting empty info for most symbols and as grievejia says it's formatted with human consumption in mind and only gives minimal info. e.g. {'contents': '```\nItemCollector\n```'} where pyre query gives fully qualified type.

Yeah I don't think LSP is a good protocol for exposing typing information. As Zsolt mentioned, the best one could do is to ask for hover results, but hover texts are usually tuned towards human consumption (i.e. potentially including a lot of pretty format artifacts and free-formed texts) rather than machine consumptions.

Hover doesn't look like the way to go, and reading more closely through the spec I'm growing uncertain if any of the other requests are fitting for getting the type info. E.g. textDocument/documentSymbol does include detail and kind - but I'm not sure if those are suitable for stuffing in [machine-readable] type hints.
textDocument/typeDefinition initially sounded like exactly what we want, but it's all about giving the location of a type definition ... which is kinda weird in a python typing context, and while one can give locations of symbols in the standard library I don't see how to resolve more complex types with a list of locations.

@erictraut does pyright-langserver expose type hints for symbols through LSP in any way? I tried figuring it out from both testing and reading documentation/source/tests but didn't get anywhere.

[jakkdl]

Randomly found https://github.com/python/mypy/blob/cfec71798175fcbf030d6d114750d6fac454b3c9/misc/find_type.py which works by copying the current file to a temp file, shoving in a line with a reveal_type, and running mypy on it. Very elegant 😅
Somewhat doable to do the same, except shoving in reveal_types for everything, but idk if that's a path worth going down. Though would be supported by all type checkers and not rely on any APIs or internal functionality that could get changed.

[erictraut]

Pyright's LSP supports hover text. As was mentioned above, hover text output is meant to be human-readable, so it's not a good format for tooling. It sounds like you're attempting to use the language server interface for a purpose it was not designed, so you will likely be disappointed going down that path. If you want access to the underlying type information produced by a static type analyzer, I think you'd be better off calling the analyzer directly and consuming its output in "native" data structures rather than the text representations used in the LSP.

[jakkdl]

Okay, LSP is out of the question then.

So if we still want the functionality, the alternatives as far as I can tell are:

mypy

1. Write a visitor that adds a ton of reveal_types in (a copy of) the file, which we then pass into the type checker and parse the command line output. Messy, probably quite slow, might not be possible to get type data on sub-expressions nested inside other expressions (hmm, maybe with liberal use of the walrus operator...), but does not rely on any internal APIs that can change.
2. manually parse json files in .mypy_cache. Fast, but requires some code to properly parse the stored type. Requires ongoing maintenance if format of the cache changes, though that should be minimal. Not entirely sure if/how it handles sub-expressions.
3. Call internal mypy function build.build and parse the BuildResult. Should be relatively straightforward, but will ofc break if mypy internals get changed. Pretty much same as above otherwise.

pyright

1. Same as 1. above, which gives more or less universal support for all type checkers.
2. interface with the typescript internals ... somehow. After a brief look into the code I'm not seeing any entry points that expose all the data.