Progress Broken in Nvim

[ibrokemypie]

I have pylsp set up in neovim through lspconfig, as well as the plugin fidget.nvim to show me lsp progress. This was working fine previously, but has since stopped working, filling my lsp log with messages such as the below. This is a pain because I want to know the progress of the autoimport cache as that feature doesn't seem to be working for me right now.

python-lsp-server version 1.7.2-1
neovim version 0.9.0
nvim-lspconfig commit 427378a

[ERROR][2023-04-27 16:12:22] .../vim/lsp/rpc.lua:734	"rpc"	"pylsp"	"stderr"	"2023-04-27 16:12:22,272 AEST - WARNING - pylsp.workspace - There was an error while trying to initialize progress reporting.Likely progress reporting was used in a synchronous LSP handler, which is not supported by progress reporting yet.\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/site-packages/pylsp/workspace.py\", line 148, in report_progress\n    token = self._progress_begin(title, message, percentage)\n  File \"/usr/lib/python3.10/site-packages/pylsp/workspace.py\", line 185, in _progress_begin\n    self._endpoint.request(self.M_INITIALIZE_PROGRESS, {'token': token}).result(timeout=1.0)\n  File \"/usr/lib/python3.10/concurrent/futures/_base.py\", line 460, in result\n    raise TimeoutError()\nconcurrent.futures._base.TimeoutError\n"
[ERROR][2023-04-27 16:12:31] .../vim/lsp/rpc.lua:734	"rpc"	"pylsp"	"stderr"	"2023-04-27 16:12:31,401 AEST - WARNING - pylsp.workspace - There was an error while trying to initialize progress reporting.Likely progress reporting was used in a synchronous LSP handler, which is not supported by progress reporting yet.\nTraceback (most recent call last):\n  File \"/usr/lib/python3.10/site-packages/pylsp/workspace.py\", line 148, in report_progress\n    token = self._progress_begin(title, message, percentage)\n  File \"/usr/lib/python3.10/site-packages/pylsp/workspace.py\", line 185, in _progress_begin\n    self._endpoint.request(self.M_INITIALIZE_PROGRESS, {'token': token}).result(timeout=1.0)\n  File \"/usr/lib/python3.10/concurrent/futures/_base.py\", line 460, in result\n    raise TimeoutError()\nconcurrent.futures._base.TimeoutError\n"

The config for python lsp is as below:

			pylsp = {
				root_dir = require("utils").get_python_root,
				settings = {
					pylsp = {
						plugins = {
							rope_autoimport = {
								enabled = true,
							},
							rope = {
								enabled = true,
							},
							black = {
								enabled = false,
							},
						},
					},
				},
			},

[ibrokemypie]

I believe the culprit is this PR #328

Reverting to 1.7.1 brings back working progress indicators

[ccordoba12]

Hey @ibrokemypie, thanks for reporting.

@syphar, could you take a look at this one?

[syphar]

@ccordoba12 yep, currently on a train / on the move but I can check it out the next days.

@ibrokemypie are all progress messages missing? diagnostics should be async, if I recall correctly.

[ibrokemypie]

@ccordoba12 yep, currently on a train / on the move but I can check it out the next days.

@ibrokemypie are all progress messages missing? diagnostics should be async, if I recall correctly.

Not sure, the autoimport caching is the only progress message I have noticed with pylsp, though that could just be because other progress messages are completed too fast to see.

[syphar]

I did some first digging.

This boils down to the past conflict between synchronous handlers, and trying to initialize the progress token, cc @ccordoba12 .

The methods used for the initialization are m_initialize, m_text_document__did_open and m_text_document__did_change, and theoretically these would have to be made async for the token initialization to work.

A first test of this made the progress reappear (but completion broke, need to check this). Of course I don't know (yet) if there are any side-effects.

[syphar]

also after digging, progress for diagnostics still works

[syphar]

Since I don't know much about lsp-jsonrpc:
@ccordoba12 what way would you propose?

• try to make the methods above async (m_initialize, m_text_document__did_open and m_text_document__did_change). I'm not sure about eventual side-effects
• switch progress-token-initialization to optional, only do it when it doesn't error? So still report progress? (goes against LSP standard)
• ?

[ccordoba12]

try to make the methods above async

I don't have time to take a look at this right now, but I don't think it'd be an easy task to make those methods async.

switch progress-token-initialization to optional, only do it when it doesn't error?

Yep, I think that's the best we can do for now, even if it doesn't follow the protocol.

[lcheylus]

Same issue with pylsp v1.8.2 on Debian Linux amd64 and Python 3.11.5.

I'm using pylsp as a LSP server for my Python code with Neovim 0.9.2 and lspconfig plugin. In my settings, I'm using Rope for code completion:

settings = {
    pylsp = {
      -- configurationSources = { 'pycodestyle' },
      plugins = {
        flake8 = { enabled = true },
        yapf = { enabled = true },
        pydocstyle = { enabled = true },
        pyflakes = { enabled = false },
        pycodestyle = { enabled = false },
        mccabe = { enabled = false },
        jedi_completion = { enabled = false },
        rope_completion = { enabled = true },
      },
    },

When I try to complete code, I have identical logs with errors :

2023-10-12 19:09:55,183 CEST - WARNING - pylsp.workspace - There was an error while trying to initialize progress reporting.Likely progress reporting was used in a synchronous LSP handler, which is not supported by progress reporting yet. To prevent waiting for the timeoutyou can set `skip_token_initialization=True`. Not every editor will show progress then, but many will.
Traceback (most recent call last):
  File "/home/fox/dev/Perso/python-lsp-server.git/venv/lib/python3.11/site-packages/pylsp/workspace.py", line 249, in _progress_begin
    ).result(timeout=1.0)
      ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 458, in result
    raise TimeoutError()
TimeoutError

In this case, LSP code completion doesn't work using Rope. If I modify my settings to use Jedi instead of Rope, this issue is not present and code completion works as expected.

[syphar]

now that's an interesting error, thank you for the reproduction steps!
I'm not sure when I can look into it, I'll try the next days. I also would be happy to review & test any PR fixing this :)

[last-partizan]

After doing git bisect (if i did everything right), suspected commit is this:

de87a80 is the first bad commit

commit de87a800b198f97517032155c077e596b7d8b608
Author: Tobias Krabel <tobiaskrabel@gmx.de>
Date:   Fri Dec 22 23:31:26 2023 +0100

    Make autoimport cache generation non-blocking (#499)

I'll try to figure out how to fix it.

[last-partizan]

#530

This is best i can do now, it does not fixes underlying problem, but reverts to using single thread by default.

This will also likely fix #529 (because database is locked by this hanging thread)