Pylance causes heavy CPU usage by setting file watchers on entire PYTHONPATH

[justin39]

Context

This relates to microsoft/vscode#223790.

Environment data

VS Code version: Code 1.91.1 (microsoft/vscode@f1e16e1, 2024-07-09T22:06:49.809Z)
OS version: Windows_NT x64 10.0.19045
Modes:
Remote OS version: Linux x64 6.1.85-ts1-amd64

Repro Steps

Set the Python interpreter to an interpreter with a large number of PYTHONPATH entries. Pylance will create recursive file watchers for every entry in the PYTHONPATH, as noted in microsoft/vscode#223790, regardless of the values of files.watcherExclude and python.analysis.exclude. With a sufficiently large PYTHONPATH pointing at enough entries, this creates significant CPU load.

Expected behavior

I would expect entries in files.watcherExclude to prevent this, but as noted in this comment in the related thread, extensions are able to bypass exclusions using the API.

Actual behavior

Pylance creates a large number of recursive file watches that peg the CPU

Logs

See microsoft/vscode#223790

Additional details

This is described in the related thread as well, but our interpreter setup employs a long PYTHONPATH for dependency resolution with nearly every entry pointing at paths on a read-only filesystem. Since these paths are read-only, we don't believe there's any value in monitoring all of these paths for file events anyways and would like the ability to exclude them.

It would be most ideal to prevent excessive file watches from hosing the CPU, but this seems like an easier task to tackle and would provide a much-needed workaround for us!

[heejaechang]

since we just use lsp libraries api (vscode api indirectly), this seems something they should take care, rather than us. otherwise, every extensions that use the api would need to implement their own mechanism to filter out.

did you open an issue to vscode about this?

...

that said, we have a hidden setting called "python.analysis.watchForLibraryChanges", if you set this to false, pylance shouldn't watch any third party libraries. you can use this hidden setting until vscode support watcherExclude for files outside of workspace root

[justin39]

@heejaechang Yes, I also felt this would belong better under the VS Code repo, but was told this was as intended in microsoft/vscode#223790 and figured I should check here.

python.analysis.watchForLibraryChanges looks like exactly what we're looking for, brilliant! I'd like to check with some other colleagues, but at least on my instance this seems to work exactly as intended. Any chance this'll become a non-hidden setting in the near future?

[heejaechang]

currently no plan, but if there are enough upvote then we will certainly consider.