Extension is watching files outside of the workspace

[MetRonnie]

Issue Type: Bug

Behaviour

Expected vs. Actual

Expected: Python extension does not watch files outside of the workspace

Actual: Log (Window):

[<timestamp>] [renderer15] [error] [File Watcher (parcel)] Inotify limit reached (ENOSPC) (path: <some/path/that/is/not/the/workspace>)

where the path is sometimes a neighbouring python project workspace or ~/.conda/envs/<env_name>/lib/python3.9. The neighbouring projects in question are not imported by any of the python files in this workspace (but they are the locations of editable pip installs in the conda environment).

Steps to reproduce:

1. Open a python project workspace

Additional context

This only started happening recently.

If I disable the Python extension for the workspace and close + reopen, I don't get the ENOSPC error until I re-enable the extension.

$ cat /proc/sys/fs/inotify/max_user_watches
32768

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.9.10
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
• Value of the python.languageServer setting: Pylance

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

N/A

User Settings

condaPath: "<placeholder>"

languageServer: "Pylance"

linting
• flake8Enabled: true
• mypyArgs: "<placeholder>"
• mypyEnabled: true
• pycodestylePath: "<placeholder>"

formatting
• autopep8Args: "<placeholder>"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true

Extension version: 2022.8.0
VS Code version: Code 1.67.2 (c3511e6c69bb39013c4a4b7b9566ec1ca73fc4d5, 2022-05-17T18:23:40.286Z)
OS version: Linux x64 3.10.0-1160.66.1.el7.x86_64
Restricted Mode: No

System Info

Item	Value
CPUs	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz (4 x 2599)
GPU Status	2d_canvas: unavailable_software canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: disabled_software multiple_raster_threads: enabled_on oop_rasterization: disabled_off opengl: disabled_off rasterization: disabled_software raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: disabled_software video_encode: disabled_software vulkan: disabled_off webgl: unavailable_software webgl2: unavailable_software
Load (avg)	2, 2, 2
Memory (System)	7.62GB (0.59GB free)
Process Argv	--unity-launch --crash-reporter-id 4cb3165e-6680-471f-992f-0c1e623756f7
Screen Reader	no
VM	100%
DESKTOP_SESSION	1-gnome
XDG_CURRENT_DESKTOP	GNOME
XDG_SESSION_DESKTOP	1-gnome
XDG_SESSION_TYPE	x11

[MetRonnie]

I've now identified an even worse culprit: microsoft/TypeScript#49474

Still, I wouldn't expect the python extension to watch neighbouring project folders

[sugatoray]

@MetRonnie Perhaps use this to avoid the TypeScript resource draining. I saw this happening around 10 months ago (Oct-2021) to my laptop and increasing the fan speed that created some unreasonably loud fan-noise. The following got it resolved.

{
    // This increases VS Code speed
    /*
       - source: https://dev.to/rmorabia/how-to-make-visual-studio-code-150-faster-in-large-projects-3eag
       - Why? Dangerously loud laptop fan
    */
    "typescript.disableAutomaticTypeAcquisition": true
}

[karrtikr]

The watcher set in the original bug report is caused by Pylance, transferring this.

[heejaechang]

this is by design. we watch all paths reported by python sys.path if we can. but I think we can improve it by removing any path part of multi workspace root since they should be already watched by other workspace.

[heejaechang]

tagging this issue as one of improvement for multi workspace - #2601

it is actually moved to discussion - #2614

[karrtikr]

I understand if it's by design, but if it's exceeding the watcher limits we should probably be doing something about it. Do we have to watch for all paths in sys.path, can we watch the root directory containing all environments instead?

[heejaechang]

@karrtikr it is already root folder of each importing roots (such as site-packages or python/lib folder). but we are not actual one who watch, we just ask vscode to watch those folders. and actual implementation is done by vscode.

so for file watch max limitation, actual implementation (vscode) should take care, I am not sure there is much we can do since actual implementation is owned by host (vscode or vs)

[karrtikr]

I understand we're not the actual ones who implement the watcher, but it looks like its not practical to ask to watch so many directories (I'm not sure if it's a bug). Also for me personally sys.path looks like:

['~\\OneDrive - Microsoft\\Desktop\\folders\\donj2', '~\\AppData\\Local\\Programs\\Python\\Python39\\python39.zip', '~\\AppData\\Local\\Programs\\Python\\Python39\\DLLs', '~\\AppData\\Local\\Programs\\Python\\Python39\\lib', '~\\AppData\\Local\\Programs\\Python\\Python39', '~\\OneDrive - Microsoft\\Desktop\\folders\\donj2\\.venv', '~\\OneDrive - Microsoft\\Desktop\\folders\\donj2\\.venv\\lib\\site-packages']

If we're only watching the root directories: ~\\AppData\\Local\\Programs\\Python\\Python39 and ~\\OneDrive - Microsoft\\Desktop\\folders\\donj2\\.venv, the count of the watcher is 2 instead of 7, which is something that could help.

[heejaechang]

@karrtikr if OS throws because of 7 directory watched, I think OS has an issue rather than us. also, those kind of dedup I think should be done by vscode, since we can only do over what we watch, if other extension watches same folders, we can't dedup. those kind of dedup better to be done by host (vscode) in my opinion.

that being said, there are some we can do to support multi workspace better. and I will work on those soon.

[karrtikr]

Right, I understand. We should make sure this has been reported on VSCode though.

[heejaechang]

@karrtikr I just looked at https://github.com/microsoft/vscode/wiki/File-Watcher-Issues

it looks like it is a limitation of Linux file watcher implementation. user might need to turn off file watching certain folders using vscode's files.watcherExclude options. or follow instructions shown with the warning.

[heejaechang]

@karrtikr I talked with vscode file watcher owner. he said, deduping is not there yet but they have work item to implement it, and asked us to dedup for now. I have WIP PR, will check in soon.

[karrtikr]

Great, thanks for the update.

[muncok]

I talked with vscode file watcher owner. he said, deduping is not there yet but they have work item to implement it, and asked us to dedup for now. I have WIP PR, will check in soon.

I have tried to usefiles.watcherExclude to exclude conda's env directory, but it did not work.

[bschnurr]

This issue has been fixed in version 2022.6.30, which we've just released. You can find the changelog here: CHANGELOG.md

[phord]

@bschnurr Is the fix some kind of dedup like @heejaechang mentioned, or was a different bug identified?

It seems unreasonable to me that you have to watch the hierarchy of $PYTHONPATH for this extension. Mine has only one directory in it, but that directory has 1.2 million files beneath it. Is there a real fix for this, or do I need to disable/move my python path?

[phord]

By the way, I noticed if I disable the pylance extenstion and then reload the workspace, the issue goes away. If I then re-enable pylance, the problem does not recur. So it seems this watch $PYTHONPATH is something only done during startup. Is that correct?

[MetRonnie]

@phord I think the full features of the extension don't load until you have a python file as an open tab, could that be why?

[mic-e]

I'm running Pylance 2022.6.30 using an ssh remote host.
Is it possible to blacklist Pylance from watching the standard library files?
On the remote host I'm not allowed to increase the inotify watchers limit, and Pylance uses up all resources while attempting to watch the standard library.
Pylance seems to ignore the "Watcher Exclude" setting.

[debonte]

@phord, @mic-e, the Pylance team does not typically monitor closed issues. Please open new issues if you feel there are more problems related to file watching.