"Unable to watch for file changes in this large workspace folder"

[augensalat]

Issue Type: Bug

When opening a typescript a(n empty) folder I get the error message:

Unable to watch for file changes in this large workspace folder. Please follow the instructions link to resolve this issue.

But it is not large at all. In fact it is enough to have only one not a single file in the "project" folder!

mkdir empty

Now opening this empty folder in VSCode leeds to the mentioned error message.

Before you ask:

$ cat /proc/sys/fs/inotify/max_user_watches
65536

and it does not help if set to the 512K maximum.

Findings:

• The FS watcher seems to look for a .git folder - that does not exist in the empty "project".
• The watcher obviously climbs up the FS hierarchy towards the root folder until it finds a .git folder.
• My $HOME is managed by git (for dotfiles mainly), so I have a .git directory in it.
• Workaround 1: When I temporarily remove ~/.git the error does not appear when opening the empty folder w/ VSCode.
• Workaround 2: git init in the empty folder.

Looks like the watcher is climbing up the folder structure until it finds a .git folder - in my case up to the $HOME directory that naturally contains a lot of files. This does not seem to make much sense as the .git folder does not necessarily belongs to the actual project.

I did not have the issue when I installed typescript (4.7.3) in the empty folder, created a proper tsconfig.json and ran npx tsc -w, so the error does not seem to come from the original ts watcher.

I am not sure if this is really a VSCode issue or comes from some component used by VSC.

VS Code version: Code 1.68.0 (4af164e, 2022-06-08T11:49:57.055Z)
OS version: Linux x64 5.3.18-150300.59.68-default
Restricted Mode: No

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz (8 x 4087)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: disabled_off opengl: enabled_on rasterization: disabled_software raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: disabled_software video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	1, 1, 1
Memory (System)	31.24GB (15.82GB free)
Process Argv	--unity-launch --crash-reporter-id b4d4e202-8269-4b94-ad97-a926191ab2fe
Screen Reader	no
VM	0%
DESKTOP_SESSION	/usr/share/xsessions/plasma5
XDG_CURRENT_DESKTOP	KDE
XDG_SESSION_DESKTOP	KDE
XDG_SESSION_TYPE	x11

Extensions (23)

Extension	Author (truncated)	Version
pug	ama	1.0.1
vscode-django	bat	1.10.0
path-intellisense	chr	2.8.1
vscode-eslint	dba	2.2.2
vscode-babel-coloring	dza	0.0.4
EditorConfig	Edi	0.16.4
prettier-vscode	esb	9.5.0
VS-code-vagrantfile	mar	0.0.7
vscode-docker	ms-	1.22.0
python	ms-	2022.8.0
vscode-pylance	ms-	2022.6.10
jupyter	ms-	2022.5.1001601848
jupyter-keymap	ms-	1.0.0
jupyter-renderers	ms-	1.0.8
remote-containers	ms-	0.238.2
angular2-inline	nat	0.0.17
color-highlight	nau	2.5.0
vetur	oct	0.35.0
material-icon-theme	PKi	4.18.1
LiveServer	rit	5.7.5
jinjahtml	sam	0.17.0
autoimport	ste	1.5.4
vscode-icons	vsc	11.12.0

(2 theme extensions excluded)

[watsonb]

Version: 1.68.1
Commit: 30d9c6c
Date: 2022-06-15T02:58:26.441Z
Electron: 17.4.7
Chromium: 98.0.4758.141
Node.js: 16.13.0
V8: 9.8.177.13-electron.0
OS: Linux x64 5.4.0-121-generic

VSCode file watching on Linux has become "hot garbage" in the last couple/few months. I hate to pile on here, but it is becoming unusable and cranking up the inotify max_user_watches beyond 65K is NOT a solution/workaround.

I'm a Python/Ansible user and maintain a couple hundred Ansible playbook, inventory, and role projects (each their own Git repo/project). At one time, I used to be able to "open folder" and open my "roles" directory (133 sub-folders, each a Git project) without hitting this issue. Now, I "open folder" on a single role and my watches count will easily hit 22K for an individual instance of VSCode running, no matter how many files are contained in the folder.

I've tried "open workspace" instead of "open folder", as I've read that can help, but the problem persists. As I author this comment, I currently have a single VSCode window opened via the "open workspace" method.

Using this script:

!/bin/bash

echo "number of files in cwd:"
find . -type f | wc -l

echo "directories with most files:"
find . -xdev -type d -print0 |
while IFS= read -d '' dir; do
  echo "$(find "$dir" -maxdepth 1 -print0 | grep -zc .) $dir"
done |
sort -rn |
head -50

I see that there are:

number of files in cwd:
3785
directories with most files:
259 ./.git/objects
26 ./.git/objects/91
25 ./.git/objects/73
25 ./.git/objects/3b
24 ./.git/objects/da
24 ./.git/objects/a7
# snipped for brevity
19 ./.git/objects/6f
19 ./.git/objects/5b

So nearly 3800 files, the majority being Git objects (which should be excluded by default, right?). However using this script to show current inotify consumers (https://github.com/fatso83/dotfiles/blob/master/utils/scripts/inotify-consumers):

(venv3_ansible-4.2.0) ben@knedmhils042:~/workspace/kiewit/ansible/inventories$ ~/bin/inotify-consumers.sh 

   INOTIFY   INSTANCES
   WATCHES      PER   
    COUNT     PROCESS   PID USER         COMMAND
------------------------------------------------------------
   15854         2      422034 ben         /usr/share/code/code --ms-enable-electron-run-as-node /usr/share/code/resources/app/out/bootstrap-fork --type=fileWatcher
    5976         2      422020 ben         /usr/share/code/code --ms-enable-electron-run-as-node --inspect-port=0 /usr/share/code/resources/app/out/bootstrap-fork --type=extensionHost --skipWorkspaceStorageLock
# snipped for brevity

There are a few other VSCode related entries in the output in the single digits, but the top two are the main offenders. According to this (https://github.com/microsoft/vscode/wiki/File-Watcher-Issues), it seems that these two offenders correspond to nodejs fs.watch and parcel watching files.

My user-level (and at times duplicated for the workspace itself) VSCode settings is configured as follows to try to get a handle on this:

    "files.watcherExclude": {
        "**/collections/**": true,
        "**/collections/*/**": true,
        "**/molecule/reports/**": true,
        "**/molecule/reports/*/**": true,
        "**/molecule/resources/**": true,
        "**/molecule/resources/*/**": true,
        "**/roles/**": true,
        "**/roles/*/**": true,
        "collections/**": true,
        "molecule/reports/**": true,
        "molecule/resources/**": true,
        "roles/**": true
    },

As you can see, I've tried every glob pattern I can think of with no relief.

Immediately after opening VSCode, I set logs to trace, toggled dev tools and filtered for file watcher. I see this:

TRACE [File Watcher (node.js)] Request to start watching: /home/ben/workspace/kiewit/ansible/collections/ac_kiewit_content (excludes: <none>, includes: **/.git/objects/**,**/.git/subtree-cache/**,**/node_modules/*/**,**/.hg/store/**,**/collections/**,**/collections/*/**,**/molecule/reports/**,**/molecule/reports/*/**,**/molecule/resources/**,**/molecule/resources/*/**,**/roles/**,**/roles/*/**,collections/**,molecule/reports/**,molecule/resources/**,roles/**)
log.ts:289 TRACE [File Watcher (node.js)] Started watching: '/home/ben/workspace/kiewit/ansible/collections/ac_kiewit_content'

Followed by this:

Extension Host
log.ts:289 TRACE [File Watcher (parcel)] Request to start watching: /home/ben/venv3_ansible-4.2.0/lib/python3.8/site-packages (excludes: **/.git/objects/**,**/.git/subtree-cache/**,**/node_modules/*/**,**/.hg/store/**,**/collections/**,**/collections/*/**,**/molecule/reports/**,**/molecule/reports/*/**,**/molecule/resources/**,**/molecule/resources/*/**,**/roles/**,**/roles/*/**,collections/**,molecule/reports/**,molecule/resources/**,roles/**, includes: <all>),/usr/lib/python3.8 (excludes: **/.git/objects/**,**/.git/subtree-cache/**,**/node_modules/*/**,**/.hg/store/**,**/collections/**,**/collections/*/**,**/molecule/reports/**,**/molecule/reports/*/**,**/molecule/resources/**,**/molecule/resources/*/**,**/roles/**,**/roles/*/**,collections/**,molecule/reports/**,molecule/resources/**,roles/**, includes: <all>)
log.ts:289 TRACE [File Watcher (parcel)] Started watching: '/home/ben/venv3_ansible-4.2.0/lib/python3.8/site-packages' with backend 'inotify' and native excludes '/home/ben/venv3_ansible-4.2.0/lib/python3.8/site-packages/.git/objects, /home/ben/venv3_ansible-4.2.0/lib/python3.8/site-packages/.git/subtree-cache, /home/ben/venv3_ansible-4.2.0/lib/python3.8/site-packages/node_modules, /home/ben/venv3_ansible-4.2.0/lib/python3.8/site-packages/.hg/store, /home/ben/venv3_ansible-4.2.0/lib/python3.8/site-packages/collections, /home/ben/venv3_ansible-4.2.0/lib/python3.8/site-packages/molecule/reports, /home/ben/venv3_ansible-4.2.0/lib/python3.8/site-packages/molecule/resources, /home/ben/venv3_ansible-4.2.0/lib/python3.8/site-packages/roles'
TRACE [File Watcher (parcel)] Started watching: '/usr/lib/python3.8' with backend 'inotify' and native excludes '/usr/lib/python3.8/.git/objects, /usr/lib/python3.8/.git/subtree-cache, /usr/lib/python3.8/node_modules, /usr/lib/python3.8/.hg/store, /usr/lib/python3.8/collections, /usr/lib/python3.8/molecule/reports, /usr/lib/python3.8/molecule/resources, /usr/lib/python3.8/roles'

Is it me, or is nodejs fs.watch reversing the watcherExclude and watcherInclude settings? Why is parcel watching my currently active Python virtual environment folder and the system Python folder (though it seems to be honoring the watcherExcludes)?

# current/active virtual env file count
(venv3_ansible-4.2.0) ben@knedmhils042:~/venv3_ansible-4.2.0$ ~/bin/dirs_with_files.sh 
number of files in cwd:
72322
directories with most files:
896 ./lib/python3.8/site-packages/ansible_collections/fortinet/fortimanager/plugins/modules
895 ./lib/python3.8/site-packages/ansible_collections/fortinet/fortimanager/plugins/modules/__pycache__
# snipped for brevity

# system Python file count
(venv3_ansible-4.2.0) ben@knedmhils042:/usr/lib/python3.8$ ~/bin/dirs_with_files.sh 
number of files in cwd:
1334
directories with most files:
206 .
175 ./__pycache__
# snipped for brevity

I made the switch to VSCode years ago and truly have loved using it. But the past couple/few months have been a real struggle. I typically have 2 or 3 instances of VSCode running at once (due to the inter-related nature of the things I work on), but having a 3rd instance up is guaranteed to hit the inotify user max watches limit.

I think I read somewhere about a new unified file watcher implementation vs. maintaining different ones for Windows, Linux, Mac and I certainly hope that is on its way and can provide some relief.

[denosaurtrain]

This bit me today like one of those bear trap claws. @watsonb, my logs also point to the "parcel" file watcher backend. If I'm not mistaken, that replaced vscode's old file watcher sometime in the last 6-12 months. @parcel/watcher doesn't have "real" support for globs. I confirmed that it creates file watchers for files/directories that use globs, then evaluates the events against the ignore rules after the event fires. In other words, files.watcherExclude likely won't fix the inotify limit problem.

Thankfully, the fix 🤞 is planned for the August 2022 release.

[watsonb]

@denosaurtrain let's hope there is some relief soon.

[MarkB2]

I should say at least on Ubuntu the bug still exists. I run VSCode in a container, so couldn't increase watches number, the root system is huge but mostly mounted read only. The only .git's are in my projects. The current one has 620 files but vscode watcher use 7360 ones.

[denosaurtrain]

Yup, the situation is the same AFAIK.

TL;DR: VS Code's new default watcher implementation, @parcel/watcher, creates too many inotify watchers. To the extent that the files.watcherExclude glob configuration is supported by vscode, the globs are only used to filter events after the events have fired, so that setting does not reduce the number of watch handles created.

parcel-bundler/watcher#64 was included in the August and September iteration plans, but the issue is not resolved yet and not included in the October iteration plan. Assuming that means it was dropped or postponed, I'm not sure why, nor do I know who to ask about it.

For some users, myself included, an acceptable workaround is to increase max_user_watches. In my case, I have to set it to 2M+, which can use up to 2G of memory. Some folks aren't lucky enough (a) to be able to change that setting at all and/or (b) to have sufficient memory for all those watch handles.

Caveat: Take my statements in this comment with a grain of salt. I'm sharing what I've gleaned from my research and testing of the issue, but I'm not "in the know" about any of this, and I could be completely off-base.

[ralwing]

I must admit that due to this issue I had to vscode downgrade to 1.70 :-(

[bpasero]

/duplicate #137872

[VSCodeTriageBot]

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for similar existing issues. See also our issue reporting guidelines.

Happy Coding!