kqueue: Only make a single syscall to add all the watches #454
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hey!
We use notify v4 in
turbopack
and we've recently tried updating to v5 to use thekqueue
implementation, as we found it to be much faster than FSEvents at detecting updates.However, we also noticed a very large drop in performance when watching large directories. After a bit of investigating, it turns out that
notify
makes akevent
syscall for each file it watches. Watching a folder with 15k files ends up taking about 11.5s.This PR changes this behavior to ensure we only make a single
kevent
call when adding multiple files at the same time. This improves the above case to take 900ms.I've also filed an issue with
kqueue
to fix another performance problem, which makes watching n files O(n²). This would further improve to 400ms.Finally, while profiling the above issue, I noticed that during the 12s of watching, only 2 cores of my computer were working. I have a prototype version that uses multiple threads to open files, which might be faster when watching large directories, but it would also require changes to
kqueue
, so I'm filing it away for now.