Slow listener

[brianmd]

In reference to Simple Guard Script Running at 100%+ CPU usage:
#184

It may be creating the checksum for all the files, not just the #only ones. Try the following:

start a listen in a directory with several large files and start a listener on only ruby files. (The directory I first toyed with listen on had about 20 files but totaled 8.6G, and only one ruby file of a few bytes.) The cpu will spike to 100% usage on the first file change, and will remain spiked. Remove the large files and the spike doesn't occur.

listener = Listen.to('/Users/bmd/Downloads') {|m,a,r| puts "#{m},#{a},#{r}"}
listener.start
listener.only /.rb$/

[e2]

You can disable hashing completely using LISTEN_GEM_DISABLE_HASHING with the downside that Listen may not correctly detect if something actually changed or not on (so it assumes it has and does an md5 in case it didn't).

Listen tried to "intelligently" detect whether the filesystem actually needs hashing (e.g. if you use VFAT on Linux, you'll get the same scenario).

There are faster hashing algorithms than md5, though on Linux I couldn't get any to perform better than md5 (due to it being in the kernel I guess). I don't know if there's a hashing library usable from a ruby lib that's actually faster in practice (due to the way file buffers are passed through Ruby).

Also, check the LISTEN_GEM_DEBUGGING variables to get times.

Also, make sure you're using the latest version.

I recently created: https://github.com/guard/listen/blob/master/TROUBLESHOOTING.md

Some relevant stuff is also mentioned in: https://github.com/guard/listen#performance

Let me know if there's anything I can update/change in the docs.

There also may be improvements to rb-fsevent that can make use of newer APIs on OSX, but I use Linux exclusively, so it's unlikely I'll contribute anything there, since I can't test.

If you want, you can check out https://github.com/guard/listen/blob/master/lib/listen/file.rb to know when the checksumming is triggered.

[brianmd]

Excellent information! Thanks and you can close this out.