-
Notifications
You must be signed in to change notification settings - Fork 248
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simple Guard Script Running at 100%+ CPU usage #184
Comments
Yeah Listen doesn't support watching that amount of files, try to ignore as much as you can with the |
@thibaudgg, thanks for the info, but i can't exclude that many files, as they are 90% php and 10% JS and Less files. I need them all to be built out, it's for our software that the company i work for builds. Just some general FYI, we have a version of listen running just find on our webapp. Here is the gem file from it. # Gemfile
source 'https://rubygems.org'
gem 'rake'
group :development do
platforms :ruby do
gem 'rb-fsevent', '~> 0.9.3', :require => false
end
gem 'terminal-notifier-guard', :require => false
gem 'growl', :require => false
gem 'guard', '1.4.0'
gem 'guard-shell', '0.5.1'
gem 'listen', '0.7.0'
gem 'rainbow'
gem 'git', :git => 'git@github.com:jwhitcraft/ruby-git.git'
gem 'addressable'
gem 'configliere'
end Is there any reason why the older version of guard + listen works just fine? If I go higher than version 0.7.0 it just spikes the CPU usage. |
How many php & js files do you have? |
jwhitcraft@jwhitcraft-mbpro ~/Projects/Mango/sugarcrm jwhitcraft@jwhitcraft-mbpro ~/Projects/Mango/sugarcrm jwhitcraft@jwhitcraft-mbpro ~/Projects/Mango/sugarcrm jwhitcraft@jwhitcraft-mbpro ~/Projects/Mango/sugarcrm jwhitcraft@jwhitcraft-mbpro ~/Projects/Mango/sugarcrm There is a better break down of all the files types that i need to watch. Is there a away with listen to say just look for php, js, less, hbs and json files? |
Even if you listen only these files, that's still a lot, and because Listen keeps the md5 checksum of each modified files it quite slow with so much files. |
@thibaudgg, thanks for the info. Maybe if I get some time I can dig in and hack away at it to see if i can't figure out how to increase performance for larger file sets. Thanks! |
Would be awesome! You can try to comment that line: https://github.com/guard/listen/blob/master/lib/listen/file.rb#L104, maybe it'll boost everything. |
@thibaudgg, In my limited testing, that did boost everything, there is still an initial delay of about 1 and half minutes where the cpu is at 100+%, I assume where it's indexing all the files. I'm gonna play around with it more to try and figure out a better way to handle that many files as IMO it should be supported. |
Interesting, normally timestamp and checksum should be eager loaded to avoid such initial delay, there's maybe something to fix there. |
@jwhitcraft could you try if release 2.6.0 fix it, I improve md5 checksum logic: db6dcc2 |
@thibaudgg, 2.6.0 did show a lot of improvement, it only ran at 100% for about 30 seconds I know when dealing with the amount of files that I'm dealing with it's not going to be instantaneous, but this is a far cry better thant was it was at 2.4.x But something I just noticed while using the new I sampled the Process when it's spiked and attached it here: |
Ok good news! Thanks for your feedback. On Darwin, we still need to keep every files timestamp & mode (both read from The new listener cli is just for tcp forwarding, so maybe it comes from there too. I'm sorry but I'm not sure I'll be able to found any useful info from the Process you give me. Can we close it for now? |
@thibaudgg, ya this can be closed. I'll keep playing around with it to see if i can find more speed improvements once it's running. Thank! |
I have this guard script
And when I run this command:
bundle exec Guard -w /Users/jwhitcraft/Projects/Mango
the ruby process takes up 100% on a CPU in my activity monitor.
The folder i'm watching has 18531 files in it, but if i change it to one that doesn't contain that many files, the issue goes away.
My gem file looks like this
I've tracked it down to being listen as the problem as when i just run listen, it does the same thing
The text was updated successfully, but these errors were encountered: