Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

start without running queues throws error #25

Open
schorsch opened this issue Apr 5, 2011 · 8 comments
Open

start without running queues throws error #25

schorsch opened this issue Apr 5, 2011 · 8 comments
Labels

Comments

@schorsch
Copy link

schorsch commented Apr 5, 2011

When starting for the first time without running workers an error is thrown and stops the whole pool from running:

bundle exec resque-pool
=> calls reaper

gems/resque-pool-0.2.0/lib/resque/pool.rb:243:in `reap_all_workers': You have a nil object when you didn't expect it! (NoMethodError)
The error occurred while evaluating nil.queues

Possible solution, add 'if worker' to reaper message:

log "Reaped resque worker[#{status.pid}] (status: #{status.exitstatus}) queues: #{worker.queues.join(",")}" if worker

I am not sure, but i think last week i had it running local without this error. maybe it is the startup time of the workers which prevents the reaper from seeing them.

@schorsch
Copy link
Author

schorsch commented Apr 5, 2011

Seems to be related to this one:
https://github.com/nevans/resque-pool/issues#issue/20

here is some more output, now the manager runs but kills and never restarts workers

bundle exec resque-pool
resque-pool-manager[7889]: Resque Pool running in development environment
(in /home/georg/trunk)
resque-pool-manager[7889]: started manager
resque-pool-manager[7889]: Pool contains worker PIDs: [7898, 7896, 7899]
resque-pool-worker[7896]: Starting worker minimi:7896:night_jobsresque-pool-worker[7899]: Starting worker minimi:7899:pubsub_publish
resque-pool-worker[7898]: Starting worker minimi:7898:scheduler

resque-pool-manager[7889]: WINCH: gracefully stopping all workers
resque-pool-manager[7889]: WINCH: gracefully stopping all workers
resque-pool-manager[7889]: Reaped resque worker[7899](status: 0) queues: pubsub_publish
resque-pool-manager[7889]: Reaped resque worker[7898](status: 0) queues: scheduler
resque-pool-manager[7889]: Reaped resque worker[7896](status: 0) queues: night_jobs
resque-pool-manager[7889]: WINCH: gracefully stopping all workers
resque-pool-manager[7889]: WINCH: gracefully stopping all workers

$ ps -ef f | grep [r]esque
georg 7889 2908 3 20:42 pts/0 S+ 0:06 | _ resque-pool-master: managing []

running ubuntu maverik 64bit:
ruby 1.8.7 (2010-06-23 patchlevel 299) [x86_64-linux]

@nevans
Copy link
Collaborator

nevans commented Apr 5, 2011

This might be a different issue than your original bug, but something appears to be sending the WINCH signal to your pool manager. Are you doing that manually?

@schorsch
Copy link
Author

schorsch commented Apr 6, 2011

no i am just starting the manager without args in dev mode and also deleted my bundle so no old gems are present. I searched in rp gem for WINCH but it is not used anywhere

.. whow this is funny: WINCH get send to the process when i resize my terminal window ???

@schorsch
Copy link
Author

schorsch commented Apr 6, 2011

WINCH is an abbreviation for window [size] change
http://en.wikipedia.org/wiki/SIGWINCH

@nevans
Copy link
Collaborator

nevans commented Apr 7, 2011

Well that explains the WINCH issue (which is different from the original one). I had completely forgotten WINCH's original purpose. :)

I'm copying nginx and Unicorn's signals handling, and WINCH will gracefully shutdown all workers (see "Signals" in the README). You can restart them with HUP.

As to your original bug, yeah we should guard against workers being nil.

@jtoy
Copy link

jtoy commented Aug 6, 2011

this issue has been hitting me for while and I have been running resque-pool from inside screen to monitor it :(

@nirvdrum
Copy link

I noticed that this started happening when I upgraded New Relic's rpm_contrib from 1.0.13 to 2.1.4. I haven't looked into it more than that, but yanking rpm_contrib fixed the problem. I'd imagine their resque probes are doing something they shouldn't be.

@clifff
Copy link

clifff commented Aug 15, 2012

I've been running into this issue since we use resque-pool in development, and as such, our workers are reaped frequently. If it hasn't been fixed, I may take a crack at a pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants