Allow hooks to be run before/after start/stop

[akheron]

I'm running redis with supervisord. (As of the current version) Redis doesn't save its database when it receives a TERM signal (or any other signal). This could be circumvented by having a pre-stop hook that could be used to run a command to save the database before sending the TERM signal.

I assume that this would be a useful feature for other use cases, too, so I'm proposing to add pre-start, post-start, post-start and pre-stop hooks to supervisord.

[mcdonc]

Does it save when it receives a different signal?

[akheron]

No.

[mcdonc]

This ubuntu issue seems to indicate that there exists a version of Redis which does save at TERM, FWIW: http://code.google.com/p/redis/issues/detail?id=97

[akheron]

Yes, saving on TERM has been implemented already in Redis' master branch. That's why I said "As of the current version". Regardless of that, I still think the hooks would be a feature worth discussing.

[mcdonc]

I'd suggest instead executing a script that registers a signal handler for TERM, spawns redis as a subprocess, then waits forever, and calls the save command then kills the subprocess when it gets a TERM. "pre" and "post" scripts are a little beyond supervisor's scope when it's easy enough to do this.

[akheron]

Ah, you're right. I hadn't thought of that.

[hvelarde]

Do you guys have an example of such script? I have the following use case:

Plone site with multiple instances running behind a web accelerator like Varnish, using memmon and httpok.

From time to time, memmon will need to restart an instance because of high memory consumption; Varnish has configured backend health check probes that test if the instances are available or not, and this probes have to play very nicely with the instances because requests hitting the backend are typically slow; for instance:

probe healthcheck {
    .interval = 10s; .request = "HEAD / HTTP/1.1"; .timeout = 3s;
}

As restarting a Plone instance is a time consuming process (typically, around 30 seconds), Varnish will not notice the instance is down for some time and will continue sending requests; the instance then came in and will be flooded with a lot of pending requests from Varnish.

The Varnish backend then behaves erratically for some time until the instance stabilizes.

I would like to be able to configure something like this:

• memmon detects high memory usage on instance1
• pre-hook is run: varnishadm backend.set_health instance1 sick
• instance1 is restarted
• supervisor waits and marks it as running
• post-hook is run: varnishadm backend.set_health instance1 auto

An additional step could be included before the last one to warm up the ZODB cache.