On service registration, health check should be "critical" not "unknown"

[armon]

Currently there is a race where the service is in the pool while it may be unavailable. Fix this by defaulting state to critical instead of the useless unknown state.

[plorenz]

I'm seeing similar behavior except the alarm is not showing up at (not even in state 'unknown'). The service appears with just the default alarm, then a short time later shows up with the alarm in warning state.

[plorenz]

This issue is not fixed. I just retested with the recently release 0.4.1 and saw the following:

Goal is to get a set of healthy services.

1. Set a blocking query for a service
2. Registered a new service instance with a health check that will fail
3. Query returns with service instance. The health check is not listed and thus the service shows as healthy.
4. The next query will show the service as failing, but action has already been taken with the unhealthy service.

Should this issue be reopened or should a new issue be created for this?

[ryanuber]

@plorenz we can track this issue here - I'll give your use case a shot and see if I get the same thing. Thanks!

[plorenz]

Thanks, @ryanuber. I'm using the /v1/health/service REST API since the /v1/catalog/service doesn't expose information about health checks.

[armon]

Yeah I think this was 2 different issues. The first was the use of the "unknown" state which has been fixed, the second is a race condition between the registration of the service and the health check.

[ryanuber]

I've separated this into another issue after @armon's comment - I have reproduced this and will look into it. @plorenz have a look at #413, we will track it there.