Create a "Pending" health check state

[langston-barrett]

Currently, health checks that have not had an initial run are marked as "Critical", which causes several problems as outlined in #954. While running the checks on startup and decreasing intervals can mitigate this issue, there will probably always be some period of time that Consul is up and the checks haven't been run. For this reason, it would be useful to have a "Pending" state that doesn't take services offline, and more accurately reflects the status of the checks.

[jrmithdobbs]

So they would still be returned by passingonly queries but have status pending ?

I think adding a pendingorpassing param (name unimportant) when adding the pending state and leaving the dns/http defaults as is but tunable makes more sense.

[armon]

We had this previously as the "unknown" state. We deprecated it on purpose, because it causes a lot of confusion. What does it mean for a check to be pending? Do you treat it as healthy or unhealthy? Instead I think an "initial" health state makes more sense. e.g. Treat this as unhealthy until otherwise known, or treat as healthy until otherwise failed.

[langston-barrett]

@armon I understand the confusion that this solution may cause in particular, but we need to address this problem in some form. The idea of nodes being marked critical before running checks causes severe problems on startup if you have long check intervals, as per #954. Perhaps there could be a configuration option for which "initial" state nodes are put in.

I'd be happy to open a new ticket with an outline of that suggestion, if you'd like.

[armon]

@siddharthist Agreed, could you make a ticket for the initial state in that case?

[ryanuber]

#859 actually should solve this case by allowing the ability to provide the check status when registering the check. The confusion is probably that we forgot to document it on the web docs, so we can probably just create a ticket for that instead. @siddharthist does that satisfy your use case?

[ryanuber]

Spoke too soon on the docs as well - #962 fixes the documentation up. Sorry for the ticket maze 🌀

[langston-barrett]

@ryanuber It does, thank you for pointing that out! I'll comment with the same pointer on issue #954 for reference, and then both this and that are probably ready to close.