HTTP Status code in check HTTP API

[hadrienk]

Hi,

Wouldn't it be very interesting to have the possibility to query the HTTP check API for status and get a status code that reflects the state of the check?

This would allow to define services depending on other services for example.

[cruatta]

I think this would break a lot of third party APIs that rely on standardized status codes to interpret results that consul returns via the HTTP API.

[hadrienk]

Yes that wpuld certainly be a problem.
But what meant is to add new API endpoints, or a parameter on the existing ones to activate this behaviour.

[slackpad]

Hi @hadrienk can you elaborate on your use case a little more? I think you could get this behavior now if you register a script as a health check for service A that calls into Consul and checks the health of service B, but I'm not sure if I fully understand what you'd like to do.

[hadrienk]

Hi @slackpad, I was thinking of doing this with a check scripts indeed but I came across this in the consul documentation about HTTP checks:

This type of check should be preferred over a script that uses curl or another external process to check a simple HTTP operation.

In my scenario, I have services that depend on other services to function properly. Registering a check script to check via HTTP something that is already checked and available to the consul cluster seems wrong. But maybe I am trying to achieve too much out of the tool?

Actually, stretching this idea even further, the ideal thing would probably be to be able to register dependencies on other nodes health checks or service checks that are within the cluster don't you think?

[slackpad]

Will have to gave that a little thought. It seems powerful but also potentially pretty complex. In general, Consul handles this already because if service A depends on service B and a connection to B fails, when A looks up B again in DNS it will get the address of another healthy instance. Assuming no more of B is available, A should then fail. This all happens without having to tell Consul what the graph is like.

[123BLiN]

+1
/v1/health/service/?passsing should not return 200 when result is empty
Use case - AWS ELB health checks, but @hadrienk use case seams powerfull too.

It looks like previous will not cover AWS ELB node healthcheck
Could it be new agent api endpoint, something like /v1/agent/service/?passing and /v1/agent/self?passing ?

[systeminsightsbuild]

This would be really nice for AWS ELBs.

[hewjoe]

I have the same request for an non 200 return for health check to support ELB.

My use case is setting up a Hashicorp Vault cluster as 3 independent auto-scaling groups, each running with min/max 1. Each ASG is in a different Availability Zone (AZ). Each system is initialized as part of cloudformation init and they run a script to take over the route53 DNS name for the respective node (vault1,vault2,vault3). My first try uses a round robin route53 record which includes all 3 hostnames.

Consul is running across all 3 nodes and vault is configured to use consul on localhost as the storage backend. Consul automatically configures a service for vault with health checks (thanks!)

Ideally, I would like to use an ELB to leverage the health checks consul + vault auto configured. In order to get this to work properly, I either have to have a separate service running on the box which will query consul and where down return a non-200 status code. Or better yet, if consul provides an endpoint that can return a non-200 result when querying http:8080/v1/health/service/vault?node=self&failhard=true.
This endpoint can be configured in ELB and when the vault is not in service, it will fail the health check and that instance will be cycled out and another will spin up to replace it.

So consider this another +1 for this future enhancement. I'd be happy to try it out for you!

[melo]

Any updates on this? Is this something being considered?

[slackpad]

This isn't on the roadmap but seems like a reasonable feature. We'd need to keep /v1 compatibility at this point so this would need to be done with new APIs or some kind of opt-in parameter to the old ones.

[kadaan]

@melo @hewjoe @systeminsightsbuild @hadrienk
One way to tackle this is with a sidecar. I wanted this functionality so I wrote Consulate. I'm getting ready to start testing it in my dev environment next week. I'd love any feedback.