Health Checks for K8S Deployments with no Server Ports

[cblauvelt]

I have a k8s deployment that reads data off a queue, processes the data, and injects that data back into a KV store. Because it's not serving any data, there is no '/healthz' endpoint to poll. How should I run health checks, liveness probes, and readiness probes if there is no http service? Is there an idiomatic way this should be done?

The one solution I had, which I'm not sure is viable, is to read a unix socket periodically and respond to requests. I could report some stats as well as a go/no-go report but I'm not sure how that would integrate with the rest of the docker-swarm and k8s platforms.

[BretFisher]

Typically I see apps write to a file that the health probe queries. You can have the app write a file and the probe checks the creation/modify date, or the probe check the contents of the file.

https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-a-liveness-command

[cblauvelt]

Thanks Bret. That makes a lot of sense and is much simpler than a unix socket. I ended up making a watchdog timer that writes the unix time every x seconds where x is as least 1/2 the probe interval.