Update DataChain worker liveness probe #325
Conversation
mjasion
force-pushed
the
studio-dvcx-worker-liveness-probe
branch
from
3c85aa9 to
181b4df
Compare
mjasion
force-pushed
the
studio-dvcx-worker-liveness-probe
branch
from
9a3f199 to
3e584d0
Compare
|
Waiting this PR to be merged and deployed to dev and prod: https://github.com/iterative/studio/pull/9247 |
|
@dreadatour what is the status of this? |
|
@dreadatour ping |
|
folks @dreadatour and @iterative/platform - what is the status here? |
1 similar comment
|
folks @dreadatour and @iterative/platform - what is the status here? |
|
It looks like https://github.com/iterative/studio/pull/9174 mentioned on #325 (comment) was released 4+ months (!) ago; after #438 we no longer need this liveness probe. Still, we should have some liveness probe, maybe making sure that the Celery workers are storing heartbeats into Redis or something that ensures us the pod is healthy and ready to ingest new tasks. |
0x2b3bfa0
changed the title
0x2b3bfa0
changed the title
|
So there's nothing wrong with this pull request, and we can merge it after re-designing |
|
there was some code duplication around SA liveness probes and tests on the Studio side that we also should cleanup along the way |
|
Maybe https://github.com/iterative/studio/pull/10307#discussion1683252817 can provide better readiness/liveness probes for DataChain that anything else, if it gets merged. |
|
Stale; opened #504 to track this |
This updates liveness probe command for DVCx worker k8s deployment.
Should be merged after https://github.com/iterative/studio/pull/9174 will be merged and released.