Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle crictl query error in slot reconciliation #6

Closed
bfjelds opened this issue Oct 15, 2020 · 1 comment · Fixed by #418
Closed

Handle crictl query error in slot reconciliation #6

bfjelds opened this issue Oct 15, 2020 · 1 comment · Fixed by #418
Labels
bug Something isn't working

Comments

@bfjelds
Copy link
Collaborator

bfjelds commented Oct 15, 2020

Currently, we are doing a best effort approach to slot reconciliation, which ensures that device usage on an Instance reflects the real state of which pods are using an instance.
To get the real usage, we are using crictl to query the container runtime in search of active Containers that have our slot Annotations that device-plugin adds to a pod upon an Allocate call from kubelet.
If this crictl query fails (possibly due to crictl not being mounted correctly), we are doing an early return on slot reconciliation. This should be handled more specifically in the future.

Alternatively, if Kubernetes adds Deallocate to Device-Plugin, slot reconciliation might not be needed. This is the PR for Deallocate: kubernetes/kubernetes#91190

@kate-goldenring
Copy link
Contributor

Another option would be to add crictl to the Agent containers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants