Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix: avoid ext resource less then allocated #892

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

payall4u
Copy link
Collaborator

What type of PR is this?

bugfix

What this PR does / why we need it:

When online pods start using resources, the crane agent will reduce the amount of ext resources. If an online pod uses too many resources, the ext resources may be less than allocated. This can cause pods using ext resources to enter an indeterminate state. At this time, the pods will not be evicted directly, but restarting the kubelet may cause eviction.

This should be a bug. I tried to compare the total amount of allocated ext resources when reporting ext resources. Of course, only running pods are calculated, which is consistent with the logic of the kubelet.

Which issue(s) this PR fixes:

Fixes #890

Special notes for your reviewer:

@payall4u payall4u requested a review from mfanjie January 21, 2024 03:16
@payall4u payall4u force-pushed the bugfix/avoid-ext-resource-less-then-allocated branch from 8a76c02 to 41c5a84 Compare January 21, 2024 03:28
Copy link
Contributor

github-actions bot commented Jan 21, 2024

🎉 Successfully Build Images.
Now Support ARM Platforms.
Comment Post Time: 2024-01-21 11:47
Git Version: 41c5a84

Docker Registry

Overview: https://hub.docker.com/u/gocrane

Image Pull Command
crane-agent:pr-892-41c5a84 docker pull gocrane/crane-agent:pr-892-41c5a84
dashboard:pr-892-41c5a84 docker pull gocrane/dashboard:pr-892-41c5a84
metric-adapter:pr-892-41c5a84 docker pull gocrane/metric-adapter:pr-892-41c5a84
craned:pr-892-41c5a84 docker pull gocrane/craned:pr-892-41c5a84

Quick Deploy - Helm

helm repo add crane https://finops-helm.pkg.coding.net/gocrane/gocrane
helm install crane -n crane-system --create-namespace \
                   --set craned.image.repository=gocrane/craned \
                   --set craned.image.tag=pr-892-41c5a84 \
                   --set metricAdapter.image.repository=gocrane/metric-adapter \
                   --set metricAdapter.image.tag=pr-892-41c5a84 \
                   --set craneAgent.image.repository=gocrane/crane-agent \
                   --set craneAgent.image.tag=pr-892-41c5a84 \
                   --set cranedDashboard.image.repository=gocrane/dashboard \
                   --set cranedDashboard.image.tag=pr-892-41c5a84 crane/crane

Coding Registry

Overview: https://finops.coding.net/public-artifacts/gocrane/crane/packages

Image Pull Command
crane-agent:pr-892-41c5a84 docker pull finops-docker.pkg.coding.net/gocrane/crane/crane-agent:pr-892-41c5a84
dashboard:pr-892-41c5a84 docker pull finops-docker.pkg.coding.net/gocrane/crane/dashboard:pr-892-41c5a84
metric-adapter:pr-892-41c5a84 docker pull finops-docker.pkg.coding.net/gocrane/crane/metric-adapter:pr-892-41c5a84
craned:pr-892-41c5a84 docker pull finops-docker.pkg.coding.net/gocrane/crane/craned:pr-892-41c5a84

Quick Deploy - Helm

helm repo add crane https://finops-helm.pkg.coding.net/gocrane/gocrane
helm install crane -n crane-system --create-namespace \
                   --set craned.image.repository=finops-docker.pkg.coding.net/gocrane/crane/craned \
                   --set craned.image.tag=pr-892-41c5a84 \
                   --set metricAdapter.image.repository=finops-docker.pkg.coding.net/gocrane/crane/metric-adapter \
                   --set metricAdapter.image.tag=pr-892-41c5a84 \
                   --set craneAgent.image.repository=finops-docker.pkg.coding.net/gocrane/crane/crane-agent \
                   --set craneAgent.image.tag=pr-892-41c5a84 \
                   --set cranedDashboard.image.repository=finops-docker.pkg.coding.net/gocrane/crane/dashboard \
                   --set cranedDashboard.image.tag=pr-892-41c5a84 crane/crane

Ghcr Registry

Overview: https://github.com/orgs/gocrane/packages?repo_name=crane

Image Pull Command
crane-agent:pr-892-41c5a84 docker pull ghcr.io/gocrane/crane/crane-agent:pr-892-41c5a84
dashboard:pr-892-41c5a84 docker pull ghcr.io/gocrane/crane/dashboard:pr-892-41c5a84
metric-adapter:pr-892-41c5a84 docker pull ghcr.io/gocrane/crane/metric-adapter:pr-892-41c5a84
craned:pr-892-41c5a84 docker pull ghcr.io/gocrane/crane/craned:pr-892-41c5a84

Quick Deploy - Helm

helm repo add crane https://finops-helm.pkg.coding.net/gocrane/gocrane
helm install crane -n crane-system --create-namespace \
                   --set craned.image.repository=ghcr.io/gocrane/crane/craned \
                   --set craned.image.tag=pr-892-41c5a84 \
                   --set metricAdapter.image.repository=ghcr.io/gocrane/crane/metric-adapter \
                   --set metricAdapter.image.tag=pr-892-41c5a84 \
                   --set craneAgent.image.repository=ghcr.io/gocrane/crane/crane-agent \
                   --set craneAgent.image.tag=pr-892-41c5a84 \
                   --set cranedDashboard.image.repository=ghcr.io/gocrane/crane/dashboard \
                   --set cranedDashboard.image.tag=pr-892-41c5a84 crane/crane

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

When online resource usage is too high, the agent will set ext resources to 0
1 participant