-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ScaledObject minReplicaCount is not work when it set to 0 #6414
Comments
I'm also seeing this issue. I'm running Keda 2.13.1 and using the Keda HTTP Add on at version 0.8.0.
Looking at the code that generates the HorizontalPodAutoscaler, I suppose we're setting the min pods to 0 in the HPA. Since thats a limit of the HPA, it's not super clear to me how we're supposed to scale to zero in any case. I've been diving into the code to see if we maybe delete the HPA and manually set the deployment to 0 scale, but that doesn't seem like it's happening anywhere. Is there something I'm missing here? httpscaledobjectapiVersion: http.keda.sh/v1alpha1
kind: HTTPScaledObject
metadata:
annotations:
app.kubernetes.io/version: "1733278908"
dokku.com/managed: "true"
httpscaledobject.keda.sh/skip-scaledobject-creation: "true"
meta.helm.sh/release-name: screencasts
meta.helm.sh/release-namespace: default
creationTimestamp: "2024-11-14T23:27:45Z"
finalizers:
- httpscaledobject.http.keda.sh
generation: 4
labels:
app.kubernetes.io/instance: screencasts-web
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: web
app.kubernetes.io/part-of: screencasts
name: screencasts-web
namespace: default
resourceVersion: "132041184"
uid: 95cedefd-26dc-42cb-b82b-248be8ed847d
spec:
hosts:
- screencasts.dokku.com
replicas:
max: 5
min: 0
scaleTargetRef:
apiVersion: apps/v1
deployment: ""
kind: Deployment
name: screencasts-web
port: 80
service: screencasts-web
scaledownPeriod: 120
scalingMetric:
concurrency:
targetValue: 2
status:
conditions:
- message: Identified HTTPScaledObject creation signal
reason: PendingCreation
status: Unknown
timestamp: "2024-12-11T00:59:45Z"
type: Ready
- message: Finished object creation
reason: HTTPScaledObjectIsReady
status: "True"
timestamp: "2024-12-11T00:59:45Z"
type: Ready
targetService: screencasts-web:80
targetWorkload: apps/v1/Deployment/screencasts-web
scaledobjectapiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
annotations:
app.kubernetes.io/version: "1733278908"
dokku.com/managed: "true"
meta.helm.sh/release-name: screencasts
meta.helm.sh/release-namespace: default
creationTimestamp: "2024-10-09T04:20:19Z"
finalizers:
- finalizer.keda.sh
generation: 17
labels:
app.kubernetes.io/instance: screencasts-web
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: web
app.kubernetes.io/part-of: screencasts
scaledobject.keda.sh/name: screencasts-web
name: screencasts-web
namespace: default
resourceVersion: "132046385"
uid: b5a3dac1-da92-417c-80f4-10b234321e30
spec:
cooldownPeriod: 300
fallback:
failureThreshold: 3
replicas: 0
maxReplicaCount: 5
minReplicaCount: 0
pollingInterval: 30
scaleTargetRef:
apiVersion: apps/v1
envSourceContainerName: screencasts-web
kind: Deployment
name: screencasts-web
triggers: []
status:
conditions:
- message: ScaledObject is defined correctly and is ready for scaling
reason: ScaledObjectReady
status: "True"
type: Ready
- message: Scaling is performed because triggers are active
reason: ScalerActive
status: "True"
type: Active
- status: Unknown
type: Fallback
- status: Unknown
type: Paused
hpaName: keda-hpa-screencasts-web
lastActiveTime: "2024-12-11T01:14:33Z"
originalReplicaCount: 1
scaleTargetGVKR:
group: apps
kind: Deployment
resource: deployments
version: v1
scaleTargetKind: apps/v1.Deployment Important I am not managing the HPA at all, thats all generated by Keda. It's not clear to me why there isn't an external metric there and it's referencing the CPU, but changing the max value of the ScaleObject/HTTPScaleObject appears to reflect on it, so maybe thats as expected? httpscaledobjectapiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
annotations:
app.kubernetes.io/version: "1733278908"
dokku.com/managed: "true"
meta.helm.sh/release-name: screencasts
meta.helm.sh/release-namespace: default
creationTimestamp: "2024-10-11T22:41:59Z"
labels:
app.kubernetes.io/instance: screencasts-web
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: web
app.kubernetes.io/part-of: screencasts
app.kubernetes.io/version: 2.13.1
scaledobject.keda.sh/name: screencasts-web
name: keda-hpa-screencasts-web
namespace: default
ownerReferences:
- apiVersion: keda.sh/v1alpha1
blockOwnerDeletion: true
controller: true
kind: ScaledObject
name: screencasts-web
uid: b5a3dac1-da92-417c-80f4-10b234321e30
resourceVersion: "132048058"
uid: d93e4aba-754c-4d88-91cb-c8bb72bc81b5
spec:
maxReplicas: 5
metrics:
- resource:
name: cpu
target:
averageUtilization: 80
type: Utilization
type: Resource
minReplicas: 1
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: screencasts-web
status:
conditions:
- lastTransitionTime: "2024-10-11T22:42:14Z"
message: recent recommendations were higher than current one, applying the highest
recent recommendation
reason: ScaleDownStabilized
status: "True"
type: AbleToScale
- lastTransitionTime: "2024-12-10T22:01:01Z"
message: the HPA was able to successfully calculate a replica count from cpu resource
utilization (percentage of request)
reason: ValidMetricFound
status: "True"
type: ScalingActive
- lastTransitionTime: "2024-12-11T01:16:40Z"
message: the desired count is within the acceptable range
reason: DesiredWithinRange
status: "False"
type: ScalingLimited
currentMetrics:
- resource:
current:
averageUtilization: 0
averageValue: 2m
name: cpu
type: Resource
currentReplicas: 3
desiredReplicas: 3
lastScaleTime: "2024-12-11T01:16:55Z" |
This might actually be related to having an empty triggers list? In my case, I'm manually creating the ScaledObject separate from the HTTPScaledObject, so it's not super clear what should be in the triggers list. Going to see if disabling the manual creation causes this to work as expected. |
Yeah that was the case for me. Either letting the HTTPScaledObject create the entry or adding in the entry myself allows the replica count to climb back down to zero (after hitting the cooldown). See this PR for more context. I also upgraded the chart to 2.16.0 at the same time. |
Report
you can see, min could set to 0, and when i see Scaledobjects,it's work
but when i see hpa, it's MINPODS had been set to 1, and the actual number of replicas remains at 1 and has not changed.
`apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
creationTimestamp: "2024-12-10T06:56:36Z"
finalizers:
generation: 9
labels:
app: ap-77ae9b7d
component: aihc-pom
scaledobject.keda.sh/name: aihcpom-ap-77ae9b7d
name: aihcpom-ap-77ae9b7d
namespace: aihc-pom
resourceVersion: "10652273"
uid: 092a2021-d036-429f-a42d-e7a6cfc2a051
spec:
advanced:
horizontalPodAutoscalerConfig:
behavior:
scaleDown:
stabilizationWindowSeconds: 0
scaleUp:
stabilizationWindowSeconds: 0
maxReplicaCount: 1
minReplicaCount: 0
scaleTargetRef:
apiVersion: argoproj.io/v1alpha1
kind: Rollout
name: ap-77ae9b7d
triggers:
value: "50"
metricType: Utilization
type: cpu
status:
conditions:
reason: ScaledObjectReady
status: "True"
type: Ready
reason: ScalerActive
status: "True"
type: Active
type: Fallback
hpaName: keda-hpa-aihcpom-ap-77ae9b7d
lastActiveTime: "2024-12-10T10:01:01Z"
originalReplicaCount: 1
resourceMetricNames:
scaleTargetGVKR:
group: argoproj.io
kind: Rollout
resource: rollouts
version: v1alpha1
scaleTargetKind: argoproj.io/v1alpha1.Rollout`
Expected Behavior
when metrics is low and minReplicaCount set to 0, the actual number of replicas should be set to 0.
Actual Behavior
the actual number of replicas remains at 1 .
Steps to Reproduce the Problem
Logs from KEDA operator
KEDA Version
< 2.12.0
Kubernetes Version
< 1.28
Platform
Other
Scaler Details
No response
Anything else?
No response
The text was updated successfully, but these errors were encountered: