Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using 'Value' in CPU scaler results in an error (must set either a target raw value or a target utilization) #2200

Closed
yogevyuval opened this issue Oct 17, 2021 · 8 comments · Fixed by #2218
Assignees
Labels
bug Something isn't working

Comments

@yogevyuval
Copy link

Report

When using 'Value' in the CPU scaler (in addition to SQS scaler), the keda hpa can't be intiailized, and logs show error in K8s hpa. When switching back to "Utilization" instead of "Value" this seems to work, but it doesn't suit my usecase.

My configuration is as follows:

Spec:
  Cooldown Period:    60
  Max Replica Count:  2
  Min Replica Count:  0
  Polling Interval:   30
  Scale Target Ref:
    Name:  indexer
  Triggers:
    Metadata:
      Aws Region:      us-east-1
      Identity Owner:  operator
      Queue Length:    500
      Queue URL:       https://sqs.us-east-1.amazonaws.com/some-queue
    Type:              aws-sqs-queue
    Metadata:
      Type:   Value
      Value:  80
    Type:     cpu

And this is the outcome YAML in the HPA

spec:
  maxReplicas: 2
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: service-a
  targetCPUUtilizationPercentage: 80
status:
  currentCPUUtilizationPercentage: 210
  currentReplicas: 2
  desiredReplicas: 2
  lastScaleTime: "2021-10-14T15:23:01Z"

Expected Behavior

The autoscaler should have been created with the "Value" target

Actual Behavior

The hpa fails to initialize / update

Steps to Reproduce the Problem

  1. Create an autoscaler with the attached config
  2. Helm install
  3. Check if HPA is created

Logs from KEDA operator

ERROR controllers.ScaledObjebt Failed to update HPA {"ScaledObject.Namespace": "keda", "ScaledObject.Name": "service-a", "HPA.Namespace": "keda", "HPA.Name": "keda-hpa-service-a"}"

spec.metrics[0].resource.target.averageUtilization: Required value: must set either a target raw value or a target utilization].

KEDA Version

2.4.0

Kubernetes Version

1.20

Platform

Amazon Web Services

Scaler Details

No response

Anything else?

No response

@yogevyuval yogevyuval added the bug Something isn't working label Oct 17, 2021
@JorTurFer
Copy link
Member

hi @yogevyuval
I can reproduce the problem, I will take a look at this ASAP

@JorTurFer
Copy link
Member

Hi,
I have been digging in this issue and I found these constrains in validateResourceSource()

	if src.Target.AverageUtilization == nil && src.Target.AverageValue == nil {
		allErrs = append(allErrs, field.Required(fldPath.Child("target").Child("averageUtilization"), "must set either a target raw value or a target utilization"))
	}

	if src.Target.AverageUtilization != nil && src.Target.AverageValue != nil {
		allErrs = append(allErrs, field.Forbidden(fldPath.Child("target").Child("averageValue"), "may not set both a target raw value and a target utilization"))
	}

Honestly, I didn't know about this changed behavior, but it seems that probably we should drop Value support and clarify in the documentation that Value is not a valid option. Maybe I'm wrong, but I'm getting the same error when I try to create the HPA directly with this manifest:

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: hpa-cpu
  namespace: keda
spec:
  scaleTargetRef:
    kind: Deployment
    name: keda-metrics-apiserver
    apiVersion: apps/v1
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Value
        value: 50

WDYT @zroubalik @arschles @tomkerkhove @ahmelsayed ?

@yogevyuval
Copy link
Author

@JorTurFer So you are saying that the right thing is to use 'AverageValue'?

@JorTurFer
Copy link
Member

@JorTurFer So you are saying that the right thing is to use 'AverageValue'?

AverageValue or AverageUtilization seem the only supported values in Kubernetes repository :/

@zroubalik
Copy link
Member

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-multiple-metrics-and-custom-metrics

Based on this ^, AverageValue or AverageUtilization are the only types allowed for resource metrics and it has been like that from the beginning. We should fix our docs.

@JorTurFer
Copy link
Member

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-multiple-metrics-and-custom-metrics

Based on this ^, AverageValue or AverageUtilization are the only types allowed for resource metrics and it has been like that from the beginning. We should fix our docs.

Thanks a ton for diggin in this @zroubalik !!!

@alvynabranches
Copy link

Can I get an example of passing the value?

@JorTurFer
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants