add ignoreNullValues for AWS CloudWatch Scaler #5635

robpickerill · 2024-03-28T15:03:29Z

This PR adds support for erroring the AWS Cloudwatch Scaler when the GetMetricData call returns an empty set of metrics, resulting in no scale-changing behaviour when no custom metrics are available during the Cloudwatch query window.

Checklist

When introducing a new scaler, I agree with the scaling governance policy
I have verified that my change is according to the deprecations & breaking changes policy
Tests have been added
Changelog has been updated and is aligned with our changelog requirements
A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
A PR is opened to update the documentation on (repo) (if applicable)
Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #5352

Relates to:

https://github.com/kedacore/keda-docs/pull/1346/files

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

pkg/scalers/aws_cloudwatch_scaler.go

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

robpickerill · 2024-04-03T21:57:12Z

I added end to end tests here for both minMetricValue and errorWhenNullValues to cover off both scenarios. I need to tidy them up a little, which ill do over the next few days.

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

...calers/aws/aws_cloudwatch_min_value_null_values/aws_cloudwatch_min_value_null_values_test.go

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

pkg/scalers/aws_cloudwatch_scaler.go

robpickerill · 2024-06-07T15:02:24Z

I was discussing with Jorge on slack to get the e2e tests re-ran, as the failures were upstream issues: https://kubernetes.slack.com/archives/C01JGDP8MB8/p1717129850310929?thread_ts=1715943304.582119&cid=C01JGDP8MB8

However, since then the changes for the validation have gone in, so let me fix up the conflicts here over the weekend, and then let's get e2e passing again on this branch

zroubalik · 2024-06-07T15:08:56Z

Thanks!

Signed-off-by: robpickerill <r.pickerill@gmail.com>

robpickerill · 2024-06-08T15:38:12Z

Fixed conflicts, this needs another e2e test run

zroubalik · 2024-06-10T07:42:56Z

/run-e2e aws
Update: You can check the progress here

zroubalik

@robpickerill there is also a problem in static checks

robpickerill · 2024-06-10T08:36:56Z

@robpickerill there is also a problem in static checks

Fixed, all tests should pass now.

I see e2e didn't run due to Azure cluster scaling actions failing, would it be easier for me to put the e2e tests in a different PR and unblock this PR? The unit tests here cover the error case for the scaler, but I wanted to add e2e to prove the backward compatibility for the change.

Let me know what works best for you.

Signed-off-by: robpickerill <r.pickerill@gmail.com>

zroubalik · 2024-06-10T10:57:59Z

/run-e2e aws
Update: You can check the progress here

stale · 2024-08-09T15:57:45Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

JorTurFer · 2024-08-12T11:55:06Z

/run-e2e aws
Update: You can check the progress here

robpickerill · 2024-08-12T19:10:19Z

Thanks Jorge, need anything else to merge this?

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

JorTurFer

LGTM! ❤️
Please address the docs comment and we are ready

robpickerill · 2024-08-20T08:07:01Z

Thanks Jorge, fixed up the docs, and merged main into here and docs, should be good to go now - thanks again!

JorTurFer · 2024-08-20T19:01:19Z

/run-e2e aws
Update: You can check the progress here

pkg/scalers/aws_cloudwatch_scaler.go

* add errorWhenMetricValueEmpty Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fix golangci-lint Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * improve error message for empty results Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * add error when empty metric values to changelog Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * rename errorWhenMetricValuesEmpty -> errorWhenNullValues Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * use getParameterFromConfigV2 to read config for errorWhenNullValues Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * add e2e for error state for cw, and improve e2e for min values for cw Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * remove erroneous print statement Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * remove unused vars Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * rename errorWhenMetricValuesEmpty -> ignoreNullValues Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * move towards shared package for e2e aws Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * minMetricValue optionality based on ignoreNullValues Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fail fast Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * Update tests/scalers/aws/aws_cloudwatch_ignore_null_values_false/aws_cloudwatch_ignore_null_values_false_test.go Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * Apply suggestions from code review Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fail fast Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fix broken new line Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fix broken new lines Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * assert no scaling changes in e2e, and set false for required in minMetricValue Signed-off-by: robpickerill <r.pickerill@gmail.com> * fix ci checks Signed-off-by: robpickerill <r.pickerill@gmail.com> * Update tests/scalers/aws/aws_cloudwatch_min_metric_value/aws_cloudwatch_min_metric_value_test.go fix invalid check Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fix merge conflicts Signed-off-by: robpickerill <r.pickerill@gmail.com> * fix e2e package names Signed-off-by: robpickerill <r.pickerill@gmail.com> --------- Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> Signed-off-by: robpickerill <r.pickerill@gmail.com> Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Fira Curie <fira42073@gmail.com>

* add errorWhenMetricValueEmpty Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fix golangci-lint Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * improve error message for empty results Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * add error when empty metric values to changelog Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * rename errorWhenMetricValuesEmpty -> errorWhenNullValues Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * use getParameterFromConfigV2 to read config for errorWhenNullValues Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * add e2e for error state for cw, and improve e2e for min values for cw Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * remove erroneous print statement Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * remove unused vars Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * rename errorWhenMetricValuesEmpty -> ignoreNullValues Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * move towards shared package for e2e aws Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * minMetricValue optionality based on ignoreNullValues Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fail fast Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * Update tests/scalers/aws/aws_cloudwatch_ignore_null_values_false/aws_cloudwatch_ignore_null_values_false_test.go Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * Apply suggestions from code review Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fail fast Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fix broken new line Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fix broken new lines Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * assert no scaling changes in e2e, and set false for required in minMetricValue Signed-off-by: robpickerill <r.pickerill@gmail.com> * fix ci checks Signed-off-by: robpickerill <r.pickerill@gmail.com> * Update tests/scalers/aws/aws_cloudwatch_min_metric_value/aws_cloudwatch_min_metric_value_test.go fix invalid check Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> * fix merge conflicts Signed-off-by: robpickerill <r.pickerill@gmail.com> * fix e2e package names Signed-off-by: robpickerill <r.pickerill@gmail.com> --------- Signed-off-by: Rob Pickerill <r.pickerill@gmail.com> Signed-off-by: robpickerill <r.pickerill@gmail.com> Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Jorge Turrado <jorge.turrado@scrm.lidl>

robpickerill requested a review from a team as a code owner March 28, 2024 15:03

add errorWhenMetricValueEmpty

53ea3fd

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

robpickerill force-pushed the cloudwatch-error-when-empty-metrics branch from 13c0fec to 53ea3fd Compare March 28, 2024 18:07

robpickerill added 2 commits March 28, 2024 19:09

fix golangci-lint

4174e93

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

improve error message for empty results

8e41311

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

robpickerill force-pushed the cloudwatch-error-when-empty-metrics branch 3 times, most recently from 495c201 to 8e41311 Compare March 29, 2024 18:38

add error when empty metric values to changelog

b0d4e84

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

robpickerill force-pushed the cloudwatch-error-when-empty-metrics branch from 584eba6 to b0d4e84 Compare March 29, 2024 19:01

JorTurFer reviewed Mar 30, 2024

View reviewed changes

pkg/scalers/aws_cloudwatch_scaler.go Outdated Show resolved Hide resolved

robpickerill added 3 commits March 31, 2024 16:39

rename errorWhenMetricValuesEmpty -> errorWhenNullValues

7a5f617

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

use getParameterFromConfigV2 to read config for errorWhenNullValues

96a2254

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

Merge branch 'main' into cloudwatch-error-when-empty-metrics

476b4e5

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

robpickerill changed the title ~~add errorWhenMetricValueEmpty~~ add errorWhenNullValues Mar 31, 2024

robpickerill mentioned this pull request Apr 1, 2024

Move to getParameterFromConfigV2 for input, excluding auth #5639

Closed

7 tasks

robpickerill changed the title ~~add errorWhenNullValues~~ add errorWhenNullValues for AWS CloudWatch Scaler Apr 1, 2024

robpickerill force-pushed the cloudwatch-error-when-empty-metrics branch 2 times, most recently from 53fac56 to 0ef98c5 Compare April 4, 2024 17:26

add e2e for error state for cw, and improve e2e for min values for cw

0b61f97

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

robpickerill force-pushed the cloudwatch-error-when-empty-metrics branch from 0ef98c5 to 0b61f97 Compare April 4, 2024 17:28

robpickerill added 2 commits April 4, 2024 18:34

remove erroneous print statement

c6dc524

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

remove unused vars

0dc9b77

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

robpickerill force-pushed the cloudwatch-error-when-empty-metrics branch from 68a1c71 to 0dc9b77 Compare April 4, 2024 17:44

robpickerill commented Apr 4, 2024

View reviewed changes

...calers/aws/aws_cloudwatch_min_value_null_values/aws_cloudwatch_min_value_null_values_test.go Outdated Show resolved Hide resolved

rename errorWhenMetricValuesEmpty -> ignoreNullValues

1f70736

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

robpickerill changed the title ~~add errorWhenNullValues for AWS CloudWatch Scaler~~ add ignoreNullValues for AWS CloudWatch Scaler Apr 11, 2024

move towards shared package for e2e aws

185db21

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

robpickerill commented Apr 13, 2024

View reviewed changes

pkg/scalers/aws_cloudwatch_scaler.go Outdated Show resolved Hide resolved

Merge branch 'main' into cloudwatch-error-when-empty-metrics

ddd9f63

robpickerill mentioned this pull request Jun 8, 2024

Default Values for IgnoreNullValue for AWS Cloudwatch Scaler hides errors - V3 Fix #5873

Closed

fix merge conflicts

2e76a9b

Signed-off-by: robpickerill <r.pickerill@gmail.com>

robpickerill force-pushed the cloudwatch-error-when-empty-metrics branch from 731d45c to 2e76a9b Compare June 8, 2024 15:37

zroubalik reviewed Jun 10, 2024

View reviewed changes

fix e2e package names

43a2c3f

Signed-off-by: robpickerill <r.pickerill@gmail.com>

robpickerill force-pushed the cloudwatch-error-when-empty-metrics branch from 48ccb91 to 43a2c3f Compare June 10, 2024 09:06

stale bot added the stale All issues that are marked as stale due to inactivity label Aug 9, 2024

stale bot removed the stale All issues that are marked as stale due to inactivity label Aug 12, 2024

Merge branch 'main' into cloudwatch-error-when-empty-metrics

a241593

Signed-off-by: Rob Pickerill <r.pickerill@gmail.com>

JorTurFer approved these changes Aug 19, 2024

View reviewed changes

Merge branch 'main' into cloudwatch-error-when-empty-metrics

8e436c3

Merge branch 'main' into cloudwatch-error-when-empty-metrics

d55be16

JorTurFer reviewed Aug 20, 2024

View reviewed changes

pkg/scalers/aws_cloudwatch_scaler.go Show resolved Hide resolved

JorTurFer mentioned this pull request Aug 20, 2024

add ignoreNullValues for AWS CloudWatch Scaler kedacore/keda-docs#1346

Merged

1 task

JorTurFer merged commit 0771b35 into kedacore:main Aug 20, 2024
26 checks passed

robpickerill deleted the cloudwatch-error-when-empty-metrics branch August 21, 2024 06:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add ignoreNullValues for AWS CloudWatch Scaler #5635

add ignoreNullValues for AWS CloudWatch Scaler #5635

robpickerill commented Mar 28, 2024 •

edited

Loading

robpickerill commented Apr 3, 2024

robpickerill commented Jun 7, 2024

zroubalik commented Jun 7, 2024

robpickerill commented Jun 8, 2024

zroubalik commented Jun 10, 2024 •

edited by github-actions bot

Loading

zroubalik left a comment

robpickerill commented Jun 10, 2024 •

edited

Loading

zroubalik commented Jun 10, 2024 •

edited by github-actions bot

Loading

stale bot commented Aug 9, 2024

JorTurFer commented Aug 12, 2024 •

edited by github-actions bot

Loading

robpickerill commented Aug 12, 2024

JorTurFer left a comment

robpickerill commented Aug 20, 2024

JorTurFer commented Aug 20, 2024 •

edited by github-actions bot

Loading

add ignoreNullValues for AWS CloudWatch Scaler #5635

add ignoreNullValues for AWS CloudWatch Scaler #5635

Conversation

robpickerill commented Mar 28, 2024 • edited Loading

Checklist

robpickerill commented Apr 3, 2024

robpickerill commented Jun 7, 2024

zroubalik commented Jun 7, 2024

robpickerill commented Jun 8, 2024

zroubalik commented Jun 10, 2024 • edited by github-actions bot Loading

zroubalik left a comment

Choose a reason for hiding this comment

robpickerill commented Jun 10, 2024 • edited Loading

zroubalik commented Jun 10, 2024 • edited by github-actions bot Loading

stale bot commented Aug 9, 2024

JorTurFer commented Aug 12, 2024 • edited by github-actions bot Loading

robpickerill commented Aug 12, 2024

JorTurFer left a comment

Choose a reason for hiding this comment

robpickerill commented Aug 20, 2024

JorTurFer commented Aug 20, 2024 • edited by github-actions bot Loading

robpickerill commented Mar 28, 2024 •

edited

Loading

zroubalik commented Jun 10, 2024 •

edited by github-actions bot

Loading

robpickerill commented Jun 10, 2024 •

edited

Loading

zroubalik commented Jun 10, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Aug 12, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Aug 20, 2024 •

edited by github-actions bot

Loading