feat: Provide scaler for Amazon managed service for Prometheus #5373

sguruvar · 2024-01-15T02:02:15Z

Provide a description of what has been changed

Checklist

When introducing a new scaler, I agree with the scaling governance policy
I have verified that my change is according to the deprecations & breaking changes policy
Tests have been added
Changelog has been updated and is aligned with our changelog requirements
A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
A PR is opened to update the documentation on (repo) (if applicable)
Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #2214

github-actions · 2024-01-15T02:02:26Z

Thank you for your contribution! 🙏 We will review your PR as soon as possible.

While you are waiting, make sure to:

Add an entry in our changelog in alphabetical order and link related issue
Update the documentation, if needed
Add unit & e2e tests for your changes
GitHub checks are passing
Is the DCO check failing? Here is how you can fix DCO issues

Learn more about:

Our contribution guide

pkg/scalers/aws_sigv4.go

pkg/scalers/aws_sigv4_test.go

.../scalers/aws/aws_managed_prometheus_pod_identity/aws_managed_prometheus_pod_identity_test.go

pkg/scalers/aws_sigv4.go

pkg/scalers/aws_sigv4_test.go

JorTurFer · 2024-01-15T07:17:25Z

Also, the changelog has to be updated

JorTurFer · 2024-01-15T07:21:08Z

And last but not least, please fix DCO check. You can get more info about how to solve it clicking in the check link

pkg/scalers/aws_sigv4.go

zroubalik

We should also update documentation for 2.13: https://github.com/kedacore/keda-docs

JorTurFer

You have resolved the conversations but after the push I still see the code changes. Maybe you didn't commit the change by error

sguruvar · 2024-01-15T20:40:34Z

You have resolved the conversations but after the push I still see the code changes. Maybe you didn't commit the change by error

done

Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Signed-off-by: geoffrey1330 <israelgeoffrey13@gmail.com> Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es> Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Jorge Turrado <jorge.turrado@scrm.lidl> Signed-off-by: Geoffrey Israel <israelgeoffrey13@gmail.com> Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Co-authored-by: Jorge Turrado <jorge.turrado@scrm.lidl> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

…re#5369) Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

* chore: fix jetstream flaky test Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es> * use specific instances instead of default client Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es> * Update mock responses Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es> --------- Signed-off-by: Jorge Turrado <jorge_turrado@hotmail.es> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Signed-off-by: Zbynek Roubalik <zroubalik@gmail.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Signed-off-by: Siva Guruvareddiar <gurusiva@gmail.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Siva Guruvareddiar <1725781+sguruvar@users.noreply.github.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

…ged_prometheus_pod_identity_test.go Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Siva Guruvareddiar <1725781+sguruvar@users.noreply.github.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Signed-off-by: Siva Guruvareddiar <gurusiva@gmail.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Siva Guruvareddiar <1725781+sguruvar@users.noreply.github.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

pkg/scalers/aws/aws_sigv4.go

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

JorTurFer · 2024-01-17T06:54:44Z

/run-e2e aws
Update: You can check the progress here

JorTurFer

new e2e are failing (with a panic) and I'd say that it's happening at workspace creation time

.../scalers/aws/aws_managed_prometheus_pod_identity/aws_managed_prometheus_pod_identity_test.go

tests/scalers/aws/aws_managed_prometheus/aws_managed_prometheus_test.go

.../scalers/aws/aws_managed_prometheus_pod_identity/aws_managed_prometheus_pod_identity_test.go

Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>

JorTurFer · 2024-01-17T07:59:22Z

/run-e2e aws
Update: You can check the progress here

JorTurFer · 2024-01-17T09:53:10Z

Error logs from e2e test are really weird:

=== RUN   TestScaler
    aws_managed_prometheus_test.go:138: --- setting up ---
--- FAIL: TestScaler (0.42s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x10f5ac8]

goroutine 132 [running]:
testing.tRunner.func1.2({0x126caa0, 0x2327760})
	/usr/local/go/src/testing/testing.go:1545 +0x1c4
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1548 +0x360
panic({0x126caa0?, 0x2327760?})
	/usr/local/go/src/runtime/panic.go:914 +0x218
command-line-arguments_test.TestScaler(0x4000475860)
	/__w/keda/keda/tests/scalers/aws/aws_managed_prometheus/aws_managed_prometheus_test.go:143 +0x178
testing.tRunner(0x4000475860, 0x15808b0)
	/usr/local/go/src/testing/testing.go:1595 +0xe8
created by testing.(*T).Run in goroutine 1
	/usr/local/go/src/testing/testing.go:1648 +0x33c
FAIL	command-line-arguments	0.491s
FAIL

It looks that the test can't start for any reason, but the issue looks in the test script side, not in KEDA. Could you take a look?

sguruvar · 2024-01-17T12:32:23Z

Error logs from e2e test are really weird:

=== RUN   TestScaler
    aws_managed_prometheus_test.go:138: --- setting up ---
--- FAIL: TestScaler (0.42s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x10f5ac8]

goroutine 132 [running]:
testing.tRunner.func1.2({0x126caa0, 0x2327760})
	/usr/local/go/src/testing/testing.go:1545 +0x1c4
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1548 +0x360
panic({0x126caa0?, 0x2327760?})
	/usr/local/go/src/runtime/panic.go:914 +0x218
command-line-arguments_test.TestScaler(0x4000475860)
	/__w/keda/keda/tests/scalers/aws/aws_managed_prometheus/aws_managed_prometheus_test.go:143 +0x178
testing.tRunner(0x4000475860, 0x15808b0)
	/usr/local/go/src/testing/testing.go:1595 +0xe8
created by testing.(*T).Run in goroutine 1
	/usr/local/go/src/testing/testing.go:1648 +0x33c
FAIL	command-line-arguments	0.491s
FAIL

It looks that the test can't start for any reason, but the issue looks in the test script side, not in KEDA. Could you take a look?

Am unable to reproduce this. Test goes fine in my local when I try GOOS="darwin" go test -v -tags e2e -timeout 2m scalers/aws/aws_managed_prometheus/aws_managed_prometheus_test.go and I could be able to see the AMP workspace created and removed after the test is done. I set the TF_AWS_ACCESS_KEY, TF_AWS_SECRET_KEY and TF_AWS_REGION.. Is there a good way to replicate this?

pkg/scalers/aws/aws_sigv4.go

pkg/scalers/aws/aws_sigv4_test.go

zroubalik · 2024-01-17T18:06:12Z

Error logs from e2e test are really weird:
=== RUN   TestScaler
    aws_managed_prometheus_test.go:138: --- setting up ---
--- FAIL: TestScaler (0.42s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x10f5ac8]

goroutine 132 [running]:
testing.tRunner.func1.2({0x126caa0, 0x2327760})
	/usr/local/go/src/testing/testing.go:1545 +0x1c4
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1548 +0x360
panic({0x126caa0?, 0x2327760?})
	/usr/local/go/src/runtime/panic.go:914 +0x218
command-line-arguments_test.TestScaler(0x4000475860)
	/__w/keda/keda/tests/scalers/aws/aws_managed_prometheus/aws_managed_prometheus_test.go:143 +0x178
testing.tRunner(0x4000475860, 0x15808b0)
	/usr/local/go/src/testing/testing.go:1595 +0xe8
created by testing.(*T).Run in goroutine 1
	/usr/local/go/src/testing/testing.go:1648 +0x33c
FAIL	command-line-arguments	0.491s
FAIL
It looks that the test can't start for any reason, but the issue looks in the test script side, not in KEDA. Could you take a look?
Am unable to reproduce this. Test goes fine in my local when I try GOOS="darwin" go test -v -tags e2e -timeout 2m scalers/aws/aws_managed_prometheus/aws_managed_prometheus_test.go and I could be able to see the AMP workspace created and removed after the test is done. I set the TF_AWS_ACCESS_KEY, TF_AWS_SECRET_KEY and TF_AWS_REGION.. Is there a good way to replicate this?

even if you are not able to reproduce, the test shouldn't end with panic, we should add a check there and fire proper error anyway.

tests/scalers/aws/aws_managed_prometheus/aws_managed_prometheus_test.go

.../scalers/aws/aws_managed_prometheus_pod_identity/aws_managed_prometheus_pod_identity_test.go

Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>

JorTurFer · 2024-01-17T20:46:15Z

/run-e2e aws
Update: You can check the progress here

JorTurFer · 2024-01-17T21:02:13Z

even if you are not able to reproduce, the test shouldn't end with panic, we should add a check there and fire proper error anyway.

I think that my previous code suggestion was wrong and that's why the test panic before any assertion error xD

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

sguruvar · 2024-01-17T21:16:51Z

even if you are not able to reproduce, the test shouldn't end with panic, we should add a check there and fire proper error anyway.

I think that my previous code suggestion was wrong and that's why the test panic before any assertion error xD

Looks like permission is missing to create the AMP workspace: operation error amp: CreateWorkspace, https response error StatusCode: 40***, RequestID: defa5098-e960-4***4-b684-a0c46dfdf6***d, AccessDeniedException: User: arn:aws:iam::***:user/e***e-test-user is not authorized to perform: aps:CreateWorkspace on resource: arn:aws:aps:***:***:/workspaces... can you add the permission to create and delete for this e2e-test-user ?

JorTurFer · 2024-01-17T22:03:48Z

/run-e2e aws
Update: You can check the progress here

JorTurFer · 2024-01-17T22:21:52Z

It seems that it works, so once all the commits are addressed, we can merge this

sguruvar · 2024-01-17T23:27:28Z

It seems that it works, so once all the commits are addressed, we can merge this

Thanks.. I believe all the comments are addressed. Can you help merge?

zroubalik

Massive nit, please use go convetion when writing docs on fucntions, thanks!

pkg/scalers/aws/aws_sigv4.go

Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com> Signed-off-by: Siva Guruvareddiar <1725781+sguruvar@users.noreply.github.com>

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

zroubalik · 2024-01-18T13:20:24Z

/run-e2e aws
Update: You can check the progress here

sguruvar requested a review from a team as a code owner January 15, 2024 02:02

github-actions bot added the requires-pr-review label Jan 15, 2024

sguruvar mentioned this pull request Jan 15, 2024

Scaler for Amazon managed service for Prometheus #2214

Closed

JorTurFer reviewed Jan 15, 2024

View reviewed changes

pkg/scalers/aws_sigv4_test.go Outdated Show resolved Hide resolved

zroubalik reviewed Jan 15, 2024

View reviewed changes

pkg/scalers/aws_sigv4.go Outdated Show resolved Hide resolved

zroubalik reviewed Jan 15, 2024

View reviewed changes

zroubalik mentioned this pull request Jan 15, 2024

Release: 2.13.0 #5275

Closed

25 tasks

zroubalik removed the requires-pr-review label Jan 15, 2024

JorTurFer reviewed Jan 15, 2024

View reviewed changes

dttung2905 and others added 11 commits January 15, 2024 15:25

chore: add large lag threshold test for Kafka scalers (kedacore#5361)

d5bdd71

Signed-off-by: dttung2905 <ttdao.2015@accountancy.smu.edu.sg> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

chore: bump Go to 1.21.6 (kedacore#5370)

ccccc67

Signed-off-by: Zbynek Roubalik <zroubalik@gmail.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

https://github.com/kedacore/keda/issues/2214

5c7c7f2

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

https://github.com/kedacore/keda/issues/2214

332e51e

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

https://github.com/kedacore/keda/issues/2214

1ad5671

Signed-off-by: Siva Guruvareddiar <gurusiva@gmail.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Update pkg/scalers/aws_sigv4.go

ba8f6ac

Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Siva Guruvareddiar <1725781+sguruvar@users.noreply.github.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

https://github.com/kedacore/keda/issues/2214

c29fb15

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

sguruvar force-pushed the main branch from 29252b6 to c29fb15 Compare January 15, 2024 21:27

Siva Guruvareddiar and others added 4 commits January 15, 2024 15:30

https://github.com/kedacore/keda/issues/2214

4b53d8e

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

https://github.com/kedacore/keda/issues/2214

68e52fe

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

https://github.com/kedacore/keda/issues/2214

ebaa539

Signed-off-by: Siva Guruvareddiar <gurusiva@gmail.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Update pkg/scalers/aws_sigv4.go

4bb9d44

Co-authored-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es> Signed-off-by: Siva Guruvareddiar <1725781+sguruvar@users.noreply.github.com> Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

JorTurFer reviewed Jan 16, 2024

View reviewed changes

pkg/scalers/aws/aws_sigv4.go Outdated Show resolved Hide resolved

https://github.com/kedacore/keda/issues/2214

f73c7aa

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

JorTurFer reviewed Jan 17, 2024

View reviewed changes

.../scalers/aws/aws_managed_prometheus_pod_identity/aws_managed_prometheus_pod_identity_test.go Outdated Show resolved Hide resolved

tests/scalers/aws/aws_managed_prometheus/aws_managed_prometheus_test.go Outdated Show resolved Hide resolved

JorTurFer reviewed Jan 17, 2024

View reviewed changes

tests/scalers/aws/aws_managed_prometheus/aws_managed_prometheus_test.go Outdated Show resolved Hide resolved

.../scalers/aws/aws_managed_prometheus_pod_identity/aws_managed_prometheus_pod_identity_test.go Outdated Show resolved Hide resolved

Apply suggestions from code review

c3bb6ab

Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>

zroubalik reviewed Jan 17, 2024

View reviewed changes

JorTurFer reviewed Jan 17, 2024

View reviewed changes

Apply suggestions from code review

268fe43

Signed-off-by: Jorge Turrado Ferrero <Jorge_turrado@hotmail.es>

sguruvar added 2 commits January 17, 2024 15:12

https://github.com/kedacore/keda/issues/2214

bcafe6f

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Merge branch 'main' of https://github.com/sguruvar/keda

5a5b8b2

zroubalik reviewed Jan 18, 2024

View reviewed changes

sguruvar and others added 5 commits January 18, 2024 06:59

Update pkg/scalers/aws/aws_sigv4.go

b16e7fc

Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com> Signed-off-by: Siva Guruvareddiar <1725781+sguruvar@users.noreply.github.com>

Update pkg/scalers/aws/aws_sigv4.go

d86d3e5

Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com> Signed-off-by: Siva Guruvareddiar <1725781+sguruvar@users.noreply.github.com>

Update pkg/scalers/aws/aws_sigv4.go

a564efd

Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com> Signed-off-by: Siva Guruvareddiar <1725781+sguruvar@users.noreply.github.com>

https://github.com/kedacore/keda/issues/2214

3376b82

Signed-off-by: Siva Guruvareddiar <sivagurunath@gmail.com>

Merge branch 'main' into main

31d2afa

zroubalik approved these changes Jan 18, 2024

View reviewed changes

zroubalik merged commit 49b0fd0 into kedacore:main Jan 18, 2024
20 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Provide scaler for Amazon managed service for Prometheus #5373

feat: Provide scaler for Amazon managed service for Prometheus #5373

sguruvar commented Jan 15, 2024 •

edited by zroubalik

Loading

github-actions bot commented Jan 15, 2024

JorTurFer commented Jan 15, 2024 •

edited

Loading

JorTurFer commented Jan 15, 2024

zroubalik left a comment

JorTurFer left a comment

sguruvar commented Jan 15, 2024 •

edited

Loading

JorTurFer commented Jan 17, 2024 •

edited by github-actions bot

Loading

JorTurFer left a comment

JorTurFer commented Jan 17, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jan 17, 2024

sguruvar commented Jan 17, 2024

zroubalik commented Jan 17, 2024

JorTurFer commented Jan 17, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jan 17, 2024

sguruvar commented Jan 17, 2024 •

edited

Loading

JorTurFer commented Jan 17, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jan 17, 2024

sguruvar commented Jan 17, 2024 •

edited

Loading

zroubalik left a comment

zroubalik commented Jan 18, 2024 •

edited by github-actions bot

Loading

feat: Provide scaler for Amazon managed service for Prometheus #5373

feat: Provide scaler for Amazon managed service for Prometheus #5373

Conversation

sguruvar commented Jan 15, 2024 • edited by zroubalik Loading

Checklist

github-actions bot commented Jan 15, 2024

JorTurFer commented Jan 15, 2024 • edited Loading

JorTurFer commented Jan 15, 2024

zroubalik left a comment

Choose a reason for hiding this comment

JorTurFer left a comment

Choose a reason for hiding this comment

sguruvar commented Jan 15, 2024 • edited Loading

JorTurFer commented Jan 17, 2024 • edited by github-actions bot Loading

JorTurFer left a comment

Choose a reason for hiding this comment

JorTurFer commented Jan 17, 2024 • edited by github-actions bot Loading

JorTurFer commented Jan 17, 2024

sguruvar commented Jan 17, 2024

zroubalik commented Jan 17, 2024

JorTurFer commented Jan 17, 2024 • edited by github-actions bot Loading

JorTurFer commented Jan 17, 2024

sguruvar commented Jan 17, 2024 • edited Loading

JorTurFer commented Jan 17, 2024 • edited by github-actions bot Loading

JorTurFer commented Jan 17, 2024

sguruvar commented Jan 17, 2024 • edited Loading

zroubalik left a comment

Choose a reason for hiding this comment

zroubalik commented Jan 18, 2024 • edited by github-actions bot Loading

sguruvar commented Jan 15, 2024 •

edited by zroubalik

Loading

JorTurFer commented Jan 15, 2024 •

edited

Loading

sguruvar commented Jan 15, 2024 •

edited

Loading

JorTurFer commented Jan 17, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jan 17, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jan 17, 2024 •

edited by github-actions bot

Loading

sguruvar commented Jan 17, 2024 •

edited

Loading

JorTurFer commented Jan 17, 2024 •

edited by github-actions bot

Loading

sguruvar commented Jan 17, 2024 •

edited

Loading

zroubalik commented Jan 18, 2024 •

edited by github-actions bot

Loading