Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add RabbitMQ publishRate trigger #1653

Merged
merged 5 commits into from
Mar 9, 2021
Merged

Add RabbitMQ publishRate trigger #1653

merged 5 commits into from
Mar 9, 2021

Conversation

rwkarg
Copy link
Contributor

@rwkarg rwkarg commented Mar 3, 2021

Signed-off-by: Ryan Karg rwkarg@gmail.com

Added an additional trigger for RabbitMQ scaling on publishRate.

This allows for higher throughput workloads to scale based on the incoming message rate. This is useful because the message backlog does not often correlate with the desired instance count.

Both publishRate and queueLength triggers can be specified on a single ScaledObject. I would imagine the normal case would be to use both of these triggers.

Checklist

  • Commits are signed with Developer Certificate of Origin (DCO - learn more)
  • Tests have been added
    [ ] A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
  • A PR is opened to update the documentation on (repo) (if applicable)
  • Changelog has been updated

Fixes #1643

rwkarg added a commit to rwkarg/keda-docs that referenced this pull request Mar 3, 2021
Signed-off-by: Karg <rkarg@blizzard.com>
@rwkarg
Copy link
Contributor Author

rwkarg commented Mar 4, 2021

Comparison of two different but similar workloads over 24 hrs. The orange line on the graph correlates with the instance count. Both workloads are in the 1k-3k/sec. range and have smooth rates of change over the day so the instance count would expect to be smooth as well to match that.

Using only queueLength it bounces bewteen min. instances and a too-high instance count after a backlog builds up:
image

Using publishRate (as well as queueLength) it stays steady:
image

In the second example, we actually had an impact on a dependency that cause a backlog to build up so the queueLength trigger eventually resulted in a higher instance count that you can see in the two spikes at the beginning of the graph.

Copy link
Contributor

@ahmelsayed ahmelsayed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @rwkarg

Signed-off-by: Karg <rkarg@blizzard.com>
Signed-off-by: Karg <rkarg@blizzard.com>
@zroubalik zroubalik changed the title Add publishRate trigger Add RabbitMQ publishRate trigger Mar 8, 2021
Copy link
Member

@zroubalik zroubalik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @rwkarg !

@zroubalik
Copy link
Member

Let's merge this once docs PR is finalized.

@tomkerkhove tomkerkhove merged commit 896ff4d into kedacore:main Mar 9, 2021
@tomkerkhove
Copy link
Member

Thanks a ton @rwkarg!

@rwkarg rwkarg deleted the patch-2 branch March 9, 2021 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RabbitMQ rate based trigger (Published/sec.)
5 participants