RabbitMQ rate based trigger (Published/sec.)

[rwkarg]

Proposal

Add a trigger based on the Published/sec. metric on a queue.

Use-Case

We have a workload that has a Publishing rate that varies between 1k and 3k/sec. When we are keeping up with the throughput, the Ready count on the queue is 0 regardless of the Publishing rate. Even using the Unacked count, that stay around a few hundred, independent of the Publishing rate. While handling 3k/sec. requires three times the instances as 1k/sec., the metrics I can choose now look identical. I have to wait for scaling to reach min. instances (since Ready == 0), then wait for a backlog to build up (since we're running too few instances to keep up), then have the instances scale up to something (since Ready > 0 now) and keep scaling up until we're at a high enough instance count to work through the backlog, reach Ready == 0 again and scale back down to min. instances.

This keeps bouncing between min. instances and some scaled up count over and over again.

I would expect to be able to configure two triggers

• X messages/sec. per instance to scale against the Published/sec. rate on the queue (to handle the steady-state happy path)
• Y backlog (Ready/Unacked backlog) per instance (to handle cases where a dependency slowed or stopped processing for a while or otherwise failed to keep up)

[rwkarg]

I can take a look at this.

[rwkarg]

Draft PR here: #1648

I appear to be having issues connecting to the http API over https so that's holding up verifying this locally.

[zroubalik]

Makes sense!