CLI Report of Delayed Message Counts

[RyanFrantz]

I would like to be able to report the total count of delayed messages within an exchange from the command line. Additionally, it would be great if we could display the count of delayed messages based on their intended queue.

I had a conversation with @michaelklishin about support in the CLI for finding the total count of delayed messages within an exchange. He helped me with an eval statement like the following example:

rabbitmqctl eval 'rabbit_delayed_message:messages_delayed(rabbit_misc:r(<<"vhost_name">>, exchange, <<"exchange_name">>)).'

I tested this running RabbitMQ 3.7.13 in a Docker container that I had just started:

$ docker-compose exec salsa-rabbit rabbitmqctl version
3.7.13

$ docker-compose exec salsa-rabbit rabbitmqctl eval 'rabbit_delayed_message:messages_delayed(rabbit_misc:r(<<"/">>, exchange, <<"salsa">>)).'
0

I expected the total count to be 0 since I had just launched the Rabbit instance and had no messages in it.

However, when I send a delayed message (delayed for a few hours) to Rabbit and run the same eval command, it still returns 0. I've confirmed my exchange exists and is of type x-delayed-message:

$ docker-compose exec salsa-rabbit rabbitmqctl list_exchanges
Listing exchanges for vhost / ...
name    type
amq.direct      direct
amq.match       headers
amq.headers     headers
amq.rabbitmq.trace      topic
amq.topic       topic
amq.fanout      fanout
salsa   x-delayed-message  <<<
        direct

NOTE: When I run the command for an exchange that doesn't actually exist, I get 0 as well, rather than an error stating the exchange is invalid:

$ docker-compose exec salsa-rabbit rabbitmqctl eval 'rabbit_delayed_message:messages_delayed(rabbit_misc:r(<<"/">>, exchange, <<"an_exchange_that_does_not_exist">>)).'
0

[michaelklishin]

This wasn't a part of #3 because CLI command support in plugins was introduced later in the 3.7.0 development cycle. Makes sense 👍

[adinapoli]

Hey folks,

thanks for this amazing plugin, it's really appreciated. @michaelklishin I am using the "latest and greatest" version available, i.e. 3.8.0, and I still experience the behaviour described by @RyanFrantz , i.e. the eval command always returns 0 for me. What am I doing wrong?

Furthermore, I don't see any trace of the metric (as described here). Even though my exchange is clearly the right one, I don't see anything in the dashboard:

And if I click on the +/- signs I don't see any "delay-related" column I can enable:

Am I missing something totally obvious here?

I am on a Mac OS X 10.15.6 , I have installed rabbitmq via brew and I am currently using version 3.8.3 with Erlang version 22.3.

[adinapoli]

Furthermore, I don't see any trace of the metric (as described here). Even though my exchange is clearly the right one, I don't see anything in the dashboard:

Partially replying to myself: I have upgraded to the latest RMQ and now I can at least see the counter in the page available by clicking on the desired exchange:

[michaelklishin]

This is the metric we expose. This issue is about adding a CLI tools command which would expose the same metric one way or another.

[zyh-future]

So how do we get the number of messages backlog in exchange?

[marguspeet]

It seems that in case of the latest plugin version (3.10.2) the count in the UI has disappeared.

[artkom911]

rabbitmq-delayed-message-exchange 3.10.2
RabbitMQ 3.10.0
Erlang 24.3.4

$ curl -u user:pwd http://localhost:15672/api/exchanges/%2F/example_exchange_delayed_0
{"arguments":{"x-delayed-type":"direct"},"auto_delete":true,"durable":true,"incoming":[],"internal":false,"message_stats":{"publish_out":1,"publish_out_details":{"rate":0.0}},"messages_delayed":10,"name":"example_exchange_delayed_0","outgoing":[],"type":"x-delayed-message","user_who_performed_action":"user","vhost":"/"}

http://127.0.0.1:15672/#/exchanges/%2F/example_exchange_delayed_0
messages delayed: 10

works fine

[michaelklishin]

@artkom911 thank you for triaging. It may be that the version of the plugin does not match that of the RabbitMQ release series (for example, the latest version of the plugin is used with 3.9).

Closing unless new evidence and a set of steps to reproduce emerges.

[artkom911]

btw, is there any chance we can add this information (messages_delayed) to the response on a "get queue" status request also?
http://127.0.0.1:15672/#/api/queues/%2F/example_queue_delayed_0

[artkom911]

@michaelklishin

Unfortunately, you did not want to listen to the arguments in your responses and closed the conversation. #228

The management UI that you mentioned uses the same http api, so it is completely useless in this case.

I never said that I delayed messages for months. As a matter of fact, it is mostly hours, even minutes, sometimes days. I simply have lots of them and planned to have more in feature, but hit this issue.
Also, in my case, messages don't have any information except dataId (uuid v4, if this matters). The actual data are already stored in DB.

I do have other options, not rabbit-based, but this solution was easy to try and worth the shot.

I do realize that your lack of experience building high-load systems affects your decisions and point of view. Still, I'm a bit surprised by such a reaction.

Anyway, thx for your time.