RabbitMQ regex - No matching metrics found

[alegmal]

Report

regex with rabbitmq scaler is unable to get metrics - error is: No matching metrics found (screenshots are provided).
the httpConnectionString environment variable in deployment takes its value from a secret and its of this format:
http://username:password@:/host

Strangly it also complains in keda-metrics pod logs about being unable to resolve secret "for env AUTHZ_CLIENT_ID". AUTHZ_CLIENT_ID environment variable that is not mentioned and not related to rabbit / scaling so I'm confused why its in the logs.

maybe I misconfigured something?
I was following this documentation:
https://keda.sh/docs/2.5/scalers/rabbitmq-queue/

Expected Behavior

get queues and scale on sum of messages

Actual Behavior

error getting metrics to scale

Steps to Reproduce the Problem

1. install keda 2.5.0
2. apply rabbitmq scaler with regex

Logs from KEDA operator

E1222 14:15:51.999441       1 provider.go:124] keda_metrics_adapter/provider "msg"="error getting metric for scaler" "error"="error resolving secrets for ScaleTarget: error resolving secret name &SecretKeySelector{LocalObjectReference:LocalObjectReference{Name:piper-auth0,},Key:clientId,Optional:nil,} for env AUTHZ_CLIENT_ID in namespace default"  "scaledObject.Name"="niffler" "scaledObject.Namespace"="default" "scaler"={}
E1222 14:15:51.999498       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"No matching metrics found for s0-rabbitmq--5eniffler-28-3f-21--2a-5b--5derrors-24-29--2a-24"}: No matching metrics found for s0-rabbitmq--5eniffler-28-3f-21--2a-5b--5derrors-24-29--2a-24

KEDA Version

2.5.0

Kubernetes Version

1.20

Platform

Google Cloud

Scaler Details

rabbitmq

Anything else?

No response

[JorTurFer]

hi @alegmal
Are the ScaledObject and the deployment in the same namespace?

[JorTurFer]

it's weird because I think that we don't use any variable AUTHZ_CLIENT_ID in the code :/
is it a variable in your workload?

[alegmal]

@JorTurFer

1. just double checked and its in the same namespace.
2. yes - this is env of the workload it should scale (niffler). As I wrote, I have no clue why its in the logs as the hostFromEnv is set to "MQ_CONNECTION_STRING" env

[JorTurFer]

Hi @alegmal
It's weird, I have at least 20-25 SO scaling in base of RabbitMQ with regex :(
Could you try using TriggerAuthentication instead of getting the value from the pod?
Please, to ensure that this problem is not affecting you, be sure that you update the ScaledObject instead of deleting and creating it again, or if you cannot do that, modify the name of the SO (one character is enough)

[alegmal]

@JorTurFer thanks for quick responses
I checked and generation of SO seems to be updated with new changes.
I created a new SO with different name and same regex configuration and I got the same erors.

I'll try triggerauthentication and update.

[alegmal]

@JorTurFer unfortunately triggerauthentication did not help solve the issue

E1222 14:15:51.999441       1 provider.go:124] keda_metrics_adapter/provider "msg"="error getting metric for scaler" "error"="error resolving secrets for ScaleTarget: error resolving secret name &SecretKeySelector{LocalObjectReference:LocalObjectReference{Name:piper-auth0,},Key:clientId,Optional:nil,} for env AUTHZ_CLIENT_ID in namespace default"  "scaledObject.Name"="niffler" "scaledObject.Namespace"="default" "scaler"={}

This is the is what puzzles me in this error: LocalObjectReference{Name:piper-auth0,},Key:clientId
We have a secret piper-auth0 with key clientId but nowhere its reffered to in scaledobject as seen in a screenshot.
This also happens when using triggerAuthentication.

Maybe I need to clear some cache or something?

[JorTurFer]

Hi @alegmal
I guess that you already tried updating the SO instead of deleting and creating it, but you could try also a simple restart of KEDA's pods, that will refresh the cache. (I know that it's not the best solution, but I propose it only for troubleshooting stuff)

[alegmal]

@JorTurFer so funny story happened... apparently "kubectl describe scaledobjects.keda.sh " is important in process of debugging... who knew * facepalm *

when using describe here is the problem:
error inspecting rabbitMQ: regex matches more queues than can be recovered at once

I would expect keda-metrics to output error that is as clear and relevant as the one when describing scaledobject

How much can be recovered and can I configure it?

[JorTurFer]

oh 🤦
If the error is that, it should be shown also in the metrics server logs. The default value is 100 ,and you could modify it using the tigger value pageSize

pageSize - This parameter allows setting page size. (Default: 100, Optional, Only applies when useRegex is true)

but be careful because the rabbitmq cluster usually has its own limit ( usually 500).

The parameter is described in the docs :) (v2.5)

[alegmal]

thanks, issue is solved, closing.