Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Prometheus metric for measuring the scalable object loop processing deviation #4702

Closed
JorTurFer opened this issue Jun 16, 2023 · 0 comments · Fixed by #4703
Closed
Assignees
Labels
feature-request All issues for new features that have not been committed to needs-discussion

Comments

@JorTurFer
Copy link
Member

JorTurFer commented Jun 16, 2023

Proposal

@MrsDaehin and I are working on #4411 but we have noticed that there isn't any metric
(in the operator) that we can use for measuring how KEDA is performing in terms of response time.

As the majority of the work (during the normal operation) is done here:

for {
tmr := time.NewTimer(pollingInterval)
h.checkScalers(ctx, scalableObject, scalingMutex)
select {
case <-tmr.C:
tmr.Stop()
case <-ctx.Done():
logger.V(1).Info("Context canceled")
err := h.ClearScalersCache(ctx, scalableObject)
if err != nil {
logger.Error(err, "error clearing scalers cache")
}
tmr.Stop()
return
}
}
}

I think that we can add a metric with the difference between the calculated time for next execution (now + pollingInterval) and the real execution time. If the time is close to 0 it means that we are fine, but if the new loop starting is too much later than the expected, we aren't executing the control loop on the expected time, which probably is because the CPU is overloaded

Use-Case

This metric can give as a measure about how the load is impacting in the scalable object processing loop measuing the time between loops

Is this a feature you are interested in implementing yourself?

Yes

Anything else?

No response

@JorTurFer JorTurFer added needs-discussion feature-request All issues for new features that have not been committed to labels Jun 16, 2023
@JorTurFer JorTurFer self-assigned this Jun 16, 2023
@JorTurFer JorTurFer changed the title Add a Promethean metric for measuring the scalable object loop processing deviation Add a Prometheus metric for measuring the scalable object loop processing deviation Jun 16, 2023
@JorTurFer JorTurFer moved this from To Triage to In Progress in Roadmap - KEDA Core Jun 16, 2023
@github-project-automation github-project-automation bot moved this from In Progress to Ready To Ship in Roadmap - KEDA Core Jun 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request All issues for new features that have not been committed to needs-discussion
Projects
Archived in project
1 participant