Support for scaling custom resources and statefulset

[raulsh]

Based on the docs and issues, there is no way to scale another kind of resources besides Deployments. We need scale rollouts (CRD of Argo Rollouts). Even, we use core KEDA to scale rollouts.

Use-Case

We use Argo Rollouts, so in Keda we can specify the Kind of the resource (https://keda.sh/docs/2.6/concepts/scaling-deployments/). This addon must scale any CRD/Resource with /scale subresource (Statefulset or Argo Rollout resource).

I use rollouts to deploy some HTTP workloads.

Specification

Probably same as core KEDA, with parameters to specify the kind and the apiVersion of CRDs

[tomkerkhove]

Just out of curiosity - How is HTTP scaling required for Argo Rollouts? (I'm not an argo expert)

[raulsh]

Rollouts resources are similar to native Deployment resources (rollout manage replicasets). They are managed by Argo Rollout controller. The difference is that rollouts support Blue/Green and Canary deployments (and other functions). This addon must scale any CRD/Resource with /scale subresource (Statefulset or Argo Rollout resource).

Even rollout resource specs are EXACTLY to the deployment specs: https://argoproj.github.io/argo-rollouts/migrating/#convert-deployment-to-rollout.

[tomkerkhove]

And those rollouts are being consumed over HTTP then? I'm just asking because I'm not sure why this is a problem for the HTTP add-on and why vanilla KEDA can't help you on this as that's supported there.

[raulsh]

Yes... we use to deploy HTTP workloads (APIs, monoliths, etc). But, why the question? That's not the addon purpose? or am I misunderstanding something? (i'm not a english native speaker, sorry)

In fact, this is the perfect solution to our case. Because we have HTTP workloads, but doesnt have a centralized queue to manage request and manage pending requests properly (while scaling workload). Because we have a high dynamic request throughput in our internal services. And I think this is the case that this addon want to cover.

Of course we can use vanilla Keda, but the "Interceptor" design is the solution that fit to our needs. And the main issue for us Is because we are using rollouts instead native deployments haha.

[tomkerkhove]

Yes... we use to deploy HTTP workloads (APIs, monoliths, etc). But, why the question? That's not the addon purpose? or am I misunderstanding something? (i'm not a english native speaker, sorry)

I was just curious about your scenario because my lack of experience with Argo Rollouts :)

In fact, this is the perfect solution to our case. Because we have HTTP workloads, but doesnt have a centralized queue to manage request and manage pending requests properly (while scaling workload). Because we have a high dynamic request throughput in our internal services. And I think this is the case that this addon want to cover.

Of course we can use vanilla Keda, but the "Interceptor" design is the solution that fit to our needs. And the main issue for us Is because we are using rollouts instead native deployments haha.

Then you are in the right place! Let's add this to our features and see when we can add this - Thanks!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[miguelvr]

@tomkerkhove I would like to work on this. From my pass on the code, it's a fairly big change that probably requires a design proposal. What is the process to propose a design for a new feature?

[tomkerkhove]

Sounds like a good idea! We typically use Google Docs so that everyone can add comments (example)

[prakarsh-dt]

I am a maintainer at Devtron and was exploring some kubernetes native tool that could help scale workloads from 0 on getting a HTTP request. I stumbled upon this tool and it looks promising and we have already done our POCs.

I would like to integrate KEDA HTTP Add-on ( HTTPScaledObject ) into Devtron, we would indeed need support for custom resources, particularly Argo Rollout. We are eager to contribute to this project by developing the necessary capabilities and sharing them back with the community.

Since there hasn't been any recent activity on this feature request thread for over 6 months, I would appreciate it if you could provide an update on the current status. It would be helpful if someone could guide me to any existing work that has been done in this direction. I am keen to collaborate and ensure that our integration aligns with the ongoing efforts of the project.

[tomkerkhove]

There was no progress on this one so I think we need a design proposal first before we can get started on this one

[JorTurFer]

Hi @prakarsh-dt
Are you still interested on this? I can take a look and maybe we can integrate them (I don't promise anything), of course, if you are willing to do it from your side and open a PR, we'll be happy

[prakarsh-dt]

Hi @prakarsh-dt Are you still interested on this? I can take a look and maybe we can integrate them (I don't promise anything), of course, if you are willing to do it from your side and open a PR, we'll be happy

Nope @JorTurFer I haven't made any progress on this, you can pick it.

[dja-fr]

Hi,
I have another use-case than argoCD.

I'm part of CICD team's in my company and we basicly offer one Atantis (see Atlatis run) per team.
These pod have to be always up and running because they are trigger by http webhook from gitlab.
For finops and greenIT purpose we aim to scale to 0 every instance of atlantis and scale-up just when a http call is made.
Unfortunatly the official chart use statefulset so we can't use this plugin right now.