Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to suspend an argo application #3039

Open
andye2004 opened this issue Jan 28, 2020 · 54 comments
Open

Ability to suspend an argo application #3039

andye2004 opened this issue Jan 28, 2020 · 54 comments
Labels
component:argo-cd component:core Syncing, diffing, cluster state cache enhancement New feature or request type:enhancement type:usability Enhancement of an existing feature workaround There's a workaround, might not be great, but exists

Comments

@andye2004
Copy link

Firstly, thanks for a great tool. It's made life much easier for us!

There are times where I want to 'suspend' an application. That is, I want to temporarily delete it from the cluster and then re-add it. Currently I have to delete the argo application completely and then re-add it into argo. It would be useful if there was a feature where I could suspend it in argo and that would delete the app in the cluster BUT keep the argo application definition ready to re-sync.

@andye2004 andye2004 added the enhancement New feature or request label Jan 28, 2020
@jannfis
Copy link
Member

jannfis commented Jan 28, 2020

Hm. You could have your application definition residing in a dedicated definition modeling the app of apps patterns with auto-sync disabled on the parent.

Then you could cascade-delete the application in question ("child app"), and if you want it back, you just re-sync the parent application.

@andye2004
Copy link
Author

@jannfis, thanks for the follow-up. I'll have a think on this for a few days and we'll maybe have a discussion internally re your suggestion. Doesn't feel quite right though - I'll get back to you.

@jannfis jannfis added component:core Syncing, diffing, cluster state cache type:usability Enhancement of an existing feature labels May 14, 2020
@dcharbonnier
Copy link

This "status" of disabled application could be really useful for development.
If I have two projects using the pattern app of apps :
project1 : app1, app2
project2 : app1, app2, app3

I would be able (on my development machine) to stop working on project1 and start working on project2 with 2 clicks (or one commit)
I don't see a use-case for a production environment.

@jeacott1
Copy link

+1 - I'd love to see a stop, or suspend button for each application in the UI. It would make regular but intermittent workloads much easier to manage. - to extend the notion, a schedule of suspend/resume could be added to only run workloads during certain times and then auto shut them down would be especially nice. Autoscalers would then auto reduce nodes and thus running costs.

@danielepanzeri
Copy link

+1

@Kampe
Copy link
Contributor

Kampe commented Oct 21, 2021

+1 - people with app of apps pattern with auto sync on need a way to pause both the parent and child application.

@hilenko
Copy link

hilenko commented Nov 11, 2021

+1

5 similar comments
@ghost
Copy link

ghost commented Dec 8, 2021

+1

@davem-git
Copy link

+1

@nikcollins
Copy link

+1

@marcelofabricanti
Copy link

+1

@armenr
Copy link

armenr commented Jan 21, 2022

+1

@vmironichev-nix
Copy link

+1

2 similar comments
@tanalam2411
Copy link

+1

@achetronic
Copy link

+1

@Cave-Johnson
Copy link

Cave-Johnson commented Apr 5, 2022

+1

Not for a prod deployment but for when debugging a dev cluster, pausing deployments would be super useful

@JBogossianKH
Copy link

+1 ❤️

@Va1
Copy link

Va1 commented Aug 16, 2022

+1

@kschjeld
Copy link

kschjeld commented Sep 1, 2022

+1
Much better than commenting out contents of YAML-file if you want to temporarily disable

@zoop-btc
Copy link

zoop-btc commented Oct 5, 2022

+1 would be useful

@Mahesh-Gunda-Maersk
Copy link

+1 very useful.

@filipdadgar
Copy link

+1, would love to see this feature!

@elendis
Copy link

elendis commented Jan 4, 2023

+1

@meetzuber
Copy link

meetzuber commented Jan 19, 2023

+1, It will be help full in production - DR cluster use case. We can disable the application in production cluster and start it in DR cluster in case of DR failover. Will reduce downtime.

@cedws
Copy link

cedws commented Feb 9, 2023

Can you not spam +1s? Just leave a thumbs up reaction on the issue.

@eaceaser
Copy link

+1

@crenshaw-dev
Copy link
Member

I want to make sure we're all +1'ing the same feature request.

Please react to indicate which feature you want:

  • ❤️ Delete all resources in the app
  • 🚀 Scale down all scalable resource in the app
  • 👀 Something else (please comment, too)

Please no more +1s without describing your use case. The enthusiasm is appreciated, but keeping the thread tidy is good too. 😄

@canghai118
Copy link

I want to make sure we're all +1'ing the same feature request.

Please react to indicate which feature you want:

  • ❤️ Delete all resources in the app
  • 🚀 Scale down all scalable resource in the app
  • 👀 Something else (please comment, too)

Please no more +1s without describing your use case. The enthusiasm is appreciated, but keeping the thread tidy is good too. 😄

* ❤️ Delete all resources in the app (It's better to have an option to keep the PVC or PV, so the entire applicaiton state can be resumed)

@jeacott1
Copy link

jeacott1 commented Jun 16, 2023

@canghai118 for me both options would be useful actually,
I want to be able to minimise resource usage without trashing configMaps etc so a Scale down option would also be very useful for things that can scale down.
I would also like to sometimes "Delete all resources in the app" without having to delete the app config itself. I think this is probably what most people want so you can push 'go' again and not have to reconfigure the app each time.

@jannfis
Copy link
Member

jannfis commented Jun 17, 2023

I think this should be possible meanwhile, although it's not exposed through the UI or CLI (yet): https://argo-cd.readthedocs.io/en/stable/user-guide/skip_reconcile/

This should effectively implement the requested feature?

@crenshaw-dev
Copy link
Member

@jannfis I think the issue title is too vague. Seems like most folks on the thread are interested in a per-app scale-down mechanism.

@crenshaw-dev
Copy link
Member

Ooh. With the new actions enhancements in 2.8, we should do something like this:

-- Loop over resources from the Application status. Scale down Deployments and StatefulSets.

actions = {}

if obj.status == nil then
    return actions
end

for _, resource in ipairs(obj.status.resources or {}) do
    if resource.health.status ~= "Missing" and (resource.kind == "Deployment" or resource.kind == "StatefulSet") then
        action = {}
        action.operation = "patch"
        action.resource = {}
        if resource.group == nil or resource.group == "" then
          action.resource.apiVersion = resource.version
        else
          action.resource.apiVersion = resource.group .. "/" .. resource.version
        end
        action.resource.kind = resource.kind
        action.resource.metadata = {}
        action.resource.metadata.name = resource.name
        action.resource.metadata.namespace = resource.namespace
        action.resource.spec = {}
        action.resource.spec.replicas = 0
        table.insert(actions, action)
    end
end

return actions

@sav116
Copy link

sav116 commented Jul 28, 2023

  • 1

@crenshaw-dev
Copy link
Member

Is anyone up for trying to implement the actions-based solution? I'd be happy to help.

@crenshaw-dev
Copy link
Member

crenshaw-dev commented Nov 1, 2023

For the two folks who answered that they want "delete all resources in the app," let's track that feature request here: #6079

This issue (#3039) can track "scale down all scalable resources."

@jnkxcel
Copy link

jnkxcel commented Nov 30, 2023

For my use case I work at a utility and they heavy hand the password, rotation is a pain, I used to just kubectl scale --replicas=0. But with Argo control it self heals. Which I love but now I have to remove my deploy, update password and then redeploy. Not huge lift but cumbersome for prod facing apps. This is specific to K8s plugin external-dns connecting to infoblox. I manage the external-dns plugin via an app of apps deploy so I just have to remove one line entry and argo sync so again not huge lift.

Just would be so much nicer to have scale down feature. similar thread that lead me here
#11451

@crenshaw-dev
Copy link
Member

Sounds like in your case you'd need both to both scale down and disable self-heal.

@jnkxcel
Copy link

jnkxcel commented Nov 30, 2023

Thank you I will go that route, still fairly new in the argo game.

@MioOgbeni
Copy link
Contributor

MioOgbeni commented Feb 9, 2024

Hi, how is this issue going? It would also be awesome to be able to set scale down of scalable resources by on/off office hours, or by scale down/up cron. It could be configurable by completely new ArgoCD resource or directly in Application resource.

@MaxWinterstein
Copy link

Hi, how is this issue going? It would also be awesome to be able to set scale down of scalable resources by on/off office hours, or by scale down/up cron. It could be configurable by completely new ArgoCD resource or directly in Application resource.

@MioOgbeni you might wanna have a look at https://keda.sh/ and its cron scaler.

@dcharbonnier
Copy link

an alternative, and maybe better solution is to use a generator, with a generator you can enable / disable and application with your own rules : https://argo-cd.readthedocs.io/en/stable/operator-manual/applicationset/Generators-Plugin/

@jnkxcel
Copy link

jnkxcel commented Apr 23, 2024

I found workaround if you need temp solution, just make application manual sync, select the deployment object, select to edit it, in replica key:value shift value to 0. This will effectively shift application to 0 pod/replicas. when you want to scale back up you can either manual sync, it will bring back to previous replica, or update code then refresh and sync and this will bring pods/replicas back up. This is Janky but you kind of have temp scale down method.

@evanstucker-hates-2fa
Copy link

How to disable applications, lol.

Ensure that your ApplicationSet syncPolicy.automated.prune: true, then:

$ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	renamed:    applications/matomo/config.json -> applications-disabled/matomo/config.json
	renamed:    applications/matomo/values.yaml -> applications-disabled/matomo/values.yaml

Note that this may cause gaps in your git history if you move files back and forth. Maybe a smarter person would use symlinks? This is just a joke/hack, but it works. Let's get a real feature added to do this!

@jmichaelthurman
Copy link

Yes. I need a feature equivalent to flux suspend kustomization to be able to pause healing of applications by automation that are stood-down during infrastructure maintenance windows.

For flux apps, I'm using KEDA to launch a job to issue commands & patch scaledObjects in the K8s cluster.

@crenshaw-dev
Copy link
Member

@jmichaelthurman wouldn't that just be "disable self-heal" and maybe "disable auto-sync"? If you need to prevent even imperative syncs, you could also add a sync window.

@xpxNPE
Copy link

xpxNPE commented Nov 6, 2024

hi

another use case : test/validation environment.

when we are not using our test / validation environment, the "powers that be" delete every application to free up ressources. so when we need the environement again we have to redeploy everything even if there was no change since the last deployment.

having a way to restart the application from the argoCS UI without havaing to redeploy them would be great

@crenshaw-dev
Copy link
Member

having a way to restart the application from the argoCS UI without havaing to redeploy them would be great

I'm not sure what you mean. If the powers-that-be have deleted your resources, the only way to get them back is to redeploy them.

Or are you suggesting that, if a "suspend" feature existed, you could ask the powers-that-be to use that feature instead of deleting your stuff?

@xpxNPE
Copy link

xpxNPE commented Nov 7, 2024

yes! something that will allow the "power that be" to free ressources when the application are not used, and the devellopers to restart the application easily when needed

@andrii-korotkov-verkada
Copy link
Contributor

andrii-korotkov-verkada commented Nov 15, 2024

Are you mostly concerned about Pods' resources? If so, one workaround could be to use KEDA for autoscaling and set annotations

autoscaling.keda.sh/paused-replicas: "0"
autoscaling.keda.sh/paused: "true"

https://keda.sh/docs/2.16/concepts/scaling-deployments/#pausing-autoscaling

@andrii-korotkov-verkada andrii-korotkov-verkada added the workaround There's a workaround, might not be great, but exists label Nov 15, 2024
@marcelldls
Copy link

marcelldls commented Nov 18, 2024

Our solution has been to add a removed and enabled key in the app of apps helm chart that we can "patch" from the gui for temporary changes or also commit to git. Removed can be used to skip inclusion of a child app (delete all resources). enabled can be used to scale the replica count to 0 (suspend without deleting resources). Template: https://github.com/epics-containers/ec-helm-charts/blob/main/Charts/argocd-apps/templates/_apps.tpl. Example use: https://github.com/epics-containers/p45-deployment/blob/main/apps/values.yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:argo-cd component:core Syncing, diffing, cluster state cache enhancement New feature or request type:enhancement type:usability Enhancement of an existing feature workaround There's a workaround, might not be great, but exists
Projects
None yet
Development

No branches or pull requests