-
Notifications
You must be signed in to change notification settings - Fork 564
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for ArgoCD resource hooks and sync waves #1110
Comments
Interesting thought this. I see some issues as well (for example what if the chart is already using the annotations for items within the chart). On a related note, I built a simple |
@travisghansen Awesome! Thanks for sharing your work. As you've shown in your examlpe, I suppose this proposal would add ordering to k8s resources for both ways and nothing else.
You're referring to cases like k8s resources rendered from your chart is already containing argocd-specific annotations? |
@mumoshu yeah, charts that already use the argo specific annotations correct. Understood on the 2 approaches, my plugin is attempting the former of the 2 options. You can store your configs in git and/or put a full |
@mumoshu I'm running a continuous deployment workflow with following tools.
As we want to switch from helm to argocd for the final deployment, the goal is to continue to use helmfile for manifest generation and kustomize transformers. So far, so good. We also have helm test Pod's (Provided by developer teams and with helm.sh/hook annotations) which I try to convert to argocd postSync annotations (argocd.argoproj.io/hook: PostSync). The newly JsonPatch (helm-x) integration you just released yesterday is just perfect for my use case. (thanks for that by the way) With latest version of helmfile (0.118.1), I'm facing following issue (bug ?) 'helmfile template' output works fine, outputting my helm test pod manifests. But if I add the '--output-dir' option, helmfile skips the pod manifest if there is an 'helm.sh/hook' annotations in it. Therefore I'm not able to use it or patch it with kustomize. For clarification, in the paste I did a bash script to create the kustomization bases for each release created by 'helmfile template --output-dir' and used it in a final kustomize process. Now with your helm-x integration I only need to use it for kustomize transformers not yet supported by helmfile. Back to my issue. The same skipping seems to appear when using integrated helm-x kustomize transformers (as I guess you use the output-dir behind the scene). If I remove the annotation 'helm.sh/hook', everything works nice. But I cannot remove it on upstream helm charts, and I think helm test is a really good way to provide integration tests for a dedicated application. (We use tests based on the robot testing framework). Sorry, long story and not sure if it's the right place for my issue. But as this ticket describes ArgoCD resource hooks and sync waves, would be great to consider also helm test pods. Should I open an other issue for the 'helm.sh/hook' annotations ? eg. "helmfile template --output-dir skips objects with 'helm.sh/hook' annotations" cheers |
I've read about both ArgoCD and Flux and I don't see any advantage in using either of these systems along with helmfile. Our helmfile CI/CD process is very simple:
|
Ok so I now see one huge value in using Argo CD - the UI! @mumoshu, I believe the CI system should diff against the actual state of the cluster, just like terraform plan is checking the actual resources to show a real diff. Currently helm-diff does not provide this - databus23/helm-diff#176. The main problem I see with Argo CD is this - how do you deploy the Argo CD components? |
@dudicoco I think you'd usually install and manage Argo CD itself with This way, you can also diff against (mostly) the actual cluster state with git-diff or github or diff, as ArgoCD guarantees the manifests committed to the config repo are periodically applied to the live cluster. |
Looks like proper dependencies between applications will be implemented into Argo CD: argoproj/argo-cd#3892 |
@dudicoco Thanks for sharing! The feature does seem like a better fit when you're to create Application custom resource for each sub-directory generated by When you need ordering within a single subdirectory(release), apparently we still need sync waves/resource hooks. |
i like it. |
I've often heard that people use
helmfile template
to export the whole collection of K8s manifests managed by Helmfile to a "config repo" for GitOps, so that actual deployments can be delegated to ArgoCD and Flux.The advantages of this approach are:
helmfile diff
results being posted in your Slack channel or pull request comments, they are now visible and reviewable in the pull request's "Files changed" tab would give you better reviewing experience by providing nicer coloring, layouts, inline review comments, etc.Altohugh Helmfile + GitOps do have advantages over the "Helmfile on CI" approach, there are issues, too.
For me, the biggest issue in traditional GitOps had been that there was no way to control installation/upgrade order of helm charts. Helmfile have been long supporting this via DAG(
needs: ...
) and sub-helmfiles(helmfiles: [...]
).However, I've recently got to know about ArgoCD's Resource Hooks and Sync Waves, and realized that, theoretically, it enables Helmfile to export manifests with ordering in mind.
I'd like to experiment this by introducing a new flag
--argocd-phases-and-hooks
tohelmfile template
.The flag will instruct Helmfile to add
argocd.argoproj.io/hook: PreSync
annotations to whatever manifest that has"helm.sh/hook": pre-install,pre-upgrade
, and alsoargocd.argoproj.io/sync-wave: "N"
whereN
is the globally assigned sequence number of the set of releases that are installed/upgraded concurrently.To illustrate how
N
is assigned, let's assume you have the following helmfiles:helmfile.yaml:
apps.yaml:
infra.yaml
Today,
helmfile sync
andhelmfile apply
deploys this in the following order. Note that each group of releases are deployed concurrently as they don't depend on each other.1: logging, prometheus, cilium
2. istio
3. foo-api, bar-api
4. web
helmfiel template --argocd-phases-and-hooks
will giveargocd.argoproj.io/sync-wave: "1"
annotations to any manifest that belongs tologging
,prometheus
andcilium
,argocd.argoproj.io/sync-wave: "2"
foristio
, and so on.You can git-push these manifests to a config repo, so that ArgoCD will sync those manifests in that order.
The text was updated successfully, but these errors were encountered: