[In the future] Helmfiles as a chart / Helmfile as a K8S CR

[mumoshu]

So that we can leverage everything helm provides to power helmfile :)

• User-defined Lua functions included in charts (Perhaps in Helm v3)
• Helm chart repositories to distribute versioned helmfiles for reuse
• No need to implement your own controller or operator to manage Helmfiles(in case you find it necessary)

Points must be addressed before making it real:

• Don't break existing use-cases of helmfile. Leverage Helm's ecosystem w/ minimal changes to Helmfile.
• Allow users to choose either kubectl or helmfile CLIs according to their usecases, like Helm v3 allows you to choose use helm or helm CRD.
• Reformat helmfile as a valid CR. Add apiVersion, metadata, and spec sections?
• Translate helmfile's ordering Ordering #137 into helm's ordering e.g. Proposal: sortable manifests (templates) helm/helm#3635
• Don't expect Helmfile CRD to handle every use-case of ours. CRD seems to have many edge-cases and gotchas, ones like noted in https://blog.manifold.co/the-state-of-custom-resource-definitions-in-kubernetes-5498425dd553

[faheem-nadeem]

In favor for having a CRD. It could be something like a helm operator for a gitops style workflow, as in Flux from weaveworks. Which I find to be bit limiting at this moment.

[mumoshu]

I built a K8s operator for Helmfile CRD - https://github.com/mumoshu/helmfile-operator 😃Anyone interested in trying it out?

[victornoel]

I built a K8s operator for Helmfile CRD - https://github.com/mumoshu/helmfile-operator smileyAnyone interested in trying it out?

Nice, do you plan to make this a full-fledged component of the helmfile ecosystem and maintain it in the long term or is it just an experiment?

[mumoshu]

@victornoel Hey!

Honestly saying, I'm maintaining a bunch of projects including helmfile and helmfile-operator almost as an after-work hobby.

That's being said, I can't commit maintaining it in the long term until someone hires me for maintaining it, or perhaps make it a more collaborative project hosted in CFNF or anywhere else.

But I do have the "passion" to make it a fully-featured component that complements the whole helm eco-system.

I believe you'll suffer with (1)limited feedbacks(=harder to notice and debug deployment issues) and (2)the lack of a manual approval feature when helmfile is integrated with:

• argo
• argo cd
• brigade
• circle ci
• tekton
• etc.

I've already written plans to address these in :

• feat: Surface changes and require manual approval before applying mumoshu/helmfile-operator#1
• feat: Stream error messages mumoshu/helmfile-operator#2

I tried hard to find solutions that works without putting too much engineering efforts into helmfile and helmfile-operator. But I have no way other than enhancing the helmfile-operator so far.

So I'd appreciate it if you could try it if you encountered the problem the operator is trying to solve, and give feedbacks and even better advocate for it, so that more people come and the project is likely to be maintained longer term.

The more a project attracts people, the more likely potential maintainers come :)

[victornoel]

@mumoshu sorry for taking so long to answer :)

After doing a bit of survey of existing solutions, I was wondering if it wouldn't be a better idea to integrate helmfile with the flux initiative. They just announced that they were going to join CNCF (or at least try) and detach themselves from weaveworks (in terms of governance): https://github.com/fluxcd/flux/wiki/MoveToFluxCD

[mumoshu]

@victornoel Thanks for the suggestion!

helmfile-operator is a tool comparable to helm-operator which is maintained as a part of flux. That is, I'm considering helmfile-operator as an effort to integrate helmfile into flux :)

Does it make sense to you?

Or are you suggesting to donate helmfile-operator into flux? If so, I'm fine to do so but not sure about the process and who actually want to maintain it if donated.

Any comment is welcomed.

[victornoel]

@mumoshu not donate no (well, except if you want to ^^), but yes, I was thinking about making something that would work well in the flux ecosystem. I'm not very knowledgeable about it, so I didn't realise that helmile-operator was already doing that :)

[antaloala]

@mumoshu @victornoel I just found this open issue. It was so nice for me to find it (I am some kind of new in helmfile -it is only some months I am playing with it, I am really impressed-) and I started to reason exactly as you did here above when trying to use helmfile in a GitOps scope.
I think the new Flux v2 and the GitOps toolkit is really opening the door to move this helmfile-operator to Flux.
(note old Flux v1 helm-operator was removed, replaced by a helm-controller that makes more sense); Flux v2 GitOps toolkit opens the door for new coming controllers to be added ... as the helmfile-controller !
It would be needed some integration towards the Flux v2/GitOps toolkit "source controller" (needing the HelmfileState CRD there?) to make it fully "usable" in Flux v2
What do you think?

[mumoshu]

@antaloala Hey! I've already read much about Flux v2 ecosystem and source-controller. I've also considered how I'd re-implement https://github.com/mumoshu/helmfile-operator in the Flux v2 ecosystem.

The question raised from my mind then was that- what's the concrete benefit of implementing it with source-controller?

As far as I can see, Flux v2 as a whole doesn't provide anything other than source-controller and the API to interact with Source from a custom controller perspective. For example, it would be great if they had support for nice Web UI like ArgoCD to interact/summarize Helmfile-related custom resources. But there aren't any.

Rewriting the helmfile-operator to use source does seem to allow me to reuse the some K8s controller and artifact fetching code from source-controller. But anything else?

I like Flux and Flux v2 and the new abstraction of Source/Artifact around source-controller. I'd really love to think about possibilities on how helmfile and flux v2 could enhance the whole u/x together. AFICS, there's no much benefit doing so, especially given we already have helmfile-operator.

WDYT?

[victornoel]

@mumoshu indeed, argocd seems clearly more full fledged than Fluxv2 in terms of available features. It seems one is really made with UIs in mind while the second is "just" description file-based gitops.

The only advantage of using FluxV2 over using nothing would be to get the community + all those various sources supported out of the box in a coherent ecosystem. I'm not sure it is worth the effort, as you say indeed.
And we would get the same advantage from ArgoCD, both being under CNCF incubation btw.

Actually, the more I think about it, the best would even be that both ArgoCD and FluxV2 support helmfile (like they already support helm, kustomize, etc) and that you would have none or a minimum of work with maintaining it so that you can continue focusing on helmfile itself :)

Not sure what would be the best way towards this except maybe creating issues in those project to have them start thinking about it and realize the greatness of helmfile and the impossibility of not supporting it :D

[mumoshu]

@victornoel Hey! Thanks for sharing your ideas and insights.

FWIW, ArgoCD seems to provide us enough extension points to integrate Helmfile nicely.

A few months ago I've written some outdated recommendation on how you would integrate Helmfile with ArgoCD https://github.com/roboll/helmfile#argocd-integration. The gist of the recommendation was to run helmfile template on CI to generate manifests to git-commit, and let ArgoCD Application deplos the manifests in the generated git repo content.

What I've realized afterwards though was that more people use ConfigManagementPlugin.

With that, you point ArgoCD to the repo that contains no manifests, but helmfile.yaml and its siblings. You let ArgoCD run helmfile template on sync. It's like letting ArgoCD to run helmfile template | kubectl apply -f -.

It looks more like a native integration to me. What would be possible improvements over this ConfigManaegmentPlugin? 🤔

[antaloala]

@mumoshu, @victornoel I have not read enough about ArgoCD, so let me come back to the Flux v2 proposal.
Yes, you are right, after some more reading on the GitOps toolkit provided by Flux v2 the integration with the source-controller is mainly about letting it to inspect the configured git repo and/or Helm repo to retrieve the required (from Helmfile-controller) artifacts. But I think it is not only that what we would get from a Flux v2 integrated Helmfile-controller, we would also have:

• a (git) PR triggered "helmfile apply" execution ... and/or triggered from a new published Helm chart in the source-controller watched Helm repo (for those helm releases where a chart semver range is set in the state file/helmfile yaml)
• (as another minor PROS) possibility to raise external notifications on "helmfile apply" results through the notifications-controller

The main PROS is first bullet above; Helmfile is nowadays to be invoked periodically (a cron job running the "helmfile apply") and or coding/writing the instrumentation required to watch changes in a git repo (where the state files and the state value files are placed under git revision) and so trigger the "helmfile apply"; even when having a helmfile operator we still need "someone" to push into k8s API server the "helmfile" custom objects watched by the helmfile operator's custom controller.
Without integrating that helmfile operator with a GitOps tool we cannot get that pull-based way of working.

But it is also true that once ArgoCD and Flux v2 provide support for Helm releases + allowing to connect different Helm releases across them (the "App of App" pattern in ArgoCD, in Flux v2/helm controller it is based on a HelmRelease object setting dependencies towards another HelmRelease object) it is not 100% clear to me if a GitOps integrated Helmfile operator is to be a long term solution or maybe some kind of gap filler as native helm-releases support in ArgoCD and Flux v2 evolve, getting more and more features.

What is your view on this? I must admit I do not see so clear now the long term value of having helmfile support in these CNCF GitOps projects ...

[mumoshu]

What is your view on this? I must admit I do not see so clear now the long term value of having helmfile support in these CNCF GitOps projects ...

@antaloala Hey! I believe having helmfile support would be great. I'm just saying that ArgoCD already supports it and it's great, and adding it to Flux v2 or enhancing helmfile-operator doesn't seem to work fully or worth the effort to me.

Flux v2/helm controller it is based on a HelmRelease object setting dependencies towards another HelmRelease object)

IIRC Flux v2's model doesn't allow cross-kind dependency. That is, a kustomize project can depend on another kustomize project and a helm release can depend on another helm release, but a kustomize project can not depend on a helm release. ArgoCD can handle this in another form, by using Sync Waves and Sync Phases.

Helmfile supports cross-kind dependency by needs.

But it is also true that once ArgoCD and Flux v2 provide support for Helm releases + allowing to connect different Helm releases across them (the "App of App" pattern in ArgoCD, in Flux v2/helm controller it is based on a HelmRelease object setting dependencies towards another HelmRelease object) it is not 100% clear to me if a GitOps integrated Helmfile operator is to be a long term solution or maybe some kind of gap filler as native helm-releases support in ArgoCD and Flux v2 evolve, getting more and more features.

As Flux v2 can't have cross-kind dependency, it's like comparing apple and orange. But anyway-

ArgoCD has out-of-box support for ConfigManagementPlugin, which allows you to integrate Helmfile into an arbitrary ArgoCD application. I think you can consider ArgoCD as it turns Application custom resources with config management plugins configured into something like a gitops operator for Helmfile.

ArgoCD Application + ConfigManagementPlugin looks great abstraction, allowing you to map an Application to a set of
releases rendered by helmfile, or even map each Helmfile release to a dedicated ArgoCD Application.

This opens up possibilities for you to use your Helmfile configs for local development or standalone deployment, while using the same configs to power GitOps. This isn't possible with Helm or Kustomize alone.

You can even combine Helmfile-backed Application with Helm-backed and Kustomize-bakced Applications.

What else do we need beyond that? I'm curious 🤔

[antaloala]

@mumoshu Thanks for quick reply (as usual ;-)
I was not really considering cases in which we need to cross-relate kustomize packaged (micro)services with Helm packaged ones in order to compose a cloud native application. Is that the normal case?
I am focusing on scenarios in which a cloud native application is defined as a set of related helm releases (one per each instance of a required (micro)service), each required (micro)service providing its own Helm chart (plus some container images, of course).
It is for this scenario where I think Hemfile, ArgoCD and Flux v2 overlap (all of them providing a fully declarative handling of helm releases).

I like (a lot) many of the additional feature you added to Helmfile + helm-x plugin (as the JSONpatches or StrategicMergePatches you can apply to a helm chart without having to fork it, the possibility to template the values.yaml to apply to each release, etc.) and so I thought on getting all of them in the two main CNCF GitOps projects: ArgoCD and Flux v2.
But more I read on helm support (only helm) in ArgoCD and Flux v2, more I think they are aiming to cover Helmfile scope in a GitOps context ... but I could be totally wrong ...

[mumoshu]

Is that the normal case?

@antaloala Yeah it's a normal case for me. I usually use Helm charts for external projects whose official charts are distributed, and either a kustomize project or a local helm chart for my own thing. Helmfile manages all of them by just specifying chart value to either directory or a repo/chartname string, and writing needs to make a dependency graph.

I like (a lot) many of the additional feature you added to Helmfile + helm-x plugin (as the JSONpatches or StrategicMergePatches you can apply to a helm chart without having to fork it, the possibility to template the values.yaml to apply to each release, etc.) and so I thought on getting all of them in the two main CNCF GitOps projects: ArgoCD and Flux v2.

AFAIK, ArgoCD's ConfigManagementPlugin allows you to still leverage Helmfile features like JSONOPatches and StrageticMergePatches, because it basically works by injecting helmfile template command into ArgoCD's sync process. That's why I asked what else we might need.