[Question] Any thoughts on templating values for subcharts?

[dtshepherd]

So this may be a bit crazy, but I'm wondering if there is a good pattern for templated subcharts... Say I have a chart A that uses helmfile and has a values.yaml.gotmpl file. Its directory structure might look like this:

A/
  templates/...
  Chart.yaml
  helmfile.yaml
  values.yaml
  values.yaml.gotmpl

This chart can exist on its own, so I want it to have its own helmfile and templated values.yaml input.

Then, I may want to reuse this chart in some other parent chart B (pulled in using requirements.yaml):

B/
  templates/...
  charts/
    A.tgz
  Chart.yaml
  helmfile.yaml
  requirements.yaml
  values.yaml
  values.yaml.gotmpl

The helm package command will package any files in the chart directory into the tgz, so the subchart B values.yaml.gotmpl will be packed with the chart.

Currently, when installing chart B, the values.yaml.gotmpl in chart A will be ignored. It may contain some critical environment values needed to function properly. Right now, we'll need to duplicate those values into chart B's values.yaml.gotmpl like so:

foo: {{ .Environment.Values.foo }}

A:
  bar: {{ .Environment.Values.bar }}

Most likely the A: subsection of the values.yaml.gotmpl in chart B will be a near duplicate of the values.yaml.gotmpl in chart A. It would be nice if we could recursively unpack the subcharts and make use of those values.yaml files when the parent chart is installed.

Again, this may be crazy, but I'd really like to DRY up some of these templates if possible.

[dtshepherd]

I’m also aware that I could just opt out of using subcharts and implement that same idea using a helmfile. However, it’s nice to use built in helm features and helm also supports packaging and repos of the charts/subcharts together.

[mumoshu]

@dtshepherd Hey! Thanks for sharing an interesting use-case. I'm still trying to understand, but anyway - you meant A.tgz under the charts/ directory in your example, rather than B.tgz, rght?

[dtshepherd]

Whoops, yes, just updated the comment.

[dtshepherd]

Ultimately, I think I can accomplish the same thing as subcharts with the helmfile: directive and some extra tooling in our build system to build an umbrella helmfile, but that doesn’t seem like a great solution.

I started prototyping with helmfile because helm decided they were never going to allow templated values.yaml files: helm/helm#2492

[mumoshu]

helm decided they were never going to allow templated values.yaml files

Yeah, that is one of reasons I started contributing helmfile 😄

[dtshepherd]

😂 I’m guessing that’s why 90% of us are here.

I was thinking a first pass of this feature could recursively unpack the subcharts and include their helmfile.yaml in the parent. However, the rendered subchart values would need to be nested in the generated values.yaml under a subchart named key.

[mumoshu]

Ultimately, I think I can accomplish the same thing as subcharts with the helmfile: directive and some extra tooling in our build system to build an umbrella helmfile, but that doesn’t seem like a great solution.

Would you mind sharing why you thought it isn't a great solution?

I still think helmfiles: is what you're looking for. You'd probably write B/helmfile.yaml that looks like the below to achieve your use-case:

helmfiles:
- ../A/helmfile.yaml

releases:
- name: B
   chart: ./

[dtshepherd]

Yes, your example is probably what it will look like and that's my plan right now. However, the reason I don't like that solution is the lack of chart versioning. If we make use of subcharts, we can use the requirements.yaml to define the subchart names/versions. In your example, I'd need to have our build system pull those dependencies using some outside tooling.

[mumoshu]

I'd need to have our build system pull those dependencies using some outside tooling.

Ah, so you have a hard requirement that A and B must NOT reside within the same git repo, right?

[mumoshu]

@dtshepherd I think your use-case seems slightly similar to #324 by @witten. You both package values (plus secrets in @witten's case, and values templates in your case) into chart archives.

I introduced #330 to allow you to extract chart archives with helmfile hooks. Would it work for you?

Maybe you want a more out-of-support from helmfile. I'm eager to implement one if we could come up with a greatest-common-factor feature that supports both of your use-cases. Any idea?

[dtshepherd]

Yes, I think #330 helps here.

Ultimately, I don't need most of what helmfile provides and I was just trying to shoehorn it into supporting templating values.yaml so I can encapsulate subcharts properly. Recently, I decided to start working on a prototype helm plugin that supports rendering templated values.yaml in a given chart (and its subcharts) -- keeping it simple as possible. Its based on the old helm-template plugin (https://github.com/technosophos/helm-template) and uses https://github.com/helm/helm/tree/master/pkg/chartutil to process the charts values.yaml and subcharts.

At this point, you can probably close this issue if you don't have much interest in supporting it.

[mumoshu]

Nice. The simpler the better 😄

I'm still interested in this issue. It is just that I'm not yet sure what would be the feature that fully supports this looks like.

I think your plugin would also allow me to see what it might look like when done in helmfile. Please don't hesitate to share it!

[dtshepherd]

Will do! Need to get client approval to open source :(

[Ro-Fo]

Hi, iam interested in this issue aswell.

We are looking for a way to manipulate or use the files provided via helm-git.

repositories:
- name: service
  url: git+https://gitlab.any.cloud/helm-charts.git@charts/service?ref=main

One use case would be to use the values.yaml.gotmpl from the parent repo.

charts/service/
  templates/...
  Chart.yaml
  helmfile.yaml
  values.yaml.gotmpl

releases:
- name: my-service
  namespace: localtest
  chart: service/my-parent-chart
  values:
  - values.yml.gotmpl
  - ./tmpdir/values.yaml.gotmpl

Second use case would be to put the files in a defined sub-dir under the helmfile. From this point we could make many changes with hooks. In our example we would like to run gotmpl multiple times on the same values.yml and add all variables to the current ".variables" to reduce duplication.

global:
  productstage: "{{ .values.productStage }}"
  imageversion: "1.32.12"

microserviceOne:
  image:
    tag: `"{{ .Values.imageversion }}"`
microserviceTwo:
  image:
    tag: `"{{ .Values.imageversion }}"`

In the example above, you can do that with yaml anchors, but that doesn't work over different values.yml.

Overall, we would like to reduce duplication of values.yml code and reuse configurations from different Git repos, the use case being to facilitate the development of test integration environments across multiple teams.

Thanks @mumoshu for the great work you are doing.