Proposal: Package helmfile.yaml along with charts and values files

[alebabai]

It would be great to have ability to download published charts and package local charts via single api.
For example like it was done with requirements.yaml and helm dep up to install charts from tar-balls that are stored locally.
It's possible to call helm fetch for published charts and helm dep up & helm package for local ones.

[osterman]

Can you give an example use-case for wanting to locally store the chart packages as opposed to pulling them from the remote repositories?

[mumoshu]

I'm interested to hear from you, too @alebabai.

Perhaps you're creating a bundle of all the helmfile.yaml and deps, so that you can later run helmfile sync on machines that has no internet access?

[alebabai]

Yeap, the main use-case is create some kind of bundle (.tgz for example) like an umbrella chart and install it later via helmfile charts or helmfile sync on machines without internet access.

[mumoshu]

@alebabai Very interesting!

I'm now wondering how it should work.

Say we add a new command helmfile [--environment ENV] package, what should be contained in the resulting archive?

I guess, if you are to run helmfile apply --skip-repo-update against the archive, at least helmfile should be enhanced to accept a compressed archive to extract helmfile.yaml and all the referenced sub-helmfiles and values.yaml files, and templates?

helmfile -f package.tgz apply --skip-repo-update

It's possible to call helm fetch for published charts and helm dep up & helm package for local ones

Honestly I don't know much about how helm upgrade --install work against fetched chart archives.

Do we need to check the existence of the archive and specifically call helm like helm upgrade --install ./mychart-x.y.z.tgz in order to use the fetched archive?

Or does helm upgrade --install stable/mysql picks mysql-x.y.z if it exists?

[alebabai]

@mumoshu

Say we add a new command helmfile [--environment ENV] package, what should be contained in the resulting archive?

So, i guess resulting archive should contain helmfile.yaml and all the referenced sub-helmfiles and values.yaml files, templates like you said, but also tgz-archives with charts that had been used in releases.

Do we need to check the existence of the archive and specifically call helm like helm upgrade --install ./mychart-x.y.z.tgz in order to use the fetched archive?
Or does helm upgrade --install stable/mysql picks mysql-x.y.z if it exists?

Helm picks tgz-s only for dependencies that are described in requirements.yaml, but for
helm upgrade --install you should explicitly pass tgz, like here
helm upgrade --install ./mychart-x.y.z.tgz

[alebabai]

@mumoshu

helmfile -f package.tgz apply --skip-repo-update

One detail, we can drop --skip-repo-update flag, if -f argument has .tgz extension.
I thing it will be really useful an interesting feature.

[mumoshu]

@alebabai

One detail, we can drop --skip-repo-update flag, if -f argument has .tgz extension.
I thing it will be really useful an interesting feature.

Ah, sounds good!

helm upgrade --install you should explicitly pass tgz, like here
helm upgrade --install ./mychart-x.y.z.tgz

Good to know! Thanks. So translating it to helmfile, how about helmfile -f package.tgz extracts packaged remote charts into path/to/subhelmfile/.helmfile/charts/*.tgz, and helmfile automagically uses the chart archive instead of remote charts, or even local chart directories?

[mumoshu]

The feature request about using values/secrets from within a remote chart #324 relates to this issue a bit. #324 is about packaing values/secrets into chart archives and loading it via helmfile, whereas this issue is about packaging all the deps, values/secrets yamls, and helmfile.yaml files into a single archive.

Probably this one is harder, but a more advanced solution to effectively distributing desired states of your k8s apps.

[mumoshu]

@alebabai Are you ok with packaging environment values and secrets #267 #255 of all the environments(e.g. dev, test, staging, prod) into a single helmfile archive(tgz)? Or would there be any use-cases that needs an environment specific helmfile archive?

That is, which one do you prefer between helmfile package --environment prod | hemlfile -f - apply or helmfile package | helmfile -f - --environment prod apply. The latter would be easier to implement, but the former has more isolation of archives across environments.

[alebabai]

@mumoshu

So translating it to helmfile, how about helmfile -f package.tgz extracts packaged remote charts into path/to/subhelmfile/.helmfile/charts/*.tgz, and helmfile automagically uses the chart archive instead of remote charts, or even local chart directories?

Yeap, i though this way.
Also we have package.tgz so we know exactly that there is chart archives.

[alebabai]

Are you ok with packaging environment values and secrets #267 #255 of all the environments(e.g. dev, test, staging, prod) into a single helmfile archive(tgz)? Or would there be any use-cases that needs an environment specific helmfile archive?
That is, which one do you prefer between helmfile package --environment prod | hemlfile -f - apply or helmfile package | helmfile -f - --environment prod apply. The latter would be easier to implement, but the former has more isolation of archives across environments.

In my opinion helmfile package should pick all resources. Flag --environment is more related to runtime tasks (apply, sync, ect), that apply configuration based on selected env.
But it's also possible to have all these options. For example when --environment flag is specified inpackage command it should pick values for this env, if there is no env specified - all resources shoud be selected.

[mumoshu]

Updated the issue title accordingly. Please correct me if I have any misunderstandings!

[mumoshu]

One thing I'm unsure is that how helmfile can discover executable files contained in a helmfile project to be used by {{ exec "./path/to/local/exec/file" }} exprs and command: "./path/to/local/hook/command" commands(#349).

Actually run the exec tmpl exprs and hook commands while packaging, and memorize what's called?

[osterman]

Related but totally not related, there's a terraform-bundle command to address a similar use-case for terraform plugin.

[SE2Dev]

I'm not sure if it's more related to #751 or #752 or not, but I'd like the ability to generate a tarball and values.yaml for each release as specified in helmfile.yaml so that I could deploy them via something like Rafay workloads. Ideally this would work for both local helm charts, and ones that are being pulled from Helm repositories.

[mumoshu]

@SE2Dev Can you just create a tarball from helmfile template --output-dir or helmfile template --output-dir-template output?

[SE2Dev]

@mumoshu Technically, yes, if you ran helmfile template --output-dir="gen" and got the following directory tree as a result:

.
├── gen
│   └── helmfile-36749c04-myrelease
│       └── filebeat
│           └── templates
│               ├── clusterrole.yaml
│               ├── clusterrolebinding.yaml
│               ├── configmap.yaml
│               ├── daemonset.yaml
│               ├── deployment.yaml
│               └── serviceaccount.yaml
├── helmfile.yaml
└── makefile

One could theoretically generate chart packages with the following (although this script wouldn't work for subcharts):

PACKAGE_DIR=out

for f in gen/*; do
    if [ -d "$f" ]; then
        RELEASE_DIR=$f
        for f in $RELEASE_DIR/*; do
            if [ -d "$f" ]; then
                CHART_DIR=$f
                CHART_NAME=$(basename $CHART_DIR)

                # Generate a Chart.yaml
                echo "apiVersion: v2
name: $CHART_NAME
description: Autogenerated chart
type: application
version: 0.1.0
appVersion: ""1.16.0""" > $CHART_DIR/Chart.yaml

                # Generate a chart package
                tar -zcf $PACKAGE_DIR/$CHART_NAME.tgz -C $RELEASE_DIR $CHART_NAME
            fi
        done
    fi
done

The approach above, however, results in the loss of original chart metadata like information, etc. as well as all of the Values data being baked into the generated chart(s). My goal is to effectively automate the actions below, and then iterate over each release to deploy the chart packages + values file via Rafay.

• helm package for each local chart
• helm pull for each remote chart
• helmfile write-values for each release