feat: add ability to capture finalized helmfile releases yaml to disk

[bitsofinfo]

In line w/ #751 and #629 would be great to be able to capture the finalized helmfile generated releases (to stdout or preferably just write the YAML to another output dir)

For my use case, the releases are generated and the helmfile is primarily a template. I'd like to be able to capture the finalized/generated helmfile releases and flush to YAML for both auditing purposes, offline debugging and more importantly if we want to use the generated helmfiles as future static ones. (or if I just want to re-run it manually without going through the original helmfile that generated it)

[mumoshu]

Hey! Would you mind clarifying a bit more on:

auditing purposes

Can't we use a GitOps repo containing either (1) helmfile template results or (2) helmfile.yaml and companion assets for that purpose?

offline debugging

What would be the benefit of this over --log-level=debug?

we want to use the generated helmfiles as future static ones

This is really interesting. What would be the purpose to do so?

re-run it manually without going through the original helmfile that generated it

In which circumstance would you need to do so?

[bitsofinfo]

auditing purposes

My ideal goal is that my team can run helmfile to generate and apply a set of releases, using a single command (without need for them to go write a wrapper or parse output etc), or at most 2 commands (apply/sync + template)

1. apply intended state to the target k8s cluster
2. capture generated YAML and catpure/write to disk/git (feat: add '--output-dir' option to other options (apply/sync etc) write generated YAML output to disk #751)
3. capture the set of releases: the helmfile ended up generating and also capture/flush to disk/git (this issue)

What would be the benefit of this over --log-level=debug?
This is really interesting. What would be the purpose to do so?
In which circumstance would you need to do so?

I just want to easily get the releases: that will be generated from the helmfile and flush it to disk without having to write some wrapper or pipe it into yet another set of tools that parse out all the extra stuff debug generates and strip all the line numbers etc.

The helmfiles I'm working can generate a lot of releases when the team is debugging sometime or in a pinch situation it can often be helpful just be able to look at a clean dump of generated releases without all the extra processing info.

I also want to be able to save these as static releases that get generated and be able to use them as a teaching aid for helping others create their own helmfile templates.

The generated releases: that can come from a templated helmfile at this point are just a very transient side effect of the setting the log-level to debug.

[mumoshu]

Thanks! So you'll be using generated static bundles of helmfile for auditing and debugging purpose?

If so, would it be duplicate of #195, or just similar to it? I'd like to hear your opinion.

[bitsofinfo]

Yeah I don't think #195 is what is really what I'm talking about.

Let me try to clarify by referencing this example at: https://github.com/bitsofinfo/helmfile-deploy/tree/master/examples

All I am interested in is given a helmfile, that generates many helm releases:, I simply want to get a copy of the literal generated releases: that helmfile generates. I don't want any k8s YAML, nor do I want any helm commands to be executed, I also don't want to see all the un-related debug output.

Right now to get the generated releases: (without applying anything), I run this command:

./helmfile \
  --log-level debug \
  --file deployments.helmfile.yaml \
  --state-values-set targetCluster=minikube \
  --namespace bitsofinfo-apps \
  --selector context=stage-qa \
  --environment catapp \
  --state-values-set chartConfigs.appdeploy.chartValues.baseValuesRootDir=examples/chartvalues/appdeploy \
  template

I get this output back: (I cut out a large chunk for brevity)

processing file "deployments.helmfile.yaml" in directory "."
first-pass rendering starting for "deployments.helmfile.yaml.part.0": inherited=<nil>, overrode=&{catapp map[chartConfigs:map[appdeploy:map[chartValues:map[baseValuesRootDir:examples/chartvalues/appdeploy]]] targetCluster:minikube] map[]}
first-pass uses: &{catapp map[chartConfigs:map[appdeploy:map[chartValues:map[baseValuesRootDir:examples/chartvalues/appdeploy]]] targetCluster:minikube] map[]}
envvals_loader: loaded statevalues/000-globals.yaml:map[chartConfigs:map[appconduits:map[chart:bitsofinfo-appconduits/appconduits version:1.0.9] appdeploy:map[chart:bitsofinfo-appdeploy/appdeploy version:1.1.5]] releaseConfigs:map[conduits:map[selectorlabelNames:map[appClassifier:app-classifier appName:app-name appVersion:app-version]] deployments:<nil>]]
envvals_loader: loaded statevalues/001-clusters.yaml:map[clusters:map[minikube:map[helmDefaults:map[kubeContext:minikube tillerNamespace:kube-system wait:true] name:minikube]]]
envvals_loader: loaded examples/statevalues/*:map[chartConfigs:map[appconduits:map[chartValues:map[baseValues:[testapps]]] appdeploy:map[chartValues:map[baseValues:[testapps]]]]]
envvals_loader: loaded examples/statevalues/*:map[clusters:map[minikube:map[ingressControllers:map[default:map[url:https://bitsofinfo-traefik.test.local]]]]]
first-pass produced: &{catapp map[chartConfigs:map[appdeploy:map[chartValues:map[baseValuesRootDir:examples/chartvalues/appdeploy]]] targetCluster:minikube] map[chartConfigs:map[appconduits:map[chart:bitsofinfo-appconduits/appconduits chartValues:map[baseValues:[testapps]] version:1.0.9] appdeploy:map[chart:bitsofinfo-appdeploy/appdeploy chartValues:map[baseValues:[testapps]] version:1.1.5]] clusters:map[minikube:map[helmDefaults:map[kubeContext:minikube tillerNamespace:kube-system wait:true] ingressControllers:map[default:map[url:https://bitsofinfo-traefik.test.local]] name:minikube]] releaseConfigs:map[conduits:map[selectorlabelNames:map[appClassifier:app-classifier appName:app-name appVersion:app-version]] deployments:<nil>]]}
first-pass rendering result of "deployments.helmfile.yaml.part.0": {catapp map[chartConfigs:map[appdeploy:map[chartValues:map[baseValuesRootDir:examples/chartvalues/appdeploy]]] targetCluster:minikube] map[chartConfigs:map[appconduits:map[chart:bitsofinfo-appconduits/appconduits chartValues:map[baseValues:[testapps]] version:1.0.9] appdeploy:map[chart:bitsofinfo-appdeploy/appdeploy chartValues:map[baseValues:[testapps]] version:1.1.5]] clusters:map[minikube:map[helmDefaults:map[kubeContext:minikube tillerNamespace:kube-system wait:true] ingressControllers:map[default:map[url:https://bitsofinfo-traefik.test.local]] name:minikube]] releaseConfigs:map[conduits:map[selectorlabelNames:map[appClassifier:app-classifier appName:app-name appVersion:app-version]] deployments:<nil>]]}
vals:
map[chartConfigs:map[appconduits:map[chart:bitsofinfo-appconduits/appconduits chartValues:map[baseValues:[testapps]] version:1.0.9] appdeploy:map[chart:bitsofinfo-appdeploy/appdeploy chartValues:map[baseValues:[testapps] baseValuesRootDir:examples/chartvalues/appdeploy] version:1.1.5]] clusters:map[minikube:map[helmDefaults:map[kubeContext:minikube tillerNamespace:kube-system wait:true] ingressControllers:map[default:map[url:https://bitsofinfo-traefik.test.local]] name:minikube]] releaseConfigs:map[conduits:map[selectorlabelNames:map[appClassifier:app-classifier appName:app-name appVersion:app-version]] deployments:<nil>] targetCluster:minikube]
defaultVals:[statevalues/000-globals.yaml statevalues/001-clusters.yaml examples/statevalues/*]
second-pass rendering result of "deployments.helmfile.yaml.part.0":
 0: values:
 1:   - statevalues/000-globals.yaml
 2:   - statevalues/001-clusters.yaml
 3:   - examples/statevalues/*
...
<NOTE! I excluded a TON of output I am not interested in here for brevity of the copy-paste>
...
second-pass rendering result of "deployments.helmfile.yaml.part.3":
 0:
 1:
 2: releases:
 3:
 4:
 5:
 6:
 7:
 8:
 9:
10:
11:
12:   - name: catapp-stage-qa-3-0-0
13:     labels:
14:       app: catapp
15:       context: stage-qa
16:     namespace: bitsofinfo-apps
17:     chart: bitsofinfo-appdeploy/appdeploy
18:
19:     version: 1.1.5
20:
21:
22:
23:     installed: true
24:
25:
26:     values:
27:       - examples/chartvalues/appdeploy/values/testapps/values.yaml
28:
29:
30:       - app:
31:           name: catapp
32:           context: stage-qa
33:           environment: stage
34:
35:
36:         creatorId: <no value>
37:         replicaCount: 1
38:
39:         image:
40:           repository: bitsofinfo/catapp
41:           tag: "3.0.0"
42:
43:
44:
45:
46:
47:
48:
49:         bootstrapSecret:
50:           k8Secret:
51:             secretValue: "<no value>"
52:         env:
53:           INGRESS_CONTROLLER_URL:
54:
55:             value: https://bitsofinfo-traefik.test.local
56:
57:
58:
59:
60:
61:
62:
63:
64:
65:
66:
67:
68:
69:
70:
71:
72:
73:
74:   - name: catapp-stage-qa-4-0-0
75:     labels:
76:       app: catapp
77:       context: stage-qa
78:     namespace: bitsofinfo-apps
79:     chart: bitsofinfo-appdeploy/appdeploy
80:
81:     version: 1.1.5
82:
83:
84:
85:     installed: true
86:
87:
88:     values:
89:       - examples/chartvalues/appdeploy/values/testapps/values.yaml
90:
91:
92:       - app:
93:           name: catapp
94:           context: stage-qa
95:           environment: stage
96:
97:
98:         creatorId: <no value>
99:         replicaCount: 1
100:
101:         image:
102:           repository: bitsofinfo/catapp
103:           tag: "4.0.0"
104:
105:
106:
107:
108:
109:
110:
111:         bootstrapSecret:
112:           k8Secret:
113:             secretValue: "<no value>"
114:         env:
115:           INGRESS_CONTROLLER_URL:
116:
117:             value: https://bitsofinfo-traefik.test.local
118:
119:
120:
121:
122:
123:
124:
125:
126:
127:
128:
129:
130:
131:
132:
133:
134:
135:
136:   - name: catapp-stage-qa-2-0-0
137:     labels:
138:       app: catapp
139:       context: stage-qa
140:     namespace: bitsofinfo-apps
141:     chart: bitsofinfo-appdeploy/appdeploy
142:
143:     version: 1.1.5
144:
145:
146:
147:     installed: true
148:
149:
150:     values:
151:       - examples/chartvalues/appdeploy/values/testapps/values.yaml
152:
153:
154:       - app:
155:           name: catapp
156:           context: stage-qa
157:           environment: stage
158:
159:
160:         creatorId: <no value>
161:         replicaCount: 1
162:
163:         image:
164:           repository: bitsofinfo/catapp
165:           tag: "2.0.0"
166:
167:
168:
169:
170:
171:
172:
173:         bootstrapSecret:
174:           k8Secret:
175:             secretValue: "<no value>"
176:         env:
177:           INGRESS_CONTROLLER_URL:
178:
179:             value: https://bitsofinfo-traefik.test.local
180:
181:
182:
183:
184:
185:
186:
187:
188:
189:
190:
191:
192:
193:
194:
195:
196:
197:
198:   - name: catapp-stage-qa-1-0-0
199:     labels:
200:       app: catapp
201:       context: stage-qa
202:     namespace: bitsofinfo-apps
203:     chart: bitsofinfo-appdeploy/appdeploy
204:
205:     version: 1.1.5
206:
207:
208:
209:     installed: false
210:
211:
212:     values:
213:       - examples/chartvalues/appdeploy/values/testapps/values.yaml
214:
215:
216:       - app:
217:           name: catapp
218:           context: stage-qa
219:           environment: stage
220:
221:
222:         creatorId: <no value>
223:         replicaCount: 1
224:
225:         image:
226:           repository: bitsofinfo/catapp
227:           tag: "1.0.0"
228:
229:
230:
231:
232:
233:
234:
235:         bootstrapSecret:
236:           k8Secret:
237:             secretValue: "<no value>"
238:         env:
239:           INGRESS_CONTROLLER_URL:
240:
241:             value: https://bitsofinfo-traefik.test.local
242:
243:
244:
245:
246:
247:
248:
249:
250:
251:

merged environment: &{catapp map[appname:catapp chartConfigs:map[appdeploy:map[chartValues:map[baseValuesRootDir:examples/chartvalues/appdeploy values:map[env:map[INGRESS_CONTROLLER_URL:map[value:{{.helmfile.targetCluster.ingressControllers.default.url}}]] image:map[repository:bitsofinfo/catapp]]]]] environments:map[stage:map[chartConfigs:map[appdeploy:map[chartValues:map[values:map[replicaCount:1]]]] contexts:map[stage-qa:map[ingress:map[animals-current:map[mappings:[map[hosts:[map[name:animals-current.mydomain.com]] labels:map[bitsofinfo-ingress:yes] name:paths paths:[/orange /red] serviceBindings:map[current:100%]]]] animals-mixed:map[mappings:[map[hosts:[map[name:animals.mydomain.com]] labels:map[bitsofinfo-ingress:yes] name:paths paths:[/orange /red] serviceBindings:map[current:40% next:30% previous:30%]]]] animals-next:map[mappings:[map[hosts:[map[name:animals-next.mydomain.com]] labels:map[bitsofinfo-ingress:yes] name:paths paths:[/orange /red] serviceBindings:map[next:100%]]]] animals-previous:map[mappings:[map[hosts:[map[name:animals-previous.mydomain.com]] labels:map[bitsofinfo-ingress:yes] name:paths paths:[/orange /red] serviceBindings:map[previous:100%]]]] current:map[mappings:[map[hosts:[map[name:catapp-current.mydomain.com]] labels:map[bitsofinfo-ingress:yes] name:all serviceBindings:map[current:100%]]]] mixed:map[mappings:[map[hosts:[map[name:catapp-mixed.mydomain.com]] labels:map[bitsofinfo-ingress:yes] name:all serviceBindings:map[current:40% next:30% previous:30%]]]] next:map[mappings:[map[hosts:[map[name:catapp-next.mydomain.com]] labels:map[bitsofinfo-ingress:yes] name:all serviceBindings:map[next:100%]]]] previous:map[mappings:[map[hosts:[map[name:catapp-previous.mydomain.com]] labels:map[bitsofinfo-ingress:yes] name:all serviceBindings:map[previous:100%]]]]] services:[map[installed:true name:current version:3.0.0] map[installed:true name:next version:4.0.0] map[installed:true name:previous version:2.0.0] map[installed:false name:old version:1.0.0]]]]]] targetCluster:minikube] map[chartConfigs:map[appconduits:map[chart:bitsofinfo-appconduits/appconduits chartValues:map[baseValues:[testapps]] version:1.0.9] appdeploy:map[chart:bitsofinfo-appdeploy/appdeploy chartValues:map[baseValues:[testapps]] version:1.1.5]] clusters:map[minikube:map[helmDefaults:map[kubeContext:minikube tillerNamespace:kube-system wait:true] ingressControllers:map[default:map[url:https://bitsofinfo-traefik.test.local]] name:minikube]] releaseConfigs:map[conduits:map[selectorlabelNames:map[appClassifier:app-classifier appName:app-name appVersion:app-version]] deployments:<nil>]]}
4 release(s) matching context=stage-qa found in deployments.helmfile.yaml
...
< AGAIN I TRIMMED OUT A BUNCH OF OUTPUT FOR BREVITY >
...
---
apiVersion: batch/v1
kind: Job
metadata:
  name: "catappstageqa30080hookpdvalidator"
  namespace: "bitsofinfo-apps"
  labels:

    "app-name": catapp
    "app-context": stage-qa
    "app-environment": stage
    "app-version": 3.0.0
    "app-hash-id": "1426130644"

    "app-creator-id": ""
    "app.kubernetes.io/name": catapp
    "app.kubernetes.io/managed-by": Tiller
    "app.kubernetes.io/instance": catapp-stage-qa-3-0-0
    "app.kubernetes.io/version": 3.0.0
    "app.kubernetes.io/part-of": catapp
    "app.kubernetes.io/component": "app"
    "helm.sh/chart": appdeploy-1.1.5
  annotations:
    "helm.sh/hook": post-delete
    "helm.sh/hook-weight": "2"
    "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation
spec:
  backoffLimit: 0 # retry 0 times
  activeDeadlineSeconds: 300 # max run for 10 minutes (i.e. inclusive of retries)
  template:
    metadata:
      name: "catappstageqa30080hookpdvalidator"
    spec:
      restartPolicy: Never
      volumes:
        - name: config-volume
          configMap:
            name: catapp-stage-qa-3-0-0-80-hook-pd-validator
      containers:
        - name: bitsofinfo-kubernetes-helm-healthcheck-hook-0-1-18
          image: "bitsofinfo/kubernetes-helm-healthcheck-hook:0.1.18"
          command:
            - "checker.py"
          args:
            - "--check-name"

            - "catapp-stage-qa-3-0-0.bitsofinfo-apps.svc.cluster.local:80"

            - "--target-root-url"

            - "http://catapp-stage-qa-3-0-0.bitsofinfo-apps.svc.cluster.local:80"

            - "--max-retries"
            - "60"
            - "--sleep-seconds"
            - "30"
            - "--any-check-fail-exit-code"
            - "1"
            - "--extra-slack-context-props"
            - "creatorId="
            - "--checksdb-filename"
            - "/etc/hc-config/healthchecks.config.yaml"
            - "--slack-config-filename"
            - "/etc/hc-config/slackalerts.config.yaml"

          volumeMounts:
            - name: config-volume
              mountPath: /etc/hc-config

The above is a lot of output that I am not interested in.

Out of the above output, all I am interested is what follows below, without the line numbers:

 0:
 1:
 2: releases:
 3:
...
10:
11:
12:   - name: catapp-stage-qa-3-0-0
13:     labels:
14:       app: catapp
15:       context: stage-qa
16:     namespace: bitsofinfo-apps
17:     chart: bitsofinfo-appdeploy/appdeploy
18:
19:     version: 1.1.5
20:
21:
22:
23:     installed: true
24:
25:
26:     values:
27:       - examples/chartvalues/appdeploy/values/testapps/values.yaml
28:
29:
30:       - app:
31:           name: catapp
32:           context: stage-qa
33:           environment: stage
34:
35:
36:         creatorId: <no value>
37:         replicaCount: 1
38:
39:         image:
40:           repository: bitsofinfo/catapp
41:           tag: "3.0.0"
42:
...
47:
48:
49:         bootstrapSecret:
50:           k8Secret:
51:             secretValue: "<no value>"
52:         env:
53:           INGRESS_CONTROLLER_URL:
54:
55:             value: https://bitsofinfo-traefik.test.local
56:
...
72:
73:
74:   - name: catapp-stage-qa-4-0-0
75:     labels:
76:       app: catapp
77:       context: stage-qa
78:     namespace: bitsofinfo-apps
79:     chart: bitsofinfo-appdeploy/appdeploy
80:
81:     version: 1.1.5
82:
83:
84:
85:     installed: true
86:
87:
88:     values:
89:       - examples/chartvalues/appdeploy/values/testapps/values.yaml
90:
91:
92:       - app:
93:           name: catapp
94:           context: stage-qa
95:           environment: stage
96:
97:
98:         creatorId: <no value>
99:         replicaCount: 1
100:
101:         image:
102:           repository: bitsofinfo/catapp
103:           tag: "4.0.0"
104:
105:
106:
107:
108:
109:
110:
111:         bootstrapSecret:
112:           k8Secret:
113:             secretValue: "<no value>"
114:         env:
115:           INGRESS_CONTROLLER_URL:
116:
117:             value: https://bitsofinfo-traefik.test.local
118:
...
135:
136:   - name: catapp-stage-qa-2-0-0
137:     labels:
138:       app: catapp
139:       context: stage-qa
140:     namespace: bitsofinfo-apps
141:     chart: bitsofinfo-appdeploy/appdeploy
142:
143:     version: 1.1.5
144:
145:
146:
147:     installed: true
148:
149:
150:     values:
151:       - examples/chartvalues/appdeploy/values/testapps/values.yaml
152:
153:
154:       - app:
155:           name: catapp
156:           context: stage-qa
157:           environment: stage
158:
159:
160:         creatorId: <no value>
161:         replicaCount: 1
162:
163:         image:
164:           repository: bitsofinfo/catapp
165:           tag: "2.0.0"
166:
167:
168:
169:
170:
171:
172:
173:         bootstrapSecret:
174:           k8Secret:
175:             secretValue: "<no value>"
176:         env:
177:           INGRESS_CONTROLLER_URL:
178:
179:             value: https://bitsofinfo-traefik.test.local
180:
181:
...
195:
196:
197:
198:   - name: catapp-stage-qa-1-0-0
199:     labels:
200:       app: catapp
201:       context: stage-qa
202:     namespace: bitsofinfo-apps
203:     chart: bitsofinfo-appdeploy/appdeploy
204:
205:     version: 1.1.5
206:
207:
208:
209:     installed: false
210:
211:
212:     values:
213:       - examples/chartvalues/appdeploy/values/testapps/values.yaml
214:
215:
216:       - app:
217:           name: catapp
218:           context: stage-qa
219:           environment: stage
220:
221:
222:         creatorId: <no value>
223:         replicaCount: 1
224:
225:         image:
226:           repository: bitsofinfo/catapp
227:           tag: "1.0.0"
228:
229:
230:
231:
232:
233:
234:
235:         bootstrapSecret:
236:           k8Secret:
237:             secretValue: "<no value>"
238:         env:
239:           INGRESS_CONTROLLER_URL:
240:
241:             value: https://bitsofinfo-traefik.test.local
...

I guess what I want to do exactly with this releases output is not super relevant imho, but in the end, I simply would like to get this releases: output, minus the numbers.

That said, I'm well aware I could certainly pipe this output to another routine that parses out what I want, but I'm just curious if such a command line option could be added that could give me this without much trouble. If its a ton of work then just forget it I guess.

I'd qualify this as more of a nice to have.