Is this possible to clear a property previously defined ?

[eddycharly]

let's say i have something along those lines :

values:
  - nodeSelector:
      nodeType: infra
  - nodeSelector: null

in the end, nodeSelector is not reset to null.
is this possible to achieve something like this ?

[mumoshu]

@eddycharly Hey! Unfortunately no, it isn't possible and I have no idea how this can be done elegantly.

But what would be your use-case? Perhaps I can suggest alternatives.

[eddycharly]

@mumoshu Hi, my use case is this one :
I have a base layer where I describe my platform infrastructure (more or less a combination of feature flags and helm values), and a specific layer for my environments to enrich/customise what was declared in the base layer on a per environment basis.

My helmfile.yaml is a template that generates all that is necessary for helmfile based on the abstract environment definition.

In the base layer I put nodeSelector statements to drive the placement of the workloads for the releases. But now I am creating a local environment that run on a single cluster, I want to clear nodeSelector statements.

I’ll write an example in a few minutes it will be easier to understand ;)

[eddycharly]

@mumoshu let's say my environments/_base.yaml is like this :

core:
  clusterAutoscaler:
    enabled: true
    nodeSelector:
      kops.k8s.io/instancegroup: infra
  dashboard:
    enabled: true
    nodeSelector:
      kops.k8s.io/instancegroup: infra
  externalDns:
    enabled: true
    nodeSelector:
      kops.k8s.io/instancegroup: infra
  heapster:
    enabled: true
    nodeSelector:
      kops.k8s.io/instancegroup: infra
  metricsServer:
    enabled: true
    nodeSelector:
      kops.k8s.io/instancegroup: infra
  nodeProblemDetector:
    enabled: true
    nodeSelector:
      kops.k8s.io/instancegroup: infra

then my "local" environment environments/microk8s.yaml :

clusterName: microk8s
kubeContext: microk8s
core:
  clusterAutoscaler:
    enabled: false
  dashboard:
    enabled: true
    nodeSelector: null
  externalDns:
    enabled: false
  heapster:
    enabled: false
  metricsServer:
    enabled: false
  nodeProblemDetector:
    enabled: false

and finally my helmfile.yaml :

environments:
  microk8s:
    values:
      - environments/_base.yaml
      - environments/microk8s.yaml
---
repositories:
  - name: stable
    url: https://kubernetes-charts.storage.googleapis.com

helmDefaults:
  kubeContext: {{ .Values.kubeContext }}
  verify: false
  wait: true
  recreatePods: true
  force: true

releases:
# CORE
{{ if .Values.core.dashboard.enabled }}
  - {{ tpl (readFile "templates/core/dashboard.yaml") . | nindent 4 }}
{{ end }}
{{ if .Values.core.heapster.enabled }}
  - {{ tpl (readFile "templates/core/heapster.yaml") . | nindent 4 }}
{{ end }}
{{ if .Values.core.externalDns.enabled }}
  - {{ tpl (readFile "templates/core/external-dns.yaml") . | nindent 4 }}
{{ end }}
{{ if .Values.core.metricsServer.enabled }}
  - {{ tpl (readFile "templates/core/metrics-server.yaml") . | nindent 4 }}
{{ end }}
{{ if .Values.core.nodeProblemDetector.enabled }}
  - {{ tpl (readFile "templates/core/node-problem-detector.yaml") . | nindent 4 }}
{{ end }}
{{ if .Values.core.clusterAutoscaler.enabled }}
  - {{ tpl (readFile "templates/core/cluster-autoscaler.yaml") . | nindent 4 }}
{{ end }}

of course there are a lot more sections than core, i have sections for logging, monitoring, infrastructure, backups, ingress...

what i was trying to do was to define the default nodeSelectors in the environments/_base.yaml and eventually override them in the environment specific file if needed.

[mumoshu]

@eddycharly Thanks for clarifying!

I believe what you are trying to achieve is out of the scope of YAML or configs expressible with Helmfile's API.

But I believe you can still use go templates and sprig functionsou to achieve anything beyond YAML/Helmfile's API.

Try sprig's unset to unset any key without a value.

{{ $overridesFile := printf "environments/%s.yaml" .Environment.Name }}
{{ $base := readFile "environments/_base.yaml" | fromYaml }}
{{ $overrides := readFile $overridesFile | fromYaml }}
{{ $merged := mergeOverwrite (dict) $base $override }}
{{ $keys = keys $merged }}

{{ _, $key := range $keys }}
{{ $val := (pluck $key $merged | first) }}
{{ if not $val }}
{{ unset $merged $key }}
{{ end }}
{{ end }}

environments:
  {{ .Environment.Name }}:
    values:
    -
{{ toYaml $merged | indent 6 }}
---
repositories:
  - name: stable
    url: https://kubernetes-charts.storage.googleapis.com

...

[eddycharly]

@mumoshu for what it’s worth, helm handles null values as a special case, it removes a key instead of merging it https://github.com/helm/helm/blob/master/docs/chart_template_guide/values_files.md#deleting-a-default-key

[mumoshu]

@eddycharly Wow, that's good to know!

I've took another look, and it turned out darccio/mergo#115 needs to be resolved first. Helmfile uses mergo for merging values entries and mergo doesn't support overrinding will nil(yaml null)s yet.

Also - does - nodeSelector: [] instead of - nodeSelector: null work for this specific case? I guess you'd have something like {{if .Values.nodeSelector }}...{{end}} within your chart template(not helmfile template) and the if condition turns false when nodeSelector is an empty array, which means you can use empty arrays as alternatives to nulls in this case.

[eddycharly]

@mumoshu yes, using an empty array in the nodeSelector does the trick. I am not a big fan of this solution though.

For now I ended up with nodeSelector config at the environment level, not in the base definition. It’s a bit more typing but not a big deal.

Anyway I feel like I’m abusing the helmfile templating feature, i think it goes too far into abstracting helm details, and relying on helmfile templating to generate a valid helmfile based on my own definition is somewhat unadapted.

I’m thinking about moving this process in a separate job, using something like gomplate to produce the desired helmfile first, then apply the produced configuration with helmfile.

It should be easier to debug also.

[mumoshu]

@eddycharly Thanks for the confirmation and sharing your insights!

relying on helmfile templating to generate a valid helmfile based on my own definition is somewhat unadapted.
I’m thinking about moving this process in a separate job, using something like gomplate to produce the desired helmfile first, then apply the produced configuration with helmfile.

I fully agree. YAML + Go template is workable, easy to start, but not the best solution especially when you have advanced use-cases.

I'm considering various options like Jsonnet(#814) and CUE(#869) for that. In the context of Helmfile, I started liking CUE. If you're interested, see examples shown in the issue!

[mumoshu]

@eddycharly FWIW, helmfile build helps debugging advanced configs containing many go template expressions, by printing the rendered YAML.

Still, I'm thinking using Jsonnet or CUE is better in terms of expressiveness and readability.

[lruslan]

@mumoshu hi, in my case it seem to be override with {} does not work as well, I have following value lookup hierarchy:

    values:
    - "../values/common.yaml.gotmpl"
    - "../values/{{ requiredEnv "ENVIRONMENT" }}/common.yaml.gotmpl" 

at default level in values/common.yaml.gotmpl I have default values for resources defined:

resources:
  requests:
    cpu: 200m
    memory: 600Mi
  limits:
    cpu: 400m
    memory: 800Mi

and then under specific environement layer I'm trying to override resources with empty value {}: values/production/common.yaml.gotmpl

resources: {}

Unfortunately resulting template for the production will not have resources set to {} but will contain values from common.yaml defined at default level: values/common.yaml.gotmpl

I assume this is result of this issue : #866 (comment)
or maybe it's something entirely different?
Thanks

[eduardobaitello]

Any news about this issue?

I'm looking forward to a feature like Helm does on Deleting a default key.

[jijotj]

In my case, the sub chart I was using had a default CPU limit set in its values.yaml. Following the 4th workaround mentioned here worked for me, as given below:

helmfile:

releases:
  - name: refinery
    namespace: tracing
    chart: PARENT_CHART
    version: 1.2.4
    values: 
      - values/refinery.yaml
      - refinery:
          resources:
            limits:
              cpu: null

values/refinery.yaml

refinery:
  resources:
    requests:
      cpu: 3000m
      memory: 6Gi
    limits:
      memory: 6Gi

PARENT_CHART/values.yaml

refinery:
  resources:
    limits:
      cpu: null
      memory: 2Gi
    requests:
      cpu: 2000m
      memory: 2Gi