How to replace a variable in a template

[MeijerM1]

Hi,

Is there a way to replace a variable across a resource without using a patch.yaml or patch.json file?

Say I have the following file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment
  labels:
    app: VARIABLE_WITH_APP_NAME
  namespace: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: VARIABLE_WITH_APP_NAME
  template:
    metadata:
      labels:
        app: VARIABLE_WITH_APP_NAME
    spec:
      containers:
      - name: VARIABLE_WITH_APP_NAME
        image: image:1

Is there a way to replace VARIABLE_WITH_APP_NAME in the kustomize file in a simple way?

[k8s-ci-robot]

@MeijerM1: This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[kferrone]

Not really what the kustomize folks would intend. You could inline the patches instead of using patch.yaml, something like this

patch: |-
  - op: add
    path: /spec/foo
    value: bar

The other option for what you are doing is to simply use the commonLabels to add the labels part. Then use the replacements to reference the name on the containers.

Lastly, for those rare times nothing in Kustomize can do what I want, then I make a plugin using bash and some sed magic.

[arash-bizcover]

Lacking this feature is crazy. Customize folks don't like variables inside the Yaml. They say Yaml in Yaml out only.
https://kubectl.docs.kubernetes.io/faq/kustomize/eschewedfeatures/

But weirdly enough they have agreed to have tons of complicated transformers. and something as crazy as this:
https://kubectl.docs.kubernetes.io/guides/extending_kustomize/execpluginguidedexample/

[kferrone]

The variables idea tends to be too abstract and hard to re-patch, ie once a variable is set it usually can't be set again by something else without completely replacing the value. The plugins are very powerful and easy to use, at least in my opinion. They can be written in any language too.

If you are interested, here is my little plugin based on this example in kustomize: https://github.com/kubernetes-sigs/kustomize/blob/master/plugin/someteam.example.com/v1/sedtransformer/SedTransformer

VarTransformer

#!/usr/bin/env bash
​
set -e
​
res="$(cat $1)"
sedMap=()
​
length=$(echo "$res" | yq e '.vars | length' -)
​
for (( i=0; i<$length; i++ ))
do
   var="$(echo "$res" | yq e '.vars['$i']' -)"
   varName=$(echo "$var" | yq e '.name' -)
   if [ "$(echo "$var" | yq e '. | has("env")' -)" == 'true' ]
   then
      envVar=$(echo "$var" | yq e '.env' -)
      var="$(echo "$var" | yq e '.value = env('$envVar')' -)"
   fi
   varValue="$(echo "$var" | yq e '.value' -)"
   sedMap+=(-e "s#\$(${varName})#${varValue}#g")
done
​
# echo "${sedMap[@]}"
sed "${sedMap[@]}"

example-vars.yaml

apiVersion: someteam.example.com/v1
kind: VarTransformer
metadata:
  name: test-variables
vars:
- name: message
  value: hello world
- name: how_many
  value: 20
- name: some.url
  value: example.com
- name: whoami
  env: USER
  
- name: user.home
  env: HOME
​
# multi line value with spaces in name
- name: fancy message
  value: no multi line allowed in VarTransformer

service.yaml

apiVersion: v1
kind: Service
metadata:
  name: some-place
  labels:
    user: $(whoami)
spec:
  type: ExternalName
  sessionAffinity: None
  externalName: $(some.url)

And for a totally different way to think about variables in a patching environment:

1. make a configmap with all your variables
2. use the replacement transformer to target configmap and set variables all over your project

I totally agree kustomize should avoid template style variables. They are easy enough to implement in code and Helm is a dedicated templating platform which easily integrates with kustomize. Either you can use a helm operator or kustomizes helm inflator, see here: https://kubectl.docs.kubernetes.io/guides/extending_kustomize/builtins/#_helmchartinflationgenerator_

[natasha41575]

Thank you @kferrone, I agree with all of your points and want to emphasize that template variables are out of scope for kustomize. We provide ways to extend kustomize via plugins and KRM functions, as well as helm support if you need this kind of functionality.

[MeijerM1]

Thanks for all the input and answers everyone!

We'll be able to figure out a solution from here.

[almson]

Makes no sense to me either. I can understand wanting to make a lightweight alternative to Helm, but avoiding templating entirely?

A template is like a base class that explicitly supports inheritance and can be updated without breaking its children. By rejecting templates, I think what Kustomize is aiming for is to only allow haphazard, brittle inheritance.

Why?! Who thought of this?!

Edit: Oh yeah, I forgot, Golang doesn’t even have inheritance. That’s who thought of this. The blind leading the blind… programming in the 21st century.

[kferrone]

Ermm . . .
@almson You missed the point. There are sooo many templating engines out there, Helm is not the only one. Kustomize works like kubernetes, as in kubernetes doesn't do the ingress, nginx does the ingress and kustomize doesn't template because helm, go, handlebars, symphony, yqtt, jsonnet etc does templating. Kubernetes and kustomize are basically just yamlized orchestration. If you want either to do something it can't, simply install an operator/plugin.

If a plugin does not exist then make it yourself. I did share an extremely simple example of using sed for templating above. I even use cat for simple templating. The plugins are simply a way to pass your values to whatever tool you want, even if it's a templating tool.

Plus, templating isn't yaml, it's some templating language, kubernetes/kustomize does yaml.

[almson]

@kferrone You have it backwards. Kustomize hates templating. That is their ideological position. They hate it so much that they're deprecating vars.

Kustomize is not a templating engine--it's a hacking engine. Its purpose is to make hacky workarounds to others' half-assed deployment files. Occasionally that's useful, but it should not be the primary means of managing kubernetes. Deployment descriptors should have some sort of parametrization and/or predetermined extension points.

In any case, is it possible to declare a dependency on a plugin and have it be automatically installed from a repo? If not, then that's hardly a convenient mechanism.

[natasha41575]

@almson We are deprecating vars because our desire is to have raw yaml configuration that can still be used without having to render it first. This makes it more extensible and reusable. For those who desire this kind of behavior, we support helm configuration and provide extensibility mechanisms. I am not sure why you need additional templating support, if we already support other tools that allow templating.

In any case, is it possible to declare a dependency on a plugin and have it be automatically installed from a repo?

I suggest you take a look at the catalog proposal. You may find it interesting: kubernetes/enhancements#2908

[nolem]

@kferrone You have it backwards. Kustomize hates templating. That is their ideological position. They hate it so much that they're deprecating vars.

Kustomize is not a templating engine--it's a hacking engine. Its purpose is to make hacky workarounds to others' half-assed deployment files. Occasionally that's useful, but it should not be the primary means of managing kubernetes. Deployment descriptors should have some sort of parametrization and/or predetermined extension points.

In any case, is it possible to declare a dependency on a plugin and have it be automatically installed from a repo? If not, then that's hardly a convenient mechanism.

Im currently at a point, where i must replace a placeholder in a template string. This was quite easy with the vars approach. But since they deprecated this vars the hell broke out. For some use-cases it is an absolute desaster to prevent vars. Just to mention... tools are for people and devs, not for ideology. That they removed the namespace prefixing was not nice. But the deprecation of the vars enforced us to build hellish workarounds or to switch to another framework. So maybe a reasonable point to reconsider.

[aitorpazos]

Real life example. I have a resource (Keda ScaledObject in this case) which requires a long property value which I want to maintain in a single place. However, that value needs some tweaking to be specific to the cluster the deployment is in:

query: avg_over_time((sum(registry_http_in_flight_requests{datacenter=~"dc1.*"})/count(registry_http_in_flight_requests{datacenter=~"dc1.*"}))[90s:])

IMO it is super useful to be able to have a variable that I can override on each overlay to pass dc1, dc2, etc values. Otherwise I am forced to maintained the whole query value in multiple places or resort to external tooling/plugins for such basic thing in a tool that is meant to help managing kustomizing config for different environments.

[kferrone]

I made a variable replacer in SED. I also made variable replacer which is much more advanced in Python using the string formatter. I made a variable replacer using a cli tool called YQ. There are so many ways to replace a simple variable, if you know how to code and have an imagination.

I shared my SED replacer above. It's used in production all the time. Sooooo basic and reliable.

Kustomize has a Helm chart inflator which can be local with one chart.yaml and at least one template. I wrapped Jinja templating into a super basic plugin, requires a single my.yaml.jinja. I often make simple little cat templates in BASH, like 5min scripts with full blown access to environment variables. So much templating in Kustomize it's almost hard to choose from.

Kustomize is just an ETL tool which extracts yaml from a file or a generator. Then it transforms with either built in transformers or user defined KRM functions which can do whatever you want. Lastly it loads into kubectl.

Thinking outside the box, if you really need full blown variable replacement and live references to other resources in the cluster, check out kyverno. It's kinda like Kustomize but runs in the cluster as an AdmissionController.

Lastly one must understand a ConfigMap is a collection of variables. These variables can be placed into other resources using the built in ReplacementTransformer. Then add the following annotation to the ConfigMap to filter it out of the output: config.kubernetes.io/local-config: 'true'. Variables!

[MaurGi]

envsubst