v0.24.0: broken bash variable evaluation

[r0bj]

Expected Behavior

bash variable evaluation works fine.

Actual Behavior

bash variable evaluation is broken.

Steps to Reproduce the Problem

taskrun.yaml:

---
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: test
spec:
  taskSpec:
    steps:
    - name: test
      image: bash:5.1.8
      script: |
        #!/usr/bin/env bash
        set -xe

        var1=var1_value
        var2=var1
        echo $(eval echo \$$var2)

Taskrun result:

+ var1=var1_value
+ var2=var1
++ eval echo '$15var1'
+++ echo 5var1
+ echo 5var1
5var1

Note: Output should be var1_value instead of 5var1.

On tekton pipelines before version v0.24.0 it works as expected:

+ var1=var1_value
+ var2=var1
++ eval echo '$var1'
+++ echo var1_value
+ echo var1_value
var1_value

Additional Info

• Kubernetes version:

  Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.4-dirty", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"dirty", BuildDate:"2021-03-15T09:58:13Z", GoVersion:"go1.16.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:03:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}

• Tekton Pipeline version:

Client version: 0.18.0
Pipeline version: v0.24.0
Triggers version: v0.13.0

[vdemeester]

/cc @sbwsg
This probably relates to #3888

[ghost]

Could you post the generated pod?

Edit: nm, I'll recreate the issue on my side.

[ghost]

The script gets expanded to:

    - args:
      - -c
      - |
        tmpfile="/tekton/scripts/script-0-tqc48"
        touch ${tmpfile} && chmod +x ${tmpfile}
        cat > ${tmpfile} << 'script-heredoc-randomly-generated-vf6hm'
        #!/usr/bin/env bash
        set -xe
        var1=var1_value
        var2=var1
        echo $(eval echo \$$$$var2)

        script-heredoc-randomly-generated-vf6hm
      command:
      - sh

This is the new script behaviour, expected. Kubernetes rewrites $$ as $ so we've doubled-up the number of dollar signs to try and prevent losing characters when k8s eventually rewrites our script.

But then the output is:

 (main) λ k logs -f test-pod-sq7x2
+ var1=var1_value
+ var2=var1
++ eval echo '$16var1'
+++ echo 6var1
+ echo 6var1
6var1

Sigh.

OK I'll make a revert commit and reopen #3871

[ghost]

For someone who doesn't know the infinite reaches of bash scripting, what is eval echo \$$$$var2? Why so many dollars and a backslash, is this related to something in the way eval works?

Never mind. The 4 dollar signs are tekton. This is so confusing.

[ghost]

For posterity's sake, here's how the script is rendered prior to 0.24.0:

    - args:
      - -c
      - |
        tmpfile="/tekton/scripts/script-0-fblm6"
        touch ${tmpfile} && chmod +x ${tmpfile}
        cat > ${tmpfile} << 'script-heredoc-randomly-generated-wz24z'
        #!/usr/bin/env bash
        set -xe
        var1=var1_value
        var2=var1
        echo $(eval echo \$$var2)

        script-heredoc-randomly-generated-wz24z
      command:
      - sh

Note \$$var2. See the resulting bash output:

 (tags/v0.23.0) λ k logs -f test-pod-kr8pg
+ var1=var1_value
+ var2=var1
++ eval echo '$var1'
+++ echo var1_value
+ echo var1_value
var1_value

Kubernetes converted the two dollar signs to a single dollar sign and we somehow lost the backslash.

[ghost]

Bit more investigation: Kubernetes does replace instances of two dollar signs, "$$", with a single dollar sign, "$" but it also attempts to replace instances of "$(X)" with the contents of env var X. In this case k8s sees $( from $(eval echo \$$var2) and assumes that "eval echo $$var2" is a variable name. Of course it doesn't have an env var with that name so it simply passes the whole string, verbatim, through to the container.

Long story short the original "fix" was too naive and we should revert this change.