[Bug] check-elasticsearch-index init container does not accept elasticsearch password supplied via existingSecret

[runitmisra]

What are you really trying to do?

I use ECK Operator to run an external elasticsearch instance. I want to supply elasticsearch password using the existingSecret key and have all temporal components/jobs/containers pick it up.

Describe the bug

ECK creates a kubernetes secret for credentials. When I supply the secret to temporal values, The temporal-schema job runs fine, but check-elasticsearch-index init containers in all components(frontend, history, matching and worker) keep reporting waiting for elasticsearch index to become ready indefinitely. Looking at the curl command resulting from the templates, it does not include the elasticsearch credentials as it expects the username and password to be provided as plaintext.

I use the following config for elasticsearch:

elasticsearch:
  enabled: false
  host: elasticsearch-es-http
  scheme: http
  port: 9200
  version: "v7"
  logLevel: "error"
  username: "elastic"
  existingSecret: "elasticsearch-es-elastic-user"
  secretKey: "elastic"
  visibilityIndex: "temporal_visibility_v1_dev"

I believe this the culprit:

templates/server-deployment.yaml

        {{- if or $.Values.elasticsearch.enabled $.Values.elasticsearch.external }}
        - name: check-elasticsearch-index
          image: "{{ $.Values.admintools.image.repository }}:{{ $.Values.admintools.image.tag }}"
          imagePullPolicy: {{ $.Values.admintools.image.pullPolicy }}
          command: ['sh', '-c', 'until curl --silent --fail {{- if and $.Values.elasticsearch.username $.Values.elasticsearch.password }} --user "{{ $.Values.elasticsearch.username }}:{{ $.Values.elasticsearch.password }}" {{- end }} {{ $.Values.elasticsearch.scheme }}://{{ $.Values.elasticsearch.host }}:{{ $.Values.elasticsearch.port }}/{{ $.Values.elasticsearch.visibilityIndex }} 2>&1 > /dev/null; do echo waiting for elasticsearch index to become ready; sleep 1; done;']
        {{- end }}

This curl command template does not take credentials from existingSecret into consideration.

Minimal Reproduction

• Install elasticsearch using ECK operator
• supply kubernetes secret to temporal values (code snippet above)
• Install Temporal
• temporal-schema job runs fine
• All temporal pods with check-elasticsearch-index init container get stuck since elasticsearch does not get the required creds from curl command.

Environment/Versions

• OS and processor: Linux
• Temporal Version: 1.24.2
• Using kubernetes v1.25.3

Additional context

Happy to provide in comments if required.

[robholland]

Solution is probably to use _admintools-env like we do in the schema job, and env vars for the curl command.

[robholland]

Duplicate of: #529

[robholland]

Fixed by: #530