Question, possible to do a helm chart install with both cockroachdb & zitadel?

[lknite]

I try to install both at the same time but looks like zitadel wants to run before cockroachdb has a chance to install.

I'm wanting to install this way because of gitops.

Tried to disable the initJob, but that didn't help.

$ cat Chart.yaml 
apiVersion: v2
name: zitadel
description: A Helm chart for Kubernetes

# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application

# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
appVersion: "1.0"

dependencies:
- name: cockroachdb
  version: 11.2.1
  #version: 14.0.5
  repository: https://charts.cockroachdb.com/
- name: zitadel
  version: 8.5.0
  repository: https://charts.zitadel.com

$ cat values.yaml 
cockroachdb:

  tls:
    enabled: false

zitadel:
  initJob:
    # Once ZITADEL is installed, the initJob can be disabled.
    enabled: false

  zitadel:
    masterkey: x123456789012345678901234567891y
    configmapConfig:
      ExternalSecure: false
      ExternalDomain: 127.0.0.1.sslip.io
      TLS:
        Enabled: false
      Database:
        Cockroach:
          Host: db-cockroachdb-public

[hifabienne]

I move this into a discussion as it is a question.

[hifabienne]

@eliobischof can you help here?

[lknite]

After visiting kubecon this year I've decided to switch to yugabyte, due to the less than ideal current state of cockroachdb charts, am running into the same issue with yugabyte. Looks like zitadel is making an effort to make sure the database exists before installing but its blocking the database install. Seems like an option we'd want to be able to toggle in zitadel.

dependencies:
- name: zitadel
  version: 8.5.0
  repository: https://charts.zitadel.com 
- name: yugabyte
  version: 2024.1.3
  repository: https://charts.yugabyte.com

[eliobischof]

We don't support Yugabyte, yet

[lknite]

According to this older article, waiting for the database is a 'kubernetes anti-pattern', instead it should just fail until the database becomes available ... https://stackoverflow.com/questions/66027083/how-to-define-start-order ; could this be right?

Maybe instead of using a helm hook you could have an initContainer that waits for the database to come up?

[eliobischof]

I like the idea of having init jobs to get rid of the hooks. I'm not exactly sure yet how to do this for the setup job and deployment though. Would you mind creating an issue?

[lknite]

Issue created:
https://github.com/zitadel/zitadel/issues/8952

[lknite]

Currently working on an error:

time="2024-11-17T03:05:02Z" level=error msg="unexpected query error" caller="/home/runner/work/zitadel/zitadel/internal/database/database.go:88" error="ERROR: relation \"system.encryption_keys\" does not exist (SQLSTATE 42P01)"
Error: ID=START-aGBq2 Message=cannot create default keys Parent=(ID= Message=unable to insert new keys Parent=(ERROR: relation "system.encryption_keys" does not exist (SQLSTATE 42P01)))
Usage:
  zitadel start [flags]

This got things moving but its not fully working:

$ cat values.yaml 
yugabyte:

  preflight:
    skipUlimit: true

  resource:
    replicas:
      master: 1
      tserver: 1

    master:
      requests:
        cpu: "1"
        memory: 2Gi
      limits:
        cpu: "2"
        ## Ensure the 'memory' value is strictly in 'Gi' or 'G' format. Deviating from these formats
        ## may result in setting an incorrect value for the 'memory_limit_hard_bytes' flag.
        ## Avoid using floating numbers for the numeric part of 'memory'. Doing so may lead to
        ## the 'memory_limit_hard_bytes' being set to 0, as the function expects integer values.
        memory: 2Gi
    tserver:
      requests:
        cpu: "1"
        memory: 4Gi
      limits:
        cpu: "2"

zitadel:

  zitadel:
    masterkey: x123456789012345678901234567891y
    configmapConfig:
      ExternalSecure: false
      ExternalDomain: 127.0.0.1.sslip.io
      TLS:
        Enabled: false
      Database:
        Postgres:
          Host: yb-tservers
          Port: 5433
          Database: yugabyte
          MaxOpenConns: 20
          MaxIdleConns: 10
          MaxConnLifetime: 30m
          MaxConnIdleTime: 5m
          User:
            Username: yugabyte
            SSL:
              Mode: disable
          Admin:
            Username: yugabyte
            SSL:
              Mode: disable

    secretConfigAnnotations:
      helm.sh/hook: ""
      helm.sh/hook-delete-policy: ""
      helm.sh/hook-weight: ""

    masterkeyAnnotations:
      helm.sh/hook: "pre-install"
      helm.sh/hook-delete-policy: ""
      helm.sh/hook-weight: ""

  configMap:
    annotations:
      helm.sh/hook: "pre-install"
      helm.sh/hook-delete-policy: ""
      helm.sh/hook-weight: ""

  serviceAccount:
    # Annotations to add to the service account
    annotations:
      helm.sh/hook: "pre-install"
      helm.sh/hook-delete-policy: ""
      helm.sh/hook-weight: ""

  initJob:
    # Once ZITADEL is installed, the initJob can be disabled.
    enabled: true
    annotations:
      helm.sh/hook: ""
      helm.sh/hook-delete-policy: ""
      helm.sh/hook-weight: ""

  setupJob:
    annotations:
      helm.sh/hook: ""
      helm.sh/hook-delete-policy: ""
      helm.sh/hook-weight: ""

[lknite]

Installed keycloak instead so I can have something working while this is figured out. Could use an assist on this.