Helm Chart Refactoring Proposal: Improving Readability and Maintainability

[emmanuelmathot]

Problem Statement

The current approach of looping over API services to generate Kubernetes resources in our Helm chart makes them difficult to read and cumbersome to maintain. While this approach reduces the number of templates, it creates significant challenges:

1. Poor Readability: Templates contain complex nested conditionals and loops ($.Values 😠)
2. Difficult Maintenance: Changes to template structure affect all services
3. Limited Flexibility: Assumes all services follow the same pattern
4. Debugging Challenges: Hard to trace issues to specific services
5. Upgrade Complexity: Difficult to update individual services independently

Current Implementation

Our current Helm charts use a loop-based approach:

{{- range $serviceName, $v := .Values -}}
{{- if has $serviceName $.Values.apiServices }}
{{- if index $v "enabled" }}
apiVersion: apps/v1
kind: Deployment
# ... deployment spec with complex conditionals
{{- end }}
{{- end }}
{{- end }}

This pattern repeats across deployment, service, configmap, and HPA templates, creating a complex web of conditionals that's difficult to follow.

Proposed Refactoring

Instead of looping over services, we propose creating dedicated template files for each service while maintaining a single chart:

helm-chart/
├── eoapi/
    ├── Chart.yaml
    ├── values.yaml
    ├── templates/
        ├── _helpers.tpl
        ├── db/
        ├── pgstacboostrap/
        ├── services/
            ├── _common.tpl            # Limited common functions
            ├── raster/                # One directory per service
            │   ├── deployment.yaml
            │   ├── service.yaml
            │   ├── configmap.yaml
            │   └── hpa.yaml
            ├── stac/
            │   ├── deployment.yaml
            │   └── ...
            ├── vector/
            └── multidim/
        └── ingress.yaml               # Single ingress for all services

Key Improvements:

1. Service-Specific Template Files

Create dedicated files for each service with explicit configurations:

# templates/services/raster/deployment.yaml
{{- if .Values.raster.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: raster-{{ .Release.Name }}
    gitsha: {{ .Values.gitSha }}
  name: raster-{{ .Release.Name }}
  {{- with .Values.raster.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}
spec:
  progressDeadlineSeconds: 600
  revisionHistoryLimit: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 50%
      maxUnavailable: 0
  selector:
    matchLabels:
      app: raster-{{ .Release.Name }}
  template:
    metadata:
      labels:
        app: raster-{{ .Release.Name }}
    spec:
      serviceAccountName: eoapi-sa-{{ .Release.Name }}
      containers:
      - image: {{ .Values.raster.image.name }}:{{ .Values.raster.image.tag }}
        name: raster
        command:
          {{- toYaml .Values.raster.command | nindent 10 }}
          {{- if (and (.Values.ingress.className) (or (eq .Values.ingress.className "nginx") (eq .Values.ingress.className "traefik"))) }}
          - "--root-path=/raster"
          {{- end }}
        # Probes and other container config
        # ...
        ports:
          - containerPort: {{ .Values.service.port }}
        resources:
          {{- toYaml .Values.raster.settings.resources | nindent 10 }}
        
        {{- if .Values.postgrescluster.enabled }}
        env:
          {{- include "eoapi.pgstacSecrets" . | nindent 12 }}
        {{- end }}
        
        envFrom:
          - configMapRef:
              name: raster-envvar-configmap-{{ .Release.Name }}
        {{- if .Values.db.enabled }}
        - secretRef:
            name: pgstac-secrets-{{ .Release.Name }}
        {{- end }}
        
        {{- include "eoapi.mountServiceSecrets" (dict "service" "raster" "root" .) | nindent 8 }}
        
      {{- with .Values.raster.settings.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.raster.settings.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}
{{- end }}

2. Limited Shared Helper Functions

Create a focused _common.tpl file with minimal, highly reusable helper functions:

{{/* 
Helper function for mounting service secrets
Only extract truly common elements that are mechanical and don't need customization
*/}}
{{- define "eoapi.mountServiceSecrets" -}}
{{- $service := .service -}}
{{- $root := .root -}}
{{- if index $root.Values $service "settings" "envSecrets" }}
{{- range $secret := index $root.Values $service "settings" "envSecrets" }}
- secretRef:
    name: {{ $secret }}
{{- end }}
{{- end }}
{{- end -}}

{{/*
Helper function for common environment variables
*/}}
{{- define "eoapi.commonEnvVars" -}}
{{- $service := .service -}}
{{- $root := .root -}}
- name: SERVICE_NAME
  value: {{ $service | quote }}
- name: RELEASE_NAME
  value: {{ $root.Release.Name | quote }}
- name: GIT_SHA
  value: {{ $root.Values.gitSha | quote }}
{{- end -}}

Each service maintains its own explicit configuration while using these helpers only for the most mechanical parts.

3. Restructured Values File

Flatten the values structure for better readability:

# values.yaml
gitSha: "gitshaABC123"

service:
  port: 8080

ingress:
  enabled: true
  className: "nginx"
  # ... other ingress settings

# Database configuration
postgrescluster:
  enabled: true
  # ... database settings

# Service configurations
raster:
  enabled: true
  image:
    name: ghcr.io/stac-utils/titiler-pgstac
    tag: 1.7.1
  command:
    - "uvicorn"
    - "titiler.pgstac.main:app"
    - "--host=$(HOST)"
    - "--port=$(PORT)"
  autoscaling:
    enabled: false
    # ... autoscaling settings
  settings:
    resources:
      # ... resource settings
    envVars:
      # ... environment variables

stac:
  enabled: true
  # ... stac service settings

# ... other services

4. Single Ingress File

Create a single ingress file that references all services explicitly:

# templates/ingress.yaml
{{- if (and (.Values.ingress.enabled) (eq .Values.ingress.className "nginx")) }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-service-ingress-shared-{{ .Release.Name }}
  # ... annotations
spec:
  ingressClassName: {{ .Values.ingress.className }}
  rules:
    - http:
        paths:
          {{- if .Values.raster.enabled }}
          - pathType: ImplementationSpecific
            path: "/raster(/|$)(.*)"
            backend:
              service:
                name: raster
                port:
                  number: {{ .Values.service.port }}
          {{- end }}
          
          {{- if .Values.stac.enabled }}
          - pathType: ImplementationSpecific
            path: "/stac(/|$)(.*)"
            backend:
              service:
                name: stac
                port:
                  number: {{ .Values.service.port }}
          {{- end }}
          
          # ... other services
{{- end }}

Justification: Why More Files is Better Than Complex Templates

While the proposed approach does result in more files to maintain, the benefits significantly outweigh the drawbacks compared to the current looping approach:

1. Cognitive Overhead Reduction

Current approach: Requires developers to mentally trace through complex loops and conditionals across multiple files.

Proposed approach: Each service's configuration is explicitly defined in its own files, eliminating the mental mapping exercise.

2. Improved Debugging

Current approach:

• Helm template errors don't clearly indicate which service caused the problem
• helm template output combines all services, making it hard to isolate issues
• A single syntax error affects all services

Proposed approach:

• Errors are isolated to specific service files
• helm template output can be filtered by service
• Fixes can be applied to individual services without affecting others

3. Lower Risk Changes

Current approach:

• Changes intended for one service might accidentally affect others
• Template modifications require comprehensive testing of all services
• Difficult to implement service-specific customizations

Proposed approach:

• Changes to one service are contained within its files
• Service-specific customizations don't complicate other services
• Testing can focus on the modified service

4. True Flexibility

Current approach:

• Service-specific requirements lead to complex conditional logic
• Adding unique features to one service complicates the template for all
• The template becomes increasingly brittle as exceptions accumulate

Proposed approach:

• Each service can evolve independently
• New services can be added by copying and modifying existing patterns
• Experimental features can be tested in isolation

5. Real-World Maintenance Benefits

1. Explicit over implicit: The configurations are directly visible, not hidden behind loops
2. Onboarding advantage: New team members can understand one service completely before moving to others
3. Documentation by structure: The file organization itself documents the system architecture
4. Future-proofing: The structure can accommodate changes without growing exponentially in complexity
5. Partial deployments: Makes it easier to implement selective service deployments or upgrades

6. Balanced Approach to DRY

The proposed approach achieves a better balance:

• Explicit service configurations: Each service has its own, easy-to-read configuration
• Limited helper functions: Only the most mechanical, repetitive parts are extracted
• No complex loops: Logic is straightforward and follows the service lifecycle

Implementation Strategy

1. Create the service-specific directories and minimal helper templates
2. Migrate one service (e.g., raster) to the new structure
3. Test thoroughly
4. Migrate remaining services
5. Update the ingress configuration
6. Validate the entire solution

Conclusion

While the proposed approach does create more files, it significantly reduces the overall complexity and cognitive load. The initial investment in restructuring will pay dividends in improved maintainability, reduced debugging time, safer changes, and greater flexibility as the system evolves.

The current approach optimizes for DRY (Don't Repeat Yourself) at the expense of clarity and maintainability. The proposed approach strikes a better balance by using limited helper templates for only the most mechanical parts, while keeping service configurations explicit and isolated.

In Helm charts, as in most infrastructure as code, clarity and predictability are more valuable than brevity. Even with some repetition across service definitions, the proposed structure delivers these benefits while still extracting truly common elements into helpers.

cc @ividito @sunu @pantierra @batpad @j08lue

[batpad]

@emmanuelmathot thanks for laying that out so clearly with clear rationale. I have definitely done similar in the past with helm charts looping over things in an attempt to make things DRY and discovering things are just easier to manage and reason about with things being more explicit - especially as one needs to start making more detailed customizations per service or so.

I would very much be in favour of making the change to remove the looping through services in favour of separate folders and templates for each service, with some consolidation in common helper functions, as you have outlined 👍

[pantierra]

Yes, I agree this is a good approach and you provided good arguments!