Skip to content

Conversation

@emmanuelmathot
Copy link
Contributor

@emmanuelmathot emmanuelmathot commented Apr 29, 2025

Overview

This PR refactors our Helm chart structure from a loop-based approach to service-specific templates. The primary goal is to improve readability, maintainability, and flexibility of our Kubernetes resource definitions.

Fixes #211

Changes Made

  1. Service-Specific Directory Structure:

    • Created dedicated directories for each service (raster, stac, vector, multidim)
    • Each service directory contains its own templates (deployment.yaml, service.yaml, configmap.yaml, hpa.yaml)
  2. Common Helper Functions:

    • Created a minimal _common.tpl with focused helper functions:
      • eoapi.mountServiceSecrets - For mounting service secrets
      • eoapi.commonEnvVars - For common environment variables
      • eoapi.pgstacInitContainers - For init containers that wait for pgstac jobs
  3. Integration with Existing Helpers:

    • Used existing eoapi.postgresqlEnv helper for database environment variables
    • Maintained compatibility with other system helpers
  4. Updated Test Suites:

    • Split tests by service into separate test files (raster_tests.yaml, stac_tests.yaml, etc.)
    • Each test file now focuses only on the specific service templates
    • Kept backward compatibility by maintaining original test files with notes
    • Tests now run successfully against the new template structure
  5. Documentation:

    • Added a comprehensive README.md in the services directory explaining the refactoring approach
    • Documented the new directory structure and helper functions

Backward Compatibility

This refactoring maintains full backward compatibility with existing values files and deployments. No changes to values.yaml structure were required, and the chart can be upgraded in-place without disruption.

- Introduced a unified PostgreSQL configuration structure in values.yaml, replacing the old db configuration.
- Added new helper functions for managing PostgreSQL environment variables and secrets based on the selected configuration type (postgrescluster, external-plaintext, external-secret).
- Removed old database-related templates (ConfigMap, Deployment, PVC, Secrets, Service) that are no longer needed.
- Updated the pgstacbootstrap job and configmap templates to align with the new PostgreSQL configuration.
- Implemented validation for PostgreSQL settings to ensure required fields are set based on the selected type.
…he external secret (host, port, database) will override the corresponding values defined in external.host, external.port, and external.database.

Confirmed that the conditional blocks in deployment.yaml were already consolidated to eliminate redundancy. The file was already using a single include statement for PostgreSQL environment variables:

env:
  {{- include "eoapi.postgresqlEnv" $ | nindent 12 }}
Removed the unused eoapi.mapLegacyPostgresql helper function from _helpers.tpl as it wasn't being referenced anywhere in the codebase.
…ik, streamline values.yaml, and update related documentation and tests
@emmanuelmathot
Copy link
Contributor Author

need #219 to be merged

…for raster, stac, vector, and multidim services; update deployment and configmap tests for backward compatibility; adjust ingress and HPA tests; clean up unused configurations in test.yaml.
@emmanuelmathot emmanuelmathot requested review from Copilot and pantierra and removed request for Copilot April 29, 2025 13:38
@emmanuelmathot emmanuelmathot requested review from alukach, batpad, Copilot, ividito and sunu and removed request for pantierra April 29, 2025 13:38
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the Helm chart structure into service-specific templates to enhance clarity, maintainability, and extensibility. Key changes include the introduction of dedicated service directories with their own templates, the creation of common helper functions, and updates to tests and documentation.

Reviewed Changes

Copilot reviewed 37 out of 46 changed files in this pull request and generated no comments.

Show a summary per file
File Description
helm-chart/eoapi/templates/services/raster/configmap.yaml Added service-specific configmap for raster.
helm-chart/eoapi/templates/services/multidim/service.yaml Introduced a dedicated service template for multidim.
helm-chart/eoapi/templates/services/multidim/hpa.yaml Added HPA spec for multidim with autoscaling configuration.
helm-chart/eoapi/templates/services/multidim/deployment.yaml New deployment template for multidim with common helpers integrated.
helm-chart/eoapi/templates/services/multidim/configmap.yaml Added configmap template for multidim environment variables.
helm-chart/eoapi/templates/services/ingress.yaml Updated ingress template with unified rules for multiple services.
helm-chart/eoapi/templates/services/doc-server.yaml Refactored doc-server configmap naming and content.
helm-chart/eoapi/templates/services/README.md Provided documentation for the new service-specific templates.
helm-chart/eoapi/templates/pgstacbootstrap/job.yaml Modified hook-delete-policy for bootstrap jobs.
docs/unified-ingress.md New documentation for the unified ingress configuration.
.github/workflows/helm-tests.yml Updated test workflow script to reflect new endpoints and enhanced failure handling.
Files not reviewed (9)
  • helm-chart/eoapi/.helmignore: Language not supported
  • helm-chart/eoapi/ingress.bkup: Language not supported
  • helm-chart/eoapi/templates/_helpers.tpl: Language not supported
  • helm-chart/eoapi/templates/services/_common.tpl: Language not supported
  • helm-chart/eoapi/templates/services/configmap.yaml: Language not supported
  • helm-chart/eoapi/templates/services/deployment.yaml: Language not supported
  • helm-chart/eoapi/templates/services/hpa.yaml: Language not supported
  • helm-chart/eoapi/templates/services/ingress-nginx.yaml: Language not supported
  • helm-chart/eoapi/templates/services/ingress-traefik.yaml: Language not supported
Comments suppressed due to low confidence (3)

helm-chart/eoapi/templates/services/multidim/hpa.yaml:16

  • The values key uses 'behaviour' (British spelling) while the output YAML key is 'behavior' (American spelling). Consider aligning the spelling to avoid potential confusion.
    {{- with .Values.multidim.autoscaling.behaviour }}

helm-chart/eoapi/templates/pgstacbootstrap/job.yaml:32

  • Removing the 'hook-succeeded' option from the hook-delete-policy might leave hook objects lingering. Verify that this change is intentional and that it won't lead to stale resources.
    helm.sh/hook-delete-policy: "before-hook-creation"

.github/workflows/helm-tests.yml:208

  • Appending to /etc/hosts requires elevated privileges; ensure the CI environment permits this change or consider an alternative approach to updating host entries during tests.
echo "$PUBLICIP_VALUE eoapi.local" | sudo tee -a /etc/hosts

@emmanuelmathot emmanuelmathot added this to the Full Cloud Agnostic milestone Apr 29, 2025
…s for raster, stac, vector, and multidim services; enhance readability and maintainability.
emmanuelmathot and others added 6 commits April 30, 2025 11:39
Co-authored-by: Tarashish Mishra <sunu@users.noreply.github.com>
Co-authored-by: Tarashish Mishra <sunu@users.noreply.github.com>
Co-authored-by: Tarashish Mishra <sunu@users.noreply.github.com>
…ific ingress control

- Added documentation for STAC Auth Proxy integration
- Configured service-specific ingress settings in values.yaml
- Updated ingress template to conditionally include STAC service paths
- Provided deployment guide and network flow diagram
- Included testing and troubleshooting sections for configuration verification
Co-authored-by: Tarashish Mishra <sunu@users.noreply.github.com>
@emmanuelmathot emmanuelmathot requested a review from sunu April 30, 2025 09:42
Copy link
Member

@sunu sunu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

@emmanuelmathot emmanuelmathot merged commit 5cdf18e into main Apr 30, 2025
2 checks passed
@emmanuelmathot emmanuelmathot deleted the chart_refactor branch April 30, 2025 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Helm Chart Refactoring Proposal: Improving Readability and Maintainability

4 participants