feat: workaround for bitnami brownout (#2843)

Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>

Author: julienmancuso

deploy/cloud/helm/platform/Chart.yaml

@@ -31,7 +31,7 @@ dependencies:
     repository: https://nats-io.github.io/k8s/helm/charts/
     condition: nats.enabled
   - name: etcd
-    version: 11.1.0
+    version: 12.0.18
     repository: "https://charts.bitnami.com/bitnami"
     condition: etcd.enabled
   - name: kai-scheduler

deploy/cloud/helm/platform/README.md

@@ -45,7 +45,7 @@ The Dynamo Platform Helm chart deploys the complete Dynamo Cloud infrastructure
 | Repository | Name | Version |
 |------------|------|---------|
 | file://components/operator | dynamo-operator | 0.5.0 |
-| https://charts.bitnami.com/bitnami | etcd | 11.1.0 |
+| https://charts.bitnami.com/bitnami | etcd | 12.0.18 |
 | https://nats-io.github.io/k8s/helm/charts/ | nats | 1.3.2 |
 | oci://ghcr.io/nvidia/grove | grove(grove-charts) | v0.0.0-6e30275 |
 | oci://ghcr.io/nvidia/kai-scheduler | kai-scheduler | v0.8.4 |

deploy/cloud/helm/platform/values.yaml

@@ -125,15 +125,15 @@ kai-scheduler:
 # etcd configuration - distributed key-value store for operator state
 # For complete configuration options, see: https://github.com/bitnami/charts/tree/main/bitnami/etcd
 etcd:
-  global:
-    security:
-      allowInsecureImages: true
 
   # -- Whether to enable etcd deployment, disable if you want to use an external etcd instance
   enabled: true
 
   image:
+    # -- following bitnami announcement for brownout - https://github.com/bitnami/charts/tree/main/bitnami/etcd#%EF%B8%8F-important-notice-upcoming-changes-to-the-bitnami-catalog
+    # -- we need to use the legacy repository until we migrate to the new "secure" repository
     repository: bitnamilegacy/etcd
+    tag: 3.5.18-debian-12-r5
 
   # Persistent storage configuration for etcd data
   persistence:

docs/guides/dynamo_deploy/installation_guide.md

@@ -174,6 +174,18 @@ kubectl create secret generic hf-token-secret \
   -n ${NAMESPACE}
 ```
 
+**Bitnami etcd "unrecognized" image?**
+
+```bash
+ERROR: Original containers have been substituted for unrecognized ones. Deploying this chart with non-standard containers is likely to cause degraded security and performance, broken chart features, and missing environment variables.
+```
+This error that you might encounter during helm install is due to bitnami changing their docker repository to a [secure one](https://github.com/bitnami/charts/tree/main/bitnami/etcd#%EF%B8%8F-important-notice-upcoming-changes-to-the-bitnami-catalog).
+
+just add the following to the helm install command:
+```bash
+--set "etcd.image.repository=bitnamilegacy/etcd" --set "etcd.global.security.allowInsecureImages=true"
+```
+
 **Clean uninstall?**
 ```bash
 ./uninstall.sh  # Removes all CRDs and platform