Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support working with existing secret #201

Merged
merged 3 commits into from
Sep 10, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/lakefs/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
name: lakefs
description: A Helm chart for running LakeFS on Kubernetes
type: application
version: 0.9.25
version: 0.10.0
appVersion: 0.108.0

home: https://lakefs.io
Expand Down
36 changes: 20 additions & 16 deletions charts/lakefs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,8 @@ Before performing this upgrade, it is strongly recommended to perform these step
* Commit all uncommitted data on branches
* Create a snapshot of your database

In order to prevent loss of data during this process it is recommended to stop all the pods running `lakeFS`. This can be achieved by scaling the number of pods down to 0:
In order to prevent loss of data during this process, it is recommended to stop all the pods running `lakeFS`.
This can be achieved by scaling the number of pods down to 0:

```bash
# Stopping all pods running release my-lakefs
Expand All @@ -112,19 +113,22 @@ helm upgrade -f my-values.yaml my-lakefs lakefs/lakefs --set kv_upgrade=true

## Configurations

| **Parameter** | **Description** | **Default** |
|------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------|
| `secrets.databaseConnectionString` | PostgreSQL connection string to be used by lakeFS | |
| `secrets.authEncryptSecretKey` | A random (cryptographically safe) generated string that is used for encryption and HMAC signing | |
| `lakefsConfig` | lakeFS config YAML stringified, as shown above. See [reference](https://docs.lakefs.io/reference/configuration.html) for available configurations. | |
| `replicaCount` | Number of lakeFS pods | `1` |
| `resources` | Pod resource requests & limits | `{}` |
| `service.type` | Kuberenetes service type | ClusterIP |
| `service.port` | Kubernetes service external port | 80 |
| `extraEnvVars` | Adds additional environment variables to the deployment (in yaml syntax) | `{}` See [values.yaml](values.yaml) |
| `extraEnvVarsSecret` | Name of a Kubernetes secret containing extra environment variables | |
| `s3Fallback.enabled` | If set to true, an [S3Proxy](https://github.com/gaul/s3proxy) container will be started. Requests to lakeFS S3 gateway with a non-existing repository will be forwarded to this container. | |
| `s3Fallback.aws_access_key` | An AWS access key to be used by the S3Proxy for authentication | |
| `s3Fallback.aws_secret_key` | An AWS secret key to be used by the S3Proxy for authentication | |
| `committedLocalCacheVolume` | A volume definition to be mounted by lakeFS and used for caching committed metadata. See [here](https://kubernetes.io/docs/concepts/storage/volumes/#volume-types) for a list of supported volume types. The default values.yaml file shows an example of how to use this parameter. | |
| **Parameter** | **Description** | **Default** |
|---------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------|
| `secrets.databaseConnectionString` | PostgreSQL connection string to be used by lakeFS | |
| `secrets.authEncryptSecretKey` | A random (cryptographically safe) generated string that is used for encryption and HMAC signing | |
nopcoder marked this conversation as resolved.
Show resolved Hide resolved
| `existingSecret` | Name of existing secret to use for the chart's secrets (by default the charts create a secret to hold the authEncryptSecretKey and databaseConnectionString | |
| `secretKeys.databaseConnectionString` | Name of key in existing secret to use for databaseConnectionString. Only used when existingSecret set. | |
| `secretKeys.authEncryptSecretKey` | Name of key in existing secret to use for authEncryptSecretKey. Only used when existingSecret set. | |
nopcoder marked this conversation as resolved.
Show resolved Hide resolved
| `lakefsConfig` | lakeFS config YAML stringified, as shown above. See [reference](https://docs.lakefs.io/reference/configuration.html) for available configurations. | |
| `replicaCount` | Number of lakeFS pods | `1` |
| `resources` | Pod resource requests & limits | `{}` |
| `service.type` | Kubernetes service type | ClusterIP |
| `service.port` | Kubernetes service external port | 80 |
| `extraEnvVars` | Adds additional environment variables to the deployment (in yaml syntax) | `{}` See [values.yaml](values.yaml) |
| `extraEnvVarsSecret` | Name of a Kubernetes secret containing extra environment variables | |
| `s3Fallback.enabled` | If set to true, an [S3Proxy](https://github.com/gaul/s3proxy) container will be started. Requests to lakeFS S3 gateway with a non-existing repository will be forwarded to this container. | |
| `s3Fallback.aws_access_key` | An AWS access key to be used by the S3Proxy for authentication | |
| `s3Fallback.aws_secret_key` | An AWS secret key to be used by the S3Proxy for authentication | |
| `committedLocalCacheVolume` | A volume definition to be mounted by lakeFS and used for caching committed metadata. See [here](https://kubernetes.io/docs/concepts/storage/volumes/#volume-types) for a list of supported volume types. The default values.yaml file shows an example of how to use this parameter. | |

16 changes: 14 additions & 2 deletions charts/lakefs/templates/_env.tpl
Original file line number Diff line number Diff line change
@@ -1,13 +1,25 @@
{{- define "lakefs.env" -}}
env:
{{- if and .Values.secrets (.Values.secrets).databaseConnectionString }}
{{- if .Values.existingSecret }}
- name: LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING
valueFrom:
secretKeyRef:
name: {{ .Values.existingSecret }}
key: {{ .Values.secretKeys.databaseConnectionString }}
{{- else if and .Values.secrets (.Values.secrets).databaseConnectionString }}
- name: LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING
valueFrom:
secretKeyRef:
name: {{ include "lakefs.fullname" . }}
key: database_connection_string
{{- end }}
{{- if and .Values.secrets (.Values.secrets).authEncryptSecretKey }}
{{- if .Values.existingSecret }}
- name: LAKEFS_AUTH_ENCRYPT_SECRET_KEY
valueFrom:
secretKeyRef:
name: {{ .Values.existingSecret }}
key: {{ .Values.secretKeys.authEncryptSecretKey }}
{{- else if and .Values.secrets (.Values.secrets).authEncryptSecretKey }}
- name: LAKEFS_AUTH_ENCRYPT_SECRET_KEY
valueFrom:
secretKeyRef:
Expand Down
8 changes: 7 additions & 1 deletion charts/lakefs/templates/_fluffy.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,13 @@ env:
key: ldap_bind_password
{{- end }}
{{- end }}
{{- if and .Values.secrets (.Values.secrets).authEncryptSecretKey }}
{{- if .Values.existingSecret }}
- name: LAKEFS_AUTH_ENCRYPT_SECRET_KEY
valueFrom:
secretKeyRef:
name: {{ .Values.existingSecret }}
key: {{ .Values.secretKeys.authEncryptSecretKey }}
{{- else if and .Values.secrets (.Values.secrets).authEncryptSecretKey }}
- name: FLUFFY_AUTH_ENCRYPT_SECRET_KEY
valueFrom:
secretKeyRef:
Expand Down
8 changes: 5 additions & 3 deletions charts/lakefs/templates/secret.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{{- if .Values.secrets }}
{{- if and (.Values.secrets) (not .Values.existingSecret) }}
apiVersion: v1
kind: Secret
metadata:
Expand All @@ -9,6 +9,8 @@ type: Opaque
data:
{{- if .Values.secrets.databaseConnectionString }}
database_connection_string: {{ .Values.secrets.databaseConnectionString | default "" | b64enc }}
{{- end}}
{{- end }}
{{- if .Values.secrets.authEncryptSecretKey }}
auth_encrypt_secret_key: {{ .Values.secrets.authEncryptSecretKey | default "" | b64enc }}
{{- end}}
{{- end }}
{{- end }}
5 changes: 5 additions & 0 deletions charts/lakefs/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -108,3 +108,8 @@ fluffy:

# Start local postgres pod for quick start, not for production
useDevPostgres: true

# Name of existing secret to use
existingSecret: null
secretKeys.databaseConnectionString: database_connection_string
secretKeys.authEncryptSecretKey: auth_encrypt_secret_key