Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: SSL Docs for OPENMETADATA_OPTS #14465

Merged
merged 2 commits into from
Dec 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ If you now start Airflow with these changes, it will be running at `https://loca
Since Airflow will be using SSL, we need to update the OpenMetadata Server configuration to use the certificates
when preparing the connection to the Airflow Webserver.

The `pipelineServiceClientConfiguration` will look like the following:
For bare metal deployments, the `pipelineServiceClientConfiguration` will look like the following:

```yaml
pipelineServiceClientConfiguration:
Expand All @@ -118,6 +118,45 @@ pipelineServiceClientConfiguration:
Update the `truststorePath` and `truststorePassword` accordingly, pointing to the `keystore` in your server host
holding the certificates we created.

For docker deployments, you will provide OpenMetadata Server Application with the self signed certificates of Airflow bundled in JVM keystore.
These will be passed to the application using `AIRFLOW_TRUST_STORE_PATH` and `AIRFLOW_TRUST_STORE_PASSWORD` environment variable.

```
AIRFLOW_TRUST_STORE_PATH="<path/to/truststore.jks>"
AIRFLOW_TRUST_STORE_PASSWORD="<JVM_TRUSTSTORE_PASSWORD>"
```
Please make sure to have the the truststore file mounted and available as part of Docker Deployments.

For kubernetes deployments, update the helm values as below -

```yaml
extraEnvs:
- name: AIRFLOW_TRUST_STORE_PASSWORD
valueFrom:
secretKeyRef:
name: truststore-password-secret
key: password
- name: AIRFLOW_TRUST_STORE_PATH
value: "/etc/openmetadata/certs/truststore.jks>"
extraVolumes:
- name: jks-vol
secret:
secretName: jks-certs
extraVolumeMounts:
- name: jks-vol
mountPath: /etc/openmetadata/certs
readOnly: true
```

In the above code snippet, we are mounting the volumes of truststore file from a kubernetes secret. You can create the secret from `truststore.jks` file from the below `kubectl` command -

```bash
kubectl create secret generic jks-certs --from-file truststore.jks --namespace <NAMESPACE_NAME>
kubectl create secret generic truststore-password-secret --from-literal password=<YOUR_TRUSTSTORE_PASSWORD> --namespace <NAMESPACE_NAME>
```

Next, restart or redeploy openmetadata application to take the above configs in effect.

### Example: Setting it locally

For example, if we are running the server locally, we need to add the certificate to the JVM `cacerts` store:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -143,4 +143,22 @@ workflowConfig:
## Demo of SSL enabled with an SSO and JWT token configured

In case you are looking for a full dockerized demo of how JWT tokens, SSO configuration, and SSL enabled work together,
please visit our demo repository [here](https://github.com/open-metadata/openmetadata-demo/tree/main/sso-with-ssl).
please visit our demo repository [here](https://github.com/open-metadata/openmetadata-demo/tree/main/sso-with-ssl).

## Providing a single keystore that has all the cacerts required

This can be achieved using the `OPENMETADATA_OPTS` environment variable configuration across all the deployments.
However, for Production, we recommend you to bundle your cacerts separately for each components (like ElasticSearch/Opensearch and Airflow) and provide that to each individual configs for [openmetadata.yaml](/conf/openmetadata.yaml).
You can use this environment variable to also provide extra JVM parameters to tune the application as per your infrastructure needs.

Below is an example values to be set for the `OPENMETADATA_OPTS` environment variable to use cacerts truststore which is bundled for an organization issued certificates -

```bash
OPENMETADATA_OPTS="-Djavax.net.ssl.trustStore=<path/to/truststore/file> -Djavax.net.ssl.trustStorePassword=<TRUSTSTORE_PASSWORD>"
```

{%important%}

It is expected to have the keystore file either mounted as external volume or to be available over the filesystem where openmetadata server application will be running.

{%/important%}
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,45 @@ pipelineServiceClientConfiguration:
Update the `truststorePath` and `truststorePassword` accordingly, pointing to the `keystore` in your server host
holding the certificates we created.

For docker deployments, you will provide OpenMetadata Server Application with the self signed certificates of Airflow bundled in JVM keystore.
These will be passed to the application using `AIRFLOW_TRUST_STORE_PATH` and `AIRFLOW_TRUST_STORE_PASSWORD` environment variable.

```
AIRFLOW_TRUST_STORE_PATH="<path/to/truststore.jks>"
AIRFLOW_TRUST_STORE_PASSWORD="<JVM_TRUSTSTORE_PASSWORD>"
```
Please make sure to have the the truststore file mounted and available as part of Docker Deployments.

For kubernetes deployments, update the helm values as below -

```yaml
extraEnvs:
- name: AIRFLOW_TRUST_STORE_PASSWORD
valueFrom:
secretKeyRef:
name: truststore-password-secret
key: password
- name: AIRFLOW_TRUST_STORE_PATH
value: "/etc/openmetadata/certs/truststore.jks>"
extraVolumes:
- name: jks-vol
secret:
secretName: jks-certs
extraVolumeMounts:
- name: jks-vol
mountPath: /etc/openmetadata/certs
readOnly: true
```

In the above code snippet, we are mounting the volumes of truststore file from a kubernetes secret. You can create the secret from `truststore.jks` file from the below `kubectl` command -

```bash
kubectl create secret generic jks-certs --from-file truststore.jks --namespace <NAMESPACE_NAME>
kubectl create secret generic truststore-password-secret --from-literal password=<YOUR_TRUSTSTORE_PASSWORD> --namespace <NAMESPACE_NAME>
```

Next, restart or redeploy openmetadata application to take the above configs in effect.

### Example: Setting it locally

For example, if we are running the server locally, we need to add the certificate to the JVM `cacerts` store:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -143,4 +143,22 @@ workflowConfig:
## Demo of SSL enabled with an SSO and JWT token configured

In case you are looking for a full dockerized demo of how JWT tokens, SSO configuration, and SSL enabled work together,
please visit our demo repository [here](https://github.com/open-metadata/openmetadata-demo/tree/main/sso-with-ssl).
please visit our demo repository [here](https://github.com/open-metadata/openmetadata-demo/tree/main/sso-with-ssl).

## Providing a single keystore that has all the cacerts required

This can be achieved using the `OPENMETADATA_OPTS` environment variable configuration across all the deployments.
However, for Production, we recommend you to bundle your cacerts separately for each components (like ElasticSearch/Opensearch and Airflow) and provide that to each individual configs for [openmetadata.yaml](/conf/openmetadata.yaml).
You can use this environment variable to also provide extra JVM parameters to tune the application as per your infrastructure needs.

Below is an example values to be set for the `OPENMETADATA_OPTS` environment variable to use cacerts truststore which is bundled for an organization issued certificates -

```bash
OPENMETADATA_OPTS="-Djavax.net.ssl.trustStore=<path/to/truststore/file> -Djavax.net.ssl.trustStorePassword=<TRUSTSTORE_PASSWORD>"
```

{%important%}

It is expected to have the keystore file either mounted as external volume or to be available over the filesystem where openmetadata server application will be running.

{%/important%}
Loading