Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
401 changes: 184 additions & 217 deletions helm/polaris/README.md

Large diffs are not rendered by default.

84 changes: 25 additions & 59 deletions helm/polaris/README.md.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -35,27 +35,12 @@

{{ template "chart.homepageLine" . }}

{{ template "chart.maintainersHeader" . }}

{{- range .Maintainers }}
* [{{ .Name }}]({{ if .Url }}{{ .Url }}{{ else }}https://github.com/{{ .Name }}{{ end }})
{{- end }}

{{ template "chart.sourcesSection" . }}

{{ template "chart.requirementsSection" . }}

## Installation

### Prerequisites

When using the (deprecated) EclipseLink-backed metastore, a custom `persistence.xml` is required,
and a Kubernetes Secret must be created for it. Below is a sample command:

```bash
kubectl create secret generic polaris-secret -n polaris --from-file=persistence.xml
```

### Running locally with a Kind cluster

The below instructions assume Kind and Helm are installed.
Expand All @@ -66,27 +51,27 @@ Simply run the `run.sh` script from the Polaris repo root:
./run.sh
```

If using the EclipseLink-backed metastore, make sure to specify the `--eclipse-link-deps` option.

This script will create a Kind cluster, deploy a local Docker registry, build the Polaris Docker
images with support for Postgres and load them into the Kind cluster. (It will also create an
example Deployment and Service with in-memory storage.)

### Running locally with a Minikube cluster

The below instructions assume a Minikube cluster is already running and Helm is installed.
The below instructions assume Minikube and Helm are installed.

If necessary, build and load the Docker images with support for Postgres into Minikube:
Start the Minikube cluster, build and load image into the Minikube cluster:

```bash
eval $(minikube -p minikube docker-env)
minikube start
eval $(minikube docker-env)

./gradlew \
:polaris-server:assemble \
:polaris-server:quarkusAppPartsBuild --rerun \
:polaris-admin:assemble \
:polaris-admin:quarkusAppPartsBuild --rerun \
-Dquarkus.container-image.build=true
:polaris-server:assemble \
:polaris-server:quarkusAppPartsBuild --rerun \
:polaris-admin:assemble \
:polaris-admin:quarkusAppPartsBuild --rerun \
-Dquarkus.container-image.tag=postgres-latest \
-Dquarkus.container-image.build=true
```

### Installing the chart locally
Expand All @@ -96,15 +81,15 @@ The below instructions assume a local Kubernetes cluster is running and Helm is
#### Common setup

Create the target namespace:

```bash
kubectl create namespace polaris
```

Create all the required resources in the `polaris` namespace. This usually includes a Postgres
database and a Kubernetes Secret for the `persistence.xml` file. The Polaris chart does not create
database, Kubernetes secrets, and service accounts. The Polaris chart does not create
these resources automatically, as they are not required for all Polaris deployments. The chart will
fail if these resources are not created beforehand.
fail if these resources are not created beforehand. You can find some examples in the
`helm/polaris/ci/fixtures` directory, but beware that these are primarily intended for tests.

Below are two sample deployment models for installing the chart: one with a non-persistent backend and another with a persistent backend.

Expand All @@ -116,19 +101,17 @@ Below are two sample deployment models for installing the chart: one with a non-
#### Non-persistent backend

Install the chart with a non-persistent backend. From Polaris repo root:

```bash
helm upgrade --install --namespace polaris \
--debug --values helm/polaris/ci/simple-values.yaml \
polaris helm/polaris
--values helm/polaris/ci/simple-values.yaml \
polaris helm/polaris
```

Note: if you are running the tests on a Kind cluster started with the `run.sh` command explained
above, then you need to run `helm upgrade` as follows:

```bash
helm upgrade --install --namespace polaris \
--debug --values helm/polaris/ci/simple-values.yaml \
--values helm/polaris/ci/simple-values.yaml \
--set=image.repository=localhost:5001/apache/polaris \
polaris helm/polaris
```
Expand All @@ -139,41 +122,28 @@ helm upgrade --install --namespace polaris \
> The Postgres deployment set up in the fixtures directory is intended for testing purposes only and is not suitable for production use. For production deployments, use a managed Postgres service or a properly configured and secured Postgres instance.

Install the chart with a persistent backend. From Polaris repo root:

```bash
helm upgrade --install --namespace polaris \
--debug --values helm/polaris/ci/persistence-values.yaml \
--values helm/polaris/ci/persistence-values.yaml \
polaris helm/polaris

kubectl wait --namespace polaris --for=condition=ready pod --selector=app.kubernetes.io/name=polaris --timeout=120s
```

After deploying the chart with a persistent backend, the `persistence.xml` file, originally loaded into the Kubernetes pod via a secret, can be accessed locally if needed. This file contains the persistence configuration required for the next steps. Use the following command to retrieve it:

```bash
kubectl exec -it -n polaris $(kubectl get pod -n polaris -l app.kubernetes.io/name=polaris -o jsonpath='{.items[0].metadata.name}') -- cat /deployments/config/persistence.xml > persistence.xml
```

The `persistence.xml` file references the Postgres hostname as postgres. Update it to localhost to enable local connections:

```bash
sed -i .bak 's/postgres:/localhost:/g' persistence.xml
```

To access Polaris and Postgres locally, set up port forwarding for both services:
To access Polaris and Postgres locally, set up port forwarding for both services (This is needed for bootstrap processes):
```bash
kubectl port-forward -n polaris $(kubectl get pod -n polaris -l app.kubernetes.io/name=polaris -o jsonpath='{.items[0].metadata.name}') 8181:8181

kubectl port-forward -n polaris $(kubectl get pod -n polaris -l app.kubernetes.io/name=postgres -o jsonpath='{.items[0].metadata.name}') 5432:5432
```

Run the catalog bootstrap using the Polaris admin tool. This step initializes the catalog with the required configuration:

```bash
java -Dpolaris.persistence.eclipselink.configuration-file=./persistence.xml \
-Dpolaris.persistence.eclipselink.persistence-unit=polaris \
-jar runtime/admin/build/polaris-admin-*-runner.jar \
bootstrap -c POLARIS,root,pass -r POLARIS
container_envs=$(kubectl exec -it -n polaris $(kubectl get pod -n polaris -l app.kubernetes.io/name=polaris -o jsonpath='{.items[0].metadata.name}') -- env)
export QUARKUS_DATASOURCE_USERNAME=$(echo "$container_envs" | grep quarkus.datasource.username | awk -F '=' '{print $2}' | tr -d '\n\r')
export QUARKUS_DATASOURCE_PASSWORD=$(echo "$container_envs" | grep quarkus.datasource.password | awk -F '=' '{print $2}' | tr -d '\n\r')
export QUARKUS_DATASOURCE_JDBC_URL=$(echo "$container_envs" | grep quarkus.datasource.jdbc.url | sed 's/postgres/localhost/2' | awk -F '=' '{print $2}' | tr -d '\n\r')

java -jar runtime/admin/build/quarkus-app/quarkus-run.jar bootstrap -c POLARIS,root,pass -r POLARIS
```

### Uninstalling
Expand All @@ -198,15 +168,13 @@ The following tools are required to run the tests:
* [Chart Testing](https://github.com/helm/chart-testing)

Quick installation instructions for these tools:

```bash
helm plugin install https://github.com/helm-unittest/helm-unittest.git
brew install chart-testing
```

The integration tests also require some fixtures to be deployed. The `ci/fixtures` directory
contains the required resources. To deploy them, run the following command:

```bash
kubectl apply --namespace polaris -f helm/polaris/ci/fixtures/
kubectl wait --namespace polaris --for=condition=ready pod --selector=app.kubernetes.io/name=postgres --timeout=120s
Expand All @@ -219,7 +187,6 @@ different configurations.

Helm unit tests do not require a Kubernetes cluster. To run the unit tests, execute Helm Unit from
the Polaris repo root:

```bash
helm unittest helm/polaris
```
Expand All @@ -236,9 +203,8 @@ Integration tests require a Kubernetes cluster. See installation instructions ab
a local cluster.

Integration tests are run with the Chart Testing tool:

```bash
ct install --namespace polaris --debug --charts ./helm/polaris
ct install --namespace polaris --charts ./helm/polaris
```

Note: if you are running the tests on a Kind cluster started with the `run.sh` command explained
Expand Down
7 changes: 0 additions & 7 deletions helm/polaris/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -191,13 +191,6 @@ Prints the config volume definition for deployments and jobs.
path: symmetric.key
{{- end }}
{{- end }}
{{- if and ( eq .Values.persistence.type "eclipse-link" ) .Values.persistence.eclipseLink.secret.name }}
- secret:
name: {{ tpl .Values.persistence.eclipseLink.secret.name . }}
items:
- key: {{ tpl .Values.persistence.eclipseLink.secret.key . }}
path: persistence.xml
{{- end }}
{{- end -}}

{{/*
Expand Down
4 changes: 0 additions & 4 deletions helm/polaris/templates/configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,6 @@ data:

{{- /* Persistence */ -}}
{{- $_ = set $map "polaris.persistence.type" .Values.persistence.type -}}
{{- if and ( eq .Values.persistence.type "eclipse-link" ) .Values.persistence.eclipseLink.secret.name -}}
{{- $_ = set $map "polaris.persistence.eclipselink.persistence-unit" .Values.persistence.eclipseLink.persistenceUnit -}}
{{- $_ = set $map "polaris.persistence.eclipselink.configuration-file" (printf "%s/persistence.xml" .Values.image.configDir ) -}}
{{- end -}}

{{- /* File IO */ -}}
{{- $_ = set $map "polaris.file-io.type" .Values.fileIo.type -}}
Expand Down
8 changes: 0 additions & 8 deletions helm/polaris/tests/configmap_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -102,14 +102,6 @@ tests:
- matchRegex: { path: 'data["application.properties"]', pattern: "polaris.features.realm-overrides.\"realm1\".\"feature1\"=false" }
- matchRegex: { path: 'data["application.properties"]', pattern: "polaris.features.realm-overrides.\"realm2\".\"feature2\"=43" }

- it: should configure persistence
set:
persistence: { type: "eclipse-link", eclipseLink: { persistenceUnit: "polaris", secret: { name: "polaris-persistence" } } }
asserts:
- matchRegex: { path: 'data["application.properties"]', pattern: "polaris.persistence.type=eclipse-link" }
- matchRegex: { path: 'data["application.properties"]', pattern: "polaris.persistence.eclipselink.persistence-unit=polaris" }
- matchRegex: { path: 'data["application.properties"]', pattern: "polaris.persistence.eclipselink.configuration-file=/deployments/config/persistence.xml" }

- it: should configure relational-jdbc persistence
set:
persistence: { type: "relational-jdbc", relationalJdbc: { secret: { name: "polaris-persistence" } } }
Expand Down
58 changes: 0 additions & 58 deletions helm/polaris/tests/deployment_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -356,31 +356,6 @@ tests:
mountPath: /deployments/config
readOnly: true

- it: should evaluate template expressions in persistence secret name
set:
persistence:
type: eclipse-link
eclipseLink:
secret:
name: "{{ .Release.Name }}-persistence-secret"
asserts:
- contains:
path: spec.template.spec.volumes
content:
name: config-volume
projected:
sources:
- configMap:
items:
- key: application.properties
path: application.properties
name: polaris-release
- secret:
items:
- key: persistence.xml
path: persistence.xml
name: polaris-release-persistence-secret

# spec.template.spec.containers[0].ports
- it: should set container ports by default
asserts:
Expand Down Expand Up @@ -987,39 +962,6 @@ tests:
- key: private.key
path: private.pem

- it: should configure config volume with persistence secret
set:
image.configDir: /config/dir
persistence:
type: eclipse-link
eclipseLink:
secret:
name: polaris-persistence
key: custom.xml
asserts:
- contains:
path: spec.template.spec.containers[0].volumeMounts
content:
name: config-volume
mountPath: /config/dir
readOnly: true
- contains:
path: spec.template.spec.volumes
content:
name: config-volume
projected:
sources:
- configMap:
name: polaris-release
items:
- key: application.properties
path: application.properties
- secret:
name: polaris-persistence
items:
- key: custom.xml
path: persistence.xml

- it: should set relational-jdbc persistence environment variables
set:
persistence: { type: "relational-jdbc", relationalJdbc: { secret: { name: "polaris-persistence", username: "username", password: "password", jdbcUrl: "jdbcUrl" } } }
Expand Down
12 changes: 0 additions & 12 deletions helm/polaris/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -537,18 +537,6 @@ persistence:
# -- The secret key holding the database JDBC connection URL
jdbcUrl: jdbcUrl

# -- The configuration for the eclipse-link persistence manager.
eclipseLink:
# -- The secret name to pull persistence.xml from.
secret:
# -- The name of the secret to pull persistence.xml from.
# If not provided, the default built-in persistence.xml will be used. This is probably not what you want.
name: ~
# -- The key in the secret to pull persistence.xml from.
key: persistence.xml
# -- The persistence unit name to use.
persistenceUnit: polaris

# -- Polaris FileIO configuration.
fileIo:
# -- The type of file IO to use. Two built-in types are supported: default and wasb. The wasb one translates WASB paths to ABFS ones.
Expand Down
11 changes: 4 additions & 7 deletions run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,13 @@
# Function to display usage information
usage() {
echo "Usage: $0 [--eclipse-link-deps=<deps>] [-h|--help]"
echo " --eclipse-link-deps=<deps> EclipseLink dependencies to use, e.g."
echo " --eclipse-link-deps=com.h2database:h2:2.3.232"
echo " -h, --help Display this help message"
exit 1
}

# Parse command-line arguments
while [[ "$#" -gt 0 ]]; do
case $1 in
--eclipse-link-deps=*)
ECLIPSE_LINK_DEPS="-PeclipseLinkDeps=${1#*=}"
;;
-h|--help)
usage
;;
Expand All @@ -53,9 +48,11 @@ sh ./kind-registry.sh
# Build and deploy the server image
echo "Building polaris image..."
./gradlew \
:polaris-server:build \
:polaris-server:assemble \
:polaris-server:quarkusAppPartsBuild --rerun \
$ECLIPSE_LINK_DEPS \
:polaris-admin:assemble \
:polaris-admin:quarkusAppPartsBuild --rerun \
-Dquarkus.container-image.tag=postgres-latest \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need the special tag anymore. The default image is built with the Postgres JDBC driver afaict.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without -Dquarkus.container-image.tag=postgres-latest, it will introduce breaking changes in other getting start examples as they are all refers to postgres-latest. If this not preferred, we can remove this and update docs that refs to this. Without using this, we will have latest and <version> tags.

-Dquarkus.container-image.build=true \
-Dquarkus.container-image.registry=localhost:5001

Expand Down
Loading