Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Elasticsearch dependency for self-hosted instances #3532

Merged
merged 2 commits into from
Feb 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions self-hosting/helm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Helm chart to self-host Omnivore.

## Notes and General Information

This helm chart uses docker images from [`sejaeger/omnivore-*`](https://hub.docker.com/u/sejaeger). If you want to use the Web-UI or build your own images, checkout `../.build-and-push-images.sh`. You will find some hard-coded environment variables (e.g., `PG_DB` or `PG_USER`), please don't change them! Those are also hard-coded in the code base and changing them will likely cause problems. Please have a look at [the values file](values.yaml) and [change it accordingly](https://github.com/bjw-s/helm-charts/blob/main/charts/library/common/values.yaml) to your setup, especially: postgres hostname, elasticsearch URL, omnivore URL.
This helm chart uses docker images from [`sejaeger/omnivore-*`](https://hub.docker.com/u/sejaeger). If you want to use the Web-UI or build your own images, checkout `../.build-and-push-images.sh`. You will find some hard-coded environment variables (e.g., `PG_DB` or `PG_USER`), please don't change them! Those are also hard-coded in the code base and changing them will likely cause problems. Please have a look at [the values file](values.yaml) and [change it accordingly](https://github.com/bjw-s/helm-charts/blob/main/charts/library/common/values.yaml) to your setup, especially: postgres hostname, omnivore URL.

Omnivore requires Postgres (+vector extension!) and Elasticsearch to store its information. Please make sure to have them up and running. Using the bitnami Helm charts works perfectly fine. However, for Postgres you need to use a custom built image that contains the vector extension: [See this descriptions](https://github.com/pgvector/pgvector/issues/126#issuecomment-1589203644) for more information or simply use `sejaeger/postgres-vector` from docker hub.
Omnivore requires Postgres (+vector extension!) to store its information. Please make sure to have it up and running. Using the bitnami Helm charts works perfectly fine but you need to use a custom built image that contains the vector extension: [See this descriptions](https://github.com/pgvector/pgvector/issues/126#issuecomment-1589203644) for more information or simply use `sejaeger/postgres-vector` from docker hub.

This setup uses a couple of secrets to safely store passwords, tokens and private information. It's your responsibility to generate them and create the following secretes accordingly.

Expand All @@ -21,8 +21,6 @@ This setup uses a couple of secrets to safely store passwords, tokens and privat
* postgres-admin-user-and-password
* PGPASSWORD
* POSTGRES_USER
* elasticsearch-auth-secret
* ES_PASSWORD
* omnivore-content-fetch-verification-token
* VERIFICATION_TOKEN

Expand Down
16 changes: 6 additions & 10 deletions self-hosting/helm/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ controllers:
api:
enabled: true
type: deployment
strategy: Recreate
strategy: RollingUpdate
containers:
api:
image:
repository: sejaeger/omnivore-api
tag: e44616b01
tag: 150444efb
pullPolicy: IfNotPresent
env:
API_ENV: local
Expand All @@ -20,8 +20,6 @@ controllers:
PG_POOL_MAX: 20
CLIENT_URL: https://omnivore.example.com # URL any client can use to reach the server
GATEWAY_URL: http://omnivore-api:8080/api
ES_URL: http://elasticsearch.elasticsearch.svc.cluster.local:9200
ES_USERNAME: elastic
CONTENT_FETCH_URL: http://omnivore-content-fetch:8080/?token=$(VERIFICATION_TOKEN)
envFrom:
- secretRef:
Expand All @@ -32,15 +30,13 @@ controllers:
name: omnivore-sso-jwt-secret
- secretRef:
name: omnivore-pg-password
- secretRef:
name: elasticsearch-auth-secret
- secretRef:
name: omnivore-content-fetch-verification-token
initContainers:
migrate:
image:
repository: sejaeger/omnivore-migrate
tag: e44616b01
tag: 150444efb
pullPolicy: IfNotPresent
command: ["/bin/sh", "-c", "--"]
args: ["./packages/db/setup.sh"]
Expand All @@ -61,18 +57,18 @@ controllers:
web:
enabled: true
type: deployment
strategy: Recreate
strategy: RollingUpdate
containers:
web:
image:
repository: sejaeger/omnivore-web
tag: e44616b01
tag: 150444efb
pullPolicy: IfNotPresent

content-fetch:
enabled: true
type: deployment
strategy: Recreate
strategy: RollingUpdate
containers:
content-fetch:
image:
Expand Down
Loading