feat: build docker images in CI #231

nperez0111 · 2024-06-30T09:55:54Z

Hi, really like the project. I wanted to have docker images that were already pre-built to self-host. So, I got the project building in GitHub Container Registry & built automatically in CI.

Definitely some optimizations could be made to the Docker images to get this to work better though like:

We could leverage Docker multi-stage builds to reduce final image size
We could pick a better base image (seems like the current base image is bundling python? That doesn't seem neccessary & I could build it without it. I didn't include it in the PR because I wasn't totally sure if this was needed for a feature that I did not know about)
I found the build times to be extremely slow. I think it is because I'm doing a multiple architecture build but I actually checked and saw that yarn installs were a good chunk of the time. Could have a cache between them or honestly just switch from yarn since it is known to be slow at resolving. (I already added frozen-lockfiles which was a significant speedup)

One thing that I had to do though, we not modify the config file (since that is better handled by a volume) at build time to include the build time arg. So what I did was make env vars be a fallback for values that are actually specified in the config (so updating the config should always override an env var declarations)

You can see the docker images (and pull them if you like!) here:

nperez0111 · 2024-06-30T09:57:48Z

app.dockerfile

+ARG NEXT_PUBLIC_WS_URL=ws://127.0.0.1:3001
+ARG NEXT_PUBLIC_API_URL=http://127.0.0.1:3001/api


args can have defaults

nperez0111 · 2024-06-30T09:57:54Z

app.dockerfile

 ENV NEXT_PUBLIC_WS_URL=${NEXT_PUBLIC_WS_URL}
 ENV NEXT_PUBLIC_API_URL=${NEXT_PUBLIC_API_URL}

 WORKDIR /home/perplexica

 COPY ui /home/perplexica/

-RUN yarn install
+RUN yarn install --frozen-lockfile


nperez0111 · 2024-06-30T09:58:08Z

backend.dockerfile

@@ -1,21 +1,19 @@
 FROM nikolaik/python-nodejs:python3.12-nodejs20-bullseye

 ARG SEARXNG_API_URL
+ENV SEARXNG_API_URL=${SEARXNG_API_URL}


have the arg show up in the env

nperez0111 · 2024-06-30T09:58:24Z

docker-compose.yaml

@@ -21,6 +21,7 @@ services:
 - 3001:3001
 volumes:
 - backend-dbstore:/home/perplexica/data
+ - ./config.toml:/home/perplexica/config.toml


bind the config file into the container

nperez0111 · 2024-06-30T09:58:55Z

sample.config.toml

@@ -7,5 +7,5 @@ OPENAI = "" # OpenAI API key - sk-1234567890abcdef1234567890abcdef
 GROQ = "" # Groq API key - gsk_1234567890abcdef1234567890abcdef

 [API_ENDPOINTS]
-SEARXNG = "http://localhost:32768" # SearxNG API URL
+SEARXNG = "" # SearxNG API URL - http://localhost:32768


No need to specify it anymore, as the env var will be used if not specified

nperez0111 · 2024-06-30T09:59:05Z

src/config.ts

@@ -37,7 +37,8 @@ export const getOpenaiApiKey = () => loadConfig().API_KEYS.OPENAI;

 export const getGroqApiKey = () => loadConfig().API_KEYS.GROQ;

-export const getSearxngApiEndpoint = () => loadConfig().API_ENDPOINTS.SEARXNG;
+export const getSearxngApiEndpoint = () =>
+ loadConfig().API_ENDPOINTS.SEARXNG || process.env['SEARXNG_API_URL'];


Fallback to the env var

nperez0111 · 2024-06-30T09:59:29Z

.github/workflows/docker-build.yml

+ id: build-and-push-app
+ with:
+ context: .
+ platforms: linux/amd64,linux/arm64


Builds for multiple arches

nperez0111 · 2024-06-30T09:59:57Z

.github/workflows/docker-build.yml

+ - name: Sign the backend images with GitHub OIDC Token
+ env:
+ DIGEST: ${{ steps.build-and-push-backend.outputs.digest }}
+ TAGS: ${{ steps.meta-backend.outputs.tags }}
+ run: |
+ images=""
+ for tag in ${TAGS}; do
+ images+="${tag}@${DIGEST} "
+ done
+ cosign sign --yes ${images}
+
+ - name: Attest app
+ uses: actions/attest-build-provenance@v1
+ id: attest-app
+ with:
+ subject-name: ghcr.io/${{ env.IMAGE_NAME }}-app
+ subject-digest: ${{ steps.build-and-push-app.outputs.digest }}
+ push-to-registry: true


image signing & attestation are best practices for docker images

akramer-zibra · 2024-06-30T19:21:45Z

@ItzCrazyKns This looks like a professional enhancement to this lovely project. +1 for this

huyn3195

LGTM

ItzCrazyKns · 2024-07-01T04:54:44Z

Hey @nperez0111, thanks for the PR! All of the changes looks good but I cannot merge the CI pipeline for the docker image building as if the config.toml file doesn't gets configured correctly, the page won't load and it would throw an error, some of the fields are still not present in the config menu to be configured from the frontend so that might cause an issue, we can implement the CI part later when these things are implemented.

mictadlo · 2024-08-02T04:31:46Z

Hi,
I found here how to get searXNG running with Portainer. Now I would like to get Perplexica inside Portainer but is there a way to provide API keyes, searchXNG , Ollama URLs, port via env rather than config.toml file.

Thank you for consideration.

nperez0111 · 2024-08-02T10:34:30Z

Yea I think in general the maintainer is looking at how to store and configure these settings. I appreciate that it can be difficult since there are so many options to configure with: env vars, runtime values, config file, even the db could store settings.

I made some modifications to read from env vars since that is quite common in docker. But I'll let the maintainer cook on what he actually wants to build, support and maintain.

rqi14 · 2024-08-05T04:08:09Z

I've got a version of mine work.
https://github.com/rqi14/Perplexica-docker

nperez0111 commented Jun 30, 2024

View reviewed changes

huyn3195 approved these changes Jul 1, 2024

View reviewed changes

ItzCrazyKns mentioned this pull request Jul 20, 2024

Provide an official Docker image #276

Closed

nperez0111 added 2 commits July 22, 2024 23:11

feat: build docker images in CI

5979f83

fix: freeze lockfiles for speedup

e13e196

nperez0111 force-pushed the master branch from d0d7059 to e13e196 Compare July 22, 2024 21:12

ItzCrazyKns mentioned this pull request Aug 1, 2024

Prebuilt docker images? #298

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: build docker images in CI #231

feat: build docker images in CI #231

nperez0111 commented Jun 30, 2024 •

edited

Loading

nperez0111 Jun 30, 2024

nperez0111 Jun 30, 2024

nperez0111 Jun 30, 2024

nperez0111 Jun 30, 2024

nperez0111 Jun 30, 2024

nperez0111 Jun 30, 2024

nperez0111 Jun 30, 2024

nperez0111 Jun 30, 2024

akramer-zibra commented Jun 30, 2024 •

edited

Loading

huyn3195 left a comment

ItzCrazyKns commented Jul 1, 2024

mictadlo commented Aug 2, 2024 •

edited

Loading

nperez0111 commented Aug 2, 2024

rqi14 commented Aug 5, 2024

		ARG NEXT_PUBLIC_WS_URL=ws://127.0.0.1:3001
		ARG NEXT_PUBLIC_API_URL=http://127.0.0.1:3001/api

feat: build docker images in CI #231

Are you sure you want to change the base?

feat: build docker images in CI #231

Conversation

nperez0111 commented Jun 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akramer-zibra commented Jun 30, 2024 • edited Loading

huyn3195 left a comment

Choose a reason for hiding this comment

ItzCrazyKns commented Jul 1, 2024

mictadlo commented Aug 2, 2024 • edited Loading

nperez0111 commented Aug 2, 2024

rqi14 commented Aug 5, 2024

nperez0111 commented Jun 30, 2024 •

edited

Loading

akramer-zibra commented Jun 30, 2024 •

edited

Loading

mictadlo commented Aug 2, 2024 •

edited

Loading