Add Docker Compose support

[rodrigobdz]

Fixes #614.

Changelog

Add

• Add missing instructions on how to build Docker images using Docker Compose.
• Add Docker Compose file compose.yaml with support for .env.local. Current Dockerfile does not support .env.local.

[nsarrazin]

I'd prefer to have deployment examples in the docs rather than in the repo directly as everyone might have different deployments. Maybe we could add it to the wiki wdyt?

[rodrigobdz]

Given that the readme is already too long, I agree. Although to be consistent, it would be advisable to move also the Quickstart and Setup sections from the readme to the wiki.

I encountered several issues in these two sections due to my ARM-based architecture and suppose several users might have faced this. That's the motivation behind this ticket.

[tino]

Note that this isn't going to work for with the local docker file. The current .env file is invalid for docker compose due to the multiline stuff with backticks.

[rodrigobdz]

@tino This command should ignore the .env to avoid errors with Docker Compose:

docker compose --env-file /dev/null up --build

[morganhein]

@rodrigobdz is there another way to pass the models/prompts to chat-ui if not through the .env? I'm not seeing anything, so i'm assuming this would require some additional args to chat-ui to allow loading json files or some other configuration?

Something like:

chat-ui:
  ...
  environment:
        MONGODB_URL: mongodb://localhost:27017/
        MODELS_FILE: "./models_file.json"
        PROMPTS_FILE: "./prompts_file.json"

Or am I missing something that would facilitate this? i'm trying to build a compose I can distribute to my team, and i'd really like to use chat-ui, but so far i've had to resort to using less-awesome interfaces until this gets fixed.

[rodrigobdz]

@morganhein Is there an argument against using .env.local? The file can be distributed with your team.

Chat UI depends on .env.local being present—see below.

chat-ui/svelte.config.js

Line 6 in 795bf39

dotenv.config({ path: "./.env.local" });

chat-ui/docs/source/installation/local.md

Line 7 in 795bf39

The default config for Chat UI is stored in the `.env` file. You will need to override some values to get Chat UI to run locally. Start by creating a `.env.local` file in the root of the repository as per the [configuration section](../configuration/overview). The bare minimum config you need to get Chat UI to run locally is the following:

chat-ui/entrypoint.sh

Line 1 in 795bf39

ENV_LOCAL_PATH=/app/.env.local

[morganhein]

It does not work with docker compose? Unless I'm missing something? You can't pass an env.local to compose that has backticks that have multiline strings in it. Same as the commentor above mentioning this about the .env file.

[rodrigobdz]

It should work with suggestion made to the commentor above: #1238 (comment)

[morganhein]

This is not me being passive aggressive, but have you tested this functionality and it worked? Because I can't get it to work at all. So either there's a misconfiguration on my end, or we're talking past each other.

~/Projects/playground/sponge main* ❯ cat .env.local                                                                                                                                                                                                       08:43:52 AM
MODELS=`[
  {
    "name": "Ollama Mistral",
    "chatPromptTemplate": "<s>{{#each messages}}{{#ifUser}}[INST] {{#if @first}}{{#if @root.preprompt}}{{@root.preprompt}}\n{{/if}}{{/if}} {{content}} [/INST]{{/ifUser}}{{#ifAssistant}}{{content}}</s> {{/ifAssistant}}{{/each}}",
    "parameters": {
      "temperature": 0.1,
      "top_p": 0.95,
      "repetition_penalty": 1.2,
      "top_k": 50,
      "truncate": 3072,
      "max_new_tokens": 1024,
      "stop": ["</s>"]
    },
    "endpoints": [
      {
        "type": "ollama",
        "url" : "http://ollama:11434",
        "ollamaName" : "mistral:latest"
      }
    ]
  }
]`
~/Projects/playground/sponge main* ❯ docker compose -f docker-compose-not-working.yml up                                                                                                                                                                  08:43:58 AM
failed to read /Users/morgan/Projects/playground/sponge/.env.local: line 2: unexpected character "{" in variable name "{"

Here's the docker compose:

name: local-llm

services:
  mongodb:
    image: mongo:latest
    ports:
      - "27017"
  ollama:
    image: ollama/ollama
    ports:
      - "11434"
    volumes:
      - "model_data:/root/.ollama/models"
  chat-ui:
    image: ghcr.io/huggingface/chat-ui
    environment:
      MONGODB_URL: mongodb://mongodb:27017
    env_file: "./.env.local"
    ports:
      - "3000"
volumes:
  model_data:

And here's my full setup, if it's helpful: https://github.com/morganhein/sponge

If you pull that repo down, and run docker compose -f docker-compose-not-working.yml up you're saying this should work? This is the way you're intending to pass a local file to docker compose, is it not? It has nothing to do with the embedded .env file. That's in the container. I'm trying to pass my .env.local to docker compose, and it does not work. It does not like the multi-line strings containing backticks, per the commentor above. The failure applies to any .env passed through docker compose that contains the backticks+multline strings.

Lastly, if i'm being dense i'm sorry. Thanks for the help and your work here. Cheers.

[rodrigobdz]

It runs with the changes proposed in morganhein/sponge#1.

Two changes:

1. Revise the command you're using to run Docker Compose (add --env-file /dev/null, as suggested here)

   - docker compose -f docker-compose-not-working.yml up 
   + docker compose --file docker-compose-not-working.yml --env-file /dev/null up

2. Revise how .env.local is defined in the Docker Compose file (as specified in this PR)

   -  env_file: "./.env.local"
   + volumes:
   +  - .env.local:/app/.env.local

Please let me know if it works.

[morganhein]

Yes, mounting the env was the missing part. Thanks.