[FIX] Compose fails: free variable referenced before assignment

[ljrobins]

Describe the bug

Trying to start khoj on a linux server. Made minimal changes to the docker-compose.yml (including notably changing the external port for pgvector to avoid the host machine's postgres instance). Setup proceeds until during initialization.py something goes wrong setting up the admin user. The full logs are attached.
logs.txt

To Reproduce

Here's my docker-compose.yml:

services:
  database:
    image: ankane/pgvector
    ports:
      - "5433:5432"
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: postgres
    volumes:
      - khoj_db:/var/lib/postgresql/data/
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 30s
      timeout: 10s
      retries: 5
  sandbox:
    image: ghcr.io/khoj-ai/terrarium:latest
    ports:
      - "8080:8080"
  server:
    depends_on:
      database:
        condition: service_healthy
    # Use the following line to use the latest version of khoj. Otherwise, it will build from source. Set this to ghcr.io/khoj-ai/khoj-cloud if you want to use the prod image.
    image: ghcr.io/khoj-ai/khoj:latest
    # Uncomment the following line to build from source. This will take a few minutes. Comment the next two lines out if you want to use the offiicial image.
    # build:
      # context: .
    ports:
      # If changing the local port (left hand side), no other changes required.
      # If changing the remote port (right hand side),
      #   change the port in the args in the build section,
      #   as well as the port in the command section to match
      - "42110:42110"
    working_dir: /app
    volumes:
      - khoj_config:/root/.khoj/
      - khoj_models:/root/.cache/torch/sentence_transformers
      - khoj_models:/root/.cache/huggingface
    # Use 0.0.0.0 to explicitly set the host ip for the service on the container. https://pythonspeed.com/articles/docker-connection-refused/
    environment:
      - POSTGRES_DB=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_HOST=database
      - POSTGRES_PORT=5432
      - KHOJ_DJANGO_SECRET_KEY=secret
      - KHOJ_DEBUG=True
      - KHOJ_ADMIN_EMAIL=username@example.com
      - KHOJ_ADMIN_PASSWORD=password
      # Uncomment line below to use with Ollama running on your local machine at localhost:11434.
      # Change URL to use with other OpenAI API compatible providers like VLLM, LMStudio etc.
      - OPENAI_API_BASE=http://host.docker.internal:11434/v1/
      #
      # Uncomment appropriate lines below to use chat models by OpenAI, Anthropic, Google.
      # Ensure you set your provider specific API keys.
      # ---
      # - OPENAI_API_KEY=your_openai_api_key
      # - GEMINI_API_KEY=your_gemini_api_key
      # - ANTHROPIC_API_KEY=your_anthropic_api_key
      #
      # Uncomment appropriate lines below to enable web results with Khoj
      # Ensure you set your provider specific API keys.
      # ---
      # Free, Slower API. Does both web search and webpage read. Get API key from https://jina.ai/
      # - JINA_API_KEY=you_jina_api_key
      # Paid, Fast API. Only does web search. Get API key from https://serper.dev/
      # - SERPER_DEV_API_KEY=your_serper_dev_api_key
      # Paid, Fast, Open API. Only does webpage read. Get API key from https://firecrawl.dev/
      # - FIRECRAWL_API_KEY=your_firecrawl_api_key
      # Paid, Fast, Higher Read Success API. Only does webpage read. Get API key from https://olostep.com/
      # - OLOSTEP_API_KEY=your_olostep_api_key
      #
      # Uncomment the necessary lines below to make your instance publicly accessible.
      # Replace the KHOJ_DOMAIN with either your domain or IP address (no http/https prefix).
      # Proceed with caution, especially if you are using anonymous mode.
      # ---
      - KHOJ_NO_HTTPS=True
      - KHOJ_DOMAIN=my_domain # removed for this bug report
      # - KHOJ_DOMAIN=khoj.example.com
      # Uncomment the line below to disable telemetry.
      # Telemetry helps us prioritize feature development and understand how people are using Khoj
      # Read more at https://docs.khoj.dev/miscellaneous/telemetry
      - KHOJ_TELEMETRY_DISABLE=True
    # Comment out this line when you're using the official ghcr.io/khoj-ai/khoj-cloud prod image.
    command: --host="0.0.0.0" --port=42110 -vv --anonymous-mode --non-interactive


volumes:
  khoj_config:
  khoj_db:
  khoj_models:

Screenshots

If applicable, add screenshots to help explain your problem.

Platform

• Server:
  • Cloud-Hosted (https://app.khoj.dev)
  • [ x] Self-Hosted Docker
  • Self-Hosted Python package
  • Self-Hosted source code
• Client:
  • Obsidian
  • Emacs
  • Desktop app
  • [x ] Web browser
  • WhatsApp
• OS:
  • Windows
  • macOS
  • [ x] Linux
  • Android
  • iOS

If self-hosted

• Server Version [e.g. 1.0.1]: 1.30.10

Additional context

[debanjum]

Hey, thanks for the detailed logs. Found and fixed the undefined variable exception while handling the exception to create OpenAI models. Not sure why the OpenAI model creation failed initially for you though. But at least now the rest of the initialization sequence shouldn't fail anymore.

Can you try use the pre tag Khoj docker images to verify you're able to get your Khoj initialized?

[ljrobins]

Great! Tried the pre tagged images:

[+] Running 0/1
 ⠦ server Pulling                                                                                                                                                          0.7s 
no matching manifest for linux/amd64 in the manifest list entries

Also a bit confused on why it didn't initialize the model -- I just have a completely default installation of ollama running. ollama run <> works fine.

[debanjum]

Ok, I needed to fix our docker build but you should be able to pull the docker amd64 build using the pre tag now. If that multi-platform build doesn't work, you an also try pull the pre-amd64 tagged docker image to get unblocked.

And yeah, let me know if the Ollama connection is still failing once you have your Khoj server running. We can look at the error logs to debug why that could be happening

[ljrobins]

Thanks for fixing the image -- the pull and initialization went fine. logs2.txt are attached, along with
docker-compose.txt.

. Two remaining issues:

• Still no connection to ollama (ollama run llama3.1) works perfectly fine in the same terminal and I have not changed its port.

• I get CSRF errors on Firefox trying to log into the admin page -- seems like the CSRF cookie is never set.