Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add docker compose ollama #1823

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

hitpoint6
Copy link
Contributor

Please describe the purpose of this pull request.
Is it to add a new feature? Is it to fix a bug?

  • Enable running Letta with ollama powered model with one command

How to test
How can we test your PR during review? What commands should we run? What outcomes should we expect?

  1. Run the containers: export LETTA_LLM_MODEL=llama3; docker compose -f docker-compose-ollama.yaml up --remove-orphans. Replace llama3 with your own model id`
  2. Once the container is running, pull a model by running the docker exec -it ollama ollama run llama3. Replace llama3 with your model here.
  3. Open http://localhost:8083/
  4. Create an agent and chat with the agent. Note that the model running here is small so it is much less powerful than gpt-4o

Have you tested this PR?
Have you tested the latest commit on the PR? If so please provide outputs from your tests.

Yes, Able to create an agent and have conversations.

Related issues or PRs
Please link any related GitHub issues or PRs.

Is your PR over 500 lines of code?
If so, please break up your PR into multiple smaller PRs so that we can review them quickly, or provide justification for its length.

Additional context
Add any other context or screenshots about the PR here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant