Skip to content

dlavrenuek/llm-chat-playground

Repository files navigation

Tired of the regular GPT chats? This one has sarcasm per default!

preview

This project is a playground UI for Large Language Modals (LLMs). It utilizes:

Prerequisites

To run this project you need

Recommended (if you want to contribute)

Run locally

This playground supports a locally running Ollama server and the HuggingFace Inference API. Per default the chat will use Ollama with llama3.1 model. If it is the first time you are using the configured model, Ollama will download it when you send the first message in the conversation UI without any visible progress. You should pull the model first:

ollama pull llama3.1

You can also run a model deployed on Hugging Face by configuring an access token, see configuration options

To start the application in development mode run:

pnpm install
pnpm dev

With NPM

npm install
npm dev

With Yarn

yarn install
yarn dev

Open http://localhost:3000 in your browser to see the result.

Configuring the Playground

The playground can be customized via environment variables, with default settings defined in the .env file. To configure the playground for a local development environment, simply copy the contents of .env to a new file named .env.local, and update the values as needed. This file is excluded from version control.

Configuration variable Description Default value
SYSTEM_MESSAGE System message that defines the behaviour of the AI You are a helpful but sarcastic assistant
OLLAMA_MODEL The model used by local Ollama server llama3.1
HUGGINGFACEHUB_MODEL Hugging Face model microsoft/Phi-3-mini-4k-instruct
HUGGINGFACEHUB_API_KEY Hugging Face access token
HUGGINGFACEHUB_ENDPOINT_URL Hugging Face inference endpoint url
DB_URL Database to use. Supported configuration are:
- leve empty for local in memory storage
- mongodb://xxx/xxx for Mongodb