Docker image for the Text Generation Web UI: A Gradio web UI for Large Language Models. Supports transformers, GPTQ, llama.cpp (GGUF), Llama models

Installs

• Ubuntu 22.04 LTS
• CUDA 11.8
• Python 3.10.12
• Text Generation Web UI
• Torch 2.0.1
• xformers 0.0.22
• runpodctl
• croc
• rclone

Available on RunPod

This image is designed to work on RunPod. You can use my custom RunPod template to launch it on RunPod.

Running Locally

Install Nvidia CUDA Driver

• Linux
• Windows

Start the Docker container

docker run -d \
  --gpus all \
  -v /workspace \
  -p 3000:3001 \
  -p 5000:5001 \
  -p 5005:5006 \
  -p 8888:8888 \
  -e JUPYTER_PASSWORD=Jup1t3R! \
  ashleykza/oobabooga:1.6.2

You can obviously substitute the image name and tag with your own.

Community and Contributing

Pull requests and issues on GitHub are welcome. Bug fixes and new features are encouraged.

You can contact me and get help with deploying your container to RunPod on the RunPod Discord Server below, my username is ashleyk.

Appreciate my work?