AskTube - An AI-powered YouTube video summarizer and QA assistant powered by Retrieval Augmented Generation (RAG) 🤖

Run it entirely on your local machine with Ollama, or cloud-based models like Claude, OpenAI, Gemini, Mistral, and more

---

🏃🏽‍➡️ Demo & Screenshot

Watch "AskTube First Demo" on YouTube

demo.mp4

---

💤 Features

• Work even with unsubtitle video
• No limit video time
• Support multiple AI vendors
• Focus on RAG implemetation
• Fully run on your local machine

🤷🏽 Why does this project exist?

• I’ve seen several GitHub repositories offering AI-powered summaries for YouTube videos, but none include Q&A functionality.
• I want to implement a more comprehensive solution while also gaining experience with AI to build my own RAG application.

---

🔨 Technology

• Language: Python, JS
• Server: Python@v3.10, Bun@v1
• Framework/Lib: Sanic, Peewee, Pytubefix, Sentence Transformers, Sqlite, Chroma, NuxtJs/DaisyUI, etc.
• Embedding Provider (Analysis Provider):
  • OpenAI
  • Gemini
  • VoyageAI
  • Mistral
  • Sentence Transformers (Local)
• AI Provider:
  • OpenAI
  • Claude
  • Gemini
  • Mistral
  • Ollama (Local)
• Speech To Text:
  • Faster-Whisper (Local)
  • AssemblyAI
  • OpenAI
  • Gemini

---

🗓️ Next Todo Tasks

• Implement Speech To Text for cloud models
  • AssemblyAI
  • OpenAI
  • Gemini
• Enhance
  • Skip using RAG for short videos
  • Chat prompts, chat messages by context limit
  • RAG: Implement Query Translation
    • Multiquery
    • Fusion
    • Decomposition
    • Step back
    • HyDE

---

🚀 How to run ?

For the first time running, the program maybe a bit slow due they need to install local models.

Run on your machine

• Ensure you installed:

  • Python 3.10

    • Windows User, please download here
    • Linux, MacOS User, please use homebrew or your install package command (apt, dnf, etc)
    • Or use conda

  • Poetry

    • Windows User open Powershell and run:

    (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | py -

    • Linux, MacOS User open Terminal and run:

    curl -sSL https://install.python-poetry.org | python3 -

  • Bun

  • ffmpeg

    • MacOS User

    brew install ffmpeg

    • Linux User

    # Ubuntu
    sudo apt install ffmpeg
    # Fedora
    sudo dnf install -y ffmpeg

    • Windows, please follow this tutorial Install ffmpeg for Windows

• Clone repostiory

  git clone https://github.com/jonaskahn/asktube.git

• Create file .env in asktube/engine directory:

  • Locally
  • Free with some limitations

• Run program

  • You may need to run first:

  poetry env use python

  • Open terminal/cmd/powershell in asktube/engine directory, then run:

  poetry install && poetry run python engine/server.py

  • Open terminal/cmd/powershell in asktube/web directory, then run:

  bun install && bun run dev

• Open web: http://localhost:3000

With docker (In process)

Before You Start

1. I built these services to docker images, but if you want to build local images, please run build.local.bat for Windows or build.local.amd64.sh or build.local.aarch64.sh for MacOS, Linux
2. If you have a GPU (cuda or rocm), please refer ENV settings above, change params like above

Locally

• Use local.yaml compose file to start
• Open terminal/cmd/powershell in asktube directory

docker compose -f compose/local.yaml pull && docker compose -f compose/local.yaml up -d

• After run, you need install Ollama model qwen2 and llama3.1 for QA

docker run ollama ollama run qwen2
docker run ollama ollama run llama3.1

Free (with rate limit)

• You need to go Google Gemini and VoyageAI to register account and generate your own API keys:
  • Gemini is free with your Google Account
  • VoyageAI (recommended by Claude) gives you free 50M tokens (a huge amount) but you need to add your credit card first.
• Replace your ENV setting in docker file free and start docker
• Open terminal/cmd/powershell in asktube directory

docker compose -f compose/free.yaml pull && docker compose -f compose/free.yaml up -d

Ideal

• Using VoyageAI for embedding texts
• Using OpenAI and Claude for QA, register account and generate your own API keys
• Replace your ENV setting in docker file ideal and start docker
• Open terminal/cmd/powershell in asktube directory

docker compose -f compose/ideal.yaml pull && docker compose -f compose/ideal.yaml up -d

Result

• Open web: http://localhost:8080

---

💡 Architecture

The real implementation might differ from this art due to its complexity.

1️⃣ Extract data from given URL

2️⃣ Storing embedding chapter subtitles

3️⃣ Asking (included enrich question)

---

🪧 Notice

1. Do not use this for production. This aimed for end-users on their local machines.
2. Do not request any advanced features for management.

✍🏿 For development

• Backend
• Frontend

---

⁉️ FAQ and Troubleshooting

• FQA

---