Skip to content

Latest commit

 

History

History
287 lines (173 loc) · 9.09 KB

README.md

File metadata and controls

287 lines (173 loc) · 9.09 KB
Raw

💡 Note:

This is stil very much alive, it's just a solid code base, and as new models come out, upstream deps support them, and then this project doesnt need updates 🫀

If you want to add features, this LSP approach is solid, and I'm happy to help, I really think this is (a chunk of) the future of LLM interaction

uniteai

Your AI Stack in your Editor: Voice-to-text, Local LLM, and GPT, +more.

Package version
Join Discord

Requirements: Python 3

Editor: VSCode(ium) or Emacs or Any Editor with LSP capabilities (most).

The Mission


The future is about Humans Augmented with AIs.



We need our AI Stack (Online, or local models)

Inside a convenient interface (Text Editors > Web UIs)

Friendly with any editor (The project is an LSP and therefore highly portable)

And close to the code (It's easy to tweak and add features. All the logic happens in friendly python code, not bespoke one-off editor code).

Screencast Demo

Some Core Features

screencast.webm

Document Chat (NEW)

aka Retrieval Augmented Generation

screencast_document_chat.webm

Quickstart, installing Everything

You can install more granularly than everything, but we'll demo everything first.

1.) Make sure Python 3 + Pip is installed.

python --version
pip --version

# or

python3 --version
pip3 --version

2.) The only platform-dependent dependency right now is portaudio, and that is only needed if you want speech-to-text/transcription.

# Mac
brew install portaudio

# Ubuntu/Debian
sudo apt install portaudio19-dev

3.) Install UniteAI.

pip3 install --user "uniteai[all]"  # install deps for all features

Your editor will make use of the installed binary, it so it needs to be on your PATH. Add your python installations bin/ to your system PATH environment variable.

# Linux and Mac
~/.local/bin

# Windows
C:\Users\USERNAME\AppData\Roaming\Python\PythonXX\Scripts\

uniteai_lsp should now be callable from a fresh terminal (but you will never need to call it yourself).

4.) Optional: Start the longlived LLM server which offers your editor a connection to your local large language model.

The editor will not start up the LLM itself, so you can just run this from anywhere, and the editor will connect to it.

uniteai_llm

5.) Install in your editor:

  • For VSCode get the uniteai extension. Eg in VSCode, Ctrl-P then ext install uniteai.uniteai .

  • For VSCodium, VSCode Marketplace files are not compatible, so you'll need to either:

    • Download the prepackaged uniteai.vsix extension, then:

      codium --install-extension clients/vscode/uniteai.vsix

    • DIY:

      npm install -g @vscode/vsce
git clone https://github.com/freckletonj/uniteai
cd uniteai/clients/vscode
vsce package
codium --install-extension uniteai-version.vsix

  • For Emacs, copy the lsp-mode config to your init.el.

  • For other editors with LSP support (most do), we just need to copy the emacs/vscode configuration, and translate it to your editor. Please submit a PR with new editor configs!

6.) Config:

When you first open a compatible file in your editor, the LSP will start.

It will prompt you to create a config file in your local dir, or your home dir.

You will need to edit this file, and then restart your editor.

Granular installs

If you did pip install "uniteai[all]", ignore this section!

Your config determines what modules/features are loaded.

The following makes sure to get your dependencies for each feature. This will become more relevant when more community features are added.

Transcription dependencies

# Debian/Ubuntu
sudo apt install portaudio19-dev  # needed by PyAudio

# Mac
brew install portaudio  # needed by PyAudio

pip3 install "uniteai[transcription]"

Local LLM dependencies

pip3 install "uniteai[local_llm]"

OpenAI/ChatGPT dependencies

pip3 install "uniteai[openai]"

Keycombos

Your client configuration determines this, so if you are using the example client config examples in ./clients:

VSCode Emacs Effect
M-' Show Code Actions Menu
Ctrl-Alt-d C-c l d Do semantic search on a document
Ctrl-Alt-g C-c l g Send region to GPT, stream output to text buffer
Ctrl-Alt-c C-c l c Same, but ChatGPT
Ctrl-Alt-l C-c l l Same, but Local (eg Falcon) model
Ctrl-Alt-v C-c l v Start voice-to-text
Ctrl-Alt-s C-c l s Whatevers streaming, stop it

  • I'm still figuring out what's most ergonomic, so, I'm accepting feedback.

  • Ctrl-Alt-d on ubuntu means defaults to "minimize all windows". You can disable that.

  • Cmd-Alt-don Mac does something else too, so, we should revisit default bindings.

Retreival Augmented Generation (RAG)

For the document feature, you can reference one of multiple document types, and lookup passages with a similar "gist" to them (semantic similarity search).

Check that your .uniteai.yaml config has uniteai.document enabled.

You can use links to: YouTube (will read transcripts), Arxiv papers, PDFs, Git repos, or any HTML.

To use this feature, write some YAML, highlight it, and hit C-c l d (emacs) or C-A-d (vscode).

query:
docs:
  - title: (optional)
    url: ...
  - title: ...
    url: ...

It will take a couple minutes for long documents to get an embedding for each chunk it finds in the document, but that then gets cached and goes fast afterward.

More details.

Contributions

Why?

Because there are so many cool tools to yet be added:

  • Image creation, eg: "Write a bulleted plan for a Hero's Journey story about X, and make an image for each scene."

  • Contextualize the AI via reading my emails via POP3, and possibly responding, eg: "what was that thing my accountant told me not to forget?"

  • Ask my database natural language questions, eg: "what were my top 10% customers' top 3 favorite products?"

  • Write-ahead for tab-completion, eg: "Once upon a ____".

  • Chat with a PDF document, eg: "what do the authors mean by X?"

  • Do some searches, scrape the web, and upload it all into my db.

  • Sky's the limit.

How?

A Key goal of this project is to be Contributor-Friendly.

  • Make an Issue with your cool concept, or bug you found.

  • .todo/ is a directory of community "tickets", eg .todo/042_my_cool_feature.md. Make a ticket or take a ticket, and make a PR with your changes!

  • ./todo/README.md gives some overview of the library, and advice on building against this library.

  • a ./contrib directory is where you can add your custom feature. See ./uniteai/contrib/example.py.

  • .uniteai.yml configuration chooses which modules to load/not load.

  • The code is well-documented, robust, and simple, to reduce friction.

  • Adding a feature is as simple as writing some python code, and making use of uniteai's library to directly handle issues like concurrency and communicating/modifying the text editor.

Misc

Notes on Local LLMs

The file ./llm_server.py launches a TCP server in which the LLM weights are booted up. The lsp_server will make calls to this llm_server.

The reason is that the lsp_server lifecycle is (generally*) managed by the text editor, and LLM models can be really slow to boot up. Especially if you're developing a feature, you do not want the LLM to keep being read into your GPU each time you restart the lsp_server.

* you don't have to let the editor manage the lsp_server. For instance, eglot in emacs allows you to launch it yourself, and then the editor client can just bind to the port.

Flash Attention

Install it with:

pip install packaging ninja torch
pip install flash-attn --no-build-isolation

License

Copyright (c) Josh Freckleton. All rights reserved.

Licensed under the Apache-2.0 license.