Add option for RAG-style augmentation

[dave1010]

https://ai.meta.com/blog/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models/

https://www.promptingguide.ai/techniques/rag

Either manually or automatically provide man pages or --help output to the LLM.

Need to find examples where this is beneficial. Eg newer or esoteric software.

Some man pages are large and would likely slow down the response.

If automatic, maybe regex or maybe as the LLM if a man page would help.

Relates to #4

[vividfog]

Take a look at how continue.dev comes with its own little embeddings server. It's a new experimental feature, and it's a big piece of software. https://continue.dev/docs/walkthroughs/codebase-embeddings

Anyway, a minimal version of a similar thing could work with clipea. I would be able to essentially dump stuff into its memory, and then use slash-commands or @-tags to summon those into the prompt.

A contrived but hopefully helpful quick example of slash-commands and @-tags. The slash makes the bot do something special ("verb") and @makes it include something ("what").

?? /read all diff files in the @src folder and calculate how many new methods I ended up writing for this release

[dave1010]

Ooh, that'd be very cool!

I've had a play with ChromaDB but not done much with embeddings yet.

llm has built in support for managing embeddings now, so this may not be too difficult to do.

https://llm.datasette.io/en/stable/embeddings/index.html

How would you envisage the setup? Something like this?

?? /embed <file or dir path> <@tag> <embedding method>

Might require simonw/llm#216 for it to be effective.

[vividfog]

I don't have a strong candidate to offer. My intuition says to stay away from abstract verbs like "embed", they are better suited for llm, which is a lower level library. We at command line level could ignore what's happening behind the scenes and /study or /find or /learn as CLI users who wouldn't have to even know what's driving this magic bus. That could be an aspiration if not full reality.

Generally, the embeddings would be in two categories. The user will have some permanent knowledge they want the CLI to be aware of. It can't really be "all my files, past and future" because then the top-k chunks won't be that relevant for a specific case. Or the risk of them becoming less relevant increases.

The very first instinct would be to not add embeddings at all. Instead you could rely that llm does a good job and cross-reference your users to create what Simon calls Collections. Ask them to make collections. This will work even better, after Simon implements built-in chunked embeddings, because they need to be size-managed.

Assuming that, then this permanent knowledge could be referred at runtime by clipea.

?? /read my-todo-list and find the proxy I used to ssh to skunk labs last year, then ssh to it

Above, my-todo-list was a collection made with llm, using instructions you would have referred to in your docs.

Now, an evolution from that could be one-time or even throwaway embeddings. Perhaps another verb for that, which explicitly is documented to create a collection and throw it away.

?? /study ./docs and and /study ./src and create a command that spawns a server with distributed inference activated

Parsing multiple verbs can get tricky, and it wouldn't be a crime to just support one verb at a time.

Even if this sounds good, there's still the distinction between verb and object. Slash and @. Or maybe not. Continue.dev has made a distinction, probably for a reason, which I haven't fully analyzed. They've made some other clever choices, so I wouldn't shrug it off as YAGNI without thinking.

Since clipea is a command line tool, one could also build an argument that verbs should be options and targets should be what follows them.

?? --read my-todo-list and...

But that is venturing off from the natural-language land ("just say what you want") and in the syntax land, which kind of is what a tool like this is supposed to abstract away. SOME syntax is probably necessary, but what would be the most plain-english way to do it, that's what I suppose needs to be invented.

[ibrahimkettaneh]

I would like to boost this