[Feature Request]: RAG integration

[AgentGenie]

Is your feature request related to a problem? Please describe.

Integrate the latest RAG tech

Describe the solution you'd like

Current integration list

Falkor DB
MS GraphRAG
Current investigation list

RAPTOR https://arxiv.org/html/2401.18059v1

Additional context

No response

[qingyun-wu]

Hi @AgentGenie, thank you for suggesting this feature! We are prioritizing this feature in the next major release. Can you share your Discord ID so that I can find you on Discord? Thank you!

[anilpillai007]

RAG is a common use case, and we should make it accessible to everyone—not just engineers.

Users should be able to upload documents (PDFs), use framework like FastAgency, Waldiez and get immediate value without coding. By limiting this, we exclude non-technical users like product managers and customer service teams.

PDF support is essential—let’s provide plug-and-play functionality that addresses the most common use cases seamlessly. It should also support multiple pdfs with a default implementation (graph db, raptor etc.)
E.g.
PDF with tables, page with columnar content,
PDF with images

Later releases should also support PPTs, Word docs, etc.

[AgentGenie]

@anilpillai007 Thank you for the input. We will support most text formats soon.

For non-technical users, could you articulate your use cases? e.g. is it adhoc or a step in a workflow? do you need a UI support etc?

[anilpillai007]

@AgentGenie
I agree it is not an easy use-case. Trying to get 100% done is a last mile problem. There are companies trying to do this. I have a discord conversation in our channel. See here
Use-case: This is not for me, but for a non-tech person (like a product person)
The way I look at it is.

1. End-State: A non-tech person uploads a bunch of pdfs, and is able to ask questions. In V1. Most examples give super basic pdfs which are not practical in real world. At the minimum, it should be able to understand tables.
2. Put this in a wrapper like FastAgency, Waldiez
3. Give ample examples where a user can just clone one example and change the pdf [minimal work for end user]
4. There should be steps where an end-user can get an OPENAI api key.

This may be too much out of scope for this ticket. If so, at least 1, 2 and 3 can be accomplished if we aim non-tech users with a suitable well developed pdf parser.

Suggestion:

• A coder agent looks at the pdf file, understand its structure, and picks up one of the commonly available open-source pdf parsers LangGraph, LlamaIndex/Avanced RAG with LllamaParse, Unstructured etc.

[AgentGenie]

@anilpillai007 We are working on designing a workflow to solve the problem. My idea is to breaks it into 3 steps.

1. Parse the pdf. There are many frameworks and tools. I tried Unstructured-IO. Do you know any other open source solutions with better performance?
2. Build a metadata from parsed pdf. The metadata serves purpose to understand the structure of the PDF and locate the relevant information. Metadata system could be in form of a graph rag.
3. Once we located the relevant information, e.g. tabular data, image data, or paragraphs of text, we could further retrieve the accurate information.

Please let me know if you have any suggestions. You input is much appreciated.