How to enable LLM on top of the documents ingested

[nasreekar]

Hello,

I'm working on a project using KM with a react frontend chat UI. I've set up the repository as a local service (functioning as a backend using dotnet run - i believe serverless) and developed a frontend application using react to interact with it. I've converted the bash scripts for uploading and querying (upload.sh, ask.sh) into API calls (/upload, /ask) for the frontend to use with request bodies. Currently, everything works as expected for queries related to the ingested documents.

However, I'm encountering an issue when trying to obtain information outside the scope of the ingested documents. Is this the expected behavior?

For example, I uploaded a document containing invoice amounts in a specific currency format. When I ask the system to convert this to a different currency, I receive a "No relevant memories found" response because it's calling the /ask endpoint. Is there a way to utilize an LLM for these types of queries rather than relying solely on API responses?

I'm looking for guidance on how to enable LLM functionality for queries that go beyond the content of the ingested documents. Any advice or suggestions would be greatly appreciated.

Thank you for your time and assistance.

[dluc]

Hi @nasreekar I think what you're facing is the difference between RAG, Intent Detection and Plugins execution. RAG is a specific action to search for an aswer, assuming the user sent a question. However, often users can be just chatting, or asking for something else.
I would suggest changing the flow of your backend, adding some steps:

• When the user sends a message, run Intent Detection, ie understand what the user needs:
  • Is the user just chatting or asking to execute a command?
  • If a command, which command is it?
    • "Answer A Question about your data" is the RAG command (KM Ask API)
    • "Take your data and execute some transformation" is not RAG (not KM). In this case it depends to you. For instance, taking a document and translating the currency can be implemented in many ways, but it's not a RAG scenario.

I hope this helps and allows you to make some progress.

[nasreekar]

Hi @dluc,

Thank you for getting back to me. I appreciate it.

I realize that price conversion might not have been the best example, but I think you understood my requirement regarding the ability to chat or ask questions outside the document context.

Here are my points of focus:

1. KM Ask API - Working as expected.
2. Transformation - Is this where we can include agents, at least for the currency conversion scenario?
3. New Requirement - Implement intent detection to either call the Ask API or retrieve answers from the LLM.

Thank you for your assistance.