Question about to how to mix chat history and KM filters

[bancroftway]

@dluc I already asked a similar question before, but there has been no response so far, so let me make a differently worded attempt.

Imagine a scenario where user is uploading product related documents to KM, i.e. when calling ImportDocumentAsync, I set a tag productId on the documents uploaded.

Now when the user wants to chat, we force user to first select a product from a dropdown so we can apply the productId filter. In the code, I have a system prompt that reads something like

"Based only on facts retrieved from memory, answer the user's question.
 Facts recalled from memory: {{memory.ask}}
 User Question: {question}

The user can ask something like "Tell me the main features of product 'xxx'", in which case they will successfully get the features from the product docs. But then the user says, "Compose a sales email to a hospital administrator". This is where my issue is.

I am unable to code this scenario, which includes chat history, KM filter and SK.
I cannot figure out when using chat history (see sample code below)

1. how to apply the filter.
2. when the user asks "compose an email...", at that point, the system prompt that I have set becomes an issue because I have instructed the LLM to only look at the facts from memory. could I say something like "based on facts recalled from memory or chat history..."?
3. also, when the user asks "compose an email...", a call to KM is likely unnecessary. Is there a different/more elegant way to code this scenario, where we hit the KM Vector DB only if necessary?

In the code below, where to fit the args variable? Or is there a different/better way to handle this? Any help will is most appreciated.

public async Task<Microsoft.SemanticKernel.ChatCompletion.ChatHistory> AskQuestionFromDocuments(string question, 
    Microsoft.SemanticKernel.ChatCompletion.ChatHistory chatHistory, 
    Dictionary<string, string> tagDictionary)
{
    var skKernel = GetSkKernel();
    var kmKernel = GetKernelMemoryClient();

    var plugin = new MemoryPlugin(kmKernel, waitForIngestionToComplete: true);
    skKernel.ImportPluginFromObject(plugin, "memory");

    var skPrompt = $@"
    Question to Kernel Memory: {question}

    Kernel Memory Answer: {{memory.ask}}

    If the answer is empty say 'I don't know', otherwise reply with the answer.
    ";

    MemoryFilter memoryFilter = new MemoryFilter();
    if (tagDictionary?.Any() == true)
    {
        foreach (var kvp in tagDictionary)
        {
            memoryFilter = memoryFilter.ByTag(kvp.Key, kvp.Value);
        }
    }

    OpenAIPromptExecutionSettings settings = new()
    {
        ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions,
        Temperature = 0.70,

    };

    var args = new KernelArguments(settings)
    {
        ["input"] = question,
        [MemoryPlugin.TagsParam] = memoryFilter
    };          

    var chatCompletionService = skKernel.GetRequiredService<IChatCompletionService>();
    chatHistory.AddMessage(Microsoft.SemanticKernel.ChatCompletion.AuthorRole.User, skPrompt);
    var result = await chatCompletionService.GetChatMessageContentAsync(chatHistory, settings, skKernel);
    chatHistory.AddMessage(Microsoft.SemanticKernel.ChatCompletion.AuthorRole.Assistant, result.Content);
    return chatHistory;
}

[dluc]

Hi @bancroftway looks like you need an "intent detection" step before calling KM.
Let's take it from a person-to-person conversation, A is the end user, B is a person from customer support:

1. A says something
2. B analyzes the message: is the user asking (case 1) a question about a product, or (case 2) something else
3. Switch:
   • Case 1: search for an answer in the knowledge base
   • Case 2: inform the user they can only answer questions about products

Step 2 is what I call "intent detection", and you can implement it using LLM Function Calling, e.g. passing to an LLM "case 1" and "case 2" as functions, asking to make a choice.

This is likely about building an Agent who can have a conversation (chat history) and select plugins when necessary. In one call it can generate a chat answer or tell your app to call KM Ask (or other plugins) to generate an answer that your app can then show to the user.

flowchart TD
    classDef wrap text-wrap width:200px;

    A["Define agent with 1 plugin:<br>Answer questions about<br>a product<br>given the product ID"]:::wrap
    B["User talks to the agent"]:::wrap
    C["Agent decides whether it<br>can chat or needs<br>the plugin.<br>Agent extracts product ID<br>from user input"]:::wrap
    D["Agent decides to<br>use the plugin?"]:::wrap
    E["Agent generates a response"]:::wrap
    F["App calls plugin,<br>passing the product ID<br>chosen by the agent"]:::wrap
    G["Plugin returns an answer"]:::wrap
    H["Show answer"]:::wrap

    A --> B
    B --> C
    C --> D
    D -->|No| E
    D -->|Yes| F
    F --> G
    E --> H
    G --> H
    H --> B

Loading