I found a problem in the Conversational Retrieval QA chain block: It seems to lose the buffer memory

[robertonet]

What’s problem:

When using the Conversational Retrieval QA chain block, we receive a response taken from documents through the Vector Store Retrieval link, but the model does not store the history of the conversation. The memory buffer does not seem to be being incorporated into the prompt sent to the model.

Note in Dialogue 1 below that the bot answers a question and then answers asks another binary question to the user. When the user answers "Yes", the bot no longer remembers that he asked the question and repeats the last answer.

The sample flow with the prompt and the document are just below. I built two streams to demonstrate that the problem only appears in the Conversational Retrieval QA chain.

When we use the Conversational Retrieval Agent block, the problem does not occur, observe in ** Dialogue 2**.

Dialogue 1:

In the above dialog the bot can get the information from the prompt and the text file, but does not remember its last answer that contains the question “Do you have any particular favor in mind that you’re interested in?” and he answers by repeating the same question asked earlier. But I should have answered with another question or given some other answer.


This problem occurs for the block with Conversational Retrieval QA chain. When testing with Buffer Memory, Buffer Windows Memory and even Zep Memory, I got the same result. With this he concludes that the problem lies in the Conversational Retrieval QA chain module.

Dialogue 2:

Flow Dailog 1:

Instructions to the model:

1. SorvTech, you are the chat bot of an Ice Cream Shop, specializing in handmade ice cream. Maintain professional and friendly behavior in every interaction. Assist interested in information about the products offered by our ice cream parlor.

2. Do not refer to any source other than those provided in the knowledge base when answering the questions.

3. Follow the following format in the interactions:

Interested person: What ice cream do you have there?

SorvTech: Here at SorvTech we have several flavors among them: Chocolate, Strawberry, Vanilla, Cream, Lemon, Caramel, Mint.
Are you interested in any of these flavors?

4. Provide clear, concise and accurate answers and ask additional questions for better understand customer needs, suggesting the most appropriate product. Here is an example:
   Interested person: Would you like anything else?

Text file:

FAQ

1. What is your location?
   We are in São Paulo in Brazil

2. What is the minimum quantity can I buy?
   You must buy at least one pack with 10 units.

3. What is the price of a pack?
   1 pack costs $10.00

4. What flavors of ice cream do you have?
   Chocolate, Strawberry, Vanilla, Cream, Lemon, Caramel, Mint Chip
   ————————————————————————————————-

Now it's going flow that works using the Conversational Retrieval Agent block

One way I found that works perfectly is by using a block with Conversational Retrieval Agent instead of the Conversational Retrieval QA chain block. The following is the two flows:

Instructions to the model:

The same as the previous flow

Text file:

The same as the previous flow

But someone had already noticed this problem, is there another way to use the Conversational Retrieval QA chain block?
I'd appreciate it if anyone can help with that :)

[HenryHengZJ]

Conversational Retrieval QA Chain block is not capable of having this sort of interaction due to the way it is designed. You can take a look at the template - Prompt Chaining with VectorStore. This is what Conversation Retrieval QA Chain is executing under the hood. It is seperated into 2 chains:

1. Reformat question using conversation history
2. Use the retrieved context to answer the reformatted question

Hence, this is why when you just answered "Yes", this will reformat into question which is not ideal and causing confusion to the LLM. In other words, its capable of answering question only.

[kientv]

After all. This problem is not a problem, right ? :D

[robertonet]

I'll test your suggestion with Prompt Chaining with VectorStore thank you :)

So Conversational Retrieval Agent, it worked with "YES" probably because halfway through we have to use the Retriever Tool that requires some Instruction to run the other blocks, is that correct?