Tool Calling messages are lost in cloned MessageBag #321
Description
Hello,
I'm currently facing an issue that I'm not able to retrieve the tool messages (AssistantMessage calling the tool + ToolMessage for response), which are handled by the Toolbox/ChainProcessor.
Reason is that I'd like to cache all messages, so that they will stay in the context for consecutive calls.
My first instinct was to simply fetch them from the message bag after I get the response - however the message bag doesn't contain it anymore.
After investigating I found that this happens, because the Toolbox/ChainProcessor is cloning the message bag. Therefore these messages never are added to the original message bag and I don't have any access to them.
I patched my version, to make it work and removed this clone. I wonder however if there is a reason for this or if this could be adjusted. If not, maybe there is a better way to achieve this goal?
As far as I understand these messages should still be included later on, so that the LLM is not missing this information when continuing with additional messages.
Here is my patch:
--- ./src/Chain/Toolbox/ChainProcessor.php
+++ ./src/Chain/Toolbox/ChainProcessor.php
@@ -81,7 +81,7 @@ final class ChainProcessor implements InputProcessor, OutputProcessor, ChainAwar
private function handleToolCallsCallback(Output $output): \Closure
{
return function (ToolCallResponse $response, ?AssistantMessage $streamedAssistantResponse = null) use ($output): ResponseInterface {
- $messages = clone $output->messages;
+ $messages = $output->messages;
if (null !== $streamedAssistantResponse && '' !== $streamedAssistantResponse->content) {
$messages->add($streamedAssistantResponse);