Anthropic user content inside tool result

[sintanial]

Description

Title:

Error with Consecutive User Role Messages After Tool Role Messages in Anthropic API

Description:

When attempting to place a user role message immediately after a tool role message using the Anthropic API, an error is encountered:

AI_APICallError: messages: roles must alternate between "user" and "assistant", but found multiple "user" roles in a row.

This issue seems to be caused by the Anthropic API's requirement for roles to alternate strictly between "user" and "assistant". The current behavior does not support having a user role message following a tool role message without an intermediate assistant message.

Steps to Reproduce:

1. Make an API call using the Anthropic API.
2. Structure the messages in the following order:
   • Tool role 'tool-call' message
   • User role with 'tool-result' message
   • User role with text
3. Execute the API call.

Expected Behavior:

The API should accept the message structure, possibly by automatically incorporating the user message content within the tool result content or by providing a mechanism to handle such sequences without causing an error.

Suggested Solution:

Replace this line https://github.com/vercel/ai/blob/main/packages/anthropic/src/convert-to-anthropic-messages-prompt.ts#L74 to this code:

if (messages.length > 0 && messages[messages.length-1].role == "user" && messages[messages.length-1].content.find(part => part.type == "tool_result")) {
         messages[messages.length-1].content = messages[messages.length-1].content.concat(anthropicContent);
} else {
         messages.push({role: "user", content: anthropicContent});
}

[sintanial]

My current patch with https://github.com/ds300/patch-package if some body faced this problem too and need solution.

diff --git a/node_modules/@ai-sdk/anthropic/dist/index.js b/node_modules/@ai-sdk/anthropic/dist/index.js
index 28c5501..551491d 100644
--- a/node_modules/@ai-sdk/anthropic/dist/index.js
+++ b/node_modules/@ai-sdk/anthropic/dist/index.js
@@ -102,7 +102,11 @@ async function convertToAnthropicMessagesPrompt({
             }
           }
         }
-        messages.push({ role: "user", content: anthropicContent });
+        if (messages.length > 0 && messages[messages.length-1].role == "user" && messages[messages.length-1].content.find(part => part.type == "tool_result")) {
+          messages[messages.length-1].content = messages[messages.length-1].content.concat(anthropicContent);
+        } else {
+          messages.push({role: "user", content: anthropicContent});
+        }
         break;
       }
       case "assistant": {
diff --git a/node_modules/@ai-sdk/anthropic/dist/index.mjs b/node_modules/@ai-sdk/anthropic/dist/index.mjs
index 55d0883..e87eae6 100644
--- a/node_modules/@ai-sdk/anthropic/dist/index.mjs
+++ b/node_modules/@ai-sdk/anthropic/dist/index.mjs
@@ -82,7 +82,11 @@ async function convertToAnthropicMessagesPrompt({
             }
           }
         }
-        messages.push({ role: "user", content: anthropicContent });
+        if (messages.length > 0 && messages[messages.length-1].role == "user" && messages[messages.length-1].content.find(part => part.type == "tool_result")) {
+          messages[messages.length-1].content = messages[messages.length-1].content.concat(anthropicContent);
+        } else {
+          messages.push({role: "user", content: anthropicContent});
+        }
         break;
       }
       case "assistant": {

[EvanFabry]

Thank you, @sintanial! This worked for me.

@lgrammel - this was super useful. Might be worth taking a look at :)

[lgrammel]

User role with 'tool-result' message: this is not an allowed combination. You should be using a tool role. Also, the tool-call must be in the assistant role message. Can you provide a code snippet? Are you using typescript?

[sintanial]

In my case, i make 'ai web crawler', and this is my message seuqence:

[
  {
    "role": "system",
    "content": "<ROLE>You are an intelligent web browser capable of navigating web pages.</ROLE> ...."
  },
  {
    "role": "user",
    "content": "<PAGE_HTML>Some html of current page</PAGE_HTML>"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "tool-call",
        "toolCallId": "call_2Rc2wCK6PIP2WpeHddb7R3n3",
        "toolName": "click",
        "args": {
          "explanation": "Accept all cookies to proceed with navigation.",
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": [
      {
        "type": "tool-result",
        "toolCallId": "call_2Rc2wCK6PIP2WpeHddb7R3n3",
        "toolName": "click",
        "result": "Click completed successfully"
      }
    ]
  },
  {
    "role": "user",
    "content": "<PAGE_HTML>Some HTML here after click</PAGE_HTML>"
  }
]

As you can see, i put role 'user' just after role 'tool'. For my case it was necessary, because every time after every browser action i must to show LLM content of the current page. OpenAI and Claude allow this case and proper handle it.

For anthropic when you need to answer on 'tool-call' (which part of assistant message), you must put your 'tool-result' type inside 'user' role message content, and also if this is the 'user' role, it's allowed to adding type 'text' with any text content. The same way related to OpenAI too.

[lgrammel]

I see - so you want to provide additional information to the LLM when the user clicks accept. Have you considered the following:

[
  {
    "role": "system",
    "content": "<ROLE>You are an intelligent web browser capable of navigating web pages.</ROLE> ...."
  },
  {
    "role": "user",
    "content": "<PAGE_HTML>Some html of current page</PAGE_HTML>"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "tool-call",
        "toolCallId": "call_2Rc2wCK6PIP2WpeHddb7R3n3",
        "toolName": "click",
        "args": {
          "explanation": "Accept all cookies to proceed with navigation.",
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": [
      {
        "type": "tool-result",
        "toolCallId": "call_2Rc2wCK6PIP2WpeHddb7R3n3",
        "toolName": "click",
        "result": "<PAGE_HTML>Some HTML here after click</PAGE_HTML>"
      }
    ]
  }
]

[lgrammel]

That idea aside, this raises an important issue. It might be better to simplify the messages API for the Vercel AI SDK into user and assistant messages, deprecating tool messages.

[sintanial]

I see - so you want to provide additional information to the LLM when the user clicks accept. Have you considered the following:

[
  {
    "role": "system",
    "content": "<ROLE>You are an intelligent web browser capable of navigating web pages.</ROLE> ...."
  },
  {
    "role": "user",
    "content": "<PAGE_HTML>Some html of current page</PAGE_HTML>"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "tool-call",
        "toolCallId": "call_2Rc2wCK6PIP2WpeHddb7R3n3",
        "toolName": "click",
        "args": {
          "explanation": "Accept all cookies to proceed with navigation.",
        }
      }
    ]
  },
  {
    "role": "tool",
    "content": [
      {
        "type": "tool-result",
        "toolCallId": "call_2Rc2wCK6PIP2WpeHddb7R3n3",
        "toolName": "click",
        "result": "<PAGE_HTML>Some HTML here after click</PAGE_HTML>"
      }
    ]
  }
]

Yep, your version is good. But it has a little problem if the function must return some result (for example, collect something). Yes, you can concatenate all results and HTML page content, but a better approach for me is to separate it. I think this library should support all the flexibility of providers (Anthropic, OpenAI, etc.).

That idea aside, this raises an important issue. It might be better to simplify the messages API for the Vercel AI SDK into user and assistant messages, deprecating tool messages.

It's a good idea! Makes sense! Also, it's much more intuitive because the tool is called on the assistant side and the tool results must be on the user side. The Anthropic version of organizing this functionality is quite good.

[nikshepsvn]

Can we get this fixed? Blocks very basic Anthropic usage without the patch

[lgrammel]

#2067