Invalid data content. Content string is not a base64-encoded media.

[baptisteArno]

Description

I provide this to a model that does not support image URL:

"messages": [
  {
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": "What's this?"
      },
      {
        "type": "image",
        "image": "https://s3.eu-west-3.amazonaws.com/typebot-test/public/tmp/cm0f0q3mp0003xaaprbc72y9s/Typebot x Segment.png"
      }
    ]
  }
]

streamText fails with the following error:

InvalidDataContentError [AI_InvalidDataContentError]: Invalid data content. Content string is not a base64-encoded media.
    at convertDataContentToUint8Array (webpack-internal:///(rsc)/../../node_modules/.pnpm/ai@3.3.20_openai@4.56.1_zod@3.22.4__react@18.2.0_solid-js@1.7.8_sswr@2.1.0_svelte@4.2.18__sve_c3yht6e7jsjetu5ax435pl6h2m/node_modules/ai/dist/index.mjs:830:19)
    at eval (webpack-internal:///(rsc)/../../node_modules/.pnpm/ai@3.3.20_openai@4.56.1_zod@3.22.4__react@18.2.0_solid-js@1.7.8_sswr@2.1.0_svelte@4.2.18__sve_c3yht6e7jsjetu5ax435pl6h2m/node_modules/ai/dist/index.mjs:1031:56)
    ... 10 lines matching cause stack trace ...
    at async POST (webpack-internal:///(rsc)/./src/app/api/v2/sessions/[sessionId]/streamMessage/route.ts:33:41)
    at async /Users/baptistearno/Dev/typebot.io/node_modules/.pnpm/next@14.1.0_@babel+core@7.22.9_@opentelemetry+api@1.9.0_react-dom@18.2.0_react@18.2.0__react@18.2.0/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:6:63809
    at async eU.execute (/Users/baptistearno/Dev/typebot.io/node_modules/.pnpm/next@14.1.0_@babel+core@7.22.9_@opentelemetry+api@1.9.0_react-dom@18.2.0_react@18.2.0__react@18.2.0/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:6:53964) {
  cause: DOMException [InvalidCharacterError]: Invalid character
      at atob (node:buffer:1269:13)
      at convertBase64ToUint8Array (webpack-internal:///(rsc)/../../node_modules/.pnpm/@ai-sdk+provider-utils@1.0.17_zod@3.22.4/node_modules/@ai-sdk/provider-utils/dist/index.mjs:515:26)
      at convertDataContentToUint8Array (webpack-internal:///(rsc)/../../node_modules/.pnpm/ai@3.3.20_openai@4.56.1_zod@3.22.4__react@18.2.0_solid-js@1.7.8_sswr@2.1.0_svelte@4.2.18__sve_c3yht6e7jsjetu5ax435pl6h2m/node_modules/ai/dist/index.mjs:828:101)
      at eval (webpack-internal:///(rsc)/../../node_modules/.pnpm/ai@3.3.20_openai@4.56.1_zod@3.22.4__react@18.2.0_solid-js@1.7.8_sswr@2.1.0_svelte@4.2.18__sve_c3yht6e7jsjetu5ax435pl6h2m/node_modules/ai/dist/index.mjs:1031:56)
      at Array.map (<anonymous>)
      at convertToLanguageModelMessage (webpack-internal:///(rsc)/../../node_modules/.pnpm/ai@3.3.20_openai@4.56.1_zod@3.22.4__react@18.2.0_solid-js@1.7.8_sswr@2.1.0_svelte@4.2.18__sve_c3yht6e7jsjetu5ax435pl6h2m/node_modules/ai/dist/index.mjs:951:46)
      at eval (webpack-internal:///(rsc)/../../node_modules/.pnpm/ai@3.3.20_openai@4.56.1_zod@3.22.4__react@18.2.0_solid-js@1.7.8_sswr@2.1.0_svelte@4.2.18__sve_c3yht6e7jsjetu5ax435pl6h2m/node_modules/ai/dist/index.mjs:913:78)
      at Array.map (<anonymous>)
      at convertToLanguageModelPrompt (webpack-internal:///(rsc)/../../node_modules/.pnpm/ai@3.3.20_openai@4.56.1_zod@3.22.4__react@18.2.0_solid-js@1.7.8_sswr@2.1.0_svelte@4.2.18__sve_c3yht6e7jsjetu5ax435pl6h2m/node_modules/ai/dist/index.mjs:913:63)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async fn (webpack-internal:///(rsc)/../../node_modules/.pnpm/ai@3.3.20_openai@4.56.1_zod@3.22.4__react@18.2.0_solid-js@1.7.8_sswr@2.1.0_svelte@4.2.18__sve_c3yht6e7jsjetu5ax435pl6h2m/node_modules/ai/dist/index.mjs:3260:36)
      at async eval (webpack-internal:///(rsc)/../../node_modules/.pnpm/ai@3.3.20_openai@4.56.1_zod@3.22.4__react@18.2.0_solid-js@1.7.8_sswr@2.1.0_svelte@4.2.18__sve_c3yht6e7jsjetu5ax435pl6h2m/node_modules/ai/dist/index.mjs:298:28)
      at async streamText (webpack-internal:///(rsc)/../../packages/ai/streamText.ts:33:26)
      at async getMessageStream (webpack-internal:///(rsc)/../../packages/bot-engine/apiHandlers/getMessageStream.ts:119:39)
      at async POST (webpack-internal:///(rsc)/./src/app/api/v2/sessions/[sessionId]/streamMessage/route.ts:33:41),
  content: 'https://s3.eu-west-3.amazonaws.com/typebot-test/public/tmp/cm0f0q3mp0003xaaprbc72y9s/Typebot x Segment.png',
  [Symbol(vercel.ai.error)]: true,
  [Symbol(vercel.ai.error.AI_InvalidDataContentError)]: true
}

Code example

No response

Additional context

ai v3.3.20

Using anthropic Claude 3.5 Sonnet

@ai-sdk/anthropic v0.0.48

[lgrammel]

x Segment.png" at the end of your image

[lgrammel]

It seems like a valid image url, but the spaces need escaping. Maybe this can have better error messages.