Skip to content

Latest commit

 

History

History
26 lines (16 loc) · 1.35 KB

MODEL_BEHAVIORS.md

File metadata and controls

26 lines (16 loc) · 1.35 KB
Raw

Model Behaviors

This is to document some of the differences between model behaviors. This is specifically about how one LLM responds differently than another. This information is likely to get out of date quickly as models change. Still, it may be helpful as a reference or starting point.

Pre-filling the assistant's response

  • ChatGPT 3.5 and 4 do not complete a pre-filled assistant response as we might hope. Giving it instructions to provide an answer in an <answer>{{ANSWER}}</answer> template, then starting an assistant message with <answer> to have it complete it, it will not include the closing </answer> tag.

    • 2024-04-17

  • Anthropic's Claude 3 does respond well to pre-filled assistant responses and it is officially encouraged.

    • 2024-04-17

Image content parts

  • Anthropic will not work if the media type is included with the base64 data. ie. "data:image/jpeg;base64," <> "..."
    • Requires providing the media type as a separate option.
  • ChatGPT -
    • When base64 data given, requires providing the image data prefixed with the base64 content.
    • Ex: "data:image/jpeg;base64," <> "..."

Token Usage

GoogleAI returns token usage for each MessageDelta. They generated value increments with each message. This is an oddity with GoogleAI.

OpenAI ChatGPT, Anthropic Claude, and Bumblebee all return token usage information at the end.