Skip to content

WIP - Support thinking mode for Anthropic models #170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

rhys117
Copy link
Contributor

@rhys117 rhys117 commented May 14, 2025

Resolves #154

Description

Add thinking support for Anthropic models.

Implementation Details

  • Set thinking on the supported model in models.json
  • Expose thinking response on Message via .thinking_content

Testing

So far, this has been tested manually.

3.2.2 :001 > chat = RubyLLM::Chat.new(
3.2.2 :002 >   model: 'claude-3-7-sonnet-latest'
3.2.2 :003 > )
3.2.2 :004 >
3.2.2 :005 > chat.ask('Write a short story about a cat')
 =>
#<RubyLLM::Message:0x0000000103a1c300
 @content=
  #<RubyLLM::Content:0x0000000105534070
   @attachments=[],
   @text=
    "# The Window Watcher\n\nEvery morning at precisely 7:36, Mr. Whiskers jumped onto the windowsill of apartment 3B. This was not a casual choice—it was exactly when the sunbeam stretched across the sill's worn wood, creating the perfect heated seat.\n\nFrom this perch, he observed the world below: the mail carrier with the jingling keys, the woman who always talked on her phone too loudly, and the old man who fed pigeons despite the building's \"NO FEEDING WILDLIFE\" signs.\n\nToday, something was different. Across the street, in the window directly facing his, sat another cat—orange with white paws like tiny socks. Mr. Whiskers' tail twitched with irritation. This was *his* viewing time.\n\nFor three days, they watched each other suspiciously. On the fourth day, the orange cat failed to appear. Mr. Whiskers found himself strangely disappointed.\n\nWhen the orange cat returned a week later wearing a cone of shame, Mr. Whiskers pressed a paw against the glass, almost sympathetically. The orange cat blinked slowly in response.\n\nBy month's end, they had developed a silent friendship through glass and air and morning light—two observers sharing the quiet joy of a world that never knew it was being watched.">,
 @input_tokens=42,
 @model_id="claude-3-7-sonnet-latest",
 @output_tokens=375,
 @role=:assistant,
 @thinking_content=
  "I'm asked to write a short story about a cat. This is a straightforward creative writing request. I'll create a brief story with a feline protagonist that has some personality, a small arc, and perhaps a little message. I'll aim for something heartwarming but not overly sentimental, with some sensory details to make the cat feel real.",
 @tool_call_id=nil,
 @tool_calls=nil>

TODO:

  • Add streaming support
  • Ensure it is easily enabled/disabled by configuration
  • Ensure support for acts_as persistent models

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Claude 3.7 Thinking support and budget
1 participant