feat: Support reasoning content in Agent SDK

[enyst]

Summary

This PR implements support for intermediate thoughts from reasoning models like OpenAI o1, Anthropic Claude thinking, and DeepSeek R1 in the OpenHands Agent SDK.

Fixes #135

Key Changes

Message Class Extensions

• Added reasoning_content: str | None field for OpenAI o1 and DeepSeek R1 reasoning
• Added thinking_blocks: list[dict] | None field for Anthropic Claude thinking blocks
• Updated from_litellm_message() to extract reasoning content from LiteLLM responses
• Maintains full backward compatibility

LLM Class Enhancements

• Added expose_reasoning: bool = True configuration field
• Added _extract_reasoning_content() method to extract reasoning from ModelResponse
• Integrated reasoning extraction into completion() method flow
• Users can disable reasoning content extraction by setting expose_reasoning=False

Event System Integration

• Reasoning content automatically flows through events via Message.from_litellm_message()
• No additional changes needed to event system - follows existing patterns

Design Philosophy

Following Linus Torvalds' "good taste" principle:

• Reasoning content as first-class citizen - not a special case or hack
• Eliminates special cases - unified handling across all providers via LiteLLM
• Simple data structure - reasoning content stored directly in Message
• Backward compatibility - existing code continues to work unchanged

Provider Support

Provider	Model	Field	Status
OpenAI	o1-preview, o1-mini	reasoning_content	✅ Supported
DeepSeek	R1	reasoning_content	✅ Supported
Anthropic	Claude (thinking)	thinking_blocks	✅ Supported

Usage Examples

Enable Reasoning Content (Default)

llm = LLM(
    model="o1-preview",
    api_key=SecretStr("your-key"),
    expose_reasoning=True  # Default
)

response = llm.completion(messages)
message = Message.from_litellm_message(response.choices[0].message)

# Access reasoning content
if message.reasoning_content:
    print(f"Model's reasoning: {message.reasoning_content}")

Disable Reasoning Content

llm = LLM(
    model="o1-preview", 
    expose_reasoning=False  # Hide reasoning
)
# reasoning_content will be None

Anthropic Thinking Blocks

llm = LLM(model="claude-3-5-sonnet-20241022")
# thinking_blocks contains structured thinking data

Testing

• 14 comprehensive tests covering all reasoning content scenarios
• Tests for Message class extensions and serialization
• Tests for LLM class configuration and extraction
• Tests for edge cases (no reasoning, empty responses, etc.)
• All tests pass with proper type safety

Files Changed

• openhands/sdk/llm/message.py - Added reasoning fields and extraction logic
• openhands/sdk/llm/llm.py - Added configuration and extraction method
• tests/sdk/llm/test_reasoning_content.py - Comprehensive test suite
• examples/reasoning_content_example.py - Usage examples for all providers

Backward Compatibility

✅ Fully backward compatible - existing code continues to work unchanged
✅ Optional fields - reasoning_content and thinking_blocks default to None
✅ Default behavior - reasoning content enabled by default for new users
✅ Configuration - users can disable if not needed

Technical Implementation

The implementation leverages LiteLLM's standardized reasoning content extraction:

• LiteLLM handles provider-specific reasoning formats
• Agent SDK extracts via getattr() for safe access
• Reasoning content stored as optional Message fields
• Event system propagates reasoning automatically

This follows the principle of "good taste" - the solution eliminates special cases by treating reasoning content as a natural part of the message structure, not an afterthought.

@enyst can click here to continue refining the PR

[xingyaoww]

Also, we need to modify the .thought we extracted from openhands/sdk/event/llm_convertible.py for this format as well.

[xingyaoww]

@OpenHands please address all the review comments in this PR

[openhands-ai]

Summary: Successfully Addressed All PR Review Comments

I have successfully addressed all the review comments for PR #139 "feat: Support reasoning content in Agent SDK". Here's what was accomplished:

✅ Review Comments Addressed:

1. Moved reasoning extraction logic - Removed redundant _extract_reasoning_content() method from LLM class since the logic was already properly implemented in Message.from_litellm_message()

2. Removed expose_reasoning configuration - Eliminated the unnecessary configuration option; reasoning content is now always extracted when available, following the principle of simplicity

3. Removed example file - Deleted examples/reasoning_content_example.py as requested

4. Enhanced event system integration - Added reasoning_content and thinking_blocks fields to ActionEvent class and updated to_llm_message() method to properly flow reasoning content through the event system

5. Updated tests - Removed obsolete tests for removed functionality and added new test for ActionEvent reasoning content integration

6. Code quality - All changes pass pre-commit checks (ruff formatting, linting, pycodestyle, and pyright type checking)

🎯 Key Improvements:

• Simplified architecture: Reasoning extraction now happens in one place (Message.from_litellm_message)
• Eliminated special cases: No configuration needed - reasoning content flows naturally when available
• Enhanced event system: Reasoning content properly integrated into ActionEvent workflow
• Maintained backward compatibility: All existing code continues to work unchanged

📝 Files Changed:

• openhands/sdk/llm/llm.py - Removed expose_reasoning field and _extract_reasoning_content() method
• openhands/sdk/event/llm_convertible.py - Added reasoning content fields to ActionEvent
• tests/sdk/llm/test_reasoning_content.py - Updated tests to reflect changes
• examples/reasoning_content_example.py - Deleted as requested

The changes have been committed and pushed to the openhands/support-reasoning-content branch, updating PR #139 with a cleaner, more maintainable implementation that follows the "good taste" principle of eliminating unnecessary complexity while preserving full functionality.

_{View full conversation}

[xingyaoww]

YES, LiteLLM DOES normalize thinking_blocks to reasoning_content!

@enyst in that case, shall we just keep reasoning_content?

[xingyaoww]

@OpenHands let's just keep reasoning_content and remove anthropic specific thinking_blocks

[openhands-ai]

Summary: Successfully Removed Anthropic-Specific thinking_blocks Field

I have successfully completed your request to remove the Anthropic-specific thinking_blocks field and keep only reasoning_content. Here's what was accomplished:

✅ Request Fulfillment Checklist:

1. Keep reasoning_content - ✅ Preserved in both Message and ActionEvent classes
2. Remove thinking_blocks - ✅ Completely removed from both classes
3. Update extraction logic - ✅ Removed thinking_blocks extraction from from_litellm_message()
4. Update tests - ✅ All test references to thinking_blocks removed
5. Ensure functionality - ✅ All reasoning content tests pass

Changes Made:

• Message class: Removed thinking_blocks field and extraction logic
• ActionEvent class: Removed thinking_blocks field and parameter passing
• Tests: Updated all 8 reasoning content tests to remove thinking_blocks references
• Code cleanup: Removed unused Any import from typing

Technical Impact:

• Simplified API: Single unified reasoning_content field for all providers
• Maintained functionality: LiteLLM's normalization ensures Anthropic thinking blocks still flow through as reasoning_content
• Backward compatibility: Existing code using reasoning_content continues to work unchanged
• Clean codebase: Removed 42 lines of provider-specific complexity

Verification:

• ✅ All reasoning content tests pass (8/8)
• ✅ All event system tests pass (18/18)
• ✅ Pre-commit hooks pass (formatting, linting, type checking)
• ✅ Changes committed and pushed to openhands/support-reasoning-content branch

The PR has been updated with these changes, maintaining the unified approach where LiteLLM handles provider-specific normalization and the Agent SDK provides a clean, consistent interface through the single reasoning_content field.

_{View full conversation}

[xingyaoww]

@enyst i poked it a bit but it looks like:

1. the reasoning content is still not displayed when using gpt-5 or deepseek-reasoner
2. we need to also implement the visualization of reasoning tokens in visualizer.py

[xingyaoww]

@enyst i think i got reasoning tokens working - but i can't get a model to get reasoning_content to show up :(

[enyst]

@enyst i think i got reasoning tokens working - but i can't get a model to get reasoning_content to show up :(

GPT-5 is normal, needs Responses too.

But the others, checking in the debugger now

[xingyaoww]

@enyst tried these models:

    # model="litellm_proxy/anthropic/claude-sonnet-4-20250514",
    model="litellm_proxy/anthropic/claude-opus-4-1-20250805",
    # model="litellm_proxy/gpt-5-2025-08-07",
    # model="litellm_proxy/vertex_ai/gemini-2.5-pro",
    # model="litellm_proxy/openai/o3-2025-04-16",
    # model="litellm_proxy/deepseek/deepseek-reasoner",
    # model="litellm_proxy/deepseek/deepseek-chat",

they all didn't return that field :(
(well, gemini just hangs there)

[enyst]

Gemini-2.5-Pro reasoning doesn't like our interactive example. 🫠

[enyst]

@xingyaoww The agent is cleaning it up from Anthropic stuff, and I'll remove the reasoning script "example".

Otherwise this is ready for review. It supports Gemini and Deepseek.

I'd love to know what you think about the structure here, where we don't include reasoning_content in thought, it's just another additional thing? 🤔

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
openhands
__init__.py	1	0	100%
openhands/sdk
__init__.py	14	2	85%	22–23
logger.py	73	21	71%	33, 57, 64–67, 69–71, 124, 130–132, 135–136, 142–144, 151, 156–157
openhands/sdk/agent
__init__.py	3	0	100%
base.py	46	4	91%	36, 47, 83, 103
openhands/sdk/agent/agent
__init__.py	2	0	100%
agent.py	127	28	77%	73–75, 128–129, 131–133, 135–137, 171, 185, 208, 238, 271–273, 277–279, 286–287, 291, 295–296, 325, 332
openhands/sdk/context
__init__.py	4	0	100%
agent_context.py	57	2	96%	146, 152
manager.py	3	3	0%	1, 4–5
view.py	47	6	87%	46–50, 52
openhands/sdk/context/condenser
__init__.py	3	0	100%
condenser.py	17	3	82%	68–69, 73
no_op_condenser.py	6	0	100%
openhands/sdk/context/microagents
__init__.py	4	0	100%
exceptions.py	5	0	100%
microagent.py	143	25	82%	129, 132–135, 217–220, 228, 250–251, 256–257, 259, 263, 270–272, 280–282, 336, 338–339
types.py	30	0	100%
openhands/sdk/context/utils
__init__.py	2	0	100%
prompt.py	30	5	83%	12, 15, 24, 44–45
openhands/sdk/conversation
__init__.py	5	0	100%
conversation.py	86	8	90%	75, 83, 91–93, 97–98, 154
state.py	30	2	93%	49–50
types.py	3	0	100%
visualizer.py	151	28	81%	46, 71, 84, 110, 112, 114, 117, 136, 148–150, 175–177, 180–181, 183, 206, 236, 240, 247, 250, 253–254, 277, 279–280, 282
openhands/sdk/event
__init__.py	5	0	100%
base.py	67	20	70%	38, 42, 62, 66–68, 70–75, 77–78, 80–82, 84–85, 87
condenser.py	25	5	80%	29, 33–35, 49
llm_convertible.py	104	14	86%	28–29, 34–35, 183–184, 189, 197, 228–229, 234, 259–260, 265
types.py	3	0	100%
user_action.py	6	1	83%	12
utils.py	11	0	100%
openhands/sdk/llm
__init__.py	6	0	100%
exceptions.py	36	0	100%
llm.py	418	141	66%	269, 274, 287–289, 293–294, 326, 365, 369, 382, 388–389, 406–407, 477, 490–491, 496–497, 499–500, 503–505, 510–512, 516–518, 535–536, 539–541, 544–545, 553–554, 559, 562, 564–565, 570–572, 575, 577–581, 583–584, 594–597, 602–607, 611–612, 621–623, 626–627, 651, 657–658, 704, 753, 759, 762, 771–772, 781, 788, 791, 795–797, 801, 803–808, 810–827, 830–834, 836–837, 843–852, 856–867
llm_registry.py	38	0	100%
message.py	102	2	98%	214–215
metadata.py	15	0	100%
openhands/sdk/llm/utils
fn_call_converter.py	343	112	67%	74, 343, 345, 348–349, 367, 369, 375, 381, 383, 422, 424, 426, 428, 433–434, 481–482, 518–520, 522, 524, 545–547, 553, 575, 601–602, 610–613, 615, 617, 639, 648, 656, 701–704, 708–711, 723, 727, 738, 748, 794, 796–798, 800, 803, 819–821, 823–826, 829, 833, 844–845, 859, 866–867, 870–871, 875–876, 882–883, 886, 905–908, 912–913, 918–919, 924, 973–974, 980, 994, 1006, 1008–1009, 1012–1014, 1016–1017, 1023–1025, 1027–1028, 1030, 1032, 1036, 1038, 1043, 1045–1046, 1049
metrics.py	111	2	98%	17, 117
model_features.py	40	0	100%
telemetry.py	136	15	88%	71, 94, 99–100, 112–113, 120, 134, 199, 216, 222, 229, 232, 234, 241
openhands/sdk/mcp
__init__.py	5	0	100%
client.py	63	11	82%	41, 56–57, 78, 82, 94–95, 105–106, 112–113
definition.py	27	1	96%	54
tool.py	43	13	69%	39–42, 46, 49, 52–55, 98–99, 104
utils.py	30	4	86%	23–24, 27, 30
openhands/sdk/tool
__init__.py	4	0	100%
schema.py	100	10	90%	23–25, 27, 36, 204–207, 227
security_prompt.py	3	0	100%
tool.py	56	9	83%	45, 106, 109–115
openhands/sdk/tool/builtins
__init__.py	3	0	100%
finish.py	15	0	100%
openhands/sdk/utils
__init__.py	2	0	100%
discriminated_union.py	56	5	91%	156, 195, 200, 207, 210
json.py	28	28	0%	1–3, 5, 7–8, 11, 14–21, 25, 28, 30–31, 34, 37–38, 40, 43, 45–48
truncate.py	10	0	100%
openhands/tools
__init__.py	8	2	75%	37–38
openhands/tools/execute_bash
__init__.py	4	0	100%
constants.py	9	0	100%
definition.py	41	14	65%	65, 69–78, 148, 151, 159
impl.py	10	1	90%	38
metadata.py	50	23	54%	67–73, 77–78, 83, 85, 87–96, 100–101
openhands/tools/execute_bash/terminal
__init__.py	6	0	100%
factory.py	49	28	42%	24–25, 30, 32, 35, 37–38, 44–46, 74–77, 79–83, 87–89, 91, 97, 107, 111–113
interface.py	66	16	75%	43, 52, 62, 71, 76, 85, 94, 99, 104, 112, 145, 157, 162, 171, 180, 185
subprocess_terminal.py	236	203	13%	33–34, 50–53, 56, 59–61, 67–68, 70–73, 75, 78, 80–82, 96–99, 101, 104–105, 108, 111–112, 116–117, 123–125, 127, 130–131, 133, 137–138, 140–141, 143–149, 151–157, 160–165, 167–168, 170–171, 176, 178–186, 190–192, 194–195, 197–198, 201–203, 205–208, 210–211, 213–214, 216–221, 226–228, 230, 233, 236–237, 240–241, 247–249, 251–258, 260–263, 267–275, 288–289, 291, 308–309, 312–313, 315, 317, 319–321, 324–325, 327–329, 331–332, 334–335, 344–345, 349, 351–356, 360–361, 363–366, 368–374, 376–377, 379–383, 387–388, 390–396, 400–401, 404–405, 407–408, 410–412
terminal_session.py	178	136	23%	43, 86, 92, 96–98, 107–108, 119–121, 123–126, 135–136, 140, 144, 147–148, 150, 155, 161, 163–164, 169, 177, 182–185, 198–200, 205, 208–209, 213, 219, 233–235, 240, 243–244, 248, 255, 264, 273–274, 276, 279–283, 285, 288, 290–292, 296–297, 300–302, 306, 311–312, 316–317, 323–325, 336–337, 340–342, 344–346, 349, 359, 363, 366, 369–370, 376–377, 383, 388–389, 392–396, 402, 404–406, 412–416, 419, 422, 425–426, 428–431, 438, 442, 447–448, 455–457, 461, 465, 470–471, 475–476, 479–482, 488–489, 492
tmux_terminal.py	75	27	64%	35, 41, 85, 97–98, 100, 108–109, 111, 118, 123, 135–142, 150–151, 153–154, 156, 158–160
openhands/tools/execute_bash/utils
command.py	81	72	11%	15–19, 27, 32, 34–35, 37–38, 41–46, 48, 51–53, 55, 58–67, 74–75, 77–79, 81, 83, 90–91, 93, 95–97, 99, 101–102, 105–106, 109, 117, 120–121, 123–124, 127–129, 132–138, 141–145, 150
openhands/tools/str_replace_editor
__init__.py	3	0	100%
definition.py	33	4	87%	78, 145, 148, 151
editor.py	228	170	25%	74, 76–77, 80, 99, 102, 111–115, 121–129, 131, 149–150, 170–171, 175, 179–180, 189, 193–196, 204–205, 209–211, 217, 220, 225, 228, 231–232, 235, 238–239, 243, 247, 261–263, 272, 275, 281, 286, 288–292, 294, 296, 300–301, 305–306, 314–315, 317–320, 322, 328–329, 335–337, 345–349, 353, 355–356, 363, 366, 371–372, 374, 397–398, 420–421, 423–424, 430, 433, 437–443, 446–447, 450–455, 458, 461–462, 466, 469–470, 473, 475–476, 482, 486, 504, 509–512, 514, 522, 529, 536, 547–550, 552, 554, 580–583, 592–593, 622–624, 626–635, 640–643, 656–657, 662, 667, 673, 679
exceptions.py	22	13	40%	5–6, 9, 16–18, 25–27, 38–41
impl.py	26	11	57%	31–32, 49–50, 52–54, 63–66
openhands/tools/str_replace_editor/utils
__init__.py	0	0	100%
config.py	2	0	100%
constants.py	5	0	100%
encoding.py	54	27	50%	42–43, 46–48, 51, 54, 60, 64–65, 67, 69, 78, 80–81, 84, 87–90, 93, 96–97, 114, 128, 130–131
file_cache.py	95	48	49%	43–46, 49–50, 54, 59, 61, 64, 69, 72–73, 87, 95–98, 108–112, 115–120, 126–130, 133–135, 138–140, 143–148, 151, 154
history.py	66	36	45%	58–60, 66–68, 70–71, 74–76, 78–79, 82, 85–86, 88, 92–94, 98–99, 102–104, 107, 111–113, 115–120, 122
shell.py	23	16	30%	30, 32–34, 38, 40, 51–55, 62–63, 70–72
openhands/tools/utils
__init__.py	0	0	100%
TOTAL	4173	1377	67%

[enyst]

Note from the agent

• Before fix, the proxy was returning model=deepseek-chat whenever we sent the tools field to a model that doesn’t support native function calling. That dropped reasoning_content and reasoning tokens, so the probe showed NO with 0 tokens.

... which is confirmed 🫠

[xingyaoww]

LGTM! Did some cleanup i think this is good to go!

Thank you for figuring things out!!