feat: native tool_use for Claude and OpenAI providers#335
Merged
Conversation
Extend LlmProvider trait with supports_tool_use() and chat_with_tools() default methods. Add ToolDefinition, ChatResponse, and ToolUseRequest types to zeph-llm. Add ToolUse/ToolResult variants to MessagePart with serde support and flatten_parts() handling. Implement Anthropic tool_use format in ClaudeProvider with structured content blocks for tool_use/tool_result messages. Implement OpenAI function calling format in OpenAiProvider with tool_calls parsing and tool role messages. Add dual-mode agent loop: process_response() branches on supports_tool_use() into native tool path with structured execution loop, doom-loop detection, and context budget checks. Legacy text extraction path unchanged for Ollama/Candle providers. Delegate new trait methods through AnyProvider. Backward compatible via default implementations. Closes #254, closes #255, closes #256, closes #257, closes #258
github-actions
bot
added
enhancement
github-actions
bot
added
enhancement
Codecov Report❌ Patch coverage is ❌ Your patch status has failed because the patch coverage (52.94%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage. @@ Coverage Diff @@
## main #335 +/- ##
==========================================
- Coverage 80.28% 79.35% -0.94%
==========================================
Files 99 99
Lines 24241 25076 +835
==========================================
+ Hits 19463 19900 +437
- Misses 4778 5176 +398
🚀 New features to boost your workflow:
|
SubProvider and ModelOrchestrator were missing delegation for the new trait methods, causing the agent loop to always take the legacy text extraction path even when the underlying provider supports native tool_use. Add debug logging for path selection.
ShellExecutor was missing execute_tool_call() implementation, inheriting the default that returns None. Native tool_use calls to bash were silently producing no output. Extract command from structured params and delegate to existing execute_inner pipeline.
When the provider supports native tool_use, the tools are passed via the API tools parameter. The text-based tool catalog in the system prompt was causing Claude to respond with fenced code blocks instead of structured tool_use calls. Skip prompt-based tool instructions for native tool_use providers.
The trait default using fn -> impl Future (RPITIT) with a body was not being overridden by async fn implementations in provider structs, causing the default fallback to chat() to always be called. Change the trait definition to async fn which correctly dispatches to overrides.
…r capable providers Split BASE_PROMPT into header/legacy/native/tail sections. When provider supports tool_use, system prompt instructs to use structured tool mechanism instead of fenced code blocks. This was the root cause of Claude returning text with backtick blocks despite tools being passed via API.
Add Native Tool Use section to tools guide describing the structured API-level tool calling mechanism for Claude and OpenAI providers. Rename existing section to Legacy Text Extraction. Update README with native tool use feature entry and architecture description. Add system prompt split entry to changelog.
This was
linked to
issues
Feb 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LlmProvidertrait withsupports_tool_use()andchat_with_tools()methodsToolDefinition,ToolUseRequest,ChatResponsetypes tozeph-llmToolUseandToolResultvariants toMessagePartfor structured message partstool_useAPI inClaudeProviderOpenAiProvidersupports_tool_use/chat_with_toolsthroughModelOrchestratorandSubProviderexecute_tool_call()inShellExecutorfor structured tool invocationsschemars(#[derive(JsonSchema)])Test plan
tool_usecontent blocks, agent executes tools, sends results back, Claude produces final response