-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Add support for human-in-the-loop tool call approval #2581
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…n deferred tool calls
Docs Preview
|
tool_call_results be passed to run when message history ended on deferred tool callsdeferred_tool_results be passed to run when message history ended on deferred tool calls
|
@DouweM Thanks, after reading this commit I notice there are two different workflows:
In the second procedure, approval/denial is not an adequate abstraction, as typically they should be organized as ToolSuccessful/ToolFailed, where Successful -> tool approved, then executed, then finished Can we organize those two scenarios in one set of API? Or are they the difference between |
|
@yf-yang Yep, that's exactly the difference between approvals/calls on the DefferedTool{Requests,Results} classes, which maps to unapproved/deferred on the ToolDefinition (naming could still change). I intend to add more convenient ways to make requires_approval conditional on deps and args (e.g. letting you raise a RequiresApproval exception from inside the tool). An ApprovalRequiredToolset that could wrap an MCP server would also be useful. All of that will come after my vacation next week though :) |
deferred_tool_results be passed to run when message history ended on deferred tool calls…lRequests among output types
# Conflicts: # pydantic_ai_slim/pydantic_ai/_agent_graph.py # pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_function_toolset.py # pydantic_ai_slim/pydantic_ai/output.py # pydantic_ai_slim/pydantic_ai/result.py # pydantic_ai_slim/pydantic_ai/run.py # pydantic_ai_slim/pydantic_ai/tools.py # tests/test_agent.py
requires_approval: booltoTooland@agent.tool{_plain}raise ApprovalRequiredso they can dynamically determine if approval is required, based on e.g. tool args or explicit approval in message historyraise CallDeferredso they can dynamically determine if the tool call can be completed inside the agent loop or not, like in the case of long-running queries or background jobs.RunContext.tool_call_approved: boolto check from inside a tool function whether approval was granted (soApprovalRequiredwouldn't be raised again)deferred_tool_results: DeferredToolResultstorunanditermethods to pass in tool call IDs mapped to approvals (True/False/ToolApproved(override_args=...)/ToolDenied(message=...)) or external call results (Any/ToolReturn/ModelRetry)ApprovalRequiredToolsetthat can wrap e.g. an MCP server and require approval for either all tools, or selectively based on a function that is given the run context, tool def, and tool args.DeferredToolsettoExternalToolsetsince there are now 2 types of deferred tool calls: external and approval-requiredDeferredToolCallstoDeferredToolRequestsWith the changes in this PR, it would also be very easy to support
return EndRun(output: OutputT)from inside a tool function (returnrather thanraiseso we can enforce the generic output type when using the agent tool decorators), if we decide we want that. I think it'd be useful!