Skip to content

Parallel execution of multiple native tool_use calls #485

@bug-ops

Description

@bug-ops

Problem

When Claude returns multiple tool_use blocks in a single response (e.g., 7 parallel bash calls), the agent executes them sequentially in handle_native_tool_calls(). For N tools at ~2s each, latency is O(N) instead of O(1).

Current behavior

streaming.rs:handle_native_tool_calls iterates tool_calls with a sequential for loop, awaiting each execute_tool_call before starting the next.

Proposed solution

Replace the sequential loop with futures::future::join_all to execute all tool calls concurrently. Process results (metrics, channel sends, message building) sequentially after completion.

Key design decisions:

  • Use join_all (not JoinSet) to avoid 'static lifetime requirements on borrowed executor
  • Trust the model's parallelism signal: multiple tool_use in one response = independent calls
  • Add optional max_parallel_tools config (default 8) with buffer_unordered fallback
  • No trait changes needed: execute_tool_call(&self) is already Send + Sync

Files

  • crates/zeph-core/src/agent/streaming.rs — core change
  • crates/zeph-core/src/config.rs — optional concurrency limit
  • config/default.toml — config documentation

Plan

.local/plan/parallel-tool-execution.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions