Skip to content

Add cancel/abort for in-flight LLM requests and tool executions #502

@bug-ops

Description

@bug-ops

Problem

Currently the only way to stop an ongoing LLM request or tool execution is Ctrl+C (full shutdown). There is no way to cancel a single operation while keeping the agent running.

Use case: user sends a wrong prompt and wants to abort the LLM response or tool chain mid-flight without restarting.

Current state

  • watch::channel<bool> shutdown signal — binary, triggers full agent exit
  • Streaming path uses tokio::select! with shutdown — works but kills everything
  • Native tool loop checks *self.shutdown.borrow() per iteration — same binary flag
  • Parallel tool execution (join_all / buffered) has no per-operation cancellation
  • TUI has no cancel keybinding; CLI has no interrupt signal besides Ctrl+C

Proposed approach

1. Add CancellationToken (from tokio-util) per operation

Replace the binary shutdown watch with a two-level scheme:

  • Operation-level CancellationToken — cancels current LLM call / tool chain, agent stays alive
  • Session-level shutdown — existing behavior (Ctrl+C), also triggers operation cancel via drop_guard

2. Wire cancellation into agent loop

  • process_response_native_tools(): wrap join_all/buffered tool futures with token.cancelled()
  • process_response_streaming(): replace shutdown branch with operation token
  • call_llm_with_timeout(): add token to tokio::select!
  • Shell executor: kill child process on cancel (child.kill())

3. Channel trait: add cancel signal

// Channel trait addition
fn cancel_rx(&mut self) -> Option<&mut tokio::sync::watch::Receiver<bool>>;

Or pass CancellationToken into agent context so it's channel-agnostic.

4. TUI: Esc in Normal mode during active operation → cancel

  • When agent is processing (spinner visible), Esc sends cancel signal
  • Show [Cancelled] status in chat history
  • Agent returns to idle, ready for next input

5. CLI: Ctrl+C once → cancel operation, twice → exit

  • First Ctrl+C cancels current operation
  • Second Ctrl+C (within 2s) triggers full shutdown
  • Standard double-Ctrl+C UX pattern

Scope

  • zeph-core: CancellationToken plumbing in agent loop and streaming
  • zeph-tools: shell child process kill on cancel
  • zeph-tui: Esc keybinding + cancel indicator
  • zeph-channels: CLI double-Ctrl+C handler

Acceptance criteria

  • Esc in TUI cancels current LLM/tool operation without exiting
  • Double Ctrl+C in CLI: first cancels, second exits
  • In-flight HTTP requests (LLM) are dropped on cancel
  • Running shell commands are killed on cancel
  • Chat shows [Cancelled] for aborted operations
  • Agent resumes normal input loop after cancel

Metadata

Metadata

Assignees

No one assigned

    Labels

    llmLLM provider related

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions