Skip to content

Add Router provider with fallback chain #376

@bug-ops

Description

@bug-ops

Epic: #365

Description

Provider that cascades requests through an ordered list of providers, falling back on errors or rate limits.

Design

[llm]
provider = "router"

[llm.router]
providers = ["claude", "compatible"]
health_check_interval_secs = 60

Tasks

  • RouterProvider implementing LlmProvider trait
  • Ordered fallback: try first provider, on error try next
  • Health check: periodic warmup calls to detect provider availability
  • Mark unhealthy providers as down, retry after cooldown
  • Respect LlmError::RateLimited as trigger for fallback
  • Unit tests with mock providers (healthy, failing, rate-limited)

Metadata

Metadata

Assignees

No one assigned

    Labels

    M20Milestone 20: Provider flexibilityllmLLM provider relatedpriority/mediumMedium prioritysize/M

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions