-
Notifications
You must be signed in to change notification settings - Fork 1
Closed
Labels
M20Milestone 20: Provider flexibilityMilestone 20: Provider flexibilityllmLLM provider relatedLLM provider relatedpriority/mediumMedium priorityMedium prioritysize/M
Description
Epic: #365
Description
Provider that cascades requests through an ordered list of providers, falling back on errors or rate limits.
Design
[llm]
provider = "router"
[llm.router]
providers = ["claude", "compatible"]
health_check_interval_secs = 60Tasks
-
RouterProviderimplementingLlmProvidertrait - Ordered fallback: try first provider, on error try next
- Health check: periodic warmup calls to detect provider availability
- Mark unhealthy providers as down, retry after cooldown
- Respect
LlmError::RateLimitedas trigger for fallback - Unit tests with mock providers (healthy, failing, rate-limited)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
M20Milestone 20: Provider flexibilityMilestone 20: Provider flexibilityllmLLM provider relatedLLM provider relatedpriority/mediumMedium priorityMedium prioritysize/M