v1.35.33.dev1
What's Changed
- [UI] show exceptions by model deployments + model latencies - v0 by @ishaan-jaff in #3373
- [UI] Polish viewing Model Latencies by @ishaan-jaff in #3380
- changing ollama response parsing to expected behaviour by @TheDiscoMole in #1526
- Added cost & context metadata for openrouter/anthropic/claude-3-opus by @paul-gauthier in #3382
- fix - error sending details to log on sentry by @ishaan-jaff in #3384
Full Changelog: v1.35.33...v1.35.33.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 77 | 81.75372134047257 | 1.5592233014701338 | 0.0 | 467 | 0 | 70.36502400001154 | 514.3882039999994 |
| /health/liveliness | Passed ✅ | 61 | 63.30112607910893 | 15.278385069651675 | 0.0 | 4576 | 0 | 58.941096000012294 | 1077.0577769999932 |
| /health/readiness | Passed ✅ | 61 | 64.04839979952813 | 15.555506127514676 | 0.0 | 4659 | 0 | 59.04779399998006 | 1356.8010579999736 |
| Aggregated | Passed ✅ | 61 | 64.54817928983753 | 32.393114498636486 | 0.0 | 9702 | 0 | 58.941096000012294 | 1356.8010579999736 |
Contributors
TheDiscoMole, ishaan-jaff, and paul-gauthier