v1.35.20.dev2

What's Changed

• (feat) Langfuse - Add location logging, and add cache_hit to metadata. by @Manouchehri in #2961
• [FEAT] Add groq/llama3 by @ishaan-jaff in #3176
• [UI] Show teams as dropdown in invite user flow by @ishaan-jaff in #3180
• [FEAT] Log team alias to langfuse by @ishaan-jaff in #3175
• fix: Stream completion responses from anthropic. (Fix #3129) by @jmandel in #3174
• [Fix] - Langfuse log proxy_base_url to langfuse as a tag (if set by user) by @ishaan-jaff in #3183
• fix(caching.py): dual cache async_batch_get_cache fix + testing by @krrishdholakia in #3179
• fix(caching.py): fix redis url parsing logic to work with ssl urls by @krrishdholakia in #3173
• [Fix] completion(model="gemini/gemini-pro-1.5-latest" raises Exception by @ishaan-jaff in #3186
• fix(_redis.py): support redis ssl as a kwarg REDIS_SSL by @krrishdholakia in #3191
• FIX: ollama chat completion proxy internal server 500 by @merefield in #3189
• Disable special tokens in ollama completion when counting tokens by @rick-github in #3170
• [Fix] - /audio/transcriptions security fix by @ishaan-jaff in #3193
• [UI] - non admin flow - only Create + Test Key available by @ishaan-jaff in #3184
• Load google ADC before init AnthropicVertex by @ushuz in #3150
• Fix tool call errors using anthropic by @n1lanjan in #3118
• Fix new line issue in cohere_message_pt by @elisalimli in #3115
• fix - slack alerting show input for embedding requests by @ishaan-jaff in #3196
• fix(router.py): Make TPM limits concurrency-safe by @krrishdholakia in #3192
• [UI] - simplify "Create Key" for non admins by @ishaan-jaff in #3197
• ui - fix create key flow / cleanup non admin flow by @ishaan-jaff in #3198
• Revert "Load google ADC before init AnthropicVertex" by @ishaan-jaff in #3203
• [Feat]- show langfuse trace in slack alerts by @ishaan-jaff in #3209
• [UI] round up team spend to 2 decimals + diversify legend for team spend by @ishaan-jaff in #3210
• UI - increase default session time to 2 hours by @ishaan-jaff in #3212
• [UI-Polish] Cleanup Inputing Key Name, Team Name, User Email by @ishaan-jaff in #3218
• Update langsmith_integration.md by @bllchmbrs in #3205
• Added openrouter/meta-llama/llama-3-70b-instruct context and cost metrics by @paul-gauthier in #3223
• [UI-Fix] Show all teams on Admin UI by @ishaan-jaff in #3219
• fix(prometheus.py): add user tracking to prometheus by @krrishdholakia in #3224
• [Bug-Fix] Alerting - don't send hanging request alert on failed request by @ishaan-jaff in #3226
• improve(vertex_ai.py): Switch to simpler dict type for supporting JSON mode by @Manouchehri in #3211
• (Vertex AI) - Add frequency_penalty and presence_penalty support by @Manouchehri in #3214
• [Fix] Non-Admin SSO Login by @ishaan-jaff in #3228
• (utils.py) - Fix response_format typo for Groq by @Manouchehri in #3231
• fix(router.py) handle initial model list being empty by @krrishdholakia in #3242
• [Fix] Proxy: updating router settings from UI by @ishaan-jaff in #3245
• [Fix] Linking Langfuse Projects to Slack Alerts by @ishaan-jaff in #3244

New Contributors

• @jmandel made their first contribution in #3174
• @merefield made their first contribution in #3189
• @rick-github made their first contribution in #3170
• @n1lanjan made their first contribution in #3118
• @elisalimli made their first contribution in #3115
• @bllchmbrs made their first contribution in #3205
• @paul-gauthier made their first contribution in #3223

Full Changelog: v1.35.15-stable...v1.35.20.dev2

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	42	45.91843134549463	1.5564000119906491	0.0	466	0	35.09323800000175	641.0140170000318
/health/liveliness	Passed ✅	26	28.603328343261236	15.58737951922824	0.0	4667	0	23.41880000000174	1197.4679479999963
/health/readiness	Passed ✅	26	28.542194085641007	15.56066020571767	0.006679828377642271	4659	2	23.53846099998691	1258.7153820000196
Aggregated	Passed ✅	26	29.39826436172399	32.70443973693656	0.006679828377642271	9792	2	23.41880000000174	1258.7153820000196