Implement round robin load balancing in model pools #44
Comments
roma-glushko
added a commit
that referenced
this issue
Jan 13, 2024
roma-glushko
added a commit
that referenced
this issue
Jan 13, 2024
roma-glushko
added a commit
that referenced
this issue
Jan 14, 2024
…& created interfaces for Model & LanguageModels
roma-glushko
added a commit
that referenced
this issue
Jan 14, 2024
roma-glushko
added a commit
that referenced
this issue
Jan 14, 2024
roma-glushko
added a commit
that referenced
this issue
Jan 14, 2024
- Added a new round robin routing strategy - Build routing strategy based on provided config - Added Model & LanguageModel interfaces and refactored corresponding code to use them - Covered the priority routing by tests - Fixed issues in the priority routing as it was acting like round robin
Merged
roma-glushko
added a commit
that referenced
this issue
Jan 21, 2024
- ✨ [providers] Support for OpenAI Chat API #3 (@mkrueger12 ) - ✨ [API] #54 Unified Chat API (@mkrueger12 ) - ✨ [providers] Support for Cohere Chat API #5 (@mkrueger12 ) - ✨ [providers] Support for Azure OpenAI Chat API #4 (@mkrueger12 ) - ✨ [providers] Support for OctoML Chat API #58 (@mkrueger12 ) - ✨ [routing] The Routing Mechanism, Adaptive Health Tracking, and Fallbacks #42 #43 #51 (@roma-glushko) - ✨ [routing] Support for round robin routing strategy #44 (@roma-glushko) - ✨ [routing] Support for the least latency routing strategy #46 (@roma-glushko) - ✨ [routing] Support for weighted round robin routing strategy #45 (@roma-glushko) - ✨ [providers] Support for Anthropic Chat API #60 (@mkrueger12 ) - ✨ [docs] OpenAPI specifications #22 (@roma-glushko ) - 🔧 [chores] Inited the project #6 (@roma-glushko) - 🔊 [telemetry] Inited logging #14 (@roma-glushko) - 🔧 [chores] Inited Glide's CLI #12 (@roma-glushko) - 👷 [chores] Setup CI workflows #8 (@roma-glushko) - ⚙️ [config] Inited configs #11 (@roma-glushko) - 🔧 [chores] Automatic coverage reports #39 (@roma-glushko) - 👷 [build] Setup release workflows #9 (@roma-glushko)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: