You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Let's add a configuration knob in the wsproxy.toml to customize the load balancing algorithm for App Proxy.
I think we may introduce per-model-service load balancing customization in the future, so let's name the global configuration as "default routing algorithm" to signify that it's a global fallback.
The prior implementation will be named to "weighted-uniform-random".
The new implementation (#3198) will be named to "shuffled-weighted-round-robin".
This issue also targets to add explicit tests to the routing algorithms.
The text was updated successfully, but these errors were encountered:
Let's add a configuration knob in the
wsproxy.toml
to customize the load balancing algorithm for App Proxy.I think we may introduce per-model-service load balancing customization in the future, so let's name the global configuration as "default routing algorithm" to signify that it's a global fallback.
The prior implementation will be named to "weighted-uniform-random".
The new implementation (#3198) will be named to "shuffled-weighted-round-robin".
This issue also targets to add explicit tests to the routing algorithms.
The text was updated successfully, but these errors were encountered: