Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add configuration knobs to customize app proxy route selection #3210

Open
achimnol opened this issue Dec 5, 2024 · 0 comments · May be fixed by #3221
Open

Add configuration knobs to customize app proxy route selection #3210

achimnol opened this issue Dec 5, 2024 · 0 comments · May be fixed by #3221
Assignees
Labels
comp:appproxy Related to App Proxy component
Milestone

Comments

@achimnol
Copy link
Member

achimnol commented Dec 5, 2024

Let's add a configuration knob in the wsproxy.toml to customize the load balancing algorithm for App Proxy.

I think we may introduce per-model-service load balancing customization in the future, so let's name the global configuration as "default routing algorithm" to signify that it's a global fallback.

The prior implementation will be named to "weighted-uniform-random".
The new implementation (#3198) will be named to "shuffled-weighted-round-robin".

This issue also targets to add explicit tests to the routing algorithms.

@achimnol achimnol added the comp:appproxy Related to App Proxy component label Dec 5, 2024
@achimnol achimnol added this to the 24.09 milestone Dec 5, 2024
@achimnol achimnol self-assigned this Dec 5, 2024
@achimnol achimnol linked a pull request Dec 6, 2024 that will close this issue
2 tasks
@achimnol achimnol linked a pull request Dec 6, 2024 that will close this issue
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:appproxy Related to App Proxy component
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant