You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
we want dynamically show available models from the llm server instead of always have to hard-coded a static list.
but current it is a hard-requirement that the query model requested has to be in the list of models set in road-core configuration. and upon every chat query it validate if the requested model is defined in the list
And if there are some model specific configs in the backend that changes system prompts or other things then we need to update it so that the model coming in the request can be used instead of the default one.
Describe alternatives you've considered
alternative will be removing the static model setting in configuration and also all checks. but this requires more discussions as people have different opinion on it
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
we want dynamically show available models from the llm server instead of always have to hard-coded a static list.
but current it is a hard-requirement that the query model requested has to be in the list of models set in road-core configuration. and upon every chat query it validate if the requested model is defined in the list
service/ols/src/llms/llm_loader.py
Lines 46 to 50 in 3ea924f
see detailed description and discussion: https://redhat-internal.slack.com/archives/C07MC7G9T8A/p1732570149978479
Describe the solution you'd like
optionally disable the model check.
And if there are some model specific configs in the backend that changes system prompts or other things then we need to update it so that the model coming in the request can be used instead of the default one.
Describe alternatives you've considered
alternative will be removing the static model setting in configuration and also all checks. but this requires more discussions as people have different opinion on it
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: