Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFE] Option to disable model check #154

Open
yangcao77 opened this issue Nov 26, 2024 · 0 comments
Open

[RFE] Option to disable model check #154

yangcao77 opened this issue Nov 26, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@yangcao77
Copy link

yangcao77 commented Nov 26, 2024

Is your feature request related to a problem? Please describe.
we want dynamically show available models from the llm server instead of always have to hard-coded a static list.

but current it is a hard-requirement that the query model requested has to be in the list of models set in road-core configuration. and upon every chat query it validate if the requested model is defined in the list

if model not in provider_config.models:
raise ModelConfigMissingError(
f"Model '{model}' is not a valid model for provider '{provider}'. "
f"Valid models are: {list(provider_config.models.keys())}"
)

see detailed description and discussion: https://redhat-internal.slack.com/archives/C07MC7G9T8A/p1732570149978479

Describe the solution you'd like

optionally disable the model check.

And if there are some model specific configs in the backend that changes system prompts or other things then we need to update it so that the model coming in the request can be used instead of the default one.

Describe alternatives you've considered

alternative will be removing the static model setting in configuration and also all checks. but this requires more discussions as people have different opinion on it

Additional context
Add any other context or screenshots about the feature request here.

@yangcao77 yangcao77 added the enhancement New feature or request label Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant