-
Notifications
You must be signed in to change notification settings - Fork 692
Closed
Labels
dynamo-llmRelates to dynamo-llm componentRelates to dynamo-llm componentenhancementNew feature or requestNew feature or request
Description
Feature request
A worker should be able to 1. register itself under multiple names, and 2. the used name should be passed on to the worker. It will use the name to decide which LoRA to use.
Thoughts on 1:
register_llmcould accept either a string or an array of string- In Rust we put the handler in the map multiple times. The handler will probably be in an
Arc
Thoughts on 2:
- If we don't do pre-processing the worker should already get the model name, we pass on the request.
- If we do pre-processing, check that the model name ends up in the JSON that we send the model, with the token ids.
Metadata
Metadata
Assignees
Labels
dynamo-llmRelates to dynamo-llm componentRelates to dynamo-llm componentenhancementNew feature or requestNew feature or request