You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This allows setting the model path and target GPU device but prevents many necessary parameters from being passed to the models. For example, many of the new LLMs being released require the trust_remote_code parameter to be set. Running 4 and 8 bit quantization is another example.
The current workaround is to create the models via the auto model classes and pass those to the pipelines. This works in Python but not with application YAML.
This issue will do the following:
Add kwargs parameters to all constructors and pass those parameters to the underlying pipeline
Resolve arguments from strings as necessary (for example torch_dtype)
The text was updated successfully, but these errors were encountered:
Currently, all Hugging Face based pipelines have the following constructor.
This allows setting the model path and target GPU device but prevents many necessary parameters from being passed to the models. For example, many of the new LLMs being released require the
trust_remote_code
parameter to be set. Running 4 and 8 bit quantization is another example.The current workaround is to create the models via the auto model classes and pass those to the pipelines. This works in Python but not with application YAML.
This issue will do the following:
The text was updated successfully, but these errors were encountered: