Support for VLLM models? SmolVLM, Florence-2, etc

[cchadowitz]

I reached out on Gitter but I'm not sure if that's still actively used:

With all the LLM and VLM models announced and released recently, are there thoughts or plans around supporting those types of models in DeepDetect? I'm specifically most interested in multimodal models like the new HuggingFace SmolVLM Instruct models and Microsoft Florence-2 vision models, both based on the HuggingFace Transformers library.
SmolVLM is available as a variety of ONNX models, while Florence-2 is available as pytorch .bin, both with their own set of various configs for the model, tokenizers, preprocessors, etc, so I'm not sure how much (or which) is viable for use in DeepDetect.

[beniz]

Hi @cchadowitz we have another software stack for LLMs and VLLMs (including smolvlm), it's not been open sourced yet. If you'd like to have them inside DD, please PM us via email to see what can be done.

Sorry about the gitter, we've been busy elsewhere.

[cchadowitz]

No problem re: gitter, just wasn't sure if it was the right place still :)

I'll reach out about the LLMs/VLLMs stuff over email, thanks!