Is it possible to convert the shiny uform-gen model to gguf format?

[lin72h]

Hi Unum Team,

Firstly, I love what you are doing—pushing the software and hardware to the limit, creating a work of art.

I just found out that the new uform-gen models are based on Llama. I'm wondering, is it possible to convert the Safetenor model to gguf format and run it on the llama.cpp inference engine?

[ashvardanian]

Should be possible. The sneaky part is getting the ViT from UForm to work with llama.cpp. Do you know if anyone has previously done that, @lin72h ?

[lin72h]

The llava model already working: here's pr

and there's vit.cpp based on ggml tensor library

and @monatis have done awesome just for the llava integration and much more, he is also a usearch user.