Issues running a few Models.

[michieal]

LocalAI version:

2.30.0

Environment, CPU architecture, OS, and Version:

6.11.0-9-generic # 9-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct 14 13:19:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
(Kubuntu 24,10)
AMD Radeon Video card, but I don't have ROCm installed. (Mainly, because this version doesn't work with the AMDGPU-dkms... and it breaks my graphics. KDE Plasma.)

Describe the bug

My main issue is that the phi models just don't work at all. (The Vision capable models. like phi-3.5-vision:vllm or openvino-phi3.)
Additionally, the image generation models are all "GPU" and since I am trying to do it with CPU, those don't work.

To Reproduce

Use the web interface. Install, say, phi-3.5-vision:vllm, and try to use it. Note the lack of the "processing" indicator.

Expected behavior

I expected some kind of output... even if it was a descriptive error message.

Logs

don't know how to do that?

Additional context

I was really looking forward to the use of images, both in analyzation and generation. My video card doesn't have enough vram to hold the model's data, so even if I do have rocm installed, it would then return an out of memory error. I'm more than willing to wait for the image generations, as it doesn't have the GPU assist.
I can say that poppy_porpoise-v1.4-l3-8b-iq-imatrix, however, works just fine at understanding images. So, at least I have a model with "vision"...

Additionally, I apologize if this is a duplicate issue, as a programmer I know what it's like to have duplicate issues on your project repo.
TIA!

[michieal]

So, I have been trying to recompile this with some help from Copilot... Things run smoothly until I get to whisper.cpp ... and then it goes sideways.
And, I lost a few hours of my life trying to do that, to try to get the backend "Diffusers" so that I can try to waste more hours of my life seeing if StableDiffusion will finally work.
that, and me are getting old. lmao.