Eval bug: Qwen3-VL-8B freezes on image processing tasks

[jhemmond]

Name and Version

${llamasvr} -m ${mpath}\Qwen3-VL-8B-Instruct-UD-Q4_K_XL.gguf --mmproj ${mpath}\Qwen3-VL-8B-mmproj-BF16.gguf --no-mmap --ctx-size 16000 --jinja --temp 0.7 --top-p 0.8 --top-k 20 --repeat-penalty 1.0 --presence-penalty 1.5

Version: b6948

Windows Vulkan precompiled binaries.

Operating systems

Windows

GGML backends

Vulkan

Hardware

Ryzen 890m

Models

Qwen3-VL-8B

Problem description & steps to reproduce

Last working version is roughly b6910 and earlier. I didn't test all versions to see where the breakdown happened.

Qwen3-VL-8B freezes as processing image..
Text responses work fine. Image uploads to the webui or Open-WebUI all result in a complete freeze, and sometimes windows locks down.

First Bad Commit

No response

Relevant log output

srv  params_from_: Chat format: Hermes 2 Pro

slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1

slot launch_slot_: id  3 | task 1 | processing task

slot update_slots: id  3 | task 1 | new prompt, n_ctx_slot = 16000, n_keep = 0, task.n_tokens = 2044

slot update_slots: id  3 | task 1 | n_tokens = 0, memory_seq_rm [0, end)

slot update_slots: id  3 | task 1 | prompt processing progress, n_tokens = 10, batch.n_tokens = 10, progress = 0.004892

slot update_slots: id  3 | task 1 | n_tokens = 10, memory_seq_rm [10, end)

srv  process_chun: processing image...

[wqerrewetw]

#16878
#16921
I think the slow speed is due to image preprocessing.

[jhemmond]

#16878 #16921 I think the slow speed is due to image preprocessing.

These sound like enhancements. The b6910 release works perfectly fine. Looks like something changed in the last 4 days that caused the freeze. The freeze takes minutes and the machine locks up, so it's not working at all.

[wqerrewetw]

These sound like enhancements. The b6910 release works perfectly fine. Looks like something changed in the last 4 days that caused the freeze. The freeze takes minutes and the machine locks up, so it's not working at all.

This will scale the image dimensions (usually upscaling), which increases the required processing time. If your device is less powerful, the total processing time can become significantly longer. You could try reducing the --image-min/max-tokens value.

[wqerrewetw]

You can Try --image-min-tokens 8 --image-max-tokens 1024 or --image-min-tokens 8 --image-max-tokens 512

[jhemmond]

That makes sense. I will try your suggestion once am off work.

[jhemmond]

I tried limiting the number of tokens, and it's working but still far slower. For ex, at 512 tokens it's 3 times slower than previous llama.cpp builds.

I saw today a change that allowed Qwen3-VL model to run full res up to 4096 tokens, up from 2048. So I ran an experiment:

• Updated llama.cpp to the latest binaries from 1 hr ago
• Set max tokens to 2048 --image-max-tokens 2048
• The performance tanked and the image process took about a minute

This proves that something else was change that tanked the performance, or maybe something in my parameters is throwing it off, but you saw above I don't have anything out of the ordinary.

If a build from 4-5 days ago processes an image at 2048 max tokens in under 5 seconds, but today's build at the same resolution takes ~60 seconds, there's another issue.

I tested with Gemma3-12B and it suffers the same issue but to a lesser extent. the mmproj file for Gemma3 is smaller than Qwen3-VL-8B mmproj file, so that can have something to do with it.

[jhemmond]

@wqerrewetw I found the PR that broke everything. It's under this build b6915

16878.

Also noticed that when I set the max tokens to 512, the recognition quality tanks, and 1024 is multiple times slower, something is not right. At least with the Vulkan backend.