Eval bug: Kimi-VL-A3B-Thinking-2506 not working correctly

[Som-anon]

Name and Version

~/work/llama.cpp/build/bin/llama-server --version
version: 6291 (44b1efa41)
built with cc (GCC) 15.2.1 20250813 for x86_64-pc-linux-gnu

Operating systems

Linux

GGML backends

CPU

Hardware

i5-8250U

Models

Kimi-VL-A3B-Thinking

Problem description & steps to reproduce

python convert_hf_to_gguf.py --outfile Kimi-VL-A3B-Thinking-2506-Q8.gguf --outtype q8_0 modeldir
python convert_hf_to_gguf.py --outfile mmproj-model-f16.gguf --outtype f16 modeldir --mmproj

To generate the ggufs. Then I run it with

      --temp 0.8
      --top-p 1
      --ctx-size 102400
      --jinja
      --no-mmap

Uploaded a korean text. Prompt

Transcribe and then translate the text

Gradio example with same temp and top-p thinks in chinese and transcribes it more or less correct.

llama-server thinks in english and fucks the transcription up completely

First Bad Commit

No response

Relevant log output

./build/bin/llama-server
      --model ~/models/Kimi-VL/Kimi-VL-A3B-Thinking-2506-Q4_K_M.gguf
      --temp 0.8
      --top-p 1
      --ctx-size 102400
      --mmproj ~/models/Kimi-VL/mmproj-Kimi-VL-A3B-Thinking-2506-f16.gguf
      --jinja
      --no-mmap
main: binding port with default address family
main: HTTP server is listening, hostname: 127.0.0.1, port: 10003, http threads: 7
main: loading model
srv    load_model: loading model '/home/anon/models/Kimi-VL2/Kimi-VL-A3B-Thinking-2506-Q8.gguf'
llama_model_loader: loaded meta data with 44 key-value pairs and 430 tensors from /home/anon/models/Kimi-VL2/Kimi-VL-A3B-Thinking-2506-Q8.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = deepseek2
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Modeldir
llama_model_loader: - kv   3:                         general.size_label str              = 64x1.8B
llama_model_loader: - kv   4:                            general.license str              = mit
llama_model_loader: - kv   5:                   general.base_model.count u32              = 1
llama_model_loader: - kv   6:                  general.base_model.0.name str              = Kimi VL A3B Instruct
llama_model_loader: - kv   7:          general.base_model.0.organization str              = Moonshotai
llama_model_loader: - kv   8:              general.base_model.0.repo_url str              = https://huggingface.co/moonshotai/Kim...
llama_model_loader: - kv   9:                               general.tags arr[str,1]       = ["image-text-to-text"]
llama_model_loader: - kv  10:                      deepseek2.block_count u32              = 27
llama_model_loader: - kv  11:                   deepseek2.context_length u32              = 131072
llama_model_loader: - kv  12:                 deepseek2.embedding_length u32              = 2048
llama_model_loader: - kv  13:              deepseek2.feed_forward_length u32              = 11264
llama_model_loader: - kv  14:             deepseek2.attention.head_count u32              = 16
llama_model_loader: - kv  15:          deepseek2.attention.head_count_kv u32              = 1
llama_model_loader: - kv  16:                   deepseek2.rope.freq_base f32              = 800000.000000
llama_model_loader: - kv  17: deepseek2.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  18:                deepseek2.expert_used_count u32              = 6
llama_model_loader: - kv  19:                          general.file_type u32              = 7
llama_model_loader: - kv  20:        deepseek2.leading_dense_block_count u32              = 1
llama_model_loader: - kv  21:                       deepseek2.vocab_size u32              = 163840
llama_model_loader: - kv  22:           deepseek2.attention.kv_lora_rank u32              = 512
llama_model_loader: - kv  23:             deepseek2.attention.key_length u32              = 576
llama_model_loader: - kv  24:           deepseek2.attention.value_length u32              = 512
llama_model_loader: - kv  25:         deepseek2.attention.key_length_mla u32              = 192
llama_model_loader: - kv  26:       deepseek2.attention.value_length_mla u32              = 128
llama_model_loader: - kv  27:       deepseek2.expert_feed_forward_length u32              = 1408
llama_model_loader: - kv  28:                     deepseek2.expert_count u32              = 64
llama_model_loader: - kv  29:              deepseek2.expert_shared_count u32              = 2
llama_model_loader: - kv  30:             deepseek2.expert_weights_scale f32              = 2.446000
llama_model_loader: - kv  31:              deepseek2.expert_weights_norm bool             = true
llama_model_loader: - kv  32:               deepseek2.expert_gating_func u32              = 2
llama_model_loader: - kv  33:             deepseek2.rope.dimension_count u32              = 64
llama_model_loader: - kv  34:               general.quantization_version u32              = 2
llama_model_loader: - kv  35:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  36:                         tokenizer.ggml.pre str              = kimi-k2
srv  log_server_r: request: GET /health 127.0.0.1 503
llama_model_loader: - kv  37:                      tokenizer.ggml.tokens arr[str,163840]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  38:                  tokenizer.ggml.token_type arr[i32,163840]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  39:                      tokenizer.ggml.merges arr[str,163328]  = ["Ġ Ġ", "ĠĠ ĠĠ", "Ġ t", "i n",...
llama_model_loader: - kv  40:                tokenizer.ggml.bos_token_id u32              = 163584
llama_model_loader: - kv  41:                tokenizer.ggml.eos_token_id u32              = 163585
llama_model_loader: - kv  42:            tokenizer.ggml.padding_token_id u32              = 163839
llama_model_loader: - kv  43:                    tokenizer.chat_template str              = {%- for message in messages -%}{%- if...
llama_model_loader: - type  f32:  134 tensors
llama_model_loader: - type q8_0:  296 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q8_0
print_info: file size   = 15.80 GiB (8.51 BPW) 
load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
load: printing all EOG tokens:
load:   - 163585 ('[EOS]')
load:   - 163586 ('<|im_end|>')
load: special tokens cache size = 256
load: token to piece cache size = 1.0607 MB
print_info: arch             = deepseek2
print_info: vocab_only       = 0
print_info: n_ctx_train      = 131072
print_info: n_embd           = 2048
print_info: n_layer          = 27
print_info: n_head           = 16
print_info: n_head_kv        = 1
print_info: n_rot            = 64
print_info: n_swa            = 0
print_info: is_swa_any       = 0
print_info: n_embd_head_k    = 576
print_info: n_embd_head_v    = 512
print_info: n_gqa            = 16
print_info: n_embd_k_gqa     = 576
print_info: n_embd_v_gqa     = 512
print_info: f_norm_eps       = 0.0e+00
print_info: f_norm_rms_eps   = 1.0e-05
print_info: f_clamp_kqv      = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale    = 0.0e+00
print_info: f_attn_scale     = 0.0e+00
print_info: n_ff             = 11264
print_info: n_expert         = 64
print_info: n_expert_used    = 6
print_info: causal attn      = 1
print_info: pooling type     = 0
print_info: rope type        = 0
print_info: rope scaling     = linear
print_info: freq_base_train  = 800000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn  = 131072
print_info: rope_finetuned   = unknown
print_info: model type       = 16B
print_info: model params     = 15.96 B
print_info: general.name     = Modeldir
print_info: n_layer_dense_lead   = 1
print_info: n_lora_q             = 0
print_info: n_lora_kv            = 512
print_info: n_embd_head_k_mla    = 192
print_info: n_embd_head_v_mla    = 128
print_info: n_ff_exp             = 1408
print_info: n_expert_shared      = 2
print_info: expert_weights_scale = 2.4
print_info: expert_weights_norm  = 1
print_info: expert_gating_func   = sigmoid
print_info: rope_yarn_log_mul    = 0.0000
print_info: vocab type       = BPE
print_info: n_vocab          = 163840
print_info: n_merges         = 163328
print_info: BOS token        = 163584 '[BOS]'
print_info: EOS token        = 163585 '[EOS]'
print_info: EOT token        = 163586 '<|im_end|>'
print_info: PAD token        = 163839 '[UNK]'
print_info: LF token         = 198 'Ċ'
print_info: EOG token        = 163585 '[EOS]'
print_info: EOG token        = 163586 '<|im_end|>'
print_info: max token length = 512
load_tensors: loading model tensors, this can take a while... (mmap = false)
load_tensors:          CPU model buffer size = 16181.95 MiB
...........................srv  log_server_r: request: GET /health 127.0.0.1 503
........................................srv  log_server_r: request: GET /health 127.0.0.1 503
......................
llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 102400
llama_context: n_ctx_per_seq = 102400
llama_context: n_batch       = 2048
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = 0
llama_context: kv_unified    = false
llama_context: freq_base     = 800000.0
llama_context: freq_scale    = 1
llama_context: n_ctx_per_seq (102400) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
llama_context:        CPU  output buffer size =     0.62 MiB
llama_kv_cache:        CPU KV buffer size =  5737.50 MiB
srv  log_server_r: request: GET /health 127.0.0.1 503
llama_kv_cache: size = 5737.50 MiB (102400 cells,  27 layers,  1/1 seqs), K (f16): 3037.50 MiB, V (f16): 2700.00 MiB
llama_context:        CPU compute buffer size =  3456.38 MiB
llama_context: graph nodes  = 1974
llama_context: graph splits = 378 (with bs=512), 1 (with bs=1)
common_init_from_params: added [EOS] logit bias = -inf
common_init_from_params: added <|im_end|> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 102400
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
clip_model_loader: model name:   Modeldir
clip_model_loader: description:  
clip_model_loader: GGUF version: 3
clip_model_loader: alignment:    32
clip_model_loader: n_tensors:    443
clip_model_loader: n_kv:         26

clip_model_loader: has vision encoder
clip_ctx: CLIP using CPU backend
load_hparams: projector:          kimivl
load_hparams: n_embd:             1152
load_hparams: n_head:             16
load_hparams: n_ff:               4304
load_hparams: n_layer:            27
load_hparams: ffn_op:             gelu
load_hparams: projection_dim:     2048

--- vision hparams ---
load_hparams: image_size:         896
load_hparams: patch_size:         14
load_hparams: has_llava_proj:     0
load_hparams: minicpmv_version:   0
load_hparams: proj_scale_factor:  2
load_hparams: n_wa_pattern:       0

load_hparams: model size:         863.41 MiB
load_hparams: metadata size:      0.16 MiB
alloc_compute_meta:        CPU compute buffer size =     2.04 MiB
srv    load_model: loaded multimodal model, '/home/anon/models/Kimi-VL2/mmproj-model-f16.gguf'
srv          init: initializing slots, n_slots = 1
slot         init: id  0 | task -1 | new slot n_ctx_slot = 102400
main: model loaded
main: chat template, chat_template: {%- for message in messages -%}{%- if loop.first and messages[0]['role'] != 'system' -%}{{'<|im_system|>system<|im_middle|>You are a helpful assistant<|im_end|>'}}{%- endif -%}{%- if message['role'] == 'system' -%}{{'<|im_system|>'}}{%- endif -%}{%- if message['role'] == 'user' -%}{{'<|im_user|>'}}{%- endif -%}{%- if message['role'] == 'assistant' -%}{{'<|im_assistant|>'}}{%- endif -%}{{- message['role'] -}}{{'<|im_middle|>'}}{%- if message['content'] is string -%}{{- message['content'] + '<|im_end|>' -}}{%- else -%}{%- for content in message['content'] -%}{%- if content['type'] == 'image' or 'image' in content or 'image_url' in content -%}{{'<|media_start|>image<|media_content|><|media_pad|><|media_end|>'}}{%- else -%}{{content['text']}}{%- endif -%}{%- endfor -%}{{'<|im_end|>'}}{%- endif -%}{%- endfor -%}{%- if add_generation_prompt -%}{{'<|im_assistant|>assistant<|im_middle|>'}}{%- endif -%}, example_format: '<|im_system|>system<|im_middle|>You are a helpful assistant<|im_end|><|im_user|>user<|im_middle|>Hello<|im_end|><|im_assistant|>assistant<|im_middle|>Hi there<|im_end|><|im_user|>user<|im_middle|>How are you?<|im_end|><|im_assistant|>assistant<|im_middle|>'
main: server is listening on http://127.0.0.1:10003 - starting the main loop
srv  update_slots: all slots are idle
srv  log_server_r: request: GET /health 127.0.0.1 200
srv  params_from_: Chat format: Content-only
slot launch_slot_: id  0 | task 0 | processing task
slot update_slots: id  0 | task 0 | new prompt, n_ctx_slot = 102400, n_keep = 0, n_prompt_tokens = 194
slot update_slots: id  0 | task 0 | kv cache rm [0, end)
slot update_slots: id  0 | task 0 | prompt processing progress, n_past = 22, n_tokens = 22, progress = 0.113402
slot update_slots: id  0 | task 0 | kv cache rm [22, end)
srv  process_chun: processing image...
srv  process_chun: image processed in 13707 ms
slot update_slots: id  0 | task 0 | prompt processing progress, n_past = 194, n_tokens = 4, progress = 1.000000
slot update_slots: id  0 | task 0 | prompt done, n_past = 194, n_tokens = 4
slot      release: id  0 | task 0 | stop processing: n_past = 2676, truncated = 0
slot print_timing: id  0 | task 0 | 
prompt eval time =   14974.21 ms /   194 tokens (   77.19 ms per token,    12.96 tokens per second)
       eval time =  355905.54 ms /  2483 tokens (  143.34 ms per token,     6.98 tokens per second)
      total time =  370879.76 ms /  2677 tokens
srv  update_slots: all slots are idle
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
srv  params_from_: Chat format: Hermes 2 Pro
slot launch_slot_: id  0 | task 276 | processing task
slot update_slots: id  0 | task 276 | new prompt, n_ctx_slot = 40960, n_keep = 0, n_prompt_tokens = 43
slot update_slots: id  0 | task 276 | need to evaluate at least 1 token for each active slot, n_past = 43, n_prompt_tokens = 43
slot update_slots: id  0 | task 276 | kv cache rm [42, end)
slot update_slots: id  0 | task 276 | prompt processing progress, n_past = 43, n_tokens = 1, progress = 0.023256
slot update_slots: id  0 | task 276 | prompt done, n_past = 43, n_tokens = 1
slot      release: id  0 | task 276 | stop processing: n_past = 221, truncated = 0
slot print_timing: id  0 | task 276 | 
prompt eval time =      58.24 ms /     1 tokens (   58.24 ms per token,    17.17 tokens per second)
       eval time =    5377.56 ms /   179 tokens (   30.04 ms per token,    33.29 tokens per second)
      total time =    5435.79 ms /   180 tokens
srv  update_slots: all slots are idle
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
srv  log_server_r: request: GET / 127.0.0.1 200
srv  log_server_r: request: GET /props 127.0.0.1 200
srv  params_from_: Chat format: Content-only
slot launch_slot_: id  0 | task 2485 | processing task
slot update_slots: id  0 | task 2485 | new prompt, n_ctx_slot = 102400, n_keep = 0, n_prompt_tokens = 193
slot update_slots: id  0 | task 2485 | kv cache rm [12, end)
srv  process_chun: processing image...
srv  process_chun: image processed in 9788 ms
slot update_slots: id  0 | task 2485 | prompt processing progress, n_past = 193, n_tokens = 13, progress = 0.937824
slot update_slots: id  0 | task 2485 | prompt done, n_past = 193, n_tokens = 13
slot      release: id  0 | task 2485 | stop processing: n_past = 1896, truncated = 0
slot print_timing: id  0 | task 2485 | 
prompt eval time =   10343.06 ms /   181 tokens (   57.14 ms per token,    17.50 tokens per second)
       eval time =  220225.81 ms /  1704 tokens (  129.24 ms per token,     7.74 tokens per second)
      total time =  230568.87 ms /  1885 tokens
srv  update_slots: all slots are idle
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200

[Som-anon]

Ok I got the problem worked out a bit more (and found a problem with internvl in the process). Attached my three test images and the output of Kimi-VL

Thinking: (I'm so sorry)

Got it, let's look at the image. The figure is a shape made of horizontal and vertical lines, forming a sort of layered or stacked structure. It's a geometric figure with multiple rectangles. Let's describe it step by step. 

First, identify the components: there are several horizontal lines (top and bottom) and vertical lines on the sides. The shape has a sort of "frame" with inner rectangles. Let's count the layers. From top to bottom, there are four horizontal segments, and the left and right sides are vertical lines connecting them. So it's a rectangle with notches or a stepped appearance? Wait, no, it's more like a vertical rectangle with multiple horizontal divisions. Wait, actually, the figure is a combination of rectangles where the left and right sides are made of vertical lines, and the top and bottom are horizontal lines, creating a shape that looks like a tall, narrow shape with multiple horizontal sections. 

Wait, let's think again. The image shows a figure composed of black lines on a white background. The figure has a vertical orientation. Let's break it down: 

    The left and right sides are vertical lines, forming the left and right edges.
    The top, middle, and bottom sections have horizontal lines connecting these vertical lines, creating multiple horizontal rectangles stacked vertically.
     

So the figure is a sort of "stack" of rectangles, with the left and right sides as the borders. The shape is a polygon with vertices at the endpoints of these lines. Let's list the vertices: 

Starting from the bottom left, go up: bottom left vertical line, then the first horizontal line to the right, then the next vertical line up, then the next horizontal line to the right, and so on, until the top horizontal line, then down the right vertical line back to the bottom left. 

Wait, no, actually, the figure is a single closed shape. Let's visualize: 

The figure has a left edge (vertical line), a right edge (vertical line), and top, middle, and bottom horizontal edges. So it's a rectangle that's been divided into three horizontal sections (top, middle, bottom), each connected by vertical lines on the left and right. Wait, no, the vertical lines are continuous from top to bottom. So the left side is a single vertical line, the right side is a single vertical line, and the top, middle, bottom are horizontal lines connecting them. So the shape is like a "taller" rectangle with three horizontal segments (top, middle, bottom) and two vertical segments (left, right). So the overall shape is a hexagon? Wait, no, let's count the sides: 

Left vertical (up), top horizontal (right), right vertical (up), next top horizontal (left), middle horizontal (left), middle vertical (up), next bottom horizontal (right), bottom vertical (up), left vertical (down)? Wait, no, maybe I'm overcomplicating. 

Alternatively, the figure is a combination of rectangles. Let's see: the outer rectangle has a height of, say, 3 units (top, middle, bottom) and width of 1 unit (left to right). Then, there are inner vertical lines dividing the left and right sides into sections? Wait, no, the image shows a shape where the left and right are solid lines, and between them, there are horizontal lines at top, middle, and bottom. So the shape is a vertical strip (left to right) with three horizontal rectangles stacked vertically (top, middle, bottom). So the figure is a 3x1 rectangle stack, with the left and right edges as the borders. 

Wait, the key is to describe what the figure is. Let's think of it as a geometric shape with the following properties: 

    It's a polygon with vertices at the following points (assuming a coordinate system where bottom left is (0,0)): 
        Bottom left: (0,0)
        Bottom middle: (1,1) [wait, no, the horizontal lines are at different y-levels. Wait, maybe the vertical dimension is divided into three parts: bottom, middle, top. Let's say the bottom horizontal line is at y=1, middle at y=2, top at y=3. The left vertical line is at x=0, right at x=1.
         

    So the vertices are: 

    (0,1) [bottom left], (1,1) [bottom right], (1,2) [middle right], (0,2) [middle left], (0,3) [top left], (1,3) [top right], back to (0,1). Wait, no, that would make a shape with three horizontal lines (bottom, middle, top) and two vertical lines (left, right), connecting them. So the figure is a hexagon? Wait, no, a hexagon has six sides. Let's count: 

    From (0,1) up to (0,3) – that's the left vertical edge. 

    Then right to (1,3) – top horizontal edge. 

    Then down to (1,2) – right vertical edge. 

    Then left to (0,2) – middle horizontal edge. 

    Then up to (0,1) – left vertical edge? No, that would close the shape. Wait, no, the shape is: 

    Left vertical (0,1) to (0,3), then right horizontal (0,3) to (1,3), then down vertical (1,3) to (1,2), then left horizontal (1,2) to (0,2), then up vertical (0,2) to (0,1). So that's a pentagon? Wait, no, that's five sides: left vertical, top horizontal, right vertical, middle horizontal, left vertical. Wait, no, the last segment is from (0,2) up to (0,1), which is the same as the first segment. So the shape has five sides: left vertical, top horizontal, right vertical, middle horizontal, and the lower left vertical. Wait, no, that's actually a rectangle with a notch? No, maybe I'm getting confused. 
     

Alternatively, the figure is a "stack" of three rectangles: bottom, middle, top, each with height 1, and width 1, aligned vertically. So the total height is 3, width 1. So the shape is a 3x1 rectangle composed of three 1x1 rectangles stacked vertically. But the lines are the borders of these rectangles. 

Wait, the user is asking "What do you see?" So I need to describe the figure. Let's try to summarize: 

The figure is a geometric shape composed of black lines on a white background. It has a vertical orientation with multiple horizontal and vertical line segments. The shape consists of a left vertical line, a right vertical line, and three horizontal lines connecting them at the top, middle, and bottom positions. This creates a layered or stacked appearance with three horizontal sections (top, middle, bottom) and two vertical borders (left and right). The overall structure is a polygon with vertices formed by the intersections of these lines, resulting in a shape that resembles a tall, narrow frame with internal horizontal divisions. 

Wait, but maybe it's simpler. Let's think of it as a rectangle divided into three equal horizontal parts by two vertical lines? No, the image doesn't show vertical lines in the middle; it shows horizontal lines at top, middle, bottom, and the left and right as vertical lines. So the shape is like a "staircase" but vertical, with three steps (top, middle, bottom) and the sides as the borders. 

Alternatively, the figure is a combination of rectangles where the left and right sides are continuous vertical lines, and between them, there are horizontal lines at three levels (top, middle, bottom). So the figure is a vertical strip (left to right) with three horizontal rectangles stacked vertically (top, middle, bottom). So the shape is a 3x1 rectangle made of three 1x1 rectangles. 

But to describe it accurately, let's note the key elements: 

    Vertical orientation.
    Two vertical boundaries (left and right).
    Three horizontal boundaries (top, middle, bottom).
    The horizontal boundaries connect the vertical ones, creating a shape that looks like a tall, narrow rectangle with three horizontal sections.
     

So the final description should be concise. Let's craft it: 

The figure is a geometric shape with a vertical orientation, composed of black lines on a white background. It features two vertical side lines and three horizontal lines connecting them at the top, middle, and bottom positions, forming a layered structure with three horizontal sections and two vertical borders. 
....

I'm stopping it here since any additional output is not helping.
Answer:

The figure is a vertically oriented geometric shape composed of black lines on a white background. It features two vertical side lines (left and right) and three horizontal lines connecting these sides at the top, middle, and bottom positions. This structure creates a layered, stacked appearance with three distinct horizontal sections and two vertical borders, forming a closed polygon. 

Thinking:

    So, let's look at the image. The image has three large, rounded shapes in red, green, and blue. Each shape is a stylized, almost soccer-ball-like pattern with segments. The red shape is on the left, green in the middle, blue on the right. They have a black outline and a white background. So I need to describe what's seen. 

    First, identify the colors: red, green, blue. Then the shapes: rounded, with a segmented (like a polygon but rounded) design. The arrangement is left, middle, right. So the description would be: The image displays three large, rounded, segmented shapes in red, green, and blue, arranged horizontally against a white background, each shape featuring a black outline and a pattern resembling a stylized, spherical segment design.  

Answer:

The image displays three large, rounded, segmented shapes in red, green, and blue, arranged horizontally against a white background. Each shape features a black outline and a pattern that resembles a stylized, spherical segment design.

Thinking:

So, let's look at the image. The user provided a simple graphic with dashed lines forming a circular or oval shape, maybe like a target or a frame. The background is a solid color. Now, I need to describe what I see.  

First, observe the elements: the dashed lines create an outline, the shape is centered, the background is a muted color. So the description should capture that. 

I'll say: "The image features a series of concentric dashed lines forming an oval or circular outline against a solid, muted gray background. The dashed lines create a layered frame-like shape, with each outline slightly offset to suggest depth or multiple layers.

Answer:

The image features a series of concentric dashed lines forming an oval - like outline against a solid, muted gray background. The dashed lines create a layered, frame - like shape, with each outline slightly offset to give a sense of depth or multiple layers.

So it seems that the images are rendered distorted (I'm experimenting a bit with SVG creation, but I'm not sure if I really get what the VLM "sees" or if it just bullshits me) for the square I got

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 200" width="100" height="200">
  <!-- Draw several horizontal lines -->
  <line x1="0" y1="50" x2="100" y2="50" stroke="gray" stroke-width="2" />
  <line x1="0" y1="75" x2="100" y2="75" stroke="gray" stroke-width="2" />
  <line x1="0" y1="100" x2="100" y2="100" stroke="gray" stroke-width="2" />
  <line x1="0" y1="125" x2="100" y2="125" stroke="gray" stroke-width="2" />
  <line x1="0" y1="150" x2="100" y2="150" stroke="gray" stroke-width="2" />
  <line x1="0" y1="175" x2="100" y2="175" stroke="gray" stroke-width="2" />
  <line x1="0" y1="200" x2="100" y2="200" stroke="gray" stroke-width="2" />
  <line x1="0" y1="225" x2="100" y2="225" stroke="gray" stroke-width="2" />
</svg>

I hoped that my colored circles could "show" squashing in any dimension, but alas...

[Hoernchen]

I have a similar issue with kimi vl, tho I did not try to debug this, but it clearly sees broken images, a bit like #15474 - that patch appears to fix qwen vl, so at least gemma and qwen vl are currently usable for vision.

[BVEsun]

I am new to this model and just want to test its OCR capabilities, but I found it performs very poorly with the GGUF downloaded from:
https://huggingface.co/ggml-org/Kimi-VL-A3B-Thinking-2506-GGUF

Although it said: #15458

It seems it didn't read the text correctly, do anyone have similar issue?

[fighter3005]

I tried KimiVL 2506 on llama.cpp also, and vision is beyond bad. It keeps seeing stairways where there are
radiator for heating and can't even detect prominent text in images. This is the only time I got it to detect any text at all with this image:

I wondered if it is the model, but it seems to be a llama.cpp issue then?

[Som-anon]

I wondered if it is the model, but it seems to be a llama.cpp issue then?

Indeed it is. You can try checking out the repo and use it with transformers if you want to try to compare results.

I think it's a simple problem in the clip implementation I tried some things and got marginally better results, but still not the same as the reference I was comparing against

[KMakrinsky]

When running the original weights through transformers library, I got good results in terms of describing various graphs and extracting data from them. However, when working with gguf versions (via LM-studio + I tried converting to f16 gguf myself), there is a catastrophic drop in image processing quality.

[foldl]

I tried KimiVL 2506 on llama.cpp also, and vision is beyond bad. It keeps seeing stairways where there are radiator for heating and can't even detect prominent text in images. This is the only time I got it to detect any text at all with this image:

I wondered if it is the model, but it seems to be a llama.cpp issue then?

[fighter3005]

@foldl How many times did you try this? If it is a recent fix, it is not in ollama yet. I did a quick test and it did get the text with the 3rd try.
I also tested it with a fresh llama.cpp with vulkan backend build, but it is also only sometimes figuring it out.
Sometimes I get "Tropic", sometimes "Tropicana" or "Tropic feel" or "Tropicel". That is pretty bad, considering how clear the text is.
I tested with Q8_0 and Q4_K_M.

I mean, it is kind of funny...

[foldl]

@fighter3005 I am using chatllm.cpp. Tested the instruct and thinking variants. Both gave the correct answers.