Compile bug: How to compile llama.cpp with Vulkan for android device

[liuxiaotiao]

Git commit

902368a

Operating systems

Linux

GGML backends

Vulkan

Problem description & steps to reproduce

I tried to compile llama.cpp(b4644) using NDK 27 and Vulkan-header(v1.4.307) and encountered the following compilation issues.

First Bad Commit

No response

Compile command

cmake \
 -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
 -DANDROID_ABI=arm64-v8a \
 -DANDROID_PLATFORM=latest \
 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod \
 -DGGML_VULKAN=ON \
 -DGGML_VULKAN_CHECK_RESULTS=OFF \
 -DGGML_VULKAN_DEBUG=ON \
 -DGGML_VULKAN_MEMORY_DEBUG=ON \
 -DGGML_VULKAN_SHADER_DEBUG_INFO=ON \
 -DGGML_VULKAN_PERF=OFF \
 -DGGML_VULKAN_VALIDATE=OFF \
 -DGGML_VULKAN_RUN_TESTS=OFF \
 -DVK_USE_PLATFORM_ANDROID_KHR=ON \
 -DVulkan_INCLUDE_DIR=$Vulkan-Headers/include 
 -DGGML_CCACHE=OFF \
 -B build-android

Relevant log output

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1378:2: warning: extra ';' outside of a function is incompatible with C++98 [-Wc++98-compat-extra-semi]
 1378 | };
      |  ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
 1607 |         CREATE_FA(GGML_TYPE_F16, f16)
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
 1590 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:46:1: note: expanded from here
   46 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
 1590 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:48:1: note: expanded from here
   48 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:51:1: note: expanded from here
   51 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:53:1: note: expanded from here
   53 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
 1592 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:56:1: note: expanded from here
   56 | flash_attn_f32_f16_f16_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:214: note: expanded from macro 'CREATE_FA2'
 1592 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:58:1: note: expanded from here
   58 | flash_attn_f32_f16_f16_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:163: note: expanded from macro 'CREATE_FA2'
 1593 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:61:1: note: expanded from here
   61 | flash_attn_f32_f16_f16_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:214: note: expanded from macro 'CREATE_FA2'
 1593 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:63:1: note: expanded from here
   63 | flash_attn_f32_f16_f16_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:163: note: expanded from macro 'CREATE_FA2'
 1594 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:66:1: note: expanded from here
   66 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:214: note: expanded from macro 'CREATE_FA2'
 1594 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:68:1: note: expanded from here
   68 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:163: note: expanded from macro 'CREATE_FA2'
 1595 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:71:1: note: expanded from here
   71 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:214: note: expanded from macro 'CREATE_FA2'
 1595 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:73:1: note: expanded from here
   73 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:163: note: expanded from macro 'CREATE_FA2'
 1596 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:76:1: note: expanded from here
   76 | flash_attn_f32_f16_f16_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:214: note: expanded from macro 'CREATE_FA2'
 1596 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:78:1: note: expanded from here
   78 | flash_attn_f32_f16_f16_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:163: note: expanded from macro 'CREATE_FA2'
 1597 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:81:1: note: expanded from here
   81 | flash_attn_f32_f16_f16_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:214: note: expanded from macro 'CREATE_FA2'
 1597 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:83:1: note: expanded from here
   83 | flash_attn_f32_f16_f16_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
 1590 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:102:1: note: expanded from here
  102 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
 1590 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:104:1: note: expanded from here
  104 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:107:1: note: expanded from here
  107 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
gmake[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/build.make:160: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [CMakeFiles/Makefile2:1795: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

[barzoj]

the same issue on my end, building against Vulkan-Headers-1.3.302

[jeffbolznv]

Please attach the full log for a clean build. This looks like another "weird combination of tooling installed" issue.

[barzoj]

@jeffbolznv please take a look.

echo $ANDROID_NDK
/build/Android/Sdk/ndk/28.0.12674087/

b4651

git status
HEAD detached at b4651
nothing to commit, working tree clean

cmake   -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake   -DANDROID_ABI=arm64-v8a   -DANDROID_PLATFORM=android-33   -DCMAKE_C_FLAGS="-march=armv8.7a"   -DCMAKE_CXX_FLAGS="-march=armv8.7a"   -DGGML_VULKAN=ON -DVulkan_INCLUDE_DIR=$VULKAN/include  -B build-android 
-- The C compiler identification is Clang 19.0.0
-- The CXX compiler identification is Clang 19.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /build/Android/Sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /build/Android/Sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.48.1") 
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp=libomp  
-- Found OpenMP_CXX: -fopenmp=libomp  
-- Found OpenMP: TRUE   
-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- Adding CPU backend variant ggml-cpu:  
-- Found Vulkan: /build/Android/Sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/33/libvulkan.so (found version "1.3.302") found components: glslc missing components: glslangValidator
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 supported by glslc
-- Host compiler: /usr/bin/gcc /usr/bin/g++
-- vulkan-shaders-gen toolchain file: /build/prj/test/llama.cpp/build-android/host-toolchain.cmake
-- Including Vulkan backend
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Configuring done (1.3s)
-- Generating done (0.1s)
-- Build files have been written to: /build/prj/test/llama.cpp/build-android

cmake --build build-android/ --config Release
[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[  2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[  2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[  3%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[  3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[  4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[  4%] Linking CXX shared library ../../bin/libggml-base.so
[  4%] Built target ggml-base
[  5%] Creating directories for 'vulkan-shaders-gen'
[  5%] No download step for 'vulkan-shaders-gen'
[  6%] No update step for 'vulkan-shaders-gen'
[  6%] No patch step for 'vulkan-shaders-gen'
[  7%] Performing configure step for 'vulkan-shaders-gen'
CMake Warning (dev) in CMakeLists.txt:
  No project() command is present.  The top-level CMakeLists.txt file must
  contain a literal, direct call to the project() command.  Add a line of
  code such as

    project(ProjectName)

  near the top of the file, but after cmake_minimum_required().

  CMake is pretending there is a "project(Project)" command on the first
  line.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  cmake_minimum_required() should be called prior to this top-level project()
  call.  Please see the cmake-commands(7) manual for usage documentation of
  both commands.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- The C compiler identification is GNU 14.2.1
-- The CXX compiler identification is GNU 14.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
CMake Warning (dev) in CMakeLists.txt:
  No cmake_minimum_required command is present.  A line of code such as

    cmake_minimum_required(VERSION 3.27)

  should be added at the top of the file.  The version specified may be lower
  if you wish to support older CMake versions for this project.  For more
  information run "cmake --help-policy CMP0000".
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done (0.4s)
-- Generating done (0.0s)
-- Build files have been written to: /build/prj/test/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[  7%] Performing build step for 'vulkan-shaders-gen'
[ 50%] Building CXX object CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.o
[100%] Linking CXX executable /build/prj/test/llama.cpp/build-android/bin/vulkan-shaders-gen
[100%] Built target vulkan-shaders-gen
[  7%] Built target vulkan-shaders-gen-build
[  7%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "Release"
[  7%] Built target vulkan-shaders-gen-install
[  8%] Completed 'vulkan-shaders-gen'
[  8%] Built target vulkan-shaders-gen
[  8%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
[  8%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1379:2: warning: extra ';' outside of a function is incompatible with C++98 [-Wc++98-compat-extra-semi]
 1379 | };
      |  ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
 1608 |         CREATE_FA(GGML_TYPE_F16, f16)
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:212:1: note: expanded from here
  212 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:214:1: note: expanded from here
  214 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
 1592 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:217:1: note: expanded from here
  217 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:214: note: expanded from macro 'CREATE_FA2'
 1592 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:219:1: note: expanded from here
  219 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:163: note: expanded from macro 'CREATE_FA2'
 1593 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:222:1: note: expanded from here
  222 | flash_attn_f32_f16_f16_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:214: note: expanded from macro 'CREATE_FA2'
 1593 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:224:1: note: expanded from here
  224 | flash_attn_f32_f16_f16_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:163: note: expanded from macro 'CREATE_FA2'
 1594 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:227:1: note: expanded from here
  227 | flash_attn_f32_f16_f16_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:214: note: expanded from macro 'CREATE_FA2'
 1594 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:229:1: note: expanded from here
  229 | flash_attn_f32_f16_f16_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:163: note: expanded from macro 'CREATE_FA2'
 1595 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:232:1: note: expanded from here
  232 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:214: note: expanded from macro 'CREATE_FA2'
 1595 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:234:1: note: expanded from here
  234 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:163: note: expanded from macro 'CREATE_FA2'
 1596 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:237:1: note: expanded from here
  237 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:214: note: expanded from macro 'CREATE_FA2'
 1596 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:239:1: note: expanded from here
  239 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:163: note: expanded from macro 'CREATE_FA2'
 1597 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:242:1: note: expanded from here
  242 | flash_attn_f32_f16_f16_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:214: note: expanded from macro 'CREATE_FA2'
 1597 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:244:1: note: expanded from here
  244 | flash_attn_f32_f16_f16_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1598:163: note: expanded from macro 'CREATE_FA2'
 1598 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:247:1: note: expanded from here
  247 | flash_attn_f32_f16_f16_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1598:214: note: expanded from macro 'CREATE_FA2'
 1598 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:2:1: note: expanded from here
    2 | flash_attn_f32_f16_f16_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1602:9: note: expanded from macro 'CREATE_FA'
 1602 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:21:1: note: expanded from here
   21 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1602:9: note: expanded from macro 'CREATE_FA'
 1602 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:23:1: note: expanded from here
   23 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1602:9: note: expanded from macro 'CREATE_FA'
 1602 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
 1592 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:26:1: note: expanded from here
   26 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
gmake[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/build.make:161: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1795: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

[liuxiaotiao]

Please attach the full log for a clean build. This looks like another "weird combination of tooling installed" issue.

Hi, here are all the logs from when I try to compile the Vulkan and llama.cpp for Android.

~/test/llama.cpp-b4644$ cmake
-DCMAKE_TOOLCHAIN_FILE=/home/ubuntu/android-ndk-r27c/build/cmake/android.toolchain.cmake
-DANDROID_ABI=arm64-v8a
-DANDROID_PLATFORM=android-28
-DCMAKE_C_FLAGS="-march=armv8.7a"
-DCMAKE_CXX_FLAGS="-march=armv8.7a"
-DGGML_OPENMP=OFF
-DGGML_LLAMAFILE=OFF
-DGGML_VULKAN=1
-DVulkan_INCLUDE_DIR=/home/ubuntu/Vulkan-Headers/include
-DVulkan_LIBRARY=/home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/35/libvulkan.so
-B build-android
-- The C compiler identification is Clang 18.0.3
-- The CXX compiler identification is Clang 18.0.3
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.34.1")
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- Adding CPU backend variant ggml-cpu:
-- Found Vulkan: /home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/35/libvulkan.so
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 supported by glslc
-- Host compiler: /usr/bin/gcc /usr/bin/g++
-- vulkan-shaders-gen toolchain file: /home/ubuntu/test/llama.cpp-b4644/build-android/host-toolchain.cmake
-- Including Vulkan backend
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
CMake Warning at common/CMakeLists.txt:32 (message):
Git repository not found; to enable automatic generation of build info,
make sure Git is installed and the project is a Git repository.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android

~/test/llama.cpp-b4644$ cmake --build build-android --config Release -j8
[ 0%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o
[ 0%] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o
[ 1%] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o
[ 2%] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o
[ 3%] Creating directories for 'vulkan-shaders-gen'
[ 4%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[ 5%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[ 5%] Built target build_info
[ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[ 5%] No download step for 'vulkan-shaders-gen'
[ 6%] No update step for 'vulkan-shaders-gen'
[ 6%] Built target sha1
[ 7%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[ 7%] Built target sha256
[ 7%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[ 7%] No patch step for 'vulkan-shaders-gen'
[ 8%] Performing configure step for 'vulkan-shaders-gen'
CMake Warning (dev) in CMakeLists.txt:
No project() command is present. The top-level CMakeLists.txt file must
contain a literal, direct call to the project() command. Add a line of
code such as

project(ProjectName)

near the top of the file, but after cmake_minimum_required().

CMake is pretending there is a "project(Project)" command on the first
line.
This warning is for project developers. Use -Wno-dev to suppress it.

-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
[ 9%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as

cmake_minimum_required(VERSION 3.22)

should be added at the top of the file. The version specified may be lower
if you wish to support older CMake versions for this project. For more
information run "cmake --help-policy CMP0000".
This warning is for project developers. Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[ 9%] Performing build step for 'vulkan-shaders-gen'
gmake[3]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
[ 50%] Building CXX object CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.o
[ 9%] Built target xxhash
[100%] Linking CXX executable /home/ubuntu/test/llama.cpp-b4644/build-android/bin/vulkan-shaders-gen
[100%] Built target vulkan-shaders-gen
[ 9%] Built target vulkan-shaders-gen-build
[ 9%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "Release"
[ 9%] Built target vulkan-shaders-gen-install
[ 10%] Completed 'vulkan-shaders-gen'
[ 10%] Built target vulkan-shaders-gen
[ 10%] Linking CXX shared library ../../bin/libggml-base.so
[ 10%] Built target ggml-base
[ 11%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
[ 13%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[ 13%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[ 14%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o
[ 15%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
[ 15%] Linking CXX shared library ../../bin/libggml-cpu.so
[ 15%] Built target ggml-cpu
[ 16%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o
[ 16%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan-shaders.cpp.o
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1378:2: warning: extra ';' outside of a function is incompatible with C++98 [-Wc++98-compat-extra-semi]
1378 | };
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
1607 | CREATE_FA(GGML_TYPE_F16, f16)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
:44:1: note: expanded from here
44 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
:46:1: note: expanded from here
46 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "aligned_f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
:49:1: note: expanded from here
49 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "aligned_f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
:51:1: note: expanded from here
51 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
:54:1: note: expanded from here
54 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:214: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
:56:1: note: expanded from here
56 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:163: note: expanded from macro 'CREATE_FA2'
1593 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "aligned_f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
:59:1: note: expanded from here
59 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:214: note: expanded from macro 'CREATE_FA2'
1593 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "aligned_f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
:61:1: note: expanded from here
61 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:163: note: expanded from macro 'CREATE_FA2'
1594 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
:64:1: note: expanded from here
64 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:214: note: expanded from macro 'CREATE_FA2'
1594 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
:66:1: note: expanded from here
66 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:163: note: expanded from macro 'CREATE_FA2'
1595 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
:69:1: note: expanded from here
69 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:214: note: expanded from macro 'CREATE_FA2'
1595 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
:71:1: note: expanded from here
71 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:163: note: expanded from macro 'CREATE_FA2'
1596 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
:74:1: note: expanded from here
74 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:214: note: expanded from macro 'CREATE_FA2'
1596 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
:76:1: note: expanded from here
76 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:163: note: expanded from macro 'CREATE_FA2'
1597 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
:79:1: note: expanded from here
79 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:214: note: expanded from macro 'CREATE_FA2'
1597 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
:81:1: note: expanded from here
81 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 80)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
💯1: note: expanded from here
100 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 80)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
:102:1: note: expanded from here
102 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 80)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "aligned_f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
:105:1: note: expanded from here
105 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
gmake[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/build.make:160: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [CMakeFiles/Makefile2:1795: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

[jeffbolznv]

OK, so cmake detects coopmat2 support, but those shader are either not being generated at all, or the generation is silently crashing (seems less likely). Please check whether GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is defined when vulkan-shaders-gen.cpp is compiled. I'm thinking this could be an issue with cross compiles not picking up the cmake state.

[liuxiaotiao]

OK, so cmake detects coopmat2 support, but those shader are either not being generated at all, or the generation is silently crashing (seems less likely). Please check whether GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is defined when vulkan-shaders-gen.cpp is compiled. I'm thinking this could be an issue with cross compiles not picking up the cmake state.

Hi, GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT was not defined when compiling vulkan-shaders-gen.cpp. However, when I add
#ifdef GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT
#pragma message("GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is defined")
#else
#pragma message("GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is NOT defined")
#define GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT
#endif
at the beginning of vulkan-shaders-gen.cpp, it still has the same compilation error. Can you help check again?

[jeffbolznv]

at the beginning of vulkan-shaders-gen.cpp, it still has the same compilation error. Can you help check again?

That's surprising. Was it a clean build? I don't fully trust the dependencies (they're manually hacked together in _ggml_vk_shader_deps).

GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT gets set in ggml-vulkan/CMakeLists.txt and in normal builds that setting gets inherited into the vulkan-shaders subproject. It sounds like this cross-compile may not be getting that behavior.

Either that (or in addition to that!) the cross-compile might be picking up a different glslc that does not support coopmat2 (seems very possible if the ndk hasn't updated). Is it possible to check what version of glslc is in the NDK and also try using it to compile -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp

[barzoj]

I updated the nkd to the latest version

/build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp
vulkan-shaders/test_coopmat_support.comp:3: error: '#extension' : extension not supported: GL_KHR_cooperative_matrix
1 error generated.

hardcoding glslc executable to the one provided by ndk solves the issue.

diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
index 07403108..4e7271ff 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
@@ -1,5 +1,5 @@
 find_package (Threads REQUIRED)
-find_program(GLSLC_EXECUTABLE glslc)
+find_program(GLSLC_EXECUTABLE /build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc)
 if(NOT GLSLC_EXECUTABLE)
     message(FATAL_ERROR "glslc not found.")
 endif()

[liuxiaotiao]

I updated the nkd to the latest version

/build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp
vulkan-shaders/test_coopmat_support.comp:3: error: '#extension' : extension not supported: GL_KHR_cooperative_matrix
1 error generated.

hardcoding glslc executable to the one provided by ndk solves the issue.

diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
index 07403108..4e7271ff 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
@@ -1,5 +1,5 @@
 find_package (Threads REQUIRED)
-find_program(GLSLC_EXECUTABLE glslc)
+find_program(GLSLC_EXECUTABLE /build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc)
 if(NOT GLSLC_EXECUTABLE)
     message(FATAL_ERROR "glslc not found.")
 endif()

Hi, which version of llama.cpp did you compile, when I use this way and the latest NDK, the problem still exists.

[liuxiaotiao]

at the beginning of vulkan-shaders-gen.cpp, it still has the same compilation error. Can you help check again?

That's surprising. Was it a clean build? I don't fully trust the dependencies (they're manually hacked together in _ggml_vk_shader_deps).

GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT gets set in ggml-vulkan/CMakeLists.txt and in normal builds that setting gets inherited into the vulkan-shaders subproject. It sounds like this cross-compile may not be getting that behavior.

Either that (or in addition to that!) the cross-compile might be picking up a different glslc that does not support coopmat2 (seems very possible if the ndk hasn't updated). Is it possible to check what version of glslc is in the NDK and also try using it to compile -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp

Hi, I use clean build every time. When downgrading vulkan from 1.4 to 1.3. The compilation log is as follows:
cmake --build build-android/ --target clean
~/test/llama.cpp-b4644$ cmake -DCMAKE_TOOLCHAIN_FILE=/home/ubuntu/android-ndk-r27c/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-28 -DCMAKE_C_FLAGS="-march=armv8.7a" -DCMAKE_CXX_FLAGS="-march=armv8.7a" -DGGML_OPENMP=OFF -DGGML_LLAMAFILE=OFF -DGGML_VULKAN=1 -DVulkan_INCLUDE_DIR=/home/ubuntu/test/Vulkan-Headers/include -DVulkan_LIBRARY=/home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/35/libvulkan.so -B build-android
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- ARM detected
-- Adding CPU backend variant ggml-cpu:
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 not supported by glslc
-- Host compiler: /usr/bin/gcc /usr/bin/g++
-- vulkan-shaders-gen toolchain file: /home/ubuntu/test/llama.cpp-b4644/build-android/host-toolchain.cmake
-- Including Vulkan backend
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
CMake Warning at common/CMakeLists.txt:32 (message):
Git repository not found; to enable automatic generation of build info,
make sure Git is installed and the project is a Git repository.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android
~/test/llama.cpp-b4644$ cmake --build build-android --config Release -j8
[ 0%] Generating build details from Git
[ 1%] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o
[ 2%] Creating directories for 'vulkan-shaders-gen'
[ 3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
-- Found Git: /usr/bin/git (found version "2.34.1")
[ 4%] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o
[ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[ 5%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
fatal: not a git repository (or any of the parent directories): .git
[ 5%] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o
fatal: not a git repository (or any of the parent directories): .git
[ 5%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o
[ 5%] No download step for 'vulkan-shaders-gen'
[ 5%] Built target build_info
[ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[ 6%] No update step for 'vulkan-shaders-gen'
[ 6%] No patch step for 'vulkan-shaders-gen'
[ 6%] Built target sha1
[ 7%] Performing configure step for 'vulkan-shaders-gen'
[ 8%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
CMake Warning (dev) in CMakeLists.txt:
No project() command is present. The top-level CMakeLists.txt file must
contain a literal, direct call to the project() command. Add a line of
code such as

project(ProjectName)

near the top of the file, but after cmake_minimum_required().

CMake is pretending there is a "project(Project)" command on the first
line.
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as

cmake_minimum_required(VERSION 3.22)

should be added at the top of the file. The version specified may be lower
if you wish to support older CMake versions for this project. For more
information run "cmake --help-policy CMP0000".
This warning is for project developers. Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[ 8%] Built target sha256
[ 8%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[ 8%] Performing build step for 'vulkan-shaders-gen'
gmake[3]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
Consolidate compiler generated dependencies of target vulkan-shaders-gen
[100%] Built target vulkan-shaders-gen
[ 8%] Built target vulkan-shaders-gen-build
[ 9%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[ 9%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "Release"
[ 9%] Built target vulkan-shaders-gen-install
[ 10%] Completed 'vulkan-shaders-gen'
[ 10%] Built target vulkan-shaders-gen
[ 10%] Built target xxhash
[ 10%] Linking CXX shared library ../../bin/libggml-base.so
[ 10%] Built target ggml-base
[ 11%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
[ 11%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[ 12%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[ 12%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[ 12%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o
[ 13%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o
[ 13%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o
[ 15%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
cannot compile matmul_f32_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_cm2.spv -DACC_TYPE=float -DB_TYPE=float16_t -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_f32_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=float16_t -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_cm2.spv -DACC_TYPE=float -DB_TYPE=float16_t -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q4_1_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q4_0_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q4_1_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=float16_t -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q5_0_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_0_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q5_1_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_1_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q4_0_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q5_0_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_0_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q5_1_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_1_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q8_0_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q8_0_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q8_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q8_0_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q8_0_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q8_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q2_k_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q2_k_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q2_K=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q2_k_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q2_k_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q2_K=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

[ 15%] Linking CXX shared library ../../bin/libggml-cpu.so
[ 15%] Built target ggml-cpu

And the compilation process will be blocked at [ 15%] Built target ggml-cpu.

[jeffbolznv]

/build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp
vulkan-shaders/test_coopmat_support.comp:3: error: '#extension' : extension not supported: GL_KHR_cooperative_matrix
1 error generated.

OK, sounds like the ndk glslc is very old (GL_KHR_cooperative_matrix was release in June 2023). And the issue seems to be related to vulkan-shaders/CMakeLists.txt picking up a different glslc than ggml-vulkan/CMakeLists.txt.

hardcoding glslc executable to the one provided by ndk solves the issue.

diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
index 07403108..4e7271ff 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
@@ -1,5 +1,5 @@
 find_package (Threads REQUIRED)
-find_program(GLSLC_EXECUTABLE glslc)
+find_program(GLSLC_EXECUTABLE /build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc)
 if(NOT GLSLC_EXECUTABLE)
     message(FATAL_ERROR "glslc not found.")
 endif()

This is surprising. If this glslc doesn't support KHR_coopmat, I'd expect those compiles to fail. Does this somehow affect the logic in ggml-vulkan/CMakeLists.txt? I'm not an expert on cmake.

But it seems like there are two glslcs in the system, the one from the ndk is old, the two CMakeLists.txt files are picking up different versions, and I suspect this is somehow related to the cross-compiling or overriding the toolchain in ggml-vulkan/CMakeLists.txt.

[jeffbolznv]

Hi, I use clean build every time. When downgrading vulkan from 1.4 to 1.3. The compilation log is as follows:
...
-- GL_NV_cooperative_matrix2 not supported by glslc

This downgrade picked up a different glslc that doesn't support coopmat2 and this avoids the problem. We need the two CMakeLists.txt files to agree on which build of glslc is being used.

(Or, we need to stop relying on whatever random stuff is installed and fetch and build shaderc directly. This just keeps coming up).