illegal instruction and crash when run llama-bench (build on android device not cross platform compilation )on android

[DamonsJ]

I follow the instruction
https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#building-the-project-using-android-ndk
to build llama.cpp on android device in order to use openblas

here is the steps :

1. download termux and install :
   https://github.com/termux/termux-app/releases
   v0.118.0: termux-app_v0.118.0+github-debug_arm64-v8a.apk

2. change termux repo to USTC use termux-change-repo (because default repo is not available)

3. pkg install clang wget git cmake

4. apt install libopenblas

5. apt install ocl-icd opencl-headers opencl-clhpp clinfo

6. git clone https://github.com/ggerganov/llama.cpp

7. git clone https://github.com/CNugteren/CLBlast and compile clblast:
   cmake .
   make
   cp libclblast.so* $PREFIX/lib
   cp ./include/clblast.h ../llama.cpp

cp /data/data/com.termux/files/usr/include/openblas/cblas.h .
cp /data/data/com.termux/files/usr/include/openblas/openblas_config.h .

9. compile llama.cpp
   cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod ..
   make LLAMA_CLBLAST=1

export GGML_OPENCL_PLATFORM=0
export GGML_OPENCL_DEVICE=0
export LD_LIBRARY_PATH=/vendor/lib64:$LD_LIBRARY_PATH

11. run ./llama-bench -m ../models/qwen1_5-1_8b-chat-q2_k.gguf -p 0 -n 64,128,256 -t 4 -o json

it will give me error info : illegal instruction and then it crash

I had tried two android devices , samsung Note 20 Ultra and HUAWEI Mate 40
both of them crash and throw illegal instruction info

[Jeximo]

We should consider removing openCL instructions from the llama.cpp Android installation section. There's issues even if the illegal instruction is resolved.

Do you receive an illegal instruction on Android CPU inference? Ie. cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod, without openCL?

[DamonsJ]

We should consider removing openCL instructions from the llama.cpp Android installation section. There's issues even if the illegal instruction is resolved.

Do you receive an illegal instruction on Android CPU inference? Ie. cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod, without openCL?

Yes， I receive an illegal instruction on Android CPU inference. this is what I have done:

1. remove the build folder (after the gpu building)
   rm -rf build
2. cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod ..
3. make
4. run ./llama-bench -m ../models/qwen1_5-1_8b-chat-q2_k.gguf -p 0 -n 64,128,256 -t 4 -o json

then , I got an illegal instruction

BTW, not sure if these instructions matter:
export GGML_OPENCL_PLATFORM=0 export GGML_OPENCL_DEVICE=0 export LD_LIBRARY_PATH=/vendor/lib64:$LD_LIBRARY_PATH

[Jeximo]

BTW, not sure if these instructions matter:
export GGML_OPENCL_PLATFORM=0 export GGML_OPENCL_DEVICE=0 export LD_LIBRARY_PATH=/vendor/lib64:$LD_LIBRARY_PATH

No, it's not needed.

I'm able to reproduce this issue:

'./llama-bench -m ~/TinyLlama-1.…' terminated by signal SIGILL (Illegal instruction)

Related: main Segfault using cmake & -march=armv8.4a flag

Maybe try cmake without march=armv8.4a, for example:

cmake -B build && cd build && cmake --build . --config Release && cd bin/

[Jeximo]

It seems that passing C flags through cmake is the cause of segfault. cmake with no flags should work.

Maybe try -DCMAKE_CXX_FLAGS:STRING=-march=armv8.4a like this instead: cmake -B build -DCMAKE_CXX_FLAGS:STRING=-march=armv8.4a && cd build && cmake --build . --config Release && cd bin/

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.

[ElaineWu66]

I wonder if the author has solved the issue? Cuz I'm facing the same problem. Im running "./llama-cli -m ../model/qwen1.5-0.5b-chat-q4_k_m.gguf" on android, and it crashed with "Illegal instruction" after the line: llama_new_context_with_model: graph splits = 1