illegal instructions error on Android

[aicoat]

first thanks for the wonderful works so far !!!

i manged to compile it in Linux and windows but i have a problem with android.
i have A52 6 GB but i get "illegal instructions" error.

i compiled the source using wsl2 with ndk r25 without any errors. i moved the llama folder from sd card to "home" directory in (Termux) in order to have the execute command working. and i converted to original model using the newer source code to avoid "too old" error message but at the end i get this error.

i believe it is because of having avx, avx2 and other instruction already enabled in my build which is arm processors cant handle them but i cant figure it out how to change it to get it working on my android device.
thanks in advanced <3

[gjmulder]

Please review and use our issue template to provide more details so we can try and better understand your problem and attempt to answer you.

[himanshu09010]

the issue happens when you don't have enough RAM

[dalnk]

your a52 might just not allow termux to allocate that much ram. weird given you have 6GB

[aicoat]

@dalnk @himanshu09010 that is sad since i heard people are running it on a raspberry pi with 4gb of ram :(

[dniku]

I believe I have encountered the same problem. I have tried to compile the binary directly on my phone (Samsung S22 Ultra with Snapdragon CPU, model SM-S908E). I followed these steps in Termux:

pkg up
pkg install wget git make clang

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make
<put model into models/llama-7B/ggml-model.bin>
./main

This fails with Illegal instruction.

Running gdb ./main (and then at the gdb prompt first r and then disassemble), I see that this is because of the cnth x14 instruction in ggml_new_tensor_impl().

The cnth instruction is documented here. For some reason, the CPU in my phone doesn't support it.

[dalnk]

Did you compile with the NDK instructions first?

@aicoat on raspberrypi it's possible to enable swap which afaik I haven't gotten working on Android yet. If I ever get my hands on an a52 or similar I'll try to recreate your issue.

[dniku]

@dalnk the "illegal instruction" error arises when the source is compiled on the device itself. I'm not sure the NDK is relevant in this case.

...however, building the binary on Linux with NDK doesn't work either due to a different error: #495 (comment)

[DGdev91]

I just compiled it directly on a Xiaomi Mi 11T Pro (Snapdragon 888), and it's woking using the GPT4All model.
Pretty slow, but works.

Maybe the last commits related to ARM_NEON fixed the issue indirectly?

[dniku]

Just tried recompiling the project for the latest commit (git pull && make clean && make). Still no luck. The offending instruction is still cnth.

[dalnk]

I'm curious if this works for you when compiling locally @dniku

last time I tried it, I had success without having to compile it on NDK

https://github.com/antimatter15/alpaca.cpp

[dniku]

@dalnk compiling alpaca.cpp directly on Android produces the exact same error caused by the same instruction.