Compilation failure on aarch64-linux in ggml SIMD code

[marcom]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

Should compile on aarch64-linux

Current Behavior

[12:34:31] [  8%] Building C object CMakeFiles/ggml.dir/ggml.c.o
[12:34:31] /opt/bin/aarch64-linux-gnu-libgfortran5-cxx11/aarch64-linux-gnu-gcc --sysroot=/opt/aarch64-linux-gnu/aarch64-linux-gnu/sys-root/  -I/workspace/srcdir/llama.cpp/. -O3 -DNDEBUG -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wno-unused-function -pthread -MD -MT CMakeFiles/ggml.dir/ggml.c.o -MF CMakeFiles/ggml.dir/ggml.c.o.d -o CMakeFiles/ggml.dir/ggml.c.o -c /workspace/srcdir/llama.cpp/ggml.c
[12:34:31] /workspace/srcdir/llama.cpp/ggml.c: In function ‘dequantize_row_q4_1’:
[12:34:31] /workspace/srcdir/llama.cpp/ggml.c:1041:13: note: use -flax-vector-conversions to permit conversions between vectors with differing element types or numbers of subparts
[12:34:31]              const uint16x8_t vi_0 = vmovl_s8(vget_low_u8 (vq));
[12:34:31]              ^~~~~
[12:34:31] /workspace/srcdir/llama.cpp/ggml.c:1041:46: error: incompatible type for argument 1 of ‘vmovl_s8’
[12:34:31]              const uint16x8_t vi_0 = vmovl_s8(vget_low_u8 (vq));
[12:34:31]                                               ^~~~~~~~~~~~~~~~
[12:34:31] In file included from /workspace/srcdir/llama.cpp/ggml.c:164:
[12:34:31] /opt/aarch64-linux-gnu/lib/gcc/aarch64-linux-gnu/8.1.0/include/arm_neon.h:8839:20: note: expected ‘int8x8_t’ but argument is of type ‘uint8x8_t’
[12:34:31]  vmovl_s8 (int8x8_t a)
[12:34:31]            ~~~~~~~~~^
[12:34:31] /workspace/srcdir/llama.cpp/ggml.c:1042:46: error: incompatible type for argument 1 of ‘vmovl_s8’
[12:34:31]              const uint16x8_t vi_1 = vmovl_s8(vget_high_u8(vq));
[12:34:31]                                               ^~~~~~~~~~~~~~~~
[12:34:31] In file included from /workspace/srcdir/llama.cpp/ggml.c:164:
[12:34:31] /opt/aarch64-linux-gnu/lib/gcc/aarch64-linux-gnu/8.1.0/include/arm_neon.h:8839:20: note: expected ‘int8x8_t’ but argument is of type ‘uint8x8_t’
[12:34:31]  vmovl_s8 (int8x8_t a)
[12:34:31]            ~~~~~~~~~^
[12:34:31] make[2]: *** [CMakeFiles/ggml.dir/build.make:76: CMakeFiles/ggml.dir/ggml.c.o] Error 1
[12:34:31] make[2]: Leaving directory '/workspace/srcdir/llama.cpp/build'

Environment and Context

Julia's BinaryBuilder cross-compilation setup with gcc/g++ 8.1.0, i.e. building for a aarch64-linux-gnu target on a x86_64-linux-musl host. See JuliaPackaging/Yggdrasil#6476 for more details.

Proposed solution

Use vmovl_u8 instead of vmovl_s8? Not sure about the intention of the code, and it seems to be just a day old.

[thomasantony]

I was able to reproduce this bug here: https://github.com/thomasantony/llamacpp-python/actions/runs/4563539473/jobs/8052152944

[ashimokawa]

duplicate of #620

[marcom]

Update: there have been patches to this file in the ggml repo: ggml-org/ggml#51 and this should fix this issue as well (the changes in lines 1041/1042 in ggml.c, vmovl_s8 -> vmovl_u8).

duplicate of #620

But yeah, there should be a real fix.

from: https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html

-flax-vector-conversions

    Allow implicit conversions between vectors with differing numbers of elements and/or
    incompatible element types. This option should not be used for new code. 

[serif]

The problem persists on aarch64-linux, but I solved it by adding this to the Makefile:

CFLAGS   += -flax-vector-conversions
CXXFLAGS += -flax-vector-conversions

For myself I put that before the # Architecture specific section, outside any conditional. To integrate this for everyone, it would need to be wrapped in a conditional. I don't know if that would go inside the ifneq ($(filter aarch64%,$(UNAME_M)),) section or a new one with ifneq changed to ifeq.