Error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch #107

tofasthacker · 2023-03-13T23:20:27Z

I cloned the GitHub repository and ran the make command but was unable to get the cpp files to compile successfully. Any help or suggestion would be appreciated.

Terminal output:

brickman@Ubuntu-brickman:~/Desktop/llama.cpp$ ls
CMakeLists.txt  convert-pth-to-ggml.py  ggml.c  ggml.h  LICENSE  main.cpp  Makefile  models  quantize.cpp  quantize.sh  README.md  utils.cpp  utils.h
brickman@Ubuntu-brickman:~/Desktop/llama.cpp$ make
I llama.cpp build info: 
I UNAME_S:  Linux
I UNAME_P:  x86_64
I UNAME_M:  x86_64
I CFLAGS:   -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -mavx -mavx2 -msse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread
I LDFLAGS:  
I CC:       cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
I CXX:      g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

cc  -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -mavx -mavx2 -msse3   -c ggml.c -o ggml.o
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/include/immintrin.h:109,
                 from ggml.c:155:
ggml.c: In function ‘ggml_vec_dot_f16’:
/usr/lib/gcc/x86_64-linux-gnu/9/include/f16cintrin.h:52:1: error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1274:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1274 |             ay[j] = GGML_F16_VEC_LOAD(y + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/include/immintrin.h:109,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/9/include/f16cintrin.h:52:1: error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1273:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1273 |             ax[j] = GGML_F16_VEC_LOAD(x + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/include/immintrin.h:109,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/9/include/f16cintrin.h:52:1: error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1273:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1273 |             ax[j] = GGML_F16_VEC_LOAD(x + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/include/immintrin.h:109,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/9/include/f16cintrin.h:52:1: error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1274:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1274 |             ay[j] = GGML_F16_VEC_LOAD(y + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/include/immintrin.h:109,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/9/include/f16cintrin.h:52:1: error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1273:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1273 |             ax[j] = GGML_F16_VEC_LOAD(x + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/include/immintrin.h:109,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/9/include/f16cintrin.h:52:1: error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1274:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1274 |             ay[j] = GGML_F16_VEC_LOAD(y + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
make: *** [Makefile:186: ggml.o] Error 1
brickman@Ubuntu-brickman:~/Desktop/llama.cpp$

apaz-cli · 2023-03-14T00:25:36Z

It would be best to take this up on https://github.com/ggerganov/ggml.

G2G2G2G · 2023-03-14T02:25:30Z

what is your CPU?
type lscpu
I think you do not have AVX2 instructions.

tofasthacker · 2023-03-14T11:54:33Z

I am running ubuntu in a vm with virutual box. Here is the out put of the lscpu command. Thank you for you help so far.

brickman@Ubuntu-brickman:~/Desktop$ lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   48 bits physical, 48 bits virtual
CPU(s):                          6
On-line CPU(s) list:             0-5
Thread(s) per core:              1
Core(s) per socket:              6
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      23
Model:                           113
Model name:                      AMD Ryzen 5 3600 6-Core Processor
Stepping:                        0
CPU MHz:                         3599.936
BogoMIPS:                        7199.87
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       192 KiB
L1i cache:                       192 KiB
L2 cache:                        3 MiB
L3 cache:                        32 MiB
NUMA node0 CPU(s):               0-5
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Not affected
Vulnerability Retbleed:          Mitigation; untrained return thunk; SMT disable
                                 d
Vulnerability Spec store bypass: Not affected
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user
                                  pointer sanitization
Vulnerability Spectre v2:        Mitigation; Retpolines, STIBP disabled, RSB fil
                                 ling, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtr
                                 r pge mca cmov pat pse36 clflush mmx fxsr sse s
                                 se2 ht syscall nx mmxext fxsr_opt rdtscp lm con
                                 stant_tsc rep_good nopl nonstop_tsc cpuid extd_
                                 apicid tsc_known_freq pni pclmulqdq ssse3 cx16 
                                 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx
                                  rdrand hypervisor lahf_lm cmp_legacy cr8_legac
                                 y abm sse4a misalignsse 3dnowprefetch ssbd vmmc
                                 all fsgsbase bmi1 avx2 bmi2 rdseed clflushopt a
                                 rat
brickman@Ubuntu-brickman:~/Desktop$

gjmulder · 2023-03-14T12:51:24Z

Flags indicate your Virtualbox instance supports avx. Can you try g++10? g++10 works for me under Ubuntu 22.04 and I just compiled it successfully in Docker using Debian 10 and g++10:

root@9dc00e80510c:~/llama.cpp# cat /etc/issue
Debian GNU/Linux 10 \n \l

root@9dc00e80510c:~/llama.cpp# uname -a
Linux 9dc00e80510c 5.15.0-60-generic #66-Ubuntu SMP Fri Jan 20 14:29:49 UTC 2023 x86_64 GNU/Linux
root@9dc00e80510c:~/llama.cpp# grep -i amd /proc/cpuinfo | sort -ru
vendor_id	: AuthenticAMD
model name	: AMD Ryzen Threadripper 1950X 16-Core Processor
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate ssbd ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca sme sev
root@9dc00e80510c:~/llama.cpp# make      
I llama.cpp build info: 
I UNAME_S:  Linux
I UNAME_P:  unknown
I UNAME_M:  x86_64
I CFLAGS:   -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -mavx -mavx2 -mfma -mf16c -msse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread
I LDFLAGS:  
I CC:       cc (GCC) 10.2.0
I CXX:      g++ (GCC) 10.2.0

cc  -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -mavx -mavx2 -mfma -mf16c -msse3   -c ggml.c -o ggml.o
g++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread -c utils.cpp -o utils.o
g++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread main.cpp ggml.o utils.o -o main 
./main -h
usage: ./main [options]

options:
  -h, --help            show this help message and exit
  -i, --interactive     run in interactive mode
  --interactive-start   run in interactive mode and poll user input at startup
  -r PROMPT, --reverse-prompt PROMPT
                        in interactive mode, poll user input upon seeing PROMPT
  --color               colorise output to distinguish prompt and user input from generations
  -s SEED, --seed SEED  RNG seed (default: -1)
  -t N, --threads N     number of threads to use during computation (default: 4)
  -p PROMPT, --prompt PROMPT
                        prompt to start generation with (default: random)
  -f FNAME, --file FNAME
                        prompt file to start generation.
  -n N, --n_predict N   number of tokens to predict (default: 128)
  --top_k N             top-k sampling (default: 40)
  --top_p N             top-p sampling (default: 0.9)
  --repeat_last_n N     last n tokens to consider for penalize (default: 64)
  --repeat_penalty N    penalize repeat sequence of tokens (default: 1.3)
  --temp N              temperature (default: 0.8)
  -b N, --batch_size N  batch size for prompt processing (default: 8)
  -m FNAME, --model FNAME
                        model path (default: models/llama-7B/ggml-model.bin)

g++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread quantize.cpp ggml.o utils.o -o quantize

tofasthacker · 2023-03-14T12:55:48Z

I ran the command with g++ and this is what I got.

brickman@Ubuntu-brickman:~/Desktop/llama.cpp$ make g++
I llama.cpp build info: 
I UNAME_S:  Linux
I UNAME_P:  x86_64
I UNAME_M:  x86_64
I CFLAGS:   -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -mavx -mavx2 -msse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread
I LDFLAGS:  
I CC:       cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
I CXX:      g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

make: *** No rule to make target 'g++'.  Stop.
brickman@Ubuntu-brickman:~/Desktop/llama.cpp$

gjmulder · 2023-03-14T12:59:28Z

g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

You're using g++9.4.0. This might help

tofasthacker · 2023-03-14T13:07:48Z

I updated my g++ version and tried your command but I am still having trouble. I really appreciate your help. Thank you.

brickman@Ubuntu-brickman:~/Desktop/llama.cpp$ g++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread quantize.cpp ggml.o utils.o -o quantize 
g++: error: ggml.o: No such file or directory
g++: error: utils.o: No such file or directory
brickman@Ubuntu-brickman:~/Desktop/llama.cpp$ g++ --version
g++ (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

brickman@Ubuntu-brickman:~/Desktop/llama.cpp$

gjmulder · 2023-03-14T13:10:45Z

Now that g++ is g++10, try just typing:

make clean;make

tofasthacker · 2023-03-14T13:12:40Z

Looks like the same error.

brickman@Ubuntu-brickman:~/Desktop/llama.cpp$ make clean;make
I llama.cpp build info: 
I UNAME_S:  Linux
I UNAME_P:  x86_64
I UNAME_M:  x86_64
I CFLAGS:   -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -mavx -mavx2 -msse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread
I LDFLAGS:  
I CC:       cc (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
I CXX:      g++ (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0

rm -f *.o main quantize
I llama.cpp build info: 
I UNAME_S:  Linux
I UNAME_P:  x86_64
I UNAME_M:  x86_64
I CFLAGS:   -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -mavx -mavx2 -msse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread
I LDFLAGS:  
I CC:       cc (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
I CXX:      g++ (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0

cc  -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -mavx -mavx2 -msse3   -c ggml.c -o ggml.o
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
ggml.c: In function ‘ggml_vec_dot_f16’:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1274:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1274 |             ay[j] = GGML_F16_VEC_LOAD(y + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1273:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1273 |             ax[j] = GGML_F16_VEC_LOAD(x + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1273:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1273 |             ax[j] = GGML_F16_VEC_LOAD(x + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1274:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1274 |             ay[j] = GGML_F16_VEC_LOAD(y + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1273:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1273 |             ax[j] = GGML_F16_VEC_LOAD(x + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:911:33: note: called from here
  911 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:921:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  921 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1274:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1274 |             ay[j] = GGML_F16_VEC_LOAD(y + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
make: *** [Makefile:186: ggml.o] Error 1
brickman@Ubuntu-brickman:~/Desktop/llama.cpp$

gjmulder · 2023-03-14T13:17:40Z

It works under Docker with g++10, so it points to an issue with Virtualbox. Why do you need to use Virtualbox?

tofasthacker · 2023-03-14T13:23:41Z

Ok, I will try to run the command on a real machine in a couple of hours. Did you put your docker image on docker hub?

gjmulder · 2023-03-14T13:29:10Z

Nope, I just smoke tested the compile in a running instance. However someone just now published a pull request for Docker support here #132 🚀

tofasthacker · 2023-03-14T17:20:58Z

I was Successfully able to build the model when was not in a virtual machine. But Now I am wondering were I can download the LLAMA model.

gjmulder · 2023-03-14T17:49:54Z

I was Successfully able to build the model when was not in a virtual machine. But Now I am wondering were I can download the LLAMA model.

meta-llama/llama#92

gjmulder · 2023-03-14T18:01:58Z

I can neither confirm nor deny that that link will work 😄

For verification, and as there are some suspect models floating around, I published the md5 sums and file sizes of my confirmed working models in issue #69.

tofasthacker · 2023-03-14T18:08:16Z

Thank You for all your help.

G2G2G2G · 2023-03-15T01:47:08Z

@tofasthacker yes that place is a good source for it. I have been using those for several days and the URL remains the same

It works under Docker with g++10, so it points to an issue with Virtualbox. Why do you need to use Virtualbox?

@gjmulder I attempted to compile on a system with no AVX a week or so ago, and I got similar output to his. Which is why I originally posted I think he has no AVX support.
I think somehow virtualbox may report it, but maybe can't compile using it idk... but the output seems near identical as far as I remember, it really has like nothing to do with anything you'd think but somehow the GGML_F16_VEC_LOAD must be related to avx idk.
anyway that's where my original thought came from

ehlerst · 2023-03-16T00:18:16Z

im on bare metal have AVX (intel 2600k)

i added this to force it to compile with gcc11

it diff
diff --git a/Makefile b/Makefile
index 1601079..3c3fae9 100644
--- a/Makefile
+++ b/Makefile
@@ -183,7 +183,7 @@ default: main quantize
 #
 
 ggml.o: ggml.c ggml.h
-       $(CC)  $(CFLAGS)   -c ggml.c -o ggml.o
+        $(CC)  $(CFLAGS)  -mf16c -c ggml.c -o ggml.o
 
 utils.o: utils.cpp utils.h
        $(CXX) $(CXXFLAGS) -c utils.cpp -o utils.o

Edit: nevermind dont try this. There are 2 AVX out there, the 2600k has AVX1, this needs AVX2

tommybutler · 2023-03-23T18:09:11Z

Now that g++ is g++10, try just typing:

make clean;make

I have the same error on my Ubuntu Linux machine (physical box). I upgraded to g++10 and it makes no difference.

$ make                                                                                                                                                        
I llama.cpp build info:                                                                                                                                       
I UNAME_S:  Linux                                            
I UNAME_P:  x86_64                                             
I UNAME_M:  x86_64                                          
I CFLAGS:   -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -mavx -msse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread
I LDFLAGS:  
I CC:       cc (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
I CXX:      g++ (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0

cc  -I.              -O3 -DNDEBUG -std=c11   -fPIC -pthread -mavx -msse3   -c ggml.c -o ggml.o
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
ggml.c: In function ‘ggml_vec_dot_f16’:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:915:33: note: called from here
  915 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:925:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  925 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1319:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1319 |             ay[j] = GGML_F16_VEC_LOAD(y + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:915:33: note: called from here
  915 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:925:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  925 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1318:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1318 |             ax[j] = GGML_F16_VEC_LOAD(x + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:915:33: note: called from here
  915 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:925:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  925 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1318:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1318 |             ax[j] = GGML_F16_VEC_LOAD(x + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:915:33: note: called from here
  915 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:925:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  925 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1319:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1319 |             ay[j] = GGML_F16_VEC_LOAD(y + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:915:33: note: called from here
  915 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:925:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  925 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1318:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1318 |             ax[j] = GGML_F16_VEC_LOAD(x + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/10/include/immintrin.h:113,
                 from ggml.c:155:
/usr/lib/gcc/x86_64-linux-gnu/10/include/f16cintrin.h:52:1: error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’: target specific option mismatch
   52 | _mm256_cvtph_ps (__m128i __A)
      | ^~~~~~~~~~~~~~~
ggml.c:915:33: note: called from here
  915 | #define GGML_F32Cx8_LOAD(x)     _mm256_cvtph_ps(_mm_loadu_si128((__m128i *)(x)))
      |                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:925:37: note: in expansion of macro ‘GGML_F32Cx8_LOAD’
  925 | #define GGML_F16_VEC_LOAD(p, i)     GGML_F32Cx8_LOAD(p)
      |                                     ^~~~~~~~~~~~~~~~
ggml.c:1319:21: note: in expansion of macro ‘GGML_F16_VEC_LOAD’
 1319 |             ay[j] = GGML_F16_VEC_LOAD(y + i + j*GGML_F16_EPR, j);
      |                     ^~~~~~~~~~~~~~~~~
make: *** [Makefile:221: ggml.o] Error 1

My CPU details:

$ lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          32
On-line CPU(s) list:             0-31
Thread(s) per core:              2
Core(s) per socket:              8
Socket(s):                       2
NUMA node(s):                    2
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           45
Model name:                      Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz
Stepping:                        6
CPU MHz:                         2994.899
CPU max MHz:                     3000.0000
CPU min MHz:                     1200.0000
BogoMIPS:                        4400.12
Virtualization:                  VT-x
L1d cache:                       512 KiB
L1i cache:                       512 KiB
L2 cache:                        4 MiB
L3 cache:                        40 MiB
NUMA node0 CPU(s):               0-7,16-23
NUMA node1 CPU(s):               8-15,24-31
Vulnerability Itlb multihit:     KVM: Mitigation: Split huge pages
Vulnerability L1tf:              Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Mmio stale data:   Unknown: No mitigations
Vulnerability Retbleed:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx es
                                 t tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts md_clear flush_l1d

DJAHIDDJ13 · 2023-03-23T21:10:02Z

Try adding -march=znver1 to CFLAGS in the Makefile, I'm not exactly sure what it is adding but it worked for me

GasparQ · 2023-10-16T12:29:43Z

I have the same issue with a Fedora Silverblue Virtual Machine compiling within podman container.

To fix it, I just set the LLAMA_NATIVE option to OFF this way

mkdir build
cd build
cmake -DLLAMA_NATIVE=OFF ..
make -j

I think there might be a bug in the use of intel SIMD instructions: https://www.intel.com/content/www/us/en/docs/cpp-compiler/developer-guide-reference/2021-8/mm-fmadd-ps-mm256-fmadd-ps.html

hkevicBB · 2023-10-27T13:49:06Z

I have the same issue with a Fedora Silverblue Virtual Machine compiling within podman container.

To fix it, I just set the LLAMA_NATIVE option to OFF this way
mkdir build
cd build
cmake -DLLAMA_NATIVE=OFF ..
make -j
I think there might be a bug in the use of intel SIMD instructions: https://www.intel.com/content/www/us/en/docs/cpp-compiler/developer-guide-reference/2021-8/mm-fmadd-ps-mm256-fmadd-ps.html

If running on Kali or Debian Linux, replace the last command with
cmake --build . --config Release

With make -j my machine froze.

tofasthacker closed this as completed Mar 14, 2023

gjmulder mentioned this issue Mar 15, 2023

Unable to compile - error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’ #159

Closed

gjmulder added hardware Hardware related build Compilation issues labels Mar 15, 2023

gjmulder mentioned this issue Mar 16, 2023

Error: inlining failed in call to ‘always_inline’ ‘_mm256_cvtph_ps’ on x86_64 - better support for different x86_64 CPU instruction extensions #196

Closed

gjmulder changed the title ~~error building on linux~~ Error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch Mar 16, 2023

gjmulder added good first issue Good for newcomers duplicate This issue or pull request already exists labels Mar 16, 2023

ggml-org deleted a comment from tofasthacker Mar 23, 2023

themanyone mentioned this issue Apr 21, 2023

Segmentation Fault with 7b on Raspberry Pi 3 antimatter15/alpaca.cpp#213

Open

countzero mentioned this issue Nov 22, 2023

Custom CMAKE_ARGS are overwritten in the "backend/cpp/llama/grpc-server" target for the "llama-cpp" backend mudler/LocalAI#1317

Closed

Deadsg pushed a commit to Deadsg/llama.cpp that referenced this issue Dec 19, 2023

Disable mmap when applying lora weights. Closes ggml-org#107

7230599

kausthub-kannan mentioned this issue Aug 16, 2024

Failed to load shared library error: undefined symbol _ZNSt15__exception_ptr13exception_ptr9_M_addrefEv abetlen/llama-cpp-python#1688

Closed

Tridu33 mentioned this issue Mar 5, 2025

Bug: Can not compile iqk_mul_mat_arm82.cpp from in ubuntu20.04 aarch64 NEON error: inlining failed in call to always_inline ‘float16x4_t vmul_f16(float16x4_t, float16x4_t)’: target specific option mismatch Mozilla-Ocho/llamafile#706

Closed

Bearsaerker mentioned this issue Mar 12, 2025

Eval bug: Gemma 3 extremly slow prompt processing when using quantized kv cache. #12352

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch #107

Error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch #107

tofasthacker commented Mar 13, 2023

apaz-cli commented Mar 14, 2023

G2G2G2G commented Mar 14, 2023

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023 •

edited

Loading

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023 •

edited

Loading

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023

gjmulder commented Mar 14, 2023

tofasthacker commented Mar 14, 2023

G2G2G2G commented Mar 15, 2023 •

edited by gjmulder

Loading

ehlerst commented Mar 16, 2023 •

edited

Loading

tommybutler commented Mar 23, 2023 •

edited

Loading

DJAHIDDJ13 commented Mar 23, 2023 •

edited

Loading

GasparQ commented Oct 16, 2023

hkevicBB commented Oct 27, 2023 •

edited

Loading

Error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch #107

Error: inlining failed in call to always_inline ‘_mm256_cvtph_ps’: target specific option mismatch #107

Comments

tofasthacker commented Mar 13, 2023

apaz-cli commented Mar 14, 2023

G2G2G2G commented Mar 14, 2023

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023 • edited Loading

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023 • edited Loading

tofasthacker commented Mar 14, 2023

gjmulder commented Mar 14, 2023

gjmulder commented Mar 14, 2023

tofasthacker commented Mar 14, 2023

G2G2G2G commented Mar 15, 2023 • edited by gjmulder Loading

ehlerst commented Mar 16, 2023 • edited Loading

tommybutler commented Mar 23, 2023 • edited Loading

DJAHIDDJ13 commented Mar 23, 2023 • edited Loading

GasparQ commented Oct 16, 2023

hkevicBB commented Oct 27, 2023 • edited Loading

gjmulder commented Mar 14, 2023 •

edited

Loading

gjmulder commented Mar 14, 2023 •

edited

Loading

G2G2G2G commented Mar 15, 2023 •

edited by gjmulder

Loading

ehlerst commented Mar 16, 2023 •

edited

Loading

tommybutler commented Mar 23, 2023 •

edited

Loading

DJAHIDDJ13 commented Mar 23, 2023 •

edited

Loading

hkevicBB commented Oct 27, 2023 •

edited

Loading