Illegal instruction (core dumped)

[mehrdad2000]

Hi i've got below error when run privateGPT!

[root@ privateGPT]# python3.11 privateGPT.py
Using embedded DuckDB with persistence: data will be stored in: db
Illegal instruction (core dumped)

Any idea?
Thanks

[apcameron]

see #203

[jon2allen]

Your cpu likely isn't supported by current pytorch and torch libraries. and even if you managed to get this to work on such a processor - it would be a painful long wait before you get results.

[mehrdad2000]

@jon2allen i’m running it on esx host that use 8 core intel cpu and won't support avx2, what can i do to fix this?

[digitalw00t]

I've tried running an a Dell T620, core dump with privateGPT.py
I've tried running on a Dell R930 running proxmox with a vm that has an ubuntu 22 LTS machine, core dump
Both will ingest just fine, but core dump on execution

[finaldie]

@digitalw00t I have the same issue with Dell R730 + PVE VM. In short, the solution is: Change the VM processor type to host, then restart the VM, it would work. (you could double-check whether your CPU type is host or not, see pve reference here: https://forum.proxmox.com/threads/avx2-and-avx-flags-on-vm.87808/)

...
emb agent loaded
Using embedded DuckDB with persistence: data will be stored in: db
vector db loaded
starting pick LLM: GPT4All, model_path: models/ggml-gpt4all-j-v1.3-groovy.bin
GPT4All branch
gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx   = 2048
gptj_model_load: n_embd  = 4096
gptj_model_load: n_head  = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot   = 64
gptj_model_load: f16     = 2
gptj_model_load: ggml ctx size = 4505.45 MB
gptj_model_load: memory_size =   896.00 MB, n_mem = 57344
gptj_model_load: ................................... done
gptj_model_load: model size =  3609.38 MB / num tensors = 285
picked LLM: GPT4All

Enter a query: hi
 Hi! How can I help you today?

Btw, please verify the processor in the PVE directly and make sure the avx/avx2 is enabled:

cat /proc/cpuinfo | grep avx --color

[digitalw00t]

I'm running two Intel(R) Xeon(R) CPU E5-2640 on the T620, and according to Phind it should have that instruction set. I'll check when I get home from work, and do the proxmox update that's mentioned as well.

[digitalw00t]

Yup.. same issue, but there's no way to turn it on in the bios, so I'm guessing it just isn't supported. I thought I read that it was, but if there's no way to activate it I'm SOL.

[mibtim]

same Illegal instruction (core dumped) issue, im running on vm ubuntu 64 bit, base mem 513000mb, 6 processers

[digitalw00t]

I'm just writing it off that my cpu is old enough that it doesn't have the avx2 instruction set. I might go about recompiling torch, but not sure at this point.