Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"Not enough context memory" on raspberry pi #522

Closed
DiamondDemon669 opened this issue Mar 26, 2023 · 13 comments
Closed

"Not enough context memory" on raspberry pi #522

DiamondDemon669 opened this issue Mar 26, 2023 · 13 comments

Comments

@DiamondDemon669
Copy link

I tried loading the 7B model on a raspberry pi 4 (8GB) and it said there was not enough memory in the context pool, before segfaulting. On the raspberry pi, I am accessing the model from a hard disk on a seperate laptop, shared over sshfs, but on the laptop itself (x86 linux, 4GB RAM + 10GB swap) I access it over USB and it works perfectly fine. I am using the one before the latest versions of llama.cpp on both systems, but i changed one line of code in the raspberry pi's version. I replaced #include <immintrin.h> with #include <arm_neon.h> in ggml.c

While I was writing this I updated the raspberry pi's version of the software to the latest version and the bug still occurs. I will only disply the log from the previous version though

Log:

pi@raspberrypi:~/clones/llama.cpp $ ./main -m /path/to/network/drive/LLaMA/7B/ggml-model-q4_0.bin.tmp -i --color -t 3
main: seed = 1679831014
llama_model_load: loading model from '/path/to/network/drive/LLaMA/7B/ggml-model-q4_0.bin.tmp' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx   = 512
llama_model_load: n_embd  = 4096
llama_model_load: n_mult  = 256
llama_model_load: n_head  = 32
llama_model_load: n_layer = 32
llama_model_load: n_rot   = 128
llama_model_load: f16     = 2
llama_model_load: n_ff    = 11008
llama_model_load: n_parts = 1
llama_model_load: ggml ctx size =   0.08 MB
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 81920140, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 81936664, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10518948, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10518948, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10518948, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10518948, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28230192, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28230192, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28230192, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10551996, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10551996, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10551996, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10551996, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 10568520, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 99144, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 28263240, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 268518216, available 83199)
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 268518216, available 83199)
Segmentation fault
@slaren
Copy link
Member

slaren commented Mar 26, 2023

llama_model_load: ggml ctx size = 0.08 MB

That's obviously not right. Overflow from 32 bit size_t?

@Green-Sky
Copy link
Collaborator

very possible, we have stuff like this 6f1ee4b after all.

@prusnak
Copy link
Collaborator

prusnak commented Mar 26, 2023 •
edited
Loading

@DiamondDemon669 What does uname -m say? If it says armv7l you are using 32-bit operating system and this has a limitation that a single process cannot allocate more than 4GB. You need to use a 64-bit OS (aarch64) to be able to run llama.cpp.

@Green-Sky
Copy link
Collaborator

If it says armv7l you are using 32-bit operating system and this has a limitation that a single process cannot allocate more than 4GB

on linux, you can. But I wonder if it can't be continuous.

@prusnak
Copy link
Collaborator

prusnak commented Mar 26, 2023

on linux, you can.

Are you sure? Can you point me to the Linux documentation where it says so? I am aware only of PAE and this does not raise the limitation of max size of single process memory.

@Green-Sky
Copy link
Collaborator

hmmm, I think I am in the wrong here. Seems like it's 4G max for a single process.

@Green-Sky
Copy link
Collaborator

Also, did we stop printing the system info line? It's missing from the log.

@prusnak
Copy link
Collaborator

prusnak commented Mar 26, 2023

Also, did we stop printing the system info line? It's missing from the log.

Wasn't this only part of the make command?

@Green-Sky
Copy link
Collaborator

no, ok, it's bugging out before it would print it.

see example:

llama_model_load: model size =  4820.52 MB / num tensors = 291
llama_init_from_file: kv self size  = 1024.00 MB

system_info: n_threads = 8 / 24 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
sampling: temp = 0.800000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.100000
generate: n_ctx = 2048, n_batch = 8, n_predict = -1, n_keep = 0

@prusnak
Copy link
Collaborator

prusnak commented Mar 26, 2023

Closing as there is nothing to do here.

@prusnak prusnak closed this as completed Mar 26, 2023
@Green-Sky
Copy link
Collaborator

Closing as there is nothing to do here.

you sure?, the raspi 4 is a armv8 64bit soc

@prusnak
Copy link
Collaborator

prusnak commented Mar 26, 2023

Closing as there is nothing to do here.

you sure?, the raspi 4 is a armv8 64bit soc

Yes, OP needs to switch to 64-bit Raspberry Pi OS and the issue will disappear. If it doesn't then we can reopen the issue.

@slaren
Copy link
Member

slaren commented Mar 26, 2023

Yes, OP needs to switch to 64-bit Raspberry Pi OS and the issue will disappear. If it doesn't then we can reopen the issue.

If that's the case, shouldn't this be detected in the build, instead of allowing it to build on 32-bit systems silently?

Deadsg pushed a commit to Deadsg/llama.cpp that referenced this issue Dec 19, 2023
@abetlen
Merge pull request ggml-org#522 from bretello/llama2-70b-support
e4431a6 
Llama2 70b support
@Bearsaerker Bearsaerker mentioned this issue Mar 12, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@prusnak @slaren @Green-Sky @DiamondDemon669