GGML_ASSERT: /project/ggml/src/ggml.c:3732: ctx->mem_buffer != NULL Aborted

[hanwsf]

python privateGPT.py
llama.cpp: loading model from models/ggml-model-q4_0.bin
llama.cpp: can't use mmap because tensors are not aligned; convert to new format to avoid this
llama_model_load_internal: format = 'ggml' (old version with low tokenizer quality and no mmap support)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 1000
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 2 (mostly Q4_0)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 4113748.20 KB
llama_model_load_internal: mem required = 5809.33 MB (+ 2052.00 MB per state)
...................................................................................................
.
llama_init_from_file: kv self size = 1000.00 MB
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx = 2048
gptj_model_load: n_embd = 4096
gptj_model_load: n_head = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot = 64
gptj_model_load: f16 = 2
gptj_model_load: ggml ctx size = 4505.45 MB
GGML_ASSERT: /project/ggml/src/ggml.c:3732: ctx->mem_buffer != NULL
Aborted

Out of Ram?
Thanks!

[hanwsf]

Ubuntu18, gcc-11

[Yiran-young]

I have the same problem, did you solve it please?

[hanwsf]

Not yet. Use ubuntu:latest docker image, also see the same error:
llama_init_from_file: kv self size = 1000.00 MB
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Using embedded DuckDB with persistence: data will be stored in: db
gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx = 2048
gptj_model_load: n_embd = 4096
gptj_model_load: n_head = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot = 64
gptj_model_load: f16 = 2
gptj_model_load: ggml ctx size = 4505.45 MB
GGML_ASSERT: /project/ggml/src/ggml.c:3732: ctx->mem_buffer != NULL
Aborted

[bluuo]

I also had the same error. Making some space in my storage by deleting some files/programs fixed the issue for me.

[hanwsf]

I also had the same error. Making some space in my storage by deleting some files/programs fixed the issue for me.

how much space left on hard drive can make this work?

[hanwsf]

short of memory. Needs 16GB ram (used around 13GB).

[pbm94]

i have 246gb on the hard drive and 32gb of ram

[raikrahul]

have the same issue

[ADXXI1590]

i am using system with 16 GB RAM and there is 26GB free in HDD. still facing the same issue.. can anyone suggest how to resolve?

[myseq]

I managed to resolve the issue after increase the memory to 16GB.

In case you are like me, running Ubuntu under WSL, and trying to test privateGPT.py. In your Ubuntu VM, run 'free -h' to check your RAM size. Should be at least 16GB, like below:

─$ free -h
               total        used        free      shared  buff/cache   available
Mem:            15Gi       385Mi        14Gi       3.0Mi       447Mi        14Gi
Swap:          4.0Gi          0B       4.0Gi

If not, check your WSL config, and allocate 16GB RAM to your Ubuntu.

• At your Windows cmd prompt, such as c:\users\XXXX, create a file ".wslconfig" if it doesn't exist.
• Paste the following:

[wsl2]
memory=16GB
processors=4

• then restart your WSL, with "wsl --shutdown", and start your Ubuntu instance again.
• Run the cmd free -h and make sure you have 16GB ram.

[Dragon1573]

I'm facing the same problem. Does it mean I have to have 16GiB FREE space RAM for running PrivateGPT?

PS> python .\privateGPT.py
Using embedded DuckDB with persistence: data will be stored in: db
Found model file.
gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
gptj_model_load: n_vocab = 50400
gptj_model_load: n_ctx   = 2048
gptj_model_load: n_embd  = 4096
gptj_model_load: n_head  = 16
gptj_model_load: n_layer = 28
gptj_model_load: n_rot   = 64
gptj_model_load: f16     = 2
gptj_model_load: ggml ctx size = 5401.45 MB
GGML_ASSERT: C:\Users\circleci.PACKER-64370BA5\project\gpt4all-backend\llama.cpp\ggml.c:4411: ctx->mem_buffer != NULL

System Info

• Microsoft Windows 11 Professional 22H2
• Python for Windows v3.11.3
• 16GiB physical RAM
  • 10.4GiB (386MiB) used
  • 5.3GiB available
  • 14.9/18.3GiB commited
  • 5.4GiB cached
  • 944MiB page cache pool
  • 666MiB non-page cache pool
• 1TB physical SSD
  • C:\ 247GiB total with 108GiB available
  • D:\ 692GiB total with 600GiB available

[hanwsf]

windows 16GB is not sufficient. Add swapper more, then it might work, but very slow

…

---- 回复的原邮件 ---- | 发件人 | ***@***.***> | | 日期 | 2023年06月08日 10:41 | | 收件人 | ***@***.***> | | 抄送至 | ***@***.***>、State ***@***.***> | | 主题 | Re: [imartinez/privateGPT] GGML_ASSERT: /project/ggml/src/ggml.c:3732: ctx->mem_buffer != NULL Aborted (Issue #90) | I'm facing the same problem. Does it mean I have to have 16GiB FREE space RAM for running PrivateGPT? PS> python .\privateGPT.py Using embedded DuckDB with persistence: data will be stored in: db Found model file. gptj_model_load: loading model from 'models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ... gptj_model_load: n_vocab = 50400 gptj_model_load: n_ctx = 2048 gptj_model_load: n_embd = 4096 gptj_model_load: n_head = 16 gptj_model_load: n_layer = 28 gptj_model_load: n_rot = 64 gptj_model_load: f16 = 2 gptj_model_load: ggml ctx size = 5401.45 MB GGML_ASSERT: C:\Users\circleci.PACKER-64370BA5\project\gpt4all-backend\llama.cpp\ggml.c:4411: ctx->mem_buffer != NULL System Info Microsoft Windows 11 Professional 22H2 Python for Windows v3.11.3 16GiB physical RAM 10.4GiB (386MiB) used 5.3GiB available 14.9/18.3GiB commited 5.4GiB cached 944MiB page cache pool 666MiB non-page cache pool 1TB physical SSD C:\ 247GiB total with 108GiB available D:\ 692GiB total with 600GiB available — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[iulix21]

RTX 4090 24B video memory
64GB DDR5
i9 - 14th gen
and still have this error:
GGML_ASSERT: /tmp/tmp25xydfjh/llama_cpp_python-0.2.53/vendor/llama.cpp/ggml-cuda.cu:8620: ptr == (void *)(g_cuda_pool_addr[device] + g_cuda_pool_used[device])
make: *** [Makefile:36: run] Aborted