CUDA error 12 : invalid pitch argument

[1980Dragon]

CUDA error 12 at C:\Users\USER\llama.cpp\ggml-cuda.cu:527: invalid pitch argument

Until yesterday, the program was running normally, but today I suddenly encountered the above error, causing the model to crash. I am using Windows 10, and I compiled it with CMAKE to use cuBLAS. The program runs without any issues when a short prompt is provided, but it immediately stops working when a long prompt is entered.

I created a virtual environment and performed a clean installation and compilation, but I still encountered the same error. I would greatly appreciate it if you could provide a solution to this issue.

[1980Dragon]

~/llama.cpp$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_May__3_18:49:52_PDT_2022
Cuda compilation tools, release 11.7, V11.7.64
Build cuda_11.7.r11.7/compiler.31294372_0

My CUDA information.

CUDA error 2 at ggml-cuda.cu:623: out of memory

And this is the error message that occurred when I tried to run it in a Linux environment through WSL2. I have 8GB of VRAM.

[UbiquitousWhite]

I was having the same issue with long prompts, resulting in the same error message, also using cuBLAS.
I was able to fix this by setting the context size higher, e.g. running with "-c 2048" as a starting parameter. This was not exactly easy to figure out however - only by running it specifying a prompt file using "-f" I was able to get an actual error message out of it.

[1980Dragon]

Thank you for your response. Fortunately, my issue was resolved automatically after installing the latest version. So I will consider this issue closed.