CL_QUEUE_OUT_OF_ORDER_EXEC_MODE_ENABLE

[arch-btw]

Hi,

I've been getting this error but it's not related to macOS.

I'm running:

Model name: AMD Ryzen 5 3550H with Radeon Vega Mobile Gfx

Command:

DRI_PRIME=1 ROC_ENABLE_PRE_VEGA=1 GGML_OPENCL_PLATFORM=0 ./main -m /home/arch-btw/llama.cpp/models/Wizard-Vicuna-13B-Uncensored.ggmlv3.q4_1.bin --color -ins --n-gpu-layers 8

Output:

main: build = 602 (3b126f6)
main: seed = 1685332084
ggml_opencl: selecting platform: 'AMD Accelerated Parallel Processing'
ggml_opencl: selecting device: 'gfx803'
ggml_opencl: device FP16 support: true
ggml_opencl: (queue = clCreateCommandQueue(context, device, CL_QUEUE_OUT_OF_ORDER_EXEC_MODE_ENABLE, &err), (err != CL_INVALID_QUEUE_PROPERTIES && err != CL_INVALID_VALUE ? err : (queue = clCreateCommandQueue(context, device, 0, &err), err) )) error -6 at /home/arch-btw/Applications/llama.cpp/ggml-opencl.cpp:485

Things I tried:

• Compile with cmake instead of make (compiling works fine either way)
• Run with and without sudo
• With and without: DRI_PRIME=1, ROC_ENABLE_PRE_VEGA=1 & GGML_OPENCL_PLATFORM=0
• Used different models
• Different gpu-layers flag

Furthermore, in this thread: #1429

@swittk suggests to do the following:

Personally I just change the argument in clCreateCommandQueue in ggml-opencl.c here to simply have no flags.

queue = clCreateCommandQueue(context, device, 0, &err);

And it should compile and run fine! (Mac OS OpenCL doesn't support CL_QUEUE_OUT_OF_ORDER_EXEC_MODE_ENABLE)

I tried doing this but it's not working (I might be doing this wrong because it's not on line 214 anymore and I'm not fully sure how to change it now).

I was wondering how to apply that patch by @swittk ?

Or how to fix this error in any other way?

Thank you very much.

[arch-btw]

When I change line 485-488 to:

    CL_CHECK((queue = clCreateCommandQueue(context, device, 0, &err),
        err
    ));

I get:

ggml_opencl: (queue = clCreateCommandQueue(context, device, 0, &err), err ) error -6 at ggml-opencl.cpp:485

[swittk]

OpenCL error -6 is an out of memory error; it appears you're using a Hackintosh with an AMD processor, with integrated AMD graphics.
As far as I've looked, Vega 8 mobile can allocate at most 2GB VRAM. I don't think your machine can run LLaMA.cpp using OpenCL; you'll have to use CPU processing.

[arch-btw]

Thank you @swittk !

Sorry, I should have clarified it's not a hackintosh. But yes you're right it was an out of memory error.
I wanted to let you know that I somehow managed to fix it by downgrading to opencl-amd 5.4.1

There's a note in the AUR about it in case that helps someone else:

https://aur.archlinux.org/packages/opencl-amd

Running on a RX560X now with a whooping 4GB 🤣

Thank you again.