Manual CLBlast device and platform number

[daniandtheweb]

Using CLBlast on some devices (like my Radeon 5700XT) the program defaults to device id 0 and platform id 0, that causes the following error:

Initializing CLBlast (First Run)...
Attempting to use: Platform=0, Device=0 (If invalid, program will crash)
Using Platform: Clover Device: AMD Radeon RX 5700 XT (navi10, LLVM 15.0.7, DRM 3.52, 6.3.1-zen1-1-zen)
fatal error: cannot open file '/usr/share/clc/gfx1010-amdgcn-mesa-mesa3d.bc': No such file or directory

When I manually edit the platform id inside the source to 1 however makes the program behave as expected using the GPU for the prompt evaluation:

Initializing CLBlast (First Run)...
Attempting to use: Platform=1, Device=0 (If invalid, program will crash)
Using Platform: AMD Accelerated Parallel Processing Device: gfx1010:xnack-

My suggestion is to add a launch option to manually choose the desired platform and device id similarly to the --useclblast option of koboldcpp.

[sw]

It should be possible to set the CLBLAST_PLATFORM environment variable to achieve this, but I haven't tested it.

[SlyEcho]

There are two environment variables you can set: GGML_CLBLAST_PLATFORM and GGML_CLBLAST_DEVICE (just numbers)

(Personally I think there should be a command line argument, and it's not really CLBlast that it's controlling, but OpenCL in general. CUDA doesn't have any way to control the GPU selection right now, @slaren?)

[slaren]

CUDA doesn't have any way to control the GPU selection right now, @slaren?

The environment variable CUDA_VISIBLE_DEVICES should work.

[daniandtheweb]

Thanks @SlyEcho for the info. I also think that a command line argument would be the best but I guess that for now I can just close the issue.