ci : add cublas to windows release

[Green-Sky]

This adds a cuBLAS build to the windows ci.

edit: did a test release, please test https://github.com/ggerganov/llama.cpp/releases/tag/ci_cublas-31ff9e2

Open questions:

• the cuda dll's are huge, should we ship them? they also don't change (often)
  • it generates a separate .zip with just the cuda dlls
• do we need the blasLt dll? ironically named "lite", it's the largest dll (>400mb)
  • yes
• the toolkit install takes ages. I tried using only a select install, but that never worked.
• since it takes ages, maybe not require it for merge.
• which cuda version to use. I set it to 12.1, but that requires the very latest driver.
  • I decided on both 11.7.1 and 12.1.0
• should we enable shared for all builds, so we distribute the .dll
• other stuff i forgot, since the turn around time is >20min, it's a real hell to debug.

eula allowing redist https://docs.nvidia.com/cuda/eula/index.html#attachment-a

[slaren]

If GitHub can handle the big releases, it is probably better to distribute the DLLs than to ask users to install the CUDA toolkit. Maybe also include an additional artifact that doesn't include the DLLs so that people who already have them don't need to download them again?

[slaren]

• do we need the blasLt dll? ironically named "lite", it's the largest dll (>400mb)

I think so, the cublas DLL depends on cublaslt:

[Green-Sky]

Maybe also include an additional artifact that doesn't include the DLLs so that people who already have them don't need to download them again?

sounds good

[Green-Sky]

performing a manual release here https://github.com/ggerganov/llama.cpp/actions/runs/4875560144

edit: good i tested this, i forgot a depends

[Green-Sky]

@slaren can you try the ci binaries here https://github.com/ggerganov/llama.cpp/releases/tag/ci_cublas-45d94c8 ?

[Green-Sky]

Looks like no one is running a cuda 12 capable driver 🤣 , gonna add 11.7

[slaren]

These binaries work well for me! No issues.

[Green-Sky]

@slaren can you try the 11.7 binaries too? https://github.com/ggerganov/llama.cpp/releases/tag/ci_cublas-31ff9e2

[slaren]

Also works, but as expected I need to download the DLLs as well, as my CUDA toolkit is 12.1. The 12.1 binaries work without the DLL.

[slaren]

since it takes ages, maybe not require it for merge.

IMO it is fine as is, but we could consider running the CI tests for cuBLAS only when ggml-cuda.h/cu have been modified. However, that may not achieve much beyond testing that the builds completes until we have a model small enough to use in CI tests.

[Green-Sky]

but we could consider running the CI tests for cuBLAS only when ggml-cuda.h/cu have been modified.

afaik we cant run cuda at all in the ci. 🙂
its the install of the cuda toolkit that takes for ever.

[slaren]

Yeah of course, I don't know what I was thinking. Checking that the build completes on Windows can still be useful, usually I do my testing under Linux/WSL2. It may also help find issues with the CUDA 11.7 build.

[Green-Sky]

My plan was to do the linux side next and provide ubuntu focal based binaries. :)

[Green-Sky]

let's see if someone complains, doing linux next.

[Green-Sky]

should have done linux first, it's an order of magnitude faster...
https://github.com/Green-Sky/llama.cpp/releases/tag/ci_cublas_linux-f1758f6