From 28086b52de7ae8452594e16f64f711efdad462c5 Mon Sep 17 00:00:00 2001 From: Carsten Seeger Date: Tue, 9 May 2023 10:14:22 +0200 Subject: [PATCH 1/2] fix: missing CLBLAS documentation --- README.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/README.md b/README.md index 045f995347154..4eb71c901a537 100644 --- a/README.md +++ b/README.md @@ -299,6 +299,22 @@ Building the program with BLAS support may lead to some performance improvements cmake --build . --config Release ``` +- clBLAS + + This provides BLAS acceleration using the CUDA cores of your GPU. Make sure to have the cblas installed. + - Using `make`: + ```bash + make LLAMA_CLBLAS=1 + ``` + - Using `CMake`: + + ```bash + mkdir build + cd build + cmake .. -DLLAMA_CLBLAS=ON + cmake --build . --config Release + ``` + Note: Because llama.cpp uses multiple CUDA streams for matrix multiplication results [are not guaranteed to be reproducible](https://docs.nvidia.com/cuda/cublas/index.html#results-reproducibility). If you need reproducibility, set `GGML_CUDA_MAX_STREAMS` in the file `ggml-cuda.cu` to 1. ### Prepare Data & Run From 7cea31c568d88acda578c88f8ecaa9088cc45df8 Mon Sep 17 00:00:00 2001 From: Carsten Seeger Date: Tue, 9 May 2023 11:58:00 +0200 Subject: [PATCH 2/2] fix: typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 4eb71c901a537..e398291e7897e 100644 --- a/README.md +++ b/README.md @@ -301,7 +301,7 @@ Building the program with BLAS support may lead to some performance improvements - clBLAS - This provides BLAS acceleration using the CUDA cores of your GPU. Make sure to have the cblas installed. + This provides BLAS acceleration using the CUDA cores of your GPU. Make sure to have the clblast installed. - Using `make`: ```bash make LLAMA_CLBLAS=1