Instruction: Partial OpenCL GPU support via CLBlast #863
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9. Usage: WHISPER_CLBLAST=1 make
Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9. Usage: ``` Makefile: cd whisper.cpp WHISPER_CLBLAST=1 make CMake: cd whisper.cpp ; mkdir build ; cd build cmake -DWHISPER_CLBLAST=ON .. make ```
Added OpenCL Build Instructions
Added build instructions and examples for Make and CMake to support OpenCL enabled GPUs.
|
Does it support OpenCL on MacOS? https://developer.apple.com/opencl/ |
It uses CLBlast for OpenCL. CLBlast supports Apple OpenCL. CLBlast needs to be installed before building whisper.cpp. Try:
See details here: https://github.com/CNugteren/CLBlast/blob/master/doc/installation.md |
|
Unfortunately it doesn't build: user@pc ~/Documents/Code/whisper.cpp (git)-[master] % git pull
Already up to date.
user@pc ~/Documents/Code/whisper.cpp (git)-[master] % brew install clblast
==> Downloading https://formulae.brew.sh/api/formula.jws.json
#=#=#
==> Downloading https://formulae.brew.sh/api/cask.jws.json
#=#=#
==> Fetching clblast
==> Downloading https://ghcr.io/v2/homebrew/core/clblast/blobs/sha256:1129b3155dca
Already downloaded: /Users/user/Library/Caches/Homebrew/downloads/2a2c4fcfd2e4834fe27efd3802df3b4b6cf67522658c4c114e78d01e26f20a06--clblast--1.5.3_1.ventura.bottle.tar.gz
==> Pouring clblast--1.5.3_1.ventura.bottle.tar.gz
🍺 /usr/local/Cellar/clblast/1.5.3_1: 41 files, 11.6MB
==> Running `brew cleanup clblast`...
brew install clblast 5.00s user 1.86s system 74% cpu 9.202 total
user@pc ~/Documents/Code/whisper.cpp (git)-[master] % make clean
sysctl: unknown oid 'hw.optional.arm64'
I whisper.cpp build info:
I UNAME_S: Darwin
I UNAME_P: i386
I UNAME_M: x86_64
I CFLAGS: -I. -O3 -DNDEBUG -std=c11 -fPIC -pthread -mf16c -mfma -mavx -mavx2 -DGGML_USE_ACCELERATE
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread
I LDFLAGS: -framework Accelerate
I CC: Apple clang version 14.0.3 (clang-1403.0.22.14.1)
I CXX: Apple clang version 14.0.3 (clang-1403.0.22.14.1)
rm -f *.o main stream command talk talk-llama bench quantize libwhisper.a libwhisper.so
user@pc ~/Documents/Code/whisper.cpp (git)-[master] % WHISPER_CLBLAST=1 make -j
sysctl: unknown oid 'hw.optional.arm64'
I whisper.cpp build info:
I UNAME_S: Darwin
I UNAME_P: i386
I UNAME_M: x86_64
I CFLAGS: -I. -O3 -DNDEBUG -std=c11 -fPIC -pthread -mf16c -mfma -mavx -mavx2 -DGGML_USE_ACCELERATE -DGGML_USE_CLBLAST
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread
I LDFLAGS: -framework Accelerate -lclblast -lOpenCL
I CC: Apple clang version 14.0.3 (clang-1403.0.22.14.1)
I CXX: Apple clang version 14.0.3 (clang-1403.0.22.14.1)
cc -I. -O3 -DNDEBUG -std=c11 -fPIC -pthread -mf16c -mfma -mavx -mavx2 -DGGML_USE_ACCELERATE -DGGML_USE_CLBLAST -c ggml.c -o ggml.o
cc -I. -O3 -DNDEBUG -std=c11 -fPIC -pthread -mf16c -mfma -mavx -mavx2 -DGGML_USE_ACCELERATE -DGGML_USE_CLBLAST -c ggml-opencl.c -o ggml-opencl.o
c++ -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -pthread -c whisper.cpp -o whisper.o
ggml.c:4406:9: error: call to undeclared function 'ggml_cl_init'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
ggml_cl_init();
^
ggml.c:4406:9: note: did you mean 'ggml_init'?
ggml.c:4357:23: note: 'ggml_init' declared here
struct ggml_context * ggml_init(struct ggml_init_params params) {
^
ggml.c:8319:17: error: call to undeclared function 'ggml_cl_sgemm_wrapper'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8319:39: error: use of undeclared identifier 'GGML_BLAS_ORDER_ROW_MAJOR'
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8319:66: error: use of undeclared identifier 'GGML_BLAS_OP_N'
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8319:82: error: use of undeclared identifier 'GGML_BLAS_OP_T'
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8509:17: error: call to undeclared function 'ggml_cl_sgemm_wrapper'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8509:39: error: use of undeclared identifier 'GGML_BLAS_ORDER_ROW_MAJOR'
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8509:66: error: use of undeclared identifier 'GGML_BLAS_OP_N'
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8509:82: error: use of undeclared identifier 'GGML_BLAS_OP_T'
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8733:17: error: call to undeclared function 'ggml_cl_sgemm_wrapper'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8733:39: error: use of undeclared identifier 'GGML_BLAS_ORDER_ROW_MAJOR'
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8733:66: error: use of undeclared identifier 'GGML_BLAS_OP_N'
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
ggml.c:8733:82: error: use of undeclared identifier 'GGML_BLAS_OP_T'
ggml_cl_sgemm_wrapper(GGML_BLAS_ORDER_ROW_MAJOR, GGML_BLAS_OP_N, GGML_BLAS_OP_T,
^
13 errors generated.
make: *** [ggml.o] Error 1
make: *** Waiting for unfinished jobs....
WHISPER_CLBLAST=1 make -j 11.38s user 1.25s system 118% cpu 10.646 total
2 user@pc ~/Documents/Code/whisper.cpp (git)-[master] % |
|
Can you build the same using CMake and get back with the logs? I have no mac here. I think cblast lib is not being picked up maybe as a result of Also please create a issue for Mac eyeballs. |
jacobwu-b
pushed a commit
to jacobwu-b/Transcriptify-by-whisper.cpp
that referenced
this pull request
Oct 24, 2023
* ggml : CLBlast support as in llama.cpp Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9. Usage: WHISPER_CLBLAST=1 make * CMake/Makefile : CLBlast support as in llama.cpp Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9. Usage: ``` Makefile: cd whisper.cpp WHISPER_CLBLAST=1 make CMake: cd whisper.cpp ; mkdir build ; cd build cmake -DWHISPER_CLBLAST=ON .. make ``` * Update README.md Added OpenCL Build Instructions * Instruction: Partial OpenCL GPU support via CLBlast Added build instructions and examples for Make and CMake to support OpenCL enabled GPUs.
jacobwu-b
pushed a commit
to jacobwu-b/Transcriptify-by-whisper.cpp
that referenced
this pull request
Oct 24, 2023
* ggml : CLBlast support as in llama.cpp Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9. Usage: WHISPER_CLBLAST=1 make * CMake/Makefile : CLBlast support as in llama.cpp Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9. Usage: ``` Makefile: cd whisper.cpp WHISPER_CLBLAST=1 make CMake: cd whisper.cpp ; mkdir build ; cd build cmake -DWHISPER_CLBLAST=ON .. make ``` * Update README.md Added OpenCL Build Instructions * Instruction: Partial OpenCL GPU support via CLBlast Added build instructions and examples for Make and CMake to support OpenCL enabled GPUs.
landtanin
pushed a commit
to landtanin/whisper.cpp
that referenced
this pull request
Dec 16, 2023
* ggml : CLBlast support as in llama.cpp Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9. Usage: WHISPER_CLBLAST=1 make * CMake/Makefile : CLBlast support as in llama.cpp Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9. Usage: ``` Makefile: cd whisper.cpp WHISPER_CLBLAST=1 make CMake: cd whisper.cpp ; mkdir build ; cd build cmake -DWHISPER_CLBLAST=ON .. make ``` * Update README.md Added OpenCL Build Instructions * Instruction: Partial OpenCL GPU support via CLBlast Added build instructions and examples for Make and CMake to support OpenCL enabled GPUs.
iThalay
pushed a commit
to iThalay/whisper.cpp
that referenced
this pull request
Sep 23, 2024
* ggml : CLBlast support as in llama.cpp Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9. Usage: WHISPER_CLBLAST=1 make * CMake/Makefile : CLBlast support as in llama.cpp Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9. Usage: ``` Makefile: cd whisper.cpp WHISPER_CLBLAST=1 make CMake: cd whisper.cpp ; mkdir build ; cd build cmake -DWHISPER_CLBLAST=ON .. make ``` * Update README.md Added OpenCL Build Instructions * Instruction: Partial OpenCL GPU support via CLBlast Added build instructions and examples for Make and CMake to support OpenCL enabled GPUs.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added build instructions and examples for Make as well as CMake to support OpenCL enabled GPUs.