Skip to content

Commit

Permalink
cann : add Ascend NPU instructions (ggerganov#2410)
Browse files Browse the repository at this point in the history
  • Loading branch information
MengqingCao authored and liuyang.marshall committed Nov 2, 2024
1 parent 8be77bf commit a7bc427
Showing 1 changed file with 34 additions and 0 deletions.
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ High-performance inference of [OpenAI's Whisper](https://github.com/openai/whisp
- Support for CPU-only inference
- [Efficient GPU support for NVIDIA](https://github.com/ggerganov/whisper.cpp#nvidia-gpu-support-via-cublas)
- [OpenVINO Support](https://github.com/ggerganov/whisper.cpp#openvino-support)
- [Ascend NPU Support](https://github.com/ggerganov/whisper.cpp#ascend-npu-support)
- [C-style API](https://github.com/ggerganov/whisper.cpp/blob/master/include/whisper.h)

Supported platforms:
Expand Down Expand Up @@ -448,6 +449,39 @@ cmake -DWHISPER_MKL=ON ..
WHISPER_MKL=1 make -j
```

## Ascend NPU support

Ascend NPU provides inference acceleration via [`CANN`](https://www.hiascend.com/en/software/cann) and AI cores.

First, check if your Ascend NPU device is supported:

**Verified devices**
| Ascend NPU | Status |
|:-----------------------------:|:-------:|
| Atlas 300T A2 | Support |

Then, make sure you have installed [`CANN toolkit`](https://www.hiascend.com/en/software/cann/community) . The lasted version of CANN is recommanded.

Now build `whisper.cpp` with CANN support:

```
mkdir build
cd build
cmake .. -D GGML_CANN=on
make -j
```

Run the inference examples as usual, for example:

```
./build/bin/main -f samples/jfk.wav -m models/ggml-base.en.bin -t 8
```

*Notes:*

- If you have trouble with Ascend NPU device, please create a issue with **[CANN]** prefix/tag.
- If you run successfully with your Ascend NPU device, please help update the table `Verified devices`.

## Docker

### Prerequisites
Expand Down

0 comments on commit a7bc427

Please sign in to comment.