Thoughts on using an Integer model?

[Digipom]

There's another Android port using TensorFlow Lite by @nyadla-sys (openai/whisper#506), which runs about twice as fast as whisper.cpp on my device using the tiny model while still using the CPU. My thoughts are the speed difference might be due to:

• Model size is around 40MB instead of 77MB GGML version. I believe the data is quantized to integers.
• The model is running using XNNPACK (https://github.com/google/XNNPACK) which might be more optimized.

Using TensorFlow lite or working with the models directly is currently beyond me, so I thought I'd just open the discussion for tracking purposes.

[rjpikus]

Accuracy comparable? I'm running whisper, but model not loading on android 10 galaxy s9

[PauGa9]

Hi guys! Maybe now we can try to apply the bitnet quantization for whisper.cpp?