Streaming with whisperx

[evanbrociner]

Is there a repo or code that allows for real-time streaming with whisperx? Thank you!

[Arche151]

Whisper itself can't stream, so I dont think so, unfortunately

[fgn]

This issue from whisper.cpp comes to mind Support for realtime audio input .
It highlights some issues with doing realtime transcription with whisper in general.

[andupotorac]

Would be great to add support for streaming, because folks have been using Whisper.cpp successfully and implemented streaming - for example gladia.io.

[chengxs1994]

This submission will allow for segmented streaming using Whisperx, improving client response time, but not for underlying streaming implementations.

Pull request here

[SalomonKisters]

So this can now be used to transcribe live from an audiostream like mic input? If yes, do you maybe know how?

[roysG]

any update?

[ararTP]

Whisper itself can't stream, so I dont think so, unfortunately

I assume it is possible, sort of, because whisperx splits the audio to chunks, it can proccess each chunk individually and stream it after finished, instead of waiting for all chunks.

would be nice feature.

[douglasg14b]

This seems like an ideal project for streaming audio given how it splits audio into chunks. Which could be pretty handy for voice assistants.

Similar projects:

• https://github.com/ufal/whisper_streaming
• https://github.com/davabase/whisper_real_time
• https://github.com/collabora/WhisperLive

One problem with snippets is context loss, which some of these try and solve for in various ways.