Whisper pipeline: implement chunk streamer for long-form audio processing #1148
+209
−58
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
as-suvorov
added
category: whisper
as-suvorov
changed the title
| class StreamerBase { | ||
| public: | ||
| /// @brief put is called every time new token is decoded, | ||
| /// @return bool flag to indicate whether generation should be stopped, if return true generation stops | ||
| virtual bool put(int64_t token) = 0; | ||
|
|
||
| /// @brief end is called at the end of generation. It can be used to flush cache if your own streamer has one | ||
| virtual void end() = 0; | ||
|
|
||
| virtual ~StreamerBase() = default; |
There was a problem hiding this comment.
it's better to move dtor definition to .cpp file and export this class
it will help with RTTI issue on some platforms
| @@ -50,6 +51,7 @@ Config from_config_json_if_exists(const std::filesystem::path& models_path, cons | |||
| } | |||
|
|
|||
| ov::genai::StreamerVariant get_streamer_from_map(const ov::AnyMap& config_map); | |||
| ov::genai::ChunkStreamerVariant get_chunk_streamer_from_map(const ov::AnyMap& config_map); | |||
There was a problem hiding this comment.
it's whisper specific entity. Maybe we can move it to whisper files? the same in other places like py_utils.hpp
these utils are supposed to be generic ones
| @@ -76,6 +78,28 @@ class ConstructableStreamer: public StreamerBase { | |||
| } | |||
| }; | |||
|
|
|||
| class ConstructableChunkStreamer: public ChunkStreamerBase { | |||
There was a problem hiding this comment.
move to src/python/py_whisper_pipeline.cpp ?
| @@ -105,4 +109,7 @@ class OPENVINO_GENAI_EXPORTS WhisperPipeline { | |||
| WhisperGenerationConfig get_generation_config() const; | |||
| void set_generation_config(const WhisperGenerationConfig& config); | |||
| }; | |||
|
|
|||
| OPENVINO_GENAI_EXPORTS std::pair<std::string, Any> chunk_streamer(ChunkStreamerVariant func); | |||
| OPENVINO_GENAI_EXPORTS std::pair<std::string, Any> whisper_generation_config(const WhisperGenerationConfig& config); | |||
There was a problem hiding this comment.
I think we can overload existing function like streamer and generation_config instead of introducing Whisper specific.
Example:
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.