Refactor pipeline_demo.py to support variant EMFORMER_RNNT bundles #2203
Conversation
|
@nateanl has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
There was a problem hiding this comment.
The script looks okay, but code-wise what is the difference from the equivalent one from librispeech_emformer_rnnt?
It's actually the same, only the pipeline-related part is different. |
nateanl
force-pushed
the
rnnt_tedlium_pipeline
branch
from
ec63c19 to
6393298
Compare
nateanl
force-pushed
the
rnnt_tedlium_pipeline
branch
from
6393298 to
e53b7c5
Compare
nateanl
changed the title
I guess it's because it's run on AWS cluster, there may be some delay when printing to the screen. |
in that case, for the screen capture, can you run the script locally? that way, we can clearly show users what we mean by streaming asr and how responsive it is using the bundles (example: #2192) |
|
Sure. The demo video has been updated @hwangjeff |
|
@nateanl has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
| logger = logging.getLogger() | ||
|
|
||
|
|
||
| def get_dataset(model_type, dataset_path): |
There was a problem hiding this comment.
Instead of repeating the key validation here and there, but these options in a dictionary, sthen pass the keys to choices arguments of argpsrse. So that the closed set of available options are defined once and only once.
|
|
||
|
|
||
| def parse_args(): | ||
| parser = ArgumentParser() |
There was a problem hiding this comment.
Please add module level docstring that describes the gist of this script, then pass it to the help description so that it's easy to see what this script does.
| @@ -0,0 +1,94 @@ | |||
| import logging | |||
…ytorch#2203) Summary: We refactored the demo script that can apply RNNT decoding using both `torchaudio.pipelines.EMFORMER_RNNT_BASE_LIBRISPEECH` and `torchaudio.prototype.pipelines.EMFORMER_RNNT_BASE_TEDLIUM3` in both streaming and non-streaming mode. (The first hypothesis prediction is streaming and the second one is non-streaming). We convert each token id sequence to word pieces and then manually join the word pieces. This allows us to preserve leading whitespaces on output strings and therefore account for word breaks and continuations across token processor invocations, which is particularly useful when performing streaming ASR. https://user-images.githubusercontent.com/8653221/153627956-f0806f18-3c1c-44df-ac07-ec2def58a0cf.mov Pull Request resolved: pytorch#2203 Reviewed By: carolineechen Differential Revision: D34006388 Pulled By: nateanl fbshipit-source-id: 3d31173ee10cdab8a2f5802570e22b50fcce5632
We refactored the demo script that can apply RNNT decoding using both
torchaudio.pipelines.EMFORMER_RNNT_BASE_LIBRISPEECHandtorchaudio.prototype.pipelines.EMFORMER_RNNT_BASE_TEDLIUM3in both streaming and non-streaming mode. (The first hypothesis prediction is streaming and the second one is non-streaming).We convert each token id sequence to word pieces and then manually join the word pieces. This allows us to preserve leading whitespaces on output strings and therefore account for word breaks and continuations across token processor invocations, which is particularly useful when performing streaming ASR.
demo.mov