Flashlight and Pyctcdecode decoders #8428

karpnv · 2024-02-15T07:16:06Z

Preserve Flashlight and Pyctcdecode beamsearch with Ngram LM

Support Flashlight and Pyctcdecode decoding with pure KenLM and NeMo KenLM
Standardize API of CLI inference scripts

Collection: ASR

Changelog

Fix install script install_beamsearch_decoders.sh
Create flashlight_lexicon file during scripts/asr_language_modeling/ngram_lm/train_kenlm.py and tar it with kenlm.bin
Unify parameters for eval_beamsearch_ngram_ctc.py, speech_to_text_eval.py and training
-- Get logprobs from Hypothesis
-- Use "pyctcdecode" strategy as default beamsearch algorithm denoted as "beam"
-- Remove default seq2seq strategy
-- Check decoding_type and search_type combinations
-- Support empty string in nemo_kenlm_path and word_kenlm_path for beamsearch without LM (ZeroLM)
Fix bug with EncDecHybridRNNTCTCModel in examples/asr/transcribe_speech.py
Support AggregateTokenizer in scripts/asr_language_modeling/ngram_lm/create_lexicon_from_arpa.py

python3 scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram_ctc.py \
model_path=am_model.nemo  \
dataset_manifest=manifest.json  \
preds_output_folder=/tmp   \
ctc_decoding.strategy=flashlight \
ctc_decoding.beam.kenlm_path=am_model.kenlm \
ctc_decoding.beam.beam_size=[4]   \
ctc_decoding.beam.beam_alpha=[0.5]   \
ctc_decoding.beam.beam_beta=[0.5] \
batch_size=32  \
beam_batch_size=1 \
cuda=1

python3 examples/asr/speech_to_text_eval.py  \
model_path=am_model.nemo \ 
dataset_manifest=manifest.json \
decoder_type=ctc  
ctc_decoding.strategy=flashlight \  
ctc_decoding.beam.nemo_kenlm_path=kenlm_model.bin \
ctc_decoding.beam.beam_size=4   \
ctc_decoding.beam.beam_alpha=0.5   \
ctc_decoding.beam.beam_beta=0.5 \
ctc_decoding.beam.flashlight_cfg.lexicon_path=am_model.flashlight_lexicon \ # DEFAULT_TOKEN_OFFSET
ctc_decoding.beam.return_best_hypothesis=true \
batch_size=32  \
output_filename=/tmp/manifest_out.json 
cuda=1

PR Type:

[ V] New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Additional Information

Related to "FileNotFoundError: KenLM binary file not found at : None" thrown when decoding without N-gram LM #9067

Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com>

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

for more information, see https://pre-commit.ci

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

…v/beamsearch

for more information, see https://pre-commit.ci

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

github-actions · 2024-03-01T01:45:10Z

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions · 2024-03-09T01:42:20Z

This PR was closed because it has been inactive for 7 days since being marked as stale.

github-actions · 2024-09-10T01:56:08Z

This PR was closed because it has been inactive for 7 days since being marked as stale.

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram_transducer.py

+        if cfg.amp:
+            if torch.cuda.is_available() and hasattr(torch.cuda, 'amp') and hasattr(torch.cuda.amp, 'autocast'):
+                logging.info("AMP is enabled!\n")
+                autocast = torch.cuda.amp.autocast


scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram_transducer.py

+                autocast = torch.cuda.amp.autocast
+
+            else:
+                autocast = default_autocast


scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram_transducer.py

+                autocast = default_autocast
+        else:
+
+            autocast = default_autocast


github-actions · 2024-10-01T16:12:24Z

[🤖]: Hi @karpnv 👋,

I just wanted to let you know that, you know, a CICD pipeline for this PR just finished successfully ✨

So it might be time to merge this PR or like to get some approvals 🚀

But I'm just a 🤖 so I'll leave it you what to do next.

Have a great day!

//cc @ko3n1g

scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram_transducer.py

artbataev

Looks it is worth merging now.
@karpov-nick please, fix autocast/use_amp in scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram_transducer.py

andrusenkoau

LGTM

scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram_ctc.py

+python eval_beamsearch_ngram_ctc.py model_path=<path to the .nemo file of the model> \
+           dataset_manifest=<path to the input evaluation JSON manifest file> \
+           ctc_decoding.beam.word_kenlm_path=<path to the binary KenLM model> \
+           ctc_decoding.beam.nemo_kenlm_path=<path to the binary KenLM model> \


github-actions · 2024-10-17T01:58:54Z

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

nithinraok · 2024-10-17T16:55:58Z

can we merge this?

github-actions · 2024-11-01T02:04:15Z

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

tbartley94 · 2024-11-04T18:18:04Z

@karpnv could you fix merge conflicts so this can be merged?

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

nemo/collections/asr/parts/submodules/ctc_beam_decoding.py

+        lexicon_path = os.path.join(tmpdir.name, lexicon[0].name)
+    SaveRestoreConnector._unpack_nemo_file(path2file=kenlm_path, out_folder=tmpdir.name, members=members)
+    cfg = OmegaConf.load(config_path)
+    return tmpdir, cfg.encoding_level, kenlm_model_path, lexicon_path


nemo/collections/asr/parts/submodules/ctc_beam_decoding.py

+            try:
+                self.tmpdir, self.kenlm_encoding_level, self.kenlm_path, lexicon_path = get_nemolm(kenlm_path)
+                if not self.flashlight_cfg.lexicon_path:
+                    self.flashlight_cfg.lexicon_path = lexicon_path


karpnv and others added 25 commits January 24, 2024 00:26

install fix

38b1050

Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com>

Merge branch 'main' of github.com:NVIDIA/NeMo into karpnv/beamsearch

ecfaec7

input_manifest

711ba5a

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

output_manifest

7af6bdf

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

output_manifest

4e7d95e

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

nemo_model_file

44835c5

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

rm search_type

5f3dddc

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

rm decoding_modes

50e3087

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

caabdcf

for more information, see https://pre-commit.ci

token offset

54d5236

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

Merge branch 'karpnv/beamsearch' of github.com:NVIDIA/NeMo into karpn…

8bbb1e9

…v/beamsearch

[pre-commit.ci] auto fixes from pre-commit.com hooks

9f205f1

for more information, see https://pre-commit.ci

black

be54cc2

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

black

00f072e

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

Merge branch 'main' of github.com:NVIDIA/NeMo into karpnv/beamsearch

a87e125

black

ba9f767

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

Merge branch 'main' of github.com:NVIDIA/NeMo into karpnv/beamsearch

3f5ceab

black

adacdcd

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

batch_size

42920e3

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

merge

22ebd6e

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

black

b85ac3c

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

cfg.acoustic_batch_size

f803716

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

cfg.amp

1a2f3f4

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

pyctcdecode_cfg

06c7aae

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

Merge branch 'main' of github.com:NVIDIA/NeMo into karpnv/beamsearch

0d2afaf

github-actions bot added the ASR label Feb 15, 2024

github-actions bot added the stale label Mar 1, 2024

github-actions bot closed this Mar 9, 2024

github-actions bot added the stale label Sep 3, 2024

github-actions bot closed this Sep 10, 2024

karpnv reopened this Sep 16, 2024

github-actions bot removed the stale label Sep 17, 2024

karpnv added 2 commits September 30, 2024 05:30

return cfg

aefce97

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

resolve conflicts

db81651

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

github-advanced-security bot found potential problems Sep 30, 2024

View reviewed changes

karpnv added Run CICD and removed Run CICD labels Oct 1, 2024

artbataev reviewed Oct 2, 2024

View reviewed changes

scripts/asr_language_modeling/ngram_lm/eval_beamsearch_ngram_transducer.py Show resolved Hide resolved

artbataev previously approved these changes Oct 2, 2024

View reviewed changes

andrusenkoau previously approved these changes Oct 2, 2024

View reviewed changes

github-actions bot added the stale label Oct 17, 2024

nithinraok added Run CICD and removed Run CICD labels Oct 17, 2024

github-actions bot removed the stale label Oct 18, 2024

github-actions bot added the stale label Nov 1, 2024

tbartley94 removed the stale label Nov 4, 2024

fix docs

4f4212c

Signed-off-by: Nikolay Karpov <karpnv@gmail.com>

karpnv dismissed stale reviews from andrusenkoau and artbataev via 4f4212c November 7, 2024 14:24

karpnv added Run CICD and removed Run CICD labels Nov 7, 2024

github-advanced-security bot found potential problems Nov 7, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flashlight and Pyctcdecode decoders #8428

Flashlight and Pyctcdecode decoders #8428

karpnv commented Feb 15, 2024 •

edited

Loading

github-actions bot commented Mar 1, 2024

github-actions bot commented Mar 9, 2024

github-actions bot commented Sep 10, 2024

github-actions bot commented Oct 1, 2024

artbataev left a comment

andrusenkoau left a comment

This comment was marked as outdated.

github-actions bot commented Oct 17, 2024

nithinraok commented Oct 17, 2024

github-actions bot commented Nov 1, 2024

tbartley94 commented Nov 4, 2024

Flashlight and Pyctcdecode decoders #8428

Are you sure you want to change the base?

Flashlight and Pyctcdecode decoders #8428

Conversation

karpnv commented Feb 15, 2024 • edited Loading

Preserve Flashlight and Pyctcdecode beamsearch with Ngram LM

Changelog

Who can review?

Additional Information

github-actions bot commented Mar 1, 2024

github-actions bot commented Mar 9, 2024

github-actions bot commented Sep 10, 2024

github-actions bot commented Oct 1, 2024

artbataev left a comment

Choose a reason for hiding this comment

andrusenkoau left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

github-actions bot commented Oct 17, 2024

nithinraok commented Oct 17, 2024

github-actions bot commented Nov 1, 2024

tbartley94 commented Nov 4, 2024

karpnv commented Feb 15, 2024 •

edited

Loading