merging main to asr_normalize #6676
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#6549) (#6553) * Added fastconfomer hybrid asr models for en, es, it, de, pl, hr, ua, by * updated ASR docs with the fastconformer hybrid checkpoints * added the fastconformer RNNT and CTC models --------- Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> Co-authored-by: Kunal Dhawan <kunaldhawan97@gmail.com>
…) (#6559) Signed-off-by: fayejf <fayejf07@gmail.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>
Signed-off-by: smajumdar <titu1994@gmail.com>
* do_lowercase, rm_punctuation Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * support beam_strategy = beam Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * black Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix config and^Cunctuation capitalization Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rm math Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * update kenlm Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * black Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add opengrm Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * mv install_beamsearch_decoders Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * punctuation_to_preserve Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Only tikenizer opion Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * Black Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * DEFAULT_TOKEN_OFFSET Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * aggregate_tokenizer Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install kenlm with more than 5gram Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * install_beamsearch_decoders Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * ngram_bin_path kenlm_bin_path Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * black Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * fix greedy PC bug Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * move global params Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * fix description and perplexity Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * fix description Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * NEMO_PATH Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * nemo:23.01 Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * License Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * description Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * isinstance Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * refactor kenlm stdin Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * black Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * add cmd arg Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * use new iter_files Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * EncDecHybridRNNTCTCModel Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * punctuation Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * train_kenlm args Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * add docstrings Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add ngram_merge docs Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * ngram_prune Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rename to ngram_merge Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rename to ngram Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * add comments Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * Ngram Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * nemo_model_file Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * install_opengrm_ngram Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * install opengrm Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rename to install_opengrm.sh Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rm extra import Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * train_paths Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * text_processing Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * fix ngram_bin_path Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * DECODERS_PATH Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * farcompile Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rm text processing Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * text_processing Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * AggregateTokenizer.DummyTokenizer Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * comments Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * TextProcessingConfig Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * typo Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * doc Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * types Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * nemo_model_file Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * rm assert Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * import kenlm_utils Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * return None Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * Copyright Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * 2022 Signed-off-by: Nikolay Karpov <karpnv@gmail.com> * 2023 Signed-off-by: Nikolay Karpov <karpnv@gmail.com> --------- Signed-off-by: Nikolay Karpov <nkarpov@nvidia.com> Signed-off-by: Nikolay Karpov <karpnv@gmail.com> Co-authored-by: Nikolay Karpov <nkarpov@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Ryan <rlangman@nvidia.com>
Signed-off-by: Nikolay Karpov <karpnv@gmail.com>
* Update manifest.py Re-order the checks for faster processing audio filepaths that are already absolute paths * Update manifest.py --------- Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com>
) (#6386) * user selection should not break model max limit * eval max seq length --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Adi Renduchintala <108822655+arendu@users.noreply.github.com> Co-authored-by: Adi Renduchintala <108822655+arendu@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Eric Harper <complex451@gmail.com>
* init commit ptuning via mixin Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updates Signed-off-by: arendu <adithya.r@gmail.com> * gpt ptuning places virtual tokens on the left only Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * encoder input modified when pre_process is true Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * optimizer group and state dict updates Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * adapter ptuning working for pp>1 Signed-off-by: arendu <adithya.r@gmail.com> * adapter defaults Signed-off-by: arendu <adithya.r@gmail.com> * adapter ptuining config defaults Signed-off-by: arendu <adithya.r@gmail.com> * training works Signed-off-by: arendu <adithya.r@gmail.com> * loading and saving adapter only params during training Signed-off-by: arendu <adithya.r@gmail.com> * added checks and comments Signed-off-by: arendu <adithya.r@gmail.com> * clean up Signed-off-by: arendu <adithya.r@gmail.com> * checks for grad is None before calling all_reduce Signed-off-by: arendu <adithya.r@gmail.com> * load adapter .nemo file working Signed-off-by: arendu <adithya.r@gmail.com> * resume training for adapters Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * peft tuning Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor Signed-off-by: arendu <adithya.r@gmail.com> * file not needed Signed-off-by: arendu <adithya.r@gmail.com> * undo prompt learning dataset changes Signed-off-by: arendu <adithya.r@gmail.com> * undo updates to gpt prompt learning model Signed-off-by: arendu <adithya.r@gmail.com> * naming updates Signed-off-by: arendu <adithya.r@gmail.com> * decoding Signed-off-by: arendu <adithya.r@gmail.com> * predict_step in gpt_sft_model Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updates Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed inference from tuning config Signed-off-by: arendu <adithya.r@gmail.com> * no test in peft training Signed-off-by: arendu <adithya.r@gmail.com> * answer only loss and correct defaults for val_loss Signed-off-by: arendu <adithya.r@gmail.com> * hybrid adapters and ptuning Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * eval working.. Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * prepending tokens for ptuning Signed-off-by: arendu <adithya.r@gmail.com> * cleaned up eval config Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * clean up Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * default prompt template Signed-off-by: arendu <adithya.r@gmail.com> * Lora added Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support synamic length with GPT SFT Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * make branch functional Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * defaults to max_pad_length=False in GPT SFT dataset Signed-off-by: arendu <adithya.r@gmail.com> * adapter parallel_adapters to support Lora Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added early stopping by default Signed-off-by: arendu <adithya.r@gmail.com> * eval script for peft and eval config. bug fixes in predict step and added out_features to t5 adapter config Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updates Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updates Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * docs Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * better defaults Signed-off-by: arendu <adithya.r@gmail.com> * updates Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> * docs Signed-off-by: arendu <adithya.r@gmail.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: Adi Renduchintala <108822655+arendu@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com>
* Add patches for Virtual Parllel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* pass .scale instead of scaler object to core (#6545) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * scale changes for main Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com>
* Add docs about hybrid ASR-TTS models * Add docs about text-only datasets * Add docs about ASR-TTS checkpoints * Add docs about ASR-TTS configs and training * Clean up * ASR-TTS docs: add to api, fix imports * Clean up * Wrap optional import * Revert general ASR import --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com>
* Fix nan loss in fp32 Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* add info about SDP e.g. processor classes in docs * add link to SDP docs in README * address code review comments and add SDP overview diagram * Fix spelling typo --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com>
Signed-off-by: Igor Gitman <igitman@nvidia.com>
* Manual garbage collection with an interval * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * use trainer.global_step for tracking the interval of GC --------- Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com>
* fixes Signed-off-by: sam1373 <samuelkriman@gmail.com> * done already Signed-off-by: sam1373 <samuelkriman@gmail.com> --------- Signed-off-by: sam1373 <samuelkriman@gmail.com>
* Update manifest.py to use os.path for get_full_path Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py to get rid of pathlib Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update manifest.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com>
* fix write Signed-off-by: fayejf <fayejf07@gmail.com> * decoding ctc Signed-off-by: fayejf <fayejf07@gmail.com> * temp set rnnt decoding return_best_hypothesis to true Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to transcribe_speech as requested Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to speech_to_text_buffered_infer_rnnt as requested Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to speech_to_text_buffered_infer_ctc as requested Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * reflect change in asr_evaluator Signed-off-by: fayejf <fayejf07@gmail.com> * reflect som and vahid comment Signed-off-by: fayejf <fayejf07@gmail.com> * remove return_best_hy=true in transcribe_speech Signed-off-by: fayejf <fayejf07@gmail.com> * no text skip Signed-off-by: fayejf <fayejf07@gmail.com> * revert partial Signed-off-by: fayejf <fayejf07@gmail.com> --------- Signed-off-by: fayejf <fayejf07@gmail.com>
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
* Not doing CastToFloat by default * Added docustring * Dummy commit --------- Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com>
* add megatron_core to requirements * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <complex451@gmail.com> * remove from jenkins Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove dup Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* sft model can use this script for eval Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * please fix me Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor Signed-off-by: arendu <adithya.r@gmail.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Ryan <rlangman@nvidia.com>
Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
* ASR-TTS: support hybrid RNNT-CTC models * Do not warn on optional import * Explain adding options to config * Fix import guard docs * Add docs for ConcatDataset * Add explanation for sampling parameters * Initial docs for the enhancer model * Fix use_start_end_token parameter usage --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
* fix conversion and eval Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Working version to train conf model + save ensemble class Signed-off-by: Igor Gitman <igitman@nvidia.com> * Working version Signed-off-by: Igor Gitman <igitman@nvidia.com> * Remove copy of transcribe_speech.py Signed-off-by: Igor Gitman <igitman@nvidia.com> * Move models parameter to config Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add explicit parameters to transcribe Signed-off-by: Igor Gitman <igitman@nvidia.com> * Small cleanups Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add temperature and integration tests Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add more tests Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add pc removal config Signed-off-by: Igor Gitman <igitman@nvidia.com> * Cleanup Signed-off-by: Igor Gitman <igitman@nvidia.com> * Fix typo Signed-off-by: Igor Gitman <igitman@nvidia.com> * Address review comments Signed-off-by: Igor Gitman <igitman@nvidia.com> --------- Signed-off-by: Igor Gitman <igitman@nvidia.com>
* Patch memory used for NeMo Megatron models Signed-off-by: smajumdar <titu1994@gmail.com> * Cleanup the dtype of embeddings Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refactor util function for parsing precision Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refactor util function for parsing precision Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Try patch for Megatron O2 Signed-off-by: smajumdar <titu1994@gmail.com> * Refactor to incorporate megatron amp 02 state Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refactor to incorporate megatron amp 02 state Signed-off-by: smajumdar <titu1994@gmail.com> * Correct indent Signed-off-by: smajumdar <titu1994@gmail.com> * Correct utils import Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: arendu <adithya.r@gmail.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>
* merge lora weights into base model Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * typo fix Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor update Signed-off-by: arendu <adithya.r@gmail.com> * update copyright Signed-off-by: arendu <adithya.r@gmail.com> * eval needs to know the PEFT class Signed-off-by: arendu <adithya.r@gmail.com> * add target class in training script so that we can use it in eval Signed-off-by: arendu <adithya.r@gmail.com> * update Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update to work for tp1 Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set restore model path Signed-off-by: arendu <adithya.r@gmail.com> * peft can be none Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated merge script so that eval works easily Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * eval with peft or sft model Signed-off-by: arendu <adithya.r@gmail.com> * keep sentences in jsonl format Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * convert sft using correct classpath Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated to force sft yaml to have the correct target Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated docs Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix conversion and eval Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: ericharper <complex451@gmail.com>
* update branch Signed-off-by: ericharper <complex451@gmail.com> * Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <complex451@gmail.com> * remove from jenkins Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> * remove dup Signed-off-by: ericharper <complex451@gmail.com> * update branch Signed-off-by: ericharper <complex451@gmail.com> * [TTS] reformat NeMo versions in the tts logging messages to avoid batch process them when upgrading NeMo versions. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> --------- Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
* Implement compute confidence to properly handle blanks Signed-off-by: Igor Gitman <igitman@nvidia.com> * Implement proper confidence for transducers Signed-off-by: Igor Gitman <igitman@nvidia.com> * Implement tuning logic Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add tests for confidence tuning Signed-off-by: Igor Gitman <igitman@nvidia.com> * Remove unused imports Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add types/docs Signed-off-by: Igor Gitman <igitman@nvidia.com> * Add comment about the main conf compute loop Signed-off-by: Igor Gitman <igitman@nvidia.com> --------- Signed-off-by: Igor Gitman <igitman@nvidia.com>
* [TTS] Implement new TextToSpeech dataset Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add unit tests Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Fix defaulting of use_log_energy Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Fix TTS export test Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com>
* chatbot interface Signed-off-by: Yi Dong <yidong@nvidia.com> * latest gradio Signed-off-by: Yi Dong <yidong@nvidia.com> * default greedy Signed-off-by: Yi Dong <yidong@nvidia.com> * better chatbot Signed-off-by: Yi Dong <yidong@nvidia.com> * handle preamble Signed-off-by: Yi Dong <yidong@nvidia.com> * added chatbot training capablity Signed-off-by: Yi Dong <yidong@nvidia.com> * added chatbot ui Signed-off-by: Yi Dong <yidong@nvidia.com> * remove debug code Signed-off-by: Yi Dong <yidong@nvidia.com> * default human Signed-off-by: Yi Dong <yidong@nvidia.com> * use special token for roles Signed-off-by: Yi Dong <yidong@nvidia.com> * special tokens Signed-off-by: Yi Dong <yidong@nvidia.com> * fix name Signed-off-by: Yi Dong <yidong@nvidia.com> * new chat dataset Signed-off-by: Yi Dong <yidong@nvidia.com> * fix the system token Signed-off-by: Yi Dong <yidong@nvidia.com> * upgrade gradio Signed-off-by: Yi Dong <yidong@nvidia.com> * save the chat history Signed-off-by: Yi Dong <yidong@nvidia.com> * update ui Signed-off-by: root <you@example.com> * update chat interface Signed-off-by: Yi Dong <yidong@nvidia.com> * handles canonical form Signed-off-by: Yi Dong <yidong@nvidia.com> * new sft chatbot Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change format Signed-off-by: Yi Dong <yidong@nvidia.com> * check extra_id in the tokenizer Signed-off-by: Yi Dong <yidong@nvidia.com> * added vocab property check Signed-off-by: Yi Dong <yidong@nvidia.com> * added missing file Signed-off-by: Yi Dong <yidong@nvidia.com> --------- Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: root <you@example.com> Co-authored-by: root <you@example.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>
* Add support for RNNT/hybrid models to partial transcribe Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update transcribe_utils.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update transcribe_speech.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update transcribe_utils.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* separate_punctuation = false * ctc decoding strategy = model.decoding * transcribe(files, logprobs=True) returns logprobs --------- Signed-off-by: Nikolay Karpov <karpnv@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok>
Signed-off-by: arendu <adithya.r@gmail.com>
* [TTS] Add callback for saving audio during FastPitch training Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Allow NGC model name for vocoder Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| if path[-8:] == '.json.gz': # for Common Crawl dataset | ||
| fin = gzip.open(path, 'r') | ||
| else: | ||
| fin = open(path, 'r', encoding='utf-8') |
Check warning
Code scanning / CodeQL
File is not always closed
| from pytorch_lightning import Trainer | ||
| from pytorch_lightning.plugins.environments import TorchElasticEnvironment | ||
| from pytorch_lightning.trainer.connectors.checkpoint_connector import CheckpointConnector | ||
| from torch.utils.data import DataLoader, Dataset |
Check notice
Code scanning / CodeQL
Unused import
| @@ -32,6 +32,7 @@ | |||
| import torch | |||
| from pytorch_lightning import Trainer | |||
|
|
|||
| import nemo | |||
Check notice
Code scanning / CodeQL
Unused import
| from nemo.utils.model_utils import inject_model_parallel_rank | ||
|
|
||
| try: | ||
| from megatron.core import parallel_state |
Check notice
Code scanning / CodeQL
Unused import
| try: | ||
| from megatron.core import parallel_state | ||
|
|
||
| HAVE_MEGATRON_CORE = True |
Check notice
Code scanning / CodeQL
Unused global variable
|
|
||
| except (ImportError, ModuleNotFoundError): | ||
|
|
||
| HAVE_MEGATRON_CORE = False |
Check notice
Code scanning / CodeQL
Unused global variable
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Collection: [Note which collection this PR will affect]
Changelog
Usage
# Add a code snippet demonstrating how to use thisBefore your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information