Update PEFT Doc #8501

cuichenx · 2024-02-25T17:52:39Z

Manually cherry-pick #8262

Signed-off-by: Chen Cui <chcui@nvidia.com>

* update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>

* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com>

* update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com>

* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com>

* update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Zeeshan Patel <zeeshanp@berkeley.edu>

* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Zeeshan Patel <zeeshanp@berkeley.edu>

* update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com>

* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com>

* update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Pablo Garay <pagaray@nvidia.com>

* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Pablo Garay <pagaray@nvidia.com>

* Refactor conversion scripts one in all Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Move bert converter Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * [TTS] Add modules for mel spectrogram codec (#8238) * [TTS] Add modules for mel spectrogram codec Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add mel band validation Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add fullband mel encoder and more documentation Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com> * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> * Account for mpirun use case in get_rank (#8429) Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add settings to suppress bf16 compile errors in CI on V100 (#8481) (#8482) * Add settings to suppress bf16 compile errors in CI on V100 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix canary chunk infer bug (#8449) * fix chunk infer bug Signed-off-by: stevehuang52 <heh@nvidia.com> * add support for duration=None, add lhotse support for relative audio path Signed-off-by: stevehuang52 <heh@nvidia.com> * add tests Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> * Add Baichuan2 support (#8282) * Add Baichuan2 support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reworked MegatronPretrainingRandomBatchSampler to correctly handle epochs > 1 (#7920) * Initital commit of reworked MegatronPretrainingRandomBatchSampler Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed small length based bug Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Euynaheh <hehanyue99@outlook.com> * Add Baichuan2 support Signed-off-by: Euynaheh <hehanyue99@outlook.com> * Add NeMo to HF conversion * fix code format * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix code format * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add Baichuan jenkins test * add_BOS bug fix * Update Jenkinsfile Signed-off-by: Euynaheh <93857693+Euynaheh@users.noreply.github.com> --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Signed-off-by: Euynaheh <hehanyue99@outlook.com> Signed-off-by: Euynaheh <93857693+Euynaheh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: trias702 <25867060+trias702@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana G…

* update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com>

* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com>

* Refactor conversion scripts one in all Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Move bert converter Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * [TTS] Add modules for mel spectrogram codec (#8238) * [TTS] Add modules for mel spectrogram codec Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add mel band validation Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add fullband mel encoder and more documentation Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com> * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> * Account for mpirun use case in get_rank (#8429) Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add settings to suppress bf16 compile errors in CI on V100 (#8481) (#8482) * Add settings to suppress bf16 compile errors in CI on V100 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix canary chunk infer bug (#8449) * fix chunk infer bug Signed-off-by: stevehuang52 <heh@nvidia.com> * add support for duration=None, add lhotse support for relative audio path Signed-off-by: stevehuang52 <heh@nvidia.com> * add tests Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> * Add Baichuan2 support (#8282) * Add Baichuan2 support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reworked MegatronPretrainingRandomBatchSampler to correctly handle epochs > 1 (#7920) * Initital commit of reworked MegatronPretrainingRandomBatchSampler Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed small length based bug Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Euynaheh <hehanyue99@outlook.com> * Add Baichuan2 support Signed-off-by: Euynaheh <hehanyue99@outlook.com> * Add NeMo to HF conversion * fix code format * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix code format * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add Baichuan jenkins test * add_BOS bug fix * Update Jenkinsfile Signed-off-by: Euynaheh <93857693+Euynaheh@users.noreply.github.com> --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Signed-off-by: Euynaheh <hehanyue99@outlook.com> Signed-off-by: Euynaheh <93857693+Euynaheh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: trias702 <25867060+trias702@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana G…

cuichenx added 7 commits February 23, 2024 17:09

update peft doc

3238464

Signed-off-by: Chen Cui <chcui@nvidia.com>

remove old prompt learning doc and notebook

7edc87d

Signed-off-by: Chen Cui <chcui@nvidia.com>

fix table

e8a37c0

Signed-off-by: Chen Cui <chcui@nvidia.com>

fix table

9b6c865

Signed-off-by: Chen Cui <chcui@nvidia.com>

fix table

20dcd5d

Signed-off-by: Chen Cui <chcui@nvidia.com>

revert accidental commit

d123e03

Signed-off-by: Chen Cui <chcui@nvidia.com>

revert accidental commit

0b94330

Signed-off-by: Chen Cui <chcui@nvidia.com>

cuichenx requested a review from ericharper February 25, 2024 17:52

github-actions bot added the NLP label Feb 25, 2024

cuichenx changed the title ~~Cherry-pick 8262~~ Update PEFT Doc Feb 25, 2024

ericharper merged commit 2fbfceb into main Feb 25, 2024
7 checks passed

ericharper deleted the chcui/cherrypick-8262 branch February 25, 2024 18:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update PEFT Doc #8501

Update PEFT Doc #8501

cuichenx commented Feb 25, 2024

Update PEFT Doc #8501

Update PEFT Doc #8501

Conversation

cuichenx commented Feb 25, 2024