add alpha scaling to lora #8248
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: arendu <adithya.r@gmail.com>
Signed-off-by: arendu <adithya.r@gmail.com>
for more information, see https://pre-commit.ci
|
jenkins |
|
jenkins |
* coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com>
Signed-off-by: Michal Futrega <mfutrega@nvidia.com>
|
jenkins |
ShriyaPalsamudram
previously approved these changes
Feb 23, 2024
Signed-off-by: Michal Futrega <mfutrega@nvidia.com>
|
jenkins |
michal2409
approved these changes
Feb 25, 2024
akoumpa
added a commit
that referenced
this pull request
Feb 26, 2024
* removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>
akoumpa
added a commit
that referenced
this pull request
Feb 26, 2024
* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com>
yaoyu-33
added a commit
that referenced
this pull request
Feb 26, 2024
* removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com>
yaoyu-33
added a commit
that referenced
this pull request
Feb 26, 2024
* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com>
zpx01
pushed a commit
to zpx01/NeMo
that referenced
this pull request
Mar 8, 2024
* removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (NVIDIA#8483) * coldfix (NVIDIA#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (NVIDIA#8416) (NVIDIA#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (NVIDIA#8357) (NVIDIA#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (NVIDIA#8314) * fix path location and branch (NVIDIA#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (NVIDIA#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (NVIDIA#8279) (NVIDIA#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (NVIDIA#8302) (NVIDIA#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (NVIDIA#8334) (NVIDIA#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (NVIDIA#8354) (NVIDIA#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (NVIDIA#8400) (NVIDIA#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (NVIDIA#8427) (NVIDIA#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (NVIDIA#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (NVIDIA#8421) (NVIDIA#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (NVIDIA#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (NVIDIA#7749) (NVIDIA#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (NVIDIA#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (NVIDIA#8315) (NVIDIA#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (NVIDIA#8283) (NVIDIA#8385) * [tutorial] fixed missing RIR scripts file. (NVIDIA#8257) * add values to en tts dict (NVIDIA#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (NVIDIA#8390) (NVIDIA#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (NVIDIA#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (NVIDIA#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (NVIDIA#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (NVIDIA#8353) (NVIDIA#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (NVIDIA#8336) (NVIDIA#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (NVIDIA#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (NVIDIA#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Zeeshan Patel <zeeshanp@berkeley.edu>
zpx01
pushed a commit
to zpx01/NeMo
that referenced
this pull request
Mar 8, 2024
* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Zeeshan Patel <zeeshanp@berkeley.edu>
JRD971000
added a commit
that referenced
this pull request
Mar 15, 2024
* removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com>
JRD971000
added a commit
that referenced
this pull request
Mar 15, 2024
* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com>
pablo-garay
added a commit
that referenced
this pull request
Mar 19, 2024
* removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Pablo Garay <pagaray@nvidia.com>
pablo-garay
added a commit
that referenced
this pull request
Mar 19, 2024
* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Pablo Garay <pagaray@nvidia.com>
ericharper
added a commit
that referenced
this pull request
Mar 19, 2024
* Refactor conversion scripts one in all Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Move bert converter Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * [TTS] Add modules for mel spectrogram codec (#8238) * [TTS] Add modules for mel spectrogram codec Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add mel band validation Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add fullband mel encoder and more documentation Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com> * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> * Account for mpirun use case in get_rank (#8429) Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add settings to suppress bf16 compile errors in CI on V100 (#8481) (#8482) * Add settings to suppress bf16 compile errors in CI on V100 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix canary chunk infer bug (#8449) * fix chunk infer bug Signed-off-by: stevehuang52 <heh@nvidia.com> * add support for duration=None, add lhotse support for relative audio path Signed-off-by: stevehuang52 <heh@nvidia.com> * add tests Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> * Add Baichuan2 support (#8282) * Add Baichuan2 support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reworked MegatronPretrainingRandomBatchSampler to correctly handle epochs > 1 (#7920) * Initital commit of reworked MegatronPretrainingRandomBatchSampler Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed small length based bug Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Euynaheh <hehanyue99@outlook.com> * Add Baichuan2 support Signed-off-by: Euynaheh <hehanyue99@outlook.com> * Add NeMo to HF conversion * fix code format * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix code format * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add Baichuan jenkins test * add_BOS bug fix * Update Jenkinsfile Signed-off-by: Euynaheh <93857693+Euynaheh@users.noreply.github.com> --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Signed-off-by: Euynaheh <hehanyue99@outlook.com> Signed-off-by: Euynaheh <93857693+Euynaheh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: trias702 <25867060+trias702@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana G…
rohitrango
pushed a commit
to rohitrango/NeMo
that referenced
this pull request
Jun 25, 2024
* removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (NVIDIA#8483) * coldfix (NVIDIA#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (NVIDIA#8416) (NVIDIA#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (NVIDIA#8357) (NVIDIA#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (NVIDIA#8314) * fix path location and branch (NVIDIA#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (NVIDIA#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (NVIDIA#8279) (NVIDIA#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (NVIDIA#8302) (NVIDIA#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (NVIDIA#8334) (NVIDIA#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (NVIDIA#8354) (NVIDIA#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (NVIDIA#8400) (NVIDIA#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (NVIDIA#8427) (NVIDIA#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (NVIDIA#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (NVIDIA#8421) (NVIDIA#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (NVIDIA#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (NVIDIA#7749) (NVIDIA#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (NVIDIA#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (NVIDIA#8315) (NVIDIA#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (NVIDIA#8283) (NVIDIA#8385) * [tutorial] fixed missing RIR scripts file. (NVIDIA#8257) * add values to en tts dict (NVIDIA#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (NVIDIA#8390) (NVIDIA#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (NVIDIA#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (NVIDIA#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (NVIDIA#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (NVIDIA#8353) (NVIDIA#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (NVIDIA#8336) (NVIDIA#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (NVIDIA#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (NVIDIA#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com>
rohitrango
pushed a commit
to rohitrango/NeMo
that referenced
this pull request
Jun 25, 2024
* MoE parameter passing (#8255) * MoE parameter passing Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Pass EP/MoE params in consumer scripts. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * PR fixes Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Use latest commit of mcore-0.5 Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * CI fix Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> --------- Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Travis Bartley <tbartley@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@dgx1v-loki-21.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> Co-authored-by: Selvaraj Anandaraj <anandaraj@wisc.edu> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: tbartley94 <90423858+tbartley94@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com>
rohitrango
pushed a commit
to rohitrango/NeMo
that referenced
this pull request
Jun 25, 2024
* Refactor conversion scripts one in all Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Move bert converter Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * [TTS] Add modules for mel spectrogram codec (#8238) * [TTS] Add modules for mel spectrogram codec Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add mel band validation Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] Add fullband mel encoder and more documentation Signed-off-by: Ryan <rlangman@nvidia.com> --------- Signed-off-by: Ryan <rlangman@nvidia.com> * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> * Account for mpirun use case in get_rank (#8429) Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add settings to suppress bf16 compile errors in CI on V100 (#8481) (#8482) * Add settings to suppress bf16 compile errors in CI on V100 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Co-authored-by: Abhishree Thittenamane <47577437+athitten@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix canary chunk infer bug (#8449) * fix chunk infer bug Signed-off-by: stevehuang52 <heh@nvidia.com> * add support for duration=None, add lhotse support for relative audio path Signed-off-by: stevehuang52 <heh@nvidia.com> * add tests Signed-off-by: stevehuang52 <heh@nvidia.com> --------- Signed-off-by: stevehuang52 <heh@nvidia.com> * Add Baichuan2 support (#8282) * Add Baichuan2 support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reworked MegatronPretrainingRandomBatchSampler to correctly handle epochs > 1 (#7920) * Initital commit of reworked MegatronPretrainingRandomBatchSampler Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed small length based bug Signed-off-by: Daniel Egert <degert@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Euynaheh <hehanyue99@outlook.com> * Add Baichuan2 support Signed-off-by: Euynaheh <hehanyue99@outlook.com> * Add NeMo to HF conversion * fix code format * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix code format * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add Baichuan jenkins test * add_BOS bug fix * Update Jenkinsfile Signed-off-by: Euynaheh <93857693+Euynaheh@users.noreply.github.com> --------- Signed-off-by: Daniel Egert <degert@nvidia.com> Signed-off-by: Euynaheh <hehanyue99@outlook.com> Signed-off-by: Euynaheh <93857693+Euynaheh@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: trias702 <25867060+trias702@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> * Jiaqiz/option to disable adapters & merge all lora layers (#8029) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * use adapter only when it is enabled Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix lora merge script (#8113) Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> * add peft ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * merge lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * support/fix cpu initialization Signed-off-by: Chen Cui <chcui@nvidia.com> * add example usage Signed-off-by: Chen Cui <chcui@nvidia.com> * fix TP due to distributed checkpoint Signed-off-by: Chen Cui <chcui@nvidia.com> * updating the logic of merging lora weights for all layers, mcore only Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * merge in fp32 then cast back Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * remove ckpt to nemo Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> * fix import Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> --------- Signed-off-by: jiaqi zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Chen Cui <chcui@nvidia.com> * Update k2 version (#8478) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add mcore full TE transformer layer spec (#8328) * Add spec and implement autocast layer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * remove try-catchs, these dependecies are mandatory for this file Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Check out this cool try/except clause Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add import tests to Jenkinsfile Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Move import tests to Jenkins and remove code that is developed only for passing tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Make test robust to faulty base configs Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Use proper GPT implementation in the test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Update nemo/collections/nlp/models/language_modeling/megatron/gpt_full_te_layer_autocast_spec.py Co-authored-by: Jaemin Choi <minitu77@gmail.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add TE knobs to the copy of AutocastTransformerLayer Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Add dummy parameter to accomodated for the changes in mcore Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update mcore to 0.5.0 in Jenkins pipeline Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump mcore commit. This is commit from tot, not any release. Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Remove from the test config option that is incompatible with bias_activation_fusion Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Bump TE version in CI to 1.4 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Update test Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change precision for the test - current runnens don't support bf16 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Signed-off-by: jbaczek <45043825+jbaczek@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sudhakar Singh <sudhakars@nvidia.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> * Handle float limit_val_batches (#8426) * Handle float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Rectify reconfiguration of float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove unused imports Signed-off-by: Abhishree <abhishreetm@gmail.com> * Scale len(val_dataloader) with float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Return len(dataloader) in microbatches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add back resetting of num val samples Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix to ensure float limit_val_batches is multiple of num_micro_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Remove forcing eval samples to 1 for float limit_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Fix bug wrt 0 limiot_val_batches Signed-off-by: Abhishree <abhishreetm@gmail.com> * Add missing mock_dataset line Signed-off-by: Abhishree <abhishreetm@gmail.com> * Avoid ensuring limit_val_batches is a mutliple of microbatches for 1.0 Signed-off-by: Abhishree <abhishreetm@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore the hack forcing number of validation and test epochs to 1 Signed-off-by: Jan Baczek <jbaczek@nvidia.com> * Change limit_val_batches to 1.0 for GPT pretraining test. The integer value is covered in other tests Signed-off-by: Jan Baczek <jbaczek@nvidia.com> --------- Signed-off-by: Abhishree <abhishreetm@gmail.com> Signed-off-by: Jan Baczek <jbaczek@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Jan Baczek <jbaczek@nvidia.com> * Fix tutorial links in user guide (#8497) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> * Sequence Parallel for LoRA (#8369) * support lora + sequence parallel Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add more comments Signed-off-by: Chen Cui <chcui@nvidia.com> * add lora SP CI test Signed-off-by: Chen Cui <chcui@nvidia.com> * support lora for all linear modules as in #7988 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Call proper method to replace (#8498) Signed-off-by: Naga Venkatesh Gavini <nagavenkat9948@gmail.com> * Added memory logger (#8395) * Added memory logger Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Selvaraj Anandaraj <selvaraja@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Canary refactor for Riva (#8363) * initial commit of bleu score tracking Signed-off-by: Travis Bartley <tbartley@nvidia.com> * initial commit, refactoring aed models for riva Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updating Canary to support torch metrics Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fixes Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * missed an empty batch conditional Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Fixing dataloader issues Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Finishing merge conflict with transcribe update Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * copyright header fix Signed-off-by: Travis Bartley <tbartley@nvidia.com> * yet another merge conflict Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * making paired data management safer Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece needs bigger tokenizer... Signed-off-by: Travis Bartley <tbartley@nvidia.com> * sentencepiece tokenizer vocab needs to be +2 from vocab for canary Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Update canary tokenizer to be more generic, updated metrics to manage special tokens removal themselves. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * merge conflit Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Simplified tokenizer and corrected bug in dataloader Signed-off-by: Travis Bartley <tbartley@nvidia.com> * Cleaning up docstrings and fixing inference bug. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding example scripts Signed-off-by: Travis Bartley <tbartley@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cleaning up useless imports Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * fixing unit tests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * cfg name change Signed-off-by: Travis Bartley <tbartley@nvidia.com> * adding custom check to pass pytests Signed-off-by: Travis Bartley <tbartley@nvidia.com> * removing print script Signed-off-by: Travis Bartley <tbartley@nvidia.com> * catching bugs regarding tokens. Signed-off-by: Travis Bartley <tbartley@nvidia.com> * added docstrings and made examples scripts more generic Signed-off-by: Travis Bartley <tbartley@nvidia.com> * docstring deleted by accident Signed-off-by: Travis Bartley <tbartley@nvidia.com> * plurals in namespace Signed-off-by: Travis Bartley <tbartley@nvidia.com> * changing example script Signed-off-by: Travis Bartley <tbartley@nvidia.com> --------- Signed-off-by: Travis Bartley <tbartley@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> * add alpha scaling to lora (#8248) * removed pdeprecated eft model Signed-off-by: arendu <adithya.r@gmail.com> * add alpha Signed-off-by: arendu <adithya.r@gmail.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add alpha scaling to lora (#8483) * coldfix (#8412) Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixed errors in the CTM gen functions (#8416) (#8420) Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add change_vocabulary and save_tokenizers() support to Multitask ASR models (#8357) (#8367) * Add change_vocabulary and save_tokenizers() support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nemo/collections/asr/models/aed_multitask_models.py --------- Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * fix path location and branch (#8314) * fix path location and branch (#8304) * fix path location and branch Signed-off-by: Nithin Rao Koluguri <nithinraok> * change to a floating point number Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * updat ebranch in tutorial Signed-off-by: Nithin Rao Koluguri <nithinraok> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add TP comm overlap knobs to AutocastTransformerLayer (#8290) Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add deallocate pipeline output optimization (#8279) (#8318) * add deallocate pipeline output optimization * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * remove assertion (#8302) (#8321) Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Keep max_seqlen and cu_seqlens_argmin for later micro-batches when PP>1 (#8334) (#8346) Signed-off-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Enable megatron core loggers for GPT pretraining (#8354) (#8384) * Logging changes tested for gpt_pretraining * Additional args * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fix dreambooth data sampler issue (#8400) (#8413) * Turn on drop last * Some neva fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add ensemble decoding fix (#8427) (#8433) Signed-off-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeVA Tutorial Notebook (#8217) * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * init commit - neva tutorial Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * NeVA tutorial notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add inference via script Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * requested changes Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> * add codeblocks to run torchrun in notebook Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> --------- Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore customization doc minor fix (#8421) (#8437) Signed-off-by: Huiying Li <willwin.lee@gmail.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add `loop_labels` algorithm for TDT greedy decoding (#8215) * Add `loop_labels` algorithm for TDT greedy decoding Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use `loop_labels` by default Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Loop labels greedy decoding v2 Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments. Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched hypotheses Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add tests for batched alignments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix comment Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix test Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add computer for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix TDT decoding algorithm Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Use loop frames by default for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Remove "loop frames" implementation for TDT Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add comments Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Fix confidence. Use tensor for durations. Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add dist ckpt support for regular optimizers (#7749) (#8293) * Add dist ckpt support for regular optimizers * [tutorial] fixed missing RIR scripts file. (#8257) * fix imports * imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ci imports fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert asr notebook * revert asr notebook --------- Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Multimodal r1.23.0 bug fix (#8315) (#8339) * Rename quick-gelu * ddpm config guard * Fix ddpm edit api * Fix insert_image_token cfg issue * neva updates * reformat * Add back jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix jenkins * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bugs * Update default neva template --------- Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * mcore ds fix (#8283) (#8385) * [tutorial] fixed missing RIR scripts file. (#8257) * add values to en tts dict (#7879) * mcore ds fix * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore * revert asr files * add comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset * update mcore version * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg * update mcore commit * fix Bert unit tests * update bert tests * fix bert mcore test * fix gpt jenkins tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update apex & TE commits * revert apex installation * turn off the fusion for jenkins --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * MCore dataset compatibility for tokenizers (#8390) (#8397) * Add unique_identifiers for all tokenizers and eod for SentencePieceTokenizer * Add generalized token aliases to TokenizerSpec to conform with MegatronTokenizer's interface. Remove now-redundant individual fixes from AutoTokenizer and SentencePieceTokenizer. --------- Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Canary: inference tokenization improvements; preserving custom keys when creating tarred manifests (#8432) * Improvements for Canary: - carry over custom keys when creatin tarred manifests - selectable text field in ASR eval - get rid of prompt slicing, create proper inference prompts Signed-off-by: Piotr Żelasko <petezor@gmail.com> * set ensure_ascii=False in tarred conversion to avoid breaking tokenizers trained on UTF-8 encoding Signed-off-by: Piotr Żelasko <petezor@gmail.com> --------- Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * add sbert to IR (#8445) * add sbert to IR Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * add doc Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * fix the auto_tokenizer property method reset bug Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * addressed bot comments Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Update readme (#8440) * update Signed-off-by: eharper <eharper@nvidia.com> * udpate Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * landing pages added * landing page added for vision * landing pages updated * some minor changes to the main readme * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * update Signed-off-by: eharper <eharper@nvidia.com> * typo fixed * update Signed-off-by: eharper <eharper@nvidia.com> --------- Signed-off-by: eharper <eharper@nvidia.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * NeMo-Mistral to HF converter bugfix. (#8353) (#8442) Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Fixing mcore bert for TP, PP and SP (#8336) (#8443) * Fixing mcore bert for TP, PP and SP * Fixing mcore bert for TP, PP and SP * Fixing mcore version * Fixing mcore version * Update Jenkinsfile * Update Jenkinsfile * Update Jenkinsfile --------- Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add LoRA support to all linear layers (#7988) * Added LoRA support for the Dense layer of Attention * Added LoRA MLP support to MCore and NeMo models. * Change LoRA config default to QKV. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed bug with ddp training. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MCoreMixin chages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * using new commit of meg-LM Signed-off-by: arendu <adithya.r@gmail.com> * add cpu_offloading_num_layers to conversion script until bug in megatron is fixed Signed-off-by: Chen Cui <chcui@nvidia.com> * fix peft mixin arguments to follow mcore 0.5 Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update megatron commit to fix ci error Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * try to fix ci Signed-off-by: Chen Cui <chcui@nvidia.com> * add cfg default Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Add Neva Template for NV-DPO Models (#8358) * add/rename from nvgpt to nv_steerlm, add nv_dpo template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add nv_dpo conversation to accomendate empty system message Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * handle nv_dpo template text generation Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * add prompt string to nvgpt Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bugfix for inference prompt template Signed-off-by: HuiyingLi <willwin.lee@gmail.com> * bug fix for grabbing clean text Signed-off-by: Huiying Li <willwin.lee@gmail.com> * fix code format Signed-off-by: Huiying Li <willwin.lee@gmail.com> --------- Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * default for alpha Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> * Rebase scaling alpha Signed-off-by: Michal Futrega <mfutrega@nvidia.com> --------- Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Adi Renduchintala <adithyare@nvidia.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: arendu <adithya.r@gmail.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: arendu <adithya.r@gmail.com> Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Michal Futrega <mfutrega@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Sangkug Lym <slym@nvidia.com> Signed-off-by: Aishwarya Bhandare <abhandare@nvidia.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Signed-off-by: Pratyush Muthukumar <pannumuthu@gmail.com> Signed-off-by: Huiying Li <willwin.lee@gmail.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: Valerie Sarge <vsarge@nvidia.com> Signed-off-by: Piotr Żelasko <petezor@gmail.com> Signed-off-by: ataghibakhsh <ataghibakhsh@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Adi Renduchintala <adithyare@nvidia.com> Signed-off-by: Jiaqi Zeng <jiaqiz@nvidia.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michal Futrega <mfutrega@nvidia.com> Co-authored-by: George <37293288+Jorjeous@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Piotr Żelasko <petezor@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Sangkug Lym <slym@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: ashbhandare <ash.bhandare@gmail.com> Co-authored-by: Aishwarya Bhandare <abhandare@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <30813477+PannuMuthu@users.noreply.github.com> Co-authored-by: Pratyush Muthukumar <pmuthukumar@nvidia.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: mikolajblaz <mikolajblaz@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: dimapihtar <dpihtar@gmail.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: Valerie Sarge <vsarge@nvidia.com> Co-authored-by: Ali Taghibakhshi <71892896+JRD971000@users.noreply.github.com> Co-authored-by: ntajbakhsh <ntajbakhsh@nvidia.com> Co-authored-by: akoumpa <153118171+akoumpa@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <shanmugamr@shanmugamr-mlt.client.nvidia.com> Co-authored-by: Tugrul Konuk <ertkonuk@gmail.com> Co-authored-by: Jiaqi Zeng <jiaqiz@nvidia.com> Co-authored-by: HeyyyyyyG <49757268+HeyyyyyyG@users.noreply.github.com> Co-authored-by: Chen Cui <chcui@nvidia.com> * Update PEFT Doc (#8501) * update peft doc Signed-off-by: Chen Cui <chcui@nvidia.com> * remove old prompt learning doc and notebook Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * fix table Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> * revert accidental commit Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> * release updates (#8394) * release updates (#8378) * [tutorial] fixed missing RIR scripts file. (#8257) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * add values to en tts dict (#7879) Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> * mcore ds fix Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update mcore Signed-off-by: dimapihtar <dpihtar@gmail.com> * revert asr files Signed-off-by: dimapihtar <dpihtar@gmail.com> * add comments Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for mcore mock dataset Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore version Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update gpt cfg Signed-off-by: dimapihtar <dpihtar@gmail.com> * update mcore commit Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix Bert unit tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * update bert tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix bert mcore test Signed-off-by: dimapihtar <dpihtar@gmail.com> * fix gpt jenkins tests Signed-off-by: dimapihtar <dpihtar@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add support for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * add mock ds test Signed-off-by: dimapihtar <dpihtar@gmail.com> * add test for dict data input type Signed-off-by: dimapihtar <dpihtar@gmail.com> * mcore ds fix Signed-off-by: dimapihtar <dpihtar@gmail.com> * data input fix Signed-off-by: dimapihtar <dpihtar@gmail.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana Graterol Fuenmayor <marianag@nvidia.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Mariana <47233618+mgrafu@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@login-eos01.eos.clusters.nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> * Update megatron_gpt_model.py Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Mariana G…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
step in line with the original lora and added alpha.
Collection: [NLP]
Changelog
Usage
# Add a code snippet demonstrating how to use thisJenkins CI
To run Jenkins, a NeMo User with write access must comment
jenkinson the PR.Before your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information