[NeMo-UX] Adding MegatronParallel #8987
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| class CallbackMethods: | ||
| def on_megatron_step_start(self, *args, **kwargs) -> None: | ||
| ... |
Check notice
Code scanning / CodeQL
Statement has no effect Note
ericharper
reviewed
Apr 22, 2024
ericharper
reviewed
Apr 22, 2024
| vp_size: Optional[int] = None, | ||
| cpu: bool = False, | ||
| ) -> None: | ||
| from apex.transformer.tensor_parallel.layers import set_defaults_if_not_set_tensor_model_parallel_attributes |
There was a problem hiding this comment.
We want to remove dependency on apex, can this temporarily go in nemo? Ultimately in mcore?
There was a problem hiding this comment.
Yeah makes sense. Let's do that in a separate PR so we can update it everywhere in NeMo.
ericharper
reviewed
Apr 22, 2024
ericharper
reviewed
Apr 22, 2024
ericharper
reviewed
Apr 22, 2024
ericharper
reviewed
Apr 22, 2024
ericharper
reviewed
Apr 22, 2024
Comment on lines
+836
to
+831
| return data_iterator # TODO @tmoon: Remove | ||
| # TODO @tmoon: Use once available in Megatron-LM | ||
| # return DataIteratorList([data_iterator]) |
There was a problem hiding this comment.
Let's clean this up everywhere soon, since this is copied from elsewhere in NeMo.
ericharper
reviewed
Apr 22, 2024
Comment on lines
+879
to
+874
| return iters # TODO @tmoon: Remove | ||
| # TODO @tmoon: Use once available in Megatron-LM | ||
| # return DataIteratorList(iters) |
There was a problem hiding this comment.
This is copied from elsewhere in NeMo. If we clean this up, let's do this in a separate PR so we can do it everywhere.
ericharper
reviewed
Apr 22, 2024
Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
* Detect 'arcname' prefix in utils when handling .nemo tarball Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Address megatron_amp_O2 = True case in quantization Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add Megatron-LM to PYTHONPATH correctly in Jenkinsfile Signed-off-by: Jan Lasek <janek.lasek@gmail.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
* PR #8803: Update embedding init prototype to match mc Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR #8810: Fix import of get_gpt_layer_ammo_spec Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR #8853: Fix memory leak at loss func Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> --------- Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
* initial commit Signed-off-by: Chen Cui <chcui@nvidia.com> * enable pp support for merge script and fix output precision Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove incomplete script for next release Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
* Move cached embedding devices and dtype for onnx export consistency Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add old trt export/inference script, currently not working in latest container. Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add NeMo TRT inference pipeline and quatization workflow Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add guards to avoid undefined variables Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor fix Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add conversion script from hf sdxl to nemo sdxl Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update quantize pipeline to adapt to variable image dimension Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * update sdxl pipeline to be aware of additional emb channels Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add guards for potential local var Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * copyright header Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update calib prompt file path Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update file paths Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * minor update Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update default quantization config Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * remove unused imports/vars Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused imports Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> --------- Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
* Replaced unpacking of nemo checkpoints on export with a VFS-like TarPath object. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed the signature of ZarrPathStore.__delitem__ Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> --------- Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
* Update Latest News Adds links to articles on * NeMo framework on GKE * Responsible Gen AI using NeMo and Picasso * NeMo powering Amazon Titan foundation models Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Minor updates to latest news in README * Remove bullets * Editing text for clarity Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Format latest news as a dropdown list * Uses embedded html to format news to dropdown, hiding lengthy details * Fixes formatting of the title Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add break to improve readability of latest news image Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add LLM and MM section in latest news Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add margin in latest news expandable lists Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Remove styling of expandable list * Github appears to not render styled elements when embedded as raw html in rst Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Fold the first news item by default Signed-off-by: Shashank Verma <shashankv@nvidia.com> --------- Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
marcromeyn
force-pushed
the
nemo-ux/megatron-parallel
branch
from
cce815e to
aac093d
Compare
| # limitations under the License. | ||
|
|
||
| import math | ||
| import time |
Check notice
Code scanning / CodeQL
Unused import Note
Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
* enable using hybrid asr models in ctc segmentation tool Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
* tdt confidence fix --------- Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
marcromeyn
force-pushed
the
nemo-ux/megatron-parallel
branch
from
aac093d to
ef2dd78
Compare
marcromeyn
approved these changes
Apr 22, 2024
xingyaoww
pushed a commit
to xingyaoww/NeMo
that referenced
this pull request
Apr 23, 2024
* Adding MegatronParallel Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Minor quantization pipeline updates (NVIDIA#8924) * Detect 'arcname' prefix in utils when handling .nemo tarball Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Address megatron_amp_O2 = True case in quantization Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add Megatron-LM to PYTHONPATH correctly in Jenkinsfile Signed-off-by: Jan Lasek <janek.lasek@gmail.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix converter (NVIDIA#8960) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix memory leak at loss func (NVIDIA#8868) * PR NVIDIA#8803: Update embedding init prototype to match mc Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR NVIDIA#8810: Fix import of get_gpt_layer_ammo_spec Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR NVIDIA#8853: Fix memory leak at loss func Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> --------- Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * PP support in LoRA merge script (NVIDIA#8934) * initial commit Signed-off-by: Chen Cui <chcui@nvidia.com> * enable pp support for merge script and fix output precision Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove incomplete script for next release Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Mingyuanm/sdxl export (NVIDIA#8926) * Move cached embedding devices and dtype for onnx export consistency Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add old trt export/inference script, currently not working in latest container. Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add NeMo TRT inference pipeline and quatization workflow Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add guards to avoid undefined variables Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor fix Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add conversion script from hf sdxl to nemo sdxl Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update quantize pipeline to adapt to variable image dimension Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * update sdxl pipeline to be aware of additional emb channels Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add guards for potential local var Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * copyright header Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update calib prompt file path Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update file paths Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * minor update Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update default quantization config Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * remove unused imports/vars Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused imports Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> --------- Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Avoid unpacking NeMo checkpoints before exporting to TRT-LLM (NVIDIA#8866) * Replaced unpacking of nemo checkpoints on export with a VFS-like TarPath object. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed the signature of ZarrPathStore.__delitem__ Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> --------- Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * update (NVIDIA#8978) Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * change the condition for get qkv tensor from linear_qkv output (NVIDIA#8965) Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Update Latest News (NVIDIA#8837) * Update Latest News Adds links to articles on * NeMo framework on GKE * Responsible Gen AI using NeMo and Picasso * NeMo powering Amazon Titan foundation models Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Minor updates to latest news in README * Remove bullets * Editing text for clarity Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Format latest news as a dropdown list * Uses embedded html to format news to dropdown, hiding lengthy details * Fixes formatting of the title Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add break to improve readability of latest news image Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add LLM and MM section in latest news Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add margin in latest news expandable lists Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Remove styling of expandable list * Github appears to not render styled elements when embedded as raw html in rst Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Fold the first news item by default Signed-off-by: Shashank Verma <shashankv@nvidia.com> --------- Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix incorrect link to latest news in README (NVIDIA#8985) Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * make unit tests works Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * add pytest-mock to unit test reqs Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Enable using hybrid asr models in CTC Segmentation tool (NVIDIA#8828) * enable using hybrid asr models in ctc segmentation tool Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Add safety checks for 'data' key in MegatronGPTModel cfg (NVIDIA#8991) Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * address some comments Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * TDT confidence fix (NVIDIA#8982) * tdt confidence fix --------- Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Address PR comments Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> --------- Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: Marc Romeyn <marcromeyn@gmail.com> Co-authored-by: Jan Lasek <janek.lasek@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Ming <111467530+Victor49152@users.noreply.github.com> Co-authored-by: Alexey Panteleev <apanteleev87@gmail.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Shashank Verma <shashank3959@gmail.com> Co-authored-by: Shashank Verma <shashankv@nvidia.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Co-authored-by: Aleksandr Laptev <alaptev@nvidia.com>
alxzhang-amazon
pushed a commit
to alxzhang-amazon/NeMo
that referenced
this pull request
Apr 26, 2024
* Adding MegatronParallel Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Minor quantization pipeline updates (NVIDIA#8924) * Detect 'arcname' prefix in utils when handling .nemo tarball Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Address megatron_amp_O2 = True case in quantization Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add Megatron-LM to PYTHONPATH correctly in Jenkinsfile Signed-off-by: Jan Lasek <janek.lasek@gmail.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix converter (NVIDIA#8960) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix memory leak at loss func (NVIDIA#8868) * PR NVIDIA#8803: Update embedding init prototype to match mc Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR NVIDIA#8810: Fix import of get_gpt_layer_ammo_spec Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR NVIDIA#8853: Fix memory leak at loss func Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> --------- Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * PP support in LoRA merge script (NVIDIA#8934) * initial commit Signed-off-by: Chen Cui <chcui@nvidia.com> * enable pp support for merge script and fix output precision Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove incomplete script for next release Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Mingyuanm/sdxl export (NVIDIA#8926) * Move cached embedding devices and dtype for onnx export consistency Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add old trt export/inference script, currently not working in latest container. Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add NeMo TRT inference pipeline and quatization workflow Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add guards to avoid undefined variables Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor fix Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add conversion script from hf sdxl to nemo sdxl Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update quantize pipeline to adapt to variable image dimension Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * update sdxl pipeline to be aware of additional emb channels Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add guards for potential local var Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * copyright header Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update calib prompt file path Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update file paths Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * minor update Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update default quantization config Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * remove unused imports/vars Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused imports Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> --------- Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Avoid unpacking NeMo checkpoints before exporting to TRT-LLM (NVIDIA#8866) * Replaced unpacking of nemo checkpoints on export with a VFS-like TarPath object. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed the signature of ZarrPathStore.__delitem__ Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> --------- Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * update (NVIDIA#8978) Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * change the condition for get qkv tensor from linear_qkv output (NVIDIA#8965) Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Update Latest News (NVIDIA#8837) * Update Latest News Adds links to articles on * NeMo framework on GKE * Responsible Gen AI using NeMo and Picasso * NeMo powering Amazon Titan foundation models Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Minor updates to latest news in README * Remove bullets * Editing text for clarity Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Format latest news as a dropdown list * Uses embedded html to format news to dropdown, hiding lengthy details * Fixes formatting of the title Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add break to improve readability of latest news image Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add LLM and MM section in latest news Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add margin in latest news expandable lists Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Remove styling of expandable list * Github appears to not render styled elements when embedded as raw html in rst Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Fold the first news item by default Signed-off-by: Shashank Verma <shashankv@nvidia.com> --------- Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix incorrect link to latest news in README (NVIDIA#8985) Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * make unit tests works Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * add pytest-mock to unit test reqs Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Enable using hybrid asr models in CTC Segmentation tool (NVIDIA#8828) * enable using hybrid asr models in ctc segmentation tool Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Add safety checks for 'data' key in MegatronGPTModel cfg (NVIDIA#8991) Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * address some comments Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * TDT confidence fix (NVIDIA#8982) * tdt confidence fix --------- Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Address PR comments Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> --------- Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: Marc Romeyn <marcromeyn@gmail.com> Co-authored-by: Jan Lasek <janek.lasek@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Ming <111467530+Victor49152@users.noreply.github.com> Co-authored-by: Alexey Panteleev <apanteleev87@gmail.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Shashank Verma <shashank3959@gmail.com> Co-authored-by: Shashank Verma <shashankv@nvidia.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Co-authored-by: Aleksandr Laptev <alaptev@nvidia.com>
galv
pushed a commit
to galv/NeMo
that referenced
this pull request
Apr 29, 2024
* Adding MegatronParallel Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Minor quantization pipeline updates (NVIDIA#8924) * Detect 'arcname' prefix in utils when handling .nemo tarball Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Address megatron_amp_O2 = True case in quantization Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add Megatron-LM to PYTHONPATH correctly in Jenkinsfile Signed-off-by: Jan Lasek <janek.lasek@gmail.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix converter (NVIDIA#8960) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix memory leak at loss func (NVIDIA#8868) * PR NVIDIA#8803: Update embedding init prototype to match mc Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR NVIDIA#8810: Fix import of get_gpt_layer_ammo_spec Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR NVIDIA#8853: Fix memory leak at loss func Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> --------- Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * PP support in LoRA merge script (NVIDIA#8934) * initial commit Signed-off-by: Chen Cui <chcui@nvidia.com> * enable pp support for merge script and fix output precision Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove incomplete script for next release Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Mingyuanm/sdxl export (NVIDIA#8926) * Move cached embedding devices and dtype for onnx export consistency Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add old trt export/inference script, currently not working in latest container. Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add NeMo TRT inference pipeline and quatization workflow Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add guards to avoid undefined variables Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor fix Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add conversion script from hf sdxl to nemo sdxl Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update quantize pipeline to adapt to variable image dimension Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * update sdxl pipeline to be aware of additional emb channels Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add guards for potential local var Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * copyright header Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update calib prompt file path Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update file paths Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * minor update Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update default quantization config Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * remove unused imports/vars Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused imports Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> --------- Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Avoid unpacking NeMo checkpoints before exporting to TRT-LLM (NVIDIA#8866) * Replaced unpacking of nemo checkpoints on export with a VFS-like TarPath object. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed the signature of ZarrPathStore.__delitem__ Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> --------- Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * update (NVIDIA#8978) Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * change the condition for get qkv tensor from linear_qkv output (NVIDIA#8965) Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Update Latest News (NVIDIA#8837) * Update Latest News Adds links to articles on * NeMo framework on GKE * Responsible Gen AI using NeMo and Picasso * NeMo powering Amazon Titan foundation models Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Minor updates to latest news in README * Remove bullets * Editing text for clarity Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Format latest news as a dropdown list * Uses embedded html to format news to dropdown, hiding lengthy details * Fixes formatting of the title Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add break to improve readability of latest news image Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add LLM and MM section in latest news Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add margin in latest news expandable lists Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Remove styling of expandable list * Github appears to not render styled elements when embedded as raw html in rst Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Fold the first news item by default Signed-off-by: Shashank Verma <shashankv@nvidia.com> --------- Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix incorrect link to latest news in README (NVIDIA#8985) Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * make unit tests works Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * add pytest-mock to unit test reqs Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Enable using hybrid asr models in CTC Segmentation tool (NVIDIA#8828) * enable using hybrid asr models in ctc segmentation tool Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Add safety checks for 'data' key in MegatronGPTModel cfg (NVIDIA#8991) Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * address some comments Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * TDT confidence fix (NVIDIA#8982) * tdt confidence fix --------- Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Address PR comments Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> --------- Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: Marc Romeyn <marcromeyn@gmail.com> Co-authored-by: Jan Lasek <janek.lasek@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Ming <111467530+Victor49152@users.noreply.github.com> Co-authored-by: Alexey Panteleev <apanteleev87@gmail.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Shashank Verma <shashank3959@gmail.com> Co-authored-by: Shashank Verma <shashankv@nvidia.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Co-authored-by: Aleksandr Laptev <alaptev@nvidia.com>
suiyoubi
pushed a commit
that referenced
this pull request
May 2, 2024
* Adding MegatronParallel Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Minor quantization pipeline updates (#8924) * Detect 'arcname' prefix in utils when handling .nemo tarball Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Address megatron_amp_O2 = True case in quantization Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add Megatron-LM to PYTHONPATH correctly in Jenkinsfile Signed-off-by: Jan Lasek <janek.lasek@gmail.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix converter (#8960) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix memory leak at loss func (#8868) * PR #8803: Update embedding init prototype to match mc Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR #8810: Fix import of get_gpt_layer_ammo_spec Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR #8853: Fix memory leak at loss func Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> --------- Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * PP support in LoRA merge script (#8934) * initial commit Signed-off-by: Chen Cui <chcui@nvidia.com> * enable pp support for merge script and fix output precision Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove incomplete script for next release Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Mingyuanm/sdxl export (#8926) * Move cached embedding devices and dtype for onnx export consistency Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add old trt export/inference script, currently not working in latest container. Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add NeMo TRT inference pipeline and quatization workflow Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add guards to avoid undefined variables Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor fix Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add conversion script from hf sdxl to nemo sdxl Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update quantize pipeline to adapt to variable image dimension Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * update sdxl pipeline to be aware of additional emb channels Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add guards for potential local var Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * copyright header Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update calib prompt file path Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update file paths Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * minor update Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update default quantization config Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * remove unused imports/vars Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused imports Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> --------- Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Avoid unpacking NeMo checkpoints before exporting to TRT-LLM (#8866) * Replaced unpacking of nemo checkpoints on export with a VFS-like TarPath object. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed the signature of ZarrPathStore.__delitem__ Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> --------- Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * update (#8978) Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * change the condition for get qkv tensor from linear_qkv output (#8965) Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Update Latest News (#8837) * Update Latest News Adds links to articles on * NeMo framework on GKE * Responsible Gen AI using NeMo and Picasso * NeMo powering Amazon Titan foundation models Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Minor updates to latest news in README * Remove bullets * Editing text for clarity Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Format latest news as a dropdown list * Uses embedded html to format news to dropdown, hiding lengthy details * Fixes formatting of the title Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add break to improve readability of latest news image Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add LLM and MM section in latest news Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add margin in latest news expandable lists Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Remove styling of expandable list * Github appears to not render styled elements when embedded as raw html in rst Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Fold the first news item by default Signed-off-by: Shashank Verma <shashankv@nvidia.com> --------- Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix incorrect link to latest news in README (#8985) Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * make unit tests works Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * add pytest-mock to unit test reqs Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Enable using hybrid asr models in CTC Segmentation tool (#8828) * enable using hybrid asr models in ctc segmentation tool Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Add safety checks for 'data' key in MegatronGPTModel cfg (#8991) Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * address some comments Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * TDT confidence fix (#8982) * tdt confidence fix --------- Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Address PR comments Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> --------- Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: Marc Romeyn <marcromeyn@gmail.com> Co-authored-by: Jan Lasek <janek.lasek@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Ming <111467530+Victor49152@users.noreply.github.com> Co-authored-by: Alexey Panteleev <apanteleev87@gmail.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Shashank Verma <shashank3959@gmail.com> Co-authored-by: Shashank Verma <shashankv@nvidia.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Co-authored-by: Aleksandr Laptev <alaptev@nvidia.com> Signed-off-by: Ao Tang <aot@nvidia.com>
rohitrango
pushed a commit
to rohitrango/NeMo
that referenced
this pull request
Jun 25, 2024
* Adding MegatronParallel Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Minor quantization pipeline updates (NVIDIA#8924) * Detect 'arcname' prefix in utils when handling .nemo tarball Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Address megatron_amp_O2 = True case in quantization Signed-off-by: Jan Lasek <janek.lasek@gmail.com> * Add Megatron-LM to PYTHONPATH correctly in Jenkinsfile Signed-off-by: Jan Lasek <janek.lasek@gmail.com> --------- Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix converter (NVIDIA#8960) Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix memory leak at loss func (NVIDIA#8868) * PR NVIDIA#8803: Update embedding init prototype to match mc Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR NVIDIA#8810: Fix import of get_gpt_layer_ammo_spec Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> * PR NVIDIA#8853: Fix memory leak at loss func Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> --------- Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * PP support in LoRA merge script (NVIDIA#8934) * initial commit Signed-off-by: Chen Cui <chcui@nvidia.com> * enable pp support for merge script and fix output precision Signed-off-by: Chen Cui <chcui@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove incomplete script for next release Signed-off-by: Chen Cui <chcui@nvidia.com> --------- Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Mingyuanm/sdxl export (NVIDIA#8926) * Move cached embedding devices and dtype for onnx export consistency Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add old trt export/inference script, currently not working in latest container. Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Add NeMo TRT inference pipeline and quatization workflow Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add guards to avoid undefined variables Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * minor fix Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add conversion script from hf sdxl to nemo sdxl Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update quantize pipeline to adapt to variable image dimension Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * update sdxl pipeline to be aware of additional emb channels Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add guards for potential local var Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * copyright header Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update calib prompt file path Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * Update file paths Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * minor update Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update default quantization config Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * remove unused imports/vars Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused imports Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> --------- Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Avoid unpacking NeMo checkpoints before exporting to TRT-LLM (NVIDIA#8866) * Replaced unpacking of nemo checkpoints on export with a VFS-like TarPath object. Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed the signature of ZarrPathStore.__delitem__ Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> --------- Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * update (NVIDIA#8978) Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * change the condition for get qkv tensor from linear_qkv output (NVIDIA#8965) Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Update Latest News (NVIDIA#8837) * Update Latest News Adds links to articles on * NeMo framework on GKE * Responsible Gen AI using NeMo and Picasso * NeMo powering Amazon Titan foundation models Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Minor updates to latest news in README * Remove bullets * Editing text for clarity Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Format latest news as a dropdown list * Uses embedded html to format news to dropdown, hiding lengthy details * Fixes formatting of the title Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add break to improve readability of latest news image Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add LLM and MM section in latest news Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Add margin in latest news expandable lists Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Remove styling of expandable list * Github appears to not render styled elements when embedded as raw html in rst Signed-off-by: Shashank Verma <shashankv@nvidia.com> * Fold the first news item by default Signed-off-by: Shashank Verma <shashankv@nvidia.com> --------- Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Fix incorrect link to latest news in README (NVIDIA#8985) Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * make unit tests works Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * add pytest-mock to unit test reqs Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Enable using hybrid asr models in CTC Segmentation tool (NVIDIA#8828) * enable using hybrid asr models in ctc segmentation tool Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Add safety checks for 'data' key in MegatronGPTModel cfg (NVIDIA#8991) Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * address some comments Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * TDT confidence fix (NVIDIA#8982) * tdt confidence fix --------- Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> * Address PR comments Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> --------- Signed-off-by: Marc Romeyn <marcromeyn@gmail.com> Signed-off-by: Jan Lasek <janek.lasek@gmail.com> Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com> Signed-off-by: Jaemin Choi <jaeminc@nvidia.com> Signed-off-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Signed-off-by: Chen Cui <chcui@nvidia.com> Signed-off-by: Mingyuan Ma <mingyuanm@nvidia.com> Signed-off-by: Alexey Panteleev <alpanteleev@nvidia.com> Signed-off-by: eharper <eharper@nvidia.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Signed-off-by: Shashank Verma <shashankv@nvidia.com> Signed-off-by: Shashank Verma <shashank3959@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Aleksandr Laptev <alaptev@nvidia.com> Co-authored-by: Marc Romeyn <marcromeyn@gmail.com> Co-authored-by: Jan Lasek <janek.lasek@gmail.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Jaemin Choi <minitu77@gmail.com> Co-authored-by: Jaemin Choi <jaeminc@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Shriya Palsamudram <69161273+ShriyaPalsamudram@users.noreply.github.com> Co-authored-by: Pablo Garay <palenq@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adi Renduchintala <adithya.r@gmail.com> Co-authored-by: Ming <111467530+Victor49152@users.noreply.github.com> Co-authored-by: Alexey Panteleev <apanteleev87@gmail.com> Co-authored-by: Onur Yilmaz <35306097+oyilmaz-nvidia@users.noreply.github.com> Co-authored-by: Huiying <willwin.lee@gmail.com> Co-authored-by: Shashank Verma <shashank3959@gmail.com> Co-authored-by: Shashank Verma <shashankv@nvidia.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Co-authored-by: Aleksandr Laptev <alaptev@nvidia.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
NeMo-UX PR 1, add megatron_parallel.py and associated tests