New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for MCR-DL to Megatron-LM #15

Open

Quentin-Anthony opened this issue Feb 21, 2024 · 1 comment

Assignees

Member

Quentin-Anthony commented Feb 21, 2024

Similar to microsoft/Megatron-DeepSpeed#50

We want a Megatron fork based on MCR-DL. Need to create a new branch of https://github.com/OSU-Nowlab/Megatron-LM and replace all calls to torch.dist with mcr-dl.

I'll test once ready.

Quentin-Anthony assigned RadhaGulhane13

Contributor

RadhaGulhane13 commented Apr 20, 2024

Please find PR for this issue (support for MCR-DL to Megatron-LM) : OSU-Nowlab/Megatron-LM#1
Above PR is dependent on https://github.com/OSU-Nowlab/MCR-DL/pull/17/files (Basically, wanted to have simplified access for mcr-dl)

Requesting to review https://github.com/OSU-Nowlab/MCR-DL/pull/17/files first and then OSU-Nowlab/Megatron-LM#1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment