Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MegaBlocks release #1102

Merged
merged 6 commits into from
Apr 9, 2024
Merged

MegaBlocks release #1102

merged 6 commits into from
Apr 9, 2024

Conversation

mvpatel2000
Copy link
Collaborator

Adds MegaBlocks integration into LLMFoundry.

  • Add MegaBlocks integration
  • Add torch implementation for easier testing
  • Update HuggingFace export
  • Add tests

* V1 of MegaBlocks
---------

Co-authored-by: GitHub Actions <actions@github.com>
Co-authored-by: Abhinav Venigalla <abhi@mosaicml.com>
Co-authored-by: Abhi Venigalla <77638579+abhi-mosaic@users.noreply.github.com>
Co-authored-by: Sasha Doubov <sasha@mosaicml.com>
Co-authored-by: Abhi Venigalla <abhi.venigalla@databricks.com>
Co-authored-by: Vitaliy Chiley <6439018+vchiley@users.noreply.github.com>
Co-authored-by: Cheng Li <pistasable@gmail.com>
Co-authored-by: Ning Wang <ning.wang@databricks.com>
Co-authored-by: Irene Dea <deaairene@gmail.com>
Co-authored-by: Shashank Rajput <shashank.rajput@databricks.com>
Co-authored-by: Charles Tang <j316chuck@users.noreply.github.com>
Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com>
Co-authored-by: Shashank Rajput <144760128+ShashankMosaicML@users.noreply.github.com>
Co-authored-by: Chuck Tang <chuck.tang@databricks.com>
Co-authored-by: Jose Javier <26491792+josejg@users.noreply.github.com>
Co-authored-by: Angel Ruiz <angel.ruiz@databricks.com>
Co-authored-by: Denny Lee <denny.g.lee@gmail.com>
Co-authored-by: Jane Zhang <jane.zhang@databricks.com>
Co-authored-by: Daniel King <daniel@mosaicml.com>
Co-authored-by: Chuck Tang <chuck@mosaicml.com>
Co-authored-by: Vitaliy Chiley <vitaliy.chiley@databricks.com>
@mvpatel2000 mvpatel2000 requested a review from a team as a code owner April 9, 2024 15:20
Copy link
Contributor

@vchiley vchiley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Copy link
Contributor

@karan6181 karan6181 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank You!

Copy link
Contributor

@bandish-shah bandish-shah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mvpatel2000 mvpatel2000 merged commit 53160f4 into main Apr 9, 2024
9 checks passed
@mvpatel2000 mvpatel2000 deleted the mvpatel2000/sync branch April 9, 2024 20:49
KuuCi pushed a commit that referenced this pull request Apr 18, 2024
* [Stage] Megablocks release (#241)

* V1 of MegaBlocks
---------

* fix hf ckptr

* rename

* lint

* lint

---------

Co-authored-by: Abhinav Venigalla <abhi@mosaicml.com>
Co-authored-by: Sasha Doubov <sasha@mosaicml.com>
Co-authored-by: Cheng Li <pistasable@gmail.com>
Co-authored-by: Ning Wang <ning.wang@databricks.com>
Co-authored-by: Irene Dea <deaairene@gmail.com>
Co-authored-by: Shashank Rajput <shashank.rajput@databricks.com>
Co-authored-by: Chuck Tang <chuck.tang@databricks.com>
Co-authored-by: Jose Javier <26491792+josejg@users.noreply.github.com>
Co-authored-by: Angel Ruiz <angel.ruiz@databricks.com>
Co-authored-by: Denny Lee <denny.g.lee@gmail.com>
Co-authored-by: Jane Zhang <jane.zhang@databricks.com>
Co-authored-by: Daniel King <daniel@mosaicml.com>
Co-authored-by: Chuck Tang <chuck@mosaicml.com>
Co-authored-by: Vitaliy Chiley <vitaliy.chiley@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants