Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade to 0.13.2, merge recent LLM changes #242

Merged
merged 51 commits into from
Mar 17, 2023
Merged

Upgrade to 0.13.2, merge recent LLM changes #242

merged 51 commits into from
Mar 17, 2023

Conversation

abhi-mosaic
Copy link
Contributor

This PR upgrades to Composer 0.13.2 and loops in all pending changes from the last few weeks of LLM runs.

Going to do some LLM runs on this branch to test and send screenshots.

abhi-mosaic and others added 30 commits February 28, 2023 12:31
@abhi-mosaic
squash
6d6477f
@abhi-mosaic
Merge branch 'main' into hero-3-3
c2573ad
@abhi-mosaic
update composer commit
1dd1c59
@abhi-mosaic
Add Lion optimizer (#200)
e8ac879
@dakinggg
Merge branch 'main' into hero-3-3
46dbc41
@dakinggg
Torchmetrics upgrade (#203)
d64db2a
@dakinggg
Low precision layer norm option (#205)
ce5d36c
@dakinggg
replace speed monitor with composer one (#207)
18387de
@dakinggg
add health checker callback (#206)
b5b57cb
@abhi-mosaic
Merge branch 'main' into hero-3-3
80d1b22
@abhi-mosaic
upgrade to release 0.13 branch
0ea0e2e
@mvpatel2000
upgrade base mosaicml
4677fb4
@abhi-mosaic
add pynvml for health checker
e44f91f
@mvpatel2000
add slack sdk
b798cd0
@codestar12
bump composer
d09e3f5
@codestar12
add lion
01b1a83
@codestar12
Merge branch 'init_tests' into hero-3-3
1b4e38c
@mvpatel2000
raise timeout
7109fb6
@abhi-mosaic
add load_ignore_keys
afc5dcf
@abhi-mosaic
remove reference to speedmonitormfu
ff7b867
@abhi-mosaic
use lpln_bias fix branch
e07ba98
@growlix
need to fix super().__init in StreamingTextDataset
5a54d0f
@codestar12
set keep_zip manually to deal with bug
fc9fc4c
@codestar12
pin torchmetrics
9d553d1
@growlix
requirements.txt is pip freeze from previous hero run
44c1322
@growlix
updated requirements.txt composer version to lp-ln fix
a2092c9
@growlix
attempted requirements.txt fix
d21152e
@growlix
requirements.txt fix attempt. mlnx-tools version
20a4584
@growlix
requirements.txt fix attempt. mlnx-tools version
b7adb8d
@growlix
requirements.txt fix attempt. commented out mlnx-tools
0e9ddf9
dakinggg
dakinggg approved these changes Mar 16, 2023
View reviewed changes
Copy link
Collaborator

@dakinggg dakinggg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once you've gotten a couple runs in successfully.
Request for manual testing:

  • a mosaicgpt run with icl
  • a gptneo run with icl
  • run through the bert readme
  • run through at least the resnet imagenet readme (we can do the rest before release)

examples/bert/yamls/test/glue.yaml Outdated Show resolved Hide resolved
examples/common/optim/__init__.py Outdated Show resolved Hide resolved
examples/common/optim/lion.py Outdated Show resolved Hide resolved
examples/llm/main.py Outdated Show resolved Hide resolved
examples/llm/requirements.txt Show resolved Hide resolved
@dakinggg
Copy link
Collaborator

Also, you'll need to bump mosaicml versions in the other subfolders to increase the top level requirement

vchiley
vchiley approved these changes Mar 16, 2023
View reviewed changes
Copy link
Contributor

@vchiley vchiley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left a few nits

examples/bert/src/glue/finetuning_jobs.py Outdated Show resolved Hide resolved
examples/bert/src/hf_bert.py Show resolved Hide resolved
examples/common/builders.py Outdated Show resolved Hide resolved
examples/common/optim/__init__.py Outdated Show resolved Hide resolved
examples/common/convert_dataset.py Outdated Show resolved Hide resolved
examples/common/optim/lion.py Show resolved Hide resolved
examples/llm/icl_eval/evaluate_model.py Show resolved Hide resolved
examples/llm/main.py Outdated Show resolved Hide resolved
examples/llm/src/models/layers/gpt_blocks.py Outdated Show resolved Hide resolved
examples/llm/src/models/mosaic_gpt.py Outdated Show resolved Hide resolved
@abhi-mosaic
Copy link
Contributor Author

CI passing, just running through some test runs now...

@abhi-mosaic
Copy link
Contributor Author

Successfully ran MosaicGPT-1.3B and OPT-2.7B both with ICL eval, and BERT pretraining, all is well!

@abhi-mosaic abhi-mosaic enabled auto-merge (squash) March 17, 2023 02:10
@abhi-mosaic abhi-mosaic merged commit 3566595 into main Mar 17, 2023
@abhi-mosaic abhi-mosaic deleted the merge-hero-changes branch March 17, 2023 02:12
@vchiley vchiley mentioned this pull request Mar 17, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dakinggg dakinggg dakinggg approved these changes

@vchiley vchiley vchiley approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@abhi-mosaic @dakinggg @vchiley @mvpatel2000 @codestar12 @growlix