Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[deepspeed] supporting --adafactor #11749

Closed
stas00 opened this issue May 17, 2021 · 0 comments · Fixed by #12257
Closed

[deepspeed] supporting --adafactor #11749

stas00 opened this issue May 17, 2021 · 0 comments · Fixed by #12257
Assignees
@stas00
Labels
DeepSpeed WIP Label your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress

Comments

@stas00
Copy link
Contributor

stas00 commented May 17, 2021

It was flagged that in this example #11044 --adafactor is used, but Deepspeed doesn't get it passed since the DS config's optimizer overrides it. So need to sort it out.
@stas00 stas00 self-assigned this May 17, 2021
@stas00 stas00 added the WIP Label your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress label Jun 17, 2021
@huggingface huggingface deleted a comment from github-actions bot Jun 17, 2021
@stas00 stas00 mentioned this issue Jun 18, 2021
Merged
@stas00 stas00 mentioned this issue Jun 29, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stas00 stas00

Labels
DeepSpeed WIP Label your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[DeepSpeed] don't ignore --adafactor stas00/transformers
1 participant
@stas00