Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate Accelerate, Deepspeed #117

Merged
merged 9 commits into from
Aug 23, 2024
Merged

Integrate Accelerate, Deepspeed #117

merged 9 commits into from
Aug 23, 2024

Conversation

Adamliu1
Copy link
Owner

@Adamliu1 Adamliu1 commented Jul 9, 2024 •
edited
Loading

Partial Fix for #115, #115 will be deprecated, tasks inside #115 will be split into multiple PRs.

This PR contains code for issues #105 #113

Here are the following todos and changes made:

  • Integrate Accelerator (commented out bf16 currently, can be used as an argument in the future, so far, using full precision)
  • Fix model saving after distributed training
  • Fix logging, including default logging and wandb. But use Accelerator's wrapper
    • wandb
    • logging
  • Currently commented out mink
  • Add documentation about installing and use accelerate, deepspeed

@Adamliu1 Adamliu1 requested a review from Willmish July 9, 2024 17:07
@Adamliu1 Adamliu1 mentioned this pull request Jul 11, 2024
Merged
@Adamliu1 Adamliu1 force-pushed the adam/accelerate_and_deepspeed branch from 7cbdaf4 to aaf4f9c Compare July 28, 2024 21:51
@Adamliu1 Adamliu1 marked this pull request as ready for review July 28, 2024 22:30
@Adamliu1 Adamliu1 requested a review from Davidyz July 29, 2024 13:47
Davidyz
Davidyz previously requested changes Jul 29, 2024
View reviewed changes
llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
@Adamliu1 Adamliu1 requested a review from Davidyz July 29, 2024 15:56
@Willmish Willmish dismissed Davidyz’s stale review August 23, 2024 11:38

addressed, looking at with a-n-j and Willmish

@Willmish Willmish linked an issue Aug 23, 2024 that may be closed by this pull request
[CODE] Code modification for accelerate with distributed training #105
Closed
TheRootOf3
TheRootOf3 requested changes Aug 23, 2024
View reviewed changes
Copy link
Collaborator

@TheRootOf3 TheRootOf3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good job, comments left 🫡

llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
llm_unlearn_ucl/unlearn_harm.py Outdated Show resolved Hide resolved
Adamliu1 and others added 2 commits August 23, 2024 17:11
@Adamliu1 Adamliu1 requested a review from TheRootOf3 August 23, 2024 16:20
TheRootOf3
TheRootOf3 approved these changes Aug 23, 2024
View reviewed changes
Copy link
Collaborator

@TheRootOf3 TheRootOf3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Adamliu1 Adamliu1 merged commit 6510942 into main Aug 23, 2024
1 check passed
@Adamliu1 Adamliu1 deleted the adam/accelerate_and_deepspeed branch August 23, 2024 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@TheRootOf3 TheRootOf3 TheRootOf3 approved these changes

@Willmish Willmish Awaiting requested review from Willmish

@Davidyz Davidyz Awaiting requested review from Davidyz

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CODE] Code modification for accelerate with distributed training
3 participants
@Adamliu1 @Davidyz @TheRootOf3