Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding the new feature of FPDT #6462

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
+1,176 −2
Open

Adding the new feature of FPDT #6462

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
c076827
fix the bug of deepspeed sequence parallel working with batch size la…
Aug 3, 2024
1b8a8c1
Merge branch 'master' into master
samadejacobs Aug 6, 2024
ed34e89
apply yapf formatting
Aug 7, 2024
89b119e
Formatting fixes
loadams Aug 7, 2024
7db5798
Merge branch 'microsoft:master' into master
YJHMITWEB Aug 28, 2024
0beff24
add FPDT
Aug 29, 2024
4522ed7
Merge branch 'master' into master
YJHMITWEB Aug 29, 2024
c15d1d8
Merge branch 'master' into master
tjruwase Sep 6, 2024
69f3892
modify streams
Sep 24, 2024
8ef9f5a
modify streams
Sep 24, 2024
b43c5ec
Merge branch 'master' into master
loadams Sep 27, 2024
a55d1f5
remove duplication of alltoall
Oct 7, 2024
1cbd59d
Merge branch 'master' of github.com:YJHMITWEB/DeepSpeed
Oct 7, 2024
6bfd76f
remove duplication of pos
Oct 7, 2024
4eeadca
fix format
Oct 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion deepspeed/runtime/activation_checkpointing/checkpointing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -369,7 +369,8 @@ def is_activation_to_checkpoint(item):
Is an activation to be checkpointed
"""
global mp_size
return torch.is_tensor(item) and item.is_floating_point() and item.numel() >= mp_size
extra_flag = (not hasattr(item, 'no_checkpointing')) or (hasattr(item, 'no_checkpointing') and item.no_checkpointing == False)
return torch.is_tensor(item) and item.is_floating_point() and item.numel() >= mp_size and extra_flag


def partition_activations(args, cpu_checkpoint, contiguous_checkpoint):
Expand Down
Loading