fix sequence parallel(Ulysses) grad scale for zero0#5555

Merged

samadejacobs merged 2 commits intodeepspeedai:masterfrom

inkcherry:fix_Ulysses_grad

Jun 5, 2024

Commits on May 21, 2024

fix ds-sp grad scale for zero0
inkcherry
committed

Commits on May 28, 2024

keep the correct dp_size
inkcherry
committed