DeeperSpeed 2.0 is based off upstream DeepSpeed 0.8.3 and is intended to closely match the upstream along with:
Any additional fixes we find that are yet to be accepted into DeepSpeed
Branches for optimal performance on EleutherAI's compute providers (e.g. stability, coreweave)