[feature-fix-refactor][ShardedDDP] Make it possible to change trainability graph on the fly#369

Merged

blefaudeux merged 18 commits intomasterfrom shardedddp_nosync_fix

Feb 12, 2021

+409-259

Commits on Feb 5, 2021

better unit testing
blefaudeux
committed
Make it possible to refresh the DDP assumptions when the model has changed. Make it optional so that you save some time
blefaudeux
committed
minor cleanup, not no_sync needs to be properly checked
blefaudeux
committed

Commits on Feb 8, 2021

Merge branch 'master' into shardedddp_nosync_fix
blefaudeux
committed
first auto trainable take, may not be the fastest. Bug to be fixed with the OSS refresh, makes little sense
blefaudeux
committed

Commits on Feb 9, 2021

Commits on Feb 11, 2021