Distributed testing#6

Merged

jeffra merged 5 commits intomasterfrom

distributed_test

Feb 4, 2020

Commits on Feb 4, 2020

Adds distributed_test decorator and some unit tests.

Shaden Smith
authored and
Shaden Smith
committed
Setting NCCL backend.

Shaden Smith
authored and
Shaden Smith
committed
Parametrizes test.

Shaden Smith
authored and
Shaden Smith
committed
rank -> local_rank

Shaden Smith
authored and
Shaden Smith
committed
Temporarily disable CUDA initialization.

Shaden Smith
authored and
Shaden Smith
committed