博文

深度学习分布式训练相关介绍 - Part 1 多GPU训练
深度学习分布式训练相关介绍 - Part 2 详解分布式训练架构PS-Worker与Horovod
是时候放弃TensorFlow集群，拥抱Horovod了
Bringing HPC Techniques to Deep Learning
一文说清楚Tensorflow分布式训练必备知识
Speeding up BERT

---

paper

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Horovod: fast and easy distributed deep learning in TensorFlow
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
ResNet: Deep Residual Learning for Image Recognition
BERT: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding