Skip to content

Commit

Permalink
Merge pull request #596 from NVIDIA/gh/release
Browse files Browse the repository at this point in the history
[Transformer-XL/TF] Updated perf table
  • Loading branch information
nv-kkudrynski authored Jul 9, 2020
2 parents 3337f72 + 40c3be6 commit ced4afc
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions TensorFlow/LanguageModeling/Transformer-XL/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ to achieve state-of-the-art accuracy and is tested and maintained by NVIDIA.
* [Training accuracy: NVIDIA DGX-2 (16x V100 32G)](#training-accuracy-nvidia-dgx-2-16x-v100-32gb)
* [Base model](#base-model-2)
* [Training loss plot](#training-loss-plot)
* [Base model](#base-model-3)
* [Training stability test](#training-stability-test)
* [Base model](#base-model-4)
* [Training performance results](#training-performance-results)
Expand Down Expand Up @@ -893,8 +892,11 @@ training iterations.

|**GPUs**|**Batch Size / GPU**|**Throughput - TF32 (tok/s)**|**Throughput - Mixed precision (tok/s)**|**Throughput speedup (TF32 to Mixed precision)**|**Weak Scaling - TF32**|**Weak Scaling - Mixed precision**|
|-------:|-------------------:|----------------------------:|---------------------------------------:|-----------------------------------------------:|----------------------:|---------------------------------:|
| 1 | 16 | 34,244 | 36,455 | 1.065 | 1.000 | 6.555 |
| 8 | 32 | 224,474 | 227,502 | 1.013 | 8.636 | 6.241 |
| 1 | 16 | 25,127 | 26,130 | 1.040 | 1.000 | 1.000 |
| 1 | 32 | 30,958 | 33,117 | 1.070 | 1.000 | 1.000 |
| 1 | 64 | 34,244 | 36,455 | 1.065 | 1.000 | 1.000 |
| 8 | 16 | 157,538 | 155,656 | 0.988 | 6.270 | 5.957 |
| 8 | 32 | 224,474 | 227,502 | 1.013 | 7.251 | 6.870 |

To achieve these same results, follow the steps in the [Quick Start Guide](#quick-start-guide).

Expand Down

0 comments on commit ced4afc

Please sign in to comment.