Skip to content

Commit 80f9ee9

Browse files
Update README.md
1 parent 7cb8ad3 commit 80f9ee9

File tree

1 file changed

+0
-2
lines changed

1 file changed

+0
-2
lines changed

TensorFlow/LanguageModeling/BERT/README.md

-2
Original file line numberDiff line numberDiff line change
@@ -175,8 +175,6 @@ To enable mixed precision, you can simply add the values to the environmental va
175175
176176
#### Enabling TF32
177177
178-
This section is model specific and needs to show how to enable TF32. How is TF32 being implemented? Tweaking layers, preprocessing data, etc…
179-
180178
TensorFloat-32 (TF32) is the new math mode in [NVIDIA A100](#https://www.nvidia.com/en-us/data-center/a100/) GPUs for handling the matrix math also called tensor operations. TF32 running on Tensor Cores in A100 GPUs can provide up to 10x speedups compared to single-precision floating-point math (FP32) on Volta GPUs.
181179
182180
TF32 Tensor Cores can speed up networks using FP32, typically with no loss of accuracy. It is more robust than FP16 for models which require high dynamic range for weights or activations.

0 commit comments

Comments
 (0)