From c844a131d57d24c73924888973b5b0e12dfb4133 Mon Sep 17 00:00:00 2001 From: pritesh2000 Date: Wed, 4 Sep 2024 20:17:47 +0530 Subject: [PATCH] all typos done --- extras/pytorch_2_intro.ipynb | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/extras/pytorch_2_intro.ipynb b/extras/pytorch_2_intro.ipynb index c5300520..de57f541 100644 --- a/extras/pytorch_2_intro.ipynb +++ b/extras/pytorch_2_intro.ipynb @@ -806,7 +806,7 @@ "\n", "Let's write some code to use a larger batch size if more GPU memory is available.\n", "\n", - "> **Note:** The ideal batch size you use will depend on the specific GPU and dataset and model you're working with. The code below is specifically targeted for the A100 GPU available on Google Colab Pro. However, you may to adjust it for your own GPU. As if you set the batch size too high, you may run into CUDA out of memory errors.\n", + "> **Note:** The ideal batch size you use will depend on the specific GPU and dataset and model you're working with. The code below is specifically targeted for the A100 GPU available on Google Colab Pro. However, you may adjust it for your own GPU. As if you set the batch size too high, you may run into CUDA out of memory errors.\n", "\n", "If the total memory on the GPU available is **above 16GB**, let's use a batch size of 128 and an image size of 224 (both of these values can be increased on GPUs with more memory).\n", "\n", @@ -985,7 +985,7 @@ "\n", "As in, the transfer speed from CPU to GPU.\n", "\n", - "As we're discussed before you want to get your data to the GPU as fast as possible.\n", + "As we've discussed before you want to get your data to the GPU as fast as possible.\n", "\n", "Let's create our `DataLoaders` using `torch.utils.data.DataLoader`.\n", "\n", @@ -1123,7 +1123,7 @@ " # 5. Optimizer step\n", " optimizer.step()\n", "\n", - " # Calculate and accumulate accuracy metric across all batches\n", + " # Calculate and accumulate accuracy metrics across all batches\n", " y_pred_class = torch.argmax(torch.softmax(y_pred, dim=1), dim=1)\n", " train_acc += (y_pred_class == y).sum().item()/len(y_pred)\n", "\n", @@ -2801,7 +2801,7 @@ "\n", "### 4.4 Save multi run results to file with GPU details\n", "\n", - "Let's also save our results dataframes for experiments 3 and 4 to file to in case we'd like to inspect them later or compare them to other kinds of models." + "Let's also save our results dataframes for experiments 3 and 4 to file in case we'd like to inspect them later or compare them to other kinds of models." ] }, { @@ -2856,7 +2856,7 @@ "For even more speedups, I'd recommend researching/trying the following:\n", "\n", "* **More powerful CPUs** - I have a sneaking suspicion that Google Colab instances are limited to 2 CPU cores, speedup numbers could be improved with more CPUs. This could be tracked via the [PyTorch Profiler](https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html) (a tool to find what processes take what time).\n", - "* **Using mixed precision training** - newer GPUs have the ability to handle difference precision types (e.g. [`torch.float16`](https://pytorch.org/docs/stable/tensors.html#data-types) and [`torch.bfloat16`](https://pytorch.org/docs/stable/generated/torch.Tensor.bfloat16.html)) which enable faster training and inference. I'd suspect you'll see an even larger speedup than we've seen here by using mixed precision training. For more on this, see the [PyTorch documentation for automatic mixed precision](https://pytorch.org/docs/stable/notes/amp_examples.html#amp-examples) (also called AMP) with PyTorch. \n", + "* **Using mixed precision training** - newer GPUs have the ability to handle different precision types (e.g. [`torch.float16`](https://pytorch.org/docs/stable/tensors.html#data-types) and [`torch.bfloat16`](https://pytorch.org/docs/stable/generated/torch.Tensor.bfloat16.html)) which enable faster training and inference. I'd suspect you'll see an even larger speedup than we've seen here by using mixed precision training. For more on this, see the [PyTorch documentation for automatic mixed precision](https://pytorch.org/docs/stable/notes/amp_examples.html#amp-examples) (also called AMP) with PyTorch. \n", "* **Transformer based models may see more *relative* speedups than convolutional models** - PyTorch 2.0 includes a [stable release for accelerated transformer models](https://pytorch.org/blog/pytorch-2.0-release/#stable-accelerated-pytorch-2-transformers) (models which use the attention mechanism). The main speedups come from an improved implementation of [`scaled_dot_product_attention()`](https://pytorch.org/docs/master/generated/torch.nn.functional.scaled_dot_product_attention.html?highlight=scaled_dot_product#torch.nn.functional.scaled_dot_product_attention) which automatically selects the best version of attention to use based on the hardware you're computing on. You can see more in the [dedicated PyTorch tutorial](https://pytorch.org/tutorials/intermediate/scaled_dot_product_attention_tutorial.html). \n", "* **Train for longer** - As previously discussed, the speedups from `torch.compile()` are likely to be more noticeable when training for longer. A great exercise would be to train over a longer number of epochs, potentially on a different dataset with a different model (e.g. a transformer) and see how the speedups compare." ]