VGG Models for Image Classification

Author: oniani

README.md

@@ -1,6 +1,6 @@
 # ai
 
-_Artificial Intelligence (AI) - Contains Code for Several YouTube Series._
+From-scratch impls of AI models, approaches, tricks, and more!
 
 ## Contents
 
@@ -49,6 +49,16 @@ _Artificial Intelligence (AI) - Contains Code for Several YouTube Series._
   - [:chart_with_upwards_trend: Plot][tanh_plot]
   - [:tv: YouTube Video: Discussing and Implementing Tanh and Its Derivative Using PyTorch][tanh_youtube]
 
+### Deep Learning
+
+- Deep Learning Project Setup (+ CNN for MNIST)
+  - [:orange_book: The Reproducible MNIST][the_reproducible_mnist]
+  - [:rocket: MNIST CNN Model][mnist_cnn]
+  - [:tv: YouTube Video: Deep Learning Project Setup (+ CNN for MNIST)][deep_learning_setup_youtube]
+- VGG Models for Image Classification
+  - [:rocket: VGG Model][vgg]
+  - [:tv: YouTube Video: VGG Models for Image Classification][vgg_youtube]
+
 ### Functions
 
 - Distance Functions
@@ -67,30 +77,23 @@ _Artificial Intelligence (AI) - Contains Code for Several YouTube Series._
   - [:chart_with_upwards_trend: Convex and Non-Convex Functions][gradient_descent_plot]
   - [:tv: YouTube Video: Discussing Batch, Stochastic, and Mini-Batch Gradient Descent][gradient_descent_youtube]
 
-### Deep Learning
-
-- Deep Learning Project Setup (+ CNN for MNIST)
-  - [:orange_book: The Reproducible MNIST][the_reproducible_mnist]
-  - [:rocket: MNIST CNN Model][mnist_cnn]
-  - [:tv: YouTube Video: Deep Learning Project Setup (+ CNN for MNIST)][deep_learning_setup]
-
 ### Machine Learning Models from Scratch Using NumPy
 
 - Gaussian Naive Bayes
   - [:rocket: Implementation][gaussian_naive_bayes]
   - [:orange_book: Theory][gaussian_naive_bayes_theory]
   - [:tv: YouTube Video: Implementing Gaussian Naive Bayes from Scratch][gaussian_naive_bayes_youtube]
-  - [:tv: YouTube Video: AI/ML Model API Design and Numerical Stability (follow-up)][api_design_and_numerical_stability]
+  - [:tv: YouTube Video: AI/ML Model API Design and Numerical Stability (follow-up)][api_design_and_numerical_stability_youtube]
 - K-Nearest Neighbors (k-NN)
   - [:rocket: Implementation][k_nearest_neighbors]
   - [:orange_book: Theory][k_nearest_neighbors_theory]
   - [:tv: YouTube Video: Implementing K-Nearest Neighbors from Scratch][k_nearest_neighbors_youtube]
-  - [:tv: YouTube Video: AI/ML Model API Design and Numerical Stability (follow-up)][api_design_and_numerical_stability]
+  - [:tv: YouTube Video: AI/ML Model API Design and Numerical Stability (follow-up)][api_design_and_numerical_stability_youtube]
 - Linear Regression
   - [:rocket: Implementation][linear_regression]
   - [:orange_book: Theory][linear_regression_theory]
   - [:tv: YouTube Video: Implementing Linear Regression from Scratch][linear_regression_youtube]
-  - [:tv: YouTube Video: AI/ML Model API Design and Numerical Stability (follow-up)][api_design_and_numerical_stability]
+  - [:tv: YouTube Video: AI/ML Model API Design and Numerical Stability (follow-up)][api_design_and_numerical_stability_youtube]
 - Logistic Regression
   - [:rocket: Implementation][logistic_regression]
   - [:orange_book: Theory][logistic_regression_theory]
@@ -106,62 +109,64 @@ _Artificial Intelligence (AI) - Contains Code for Several YouTube Series._
 [MIT License][license]
 
 [license]: LICENSE
-[sigmoid]: activations/sigmoid.py
+[sigmoid]: activation/sigmoid.py
 [sigmoid_theory]: https://en.wikipedia.org/wiki/Sigmoid_function
-[sigmoid_plot]: activations/plots/sigmoid.png
+[sigmoid_plot]: activation/plots/sigmoid.png
 [sigmoid_youtube]: https://www.youtube.com/watch?v=oxC3T_-_Amw
-[relu]: activations/relu.py
+[relu]: activation/relu.py
 [relu_theory]: https://en.wikipedia.org/wiki/Rectifier_(neural_networks)
-[relu_plot]: activations/plots/relu.png
+[relu_plot]: activation/plots/relu.png
 [relu_youtube]: https://www.youtube.com/watch?v=93qjwrP7PfE
-[leaky_relu]: activations/leaky_relu.py
+[leaky_relu]: activation/leaky_relu.py
 [leaky_relu_theory]: https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Leaky_ReLU
-[leaky_relu_plot]: activations/plots/leaky_relu.png
+[leaky_relu_plot]: activation/plots/leaky_relu.png
 [leaky_relu_youtube]: https://www.youtube.com/watch?v=1HLKeWG0qnE
-[gelu]: activations/gelu.py
+[gelu]: activation/gelu.py
 [gelu_theory]: https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Gaussian-error_linear_unit_(GELU)
-[gelu_plot]: activations/plots/gelu.png
+[gelu_plot]: activation/plots/gelu.png
 [gelu_youtube]: https://www.youtube.com/watch?v=1HLKeWG0qnE
-[swish]: activations/swish.py
+[swish]: activation/swish.py
 [swish_theory]: https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#SiLU
-[swish_plot]: activations/plots/swish.png
+[swish_plot]: activation/plots/swish.png
 [swish_youtube]: https://www.youtube.com/watch?v=1HLKeWG0qnE
-[serf]: activations/serf.py
+[serf]: activation/serf.py
 [serf_theory]: https://arxiv.org/abs/2108.09598
-[serf_plot]: activations/plots/serf.png
+[serf_plot]: activation/plots/serf.png
 [serf_youtube]: https://www.youtube.com/watch?v=CLjmEuCxuT4
 [serf_reddit]: https://www.reddit.com/r/MachineLearning/comments/uhgupq/r_d_serf_activation_function_improving_swish/
-[tanh]: activations/tanh.py
+[tanh]: activation/tanh.py
 [tanh_theory]: https://en.wikipedia.org/wiki/Hyperbolic_functions#Exponential_definitions
-[tanh_plot]: activations/plots/tanh.png
+[tanh_plot]: activation/plots/tanh.png
 [tanh_youtube]: https://www.youtube.com/watch?v=MSi1tobj-jg
-[distance_functions]: functions/distance.py
+[distance_functions]: function/distance.py
 [distance_functions_theory]: https://en.wikipedia.org/wiki/Similarity_measure
 [distance_functions_youtube]: https://www.youtube.com/watch?v=50G47n42-9o
-[convolution]: functions/convolution.py
+[convolution]: function/convolution.py
 [convolution_theory]: https://en.wikipedia.org/wiki/Convolution
 [convolution_youtube]: https://www.youtube.com/watch?v=pmyulQwV62k
 [gradient_descent_theory]: theory/gradient_descent/gradient_descent.pdf
 [gradient_descent_plot]: theory/gradient_descent/convex_and_non_convex_plot.png
 [gradient_descent_youtube]: https://www.youtube.com/watch?v=mV247Fe1DJc
-[gaussian_naive_bayes]: models/gaussian_naive_bayes.py
+[gaussian_naive_bayes]: model/ml/gaussian_naive_bayes.py
 [gaussian_naive_bayes_theory]: https://en.wikipedia.org/wiki/Naive_Bayes_classifier#Gaussian_naive_Bayes
 [gaussian_naive_bayes_youtube]: https://www.youtube.com/watch?v=maJIRFeQBVI
-[k_nearest_neighbors]: models/k_nearest_neighbors.py
+[k_nearest_neighbors]: model/ml/k_nearest_neighbors.py
 [k_nearest_neighbors_theory]: https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm
 [k_nearest_neighbors_youtube]: https://www.youtube.com/watch?v=8SFTAcZb9i4
-[linear_regression]: models/linear_regression.py
+[linear_regression]: model/ml/linear_regression.py
 [linear_regression_theory]: https://en.wikipedia.org/wiki/Linear_regression
 [linear_regression_youtube]: https://www.youtube.com/watch?v=7FdQZ9r41LU
-[logistic_regression]: models/logistic_regression.py
+[logistic_regression]: model/ml/logistic_regression.py
 [logistic_regression_theory]: https://en.wikipedia.org/wiki/Logistic_regression
 [logistic_regression_computing_gradients]: theory/gradients/logistic_regression/logistic_regression.pdf
 [logistic_regression_youtube]: https://www.youtube.com/watch?v=YDa3rX9yLCE
-[k_means_clustering]: models/k_means_clustering.py
+[k_means_clustering]: model/ml/k_means_clustering.py
 [k_means_clustering_theory]: https://en.wikipedia.org/wiki/K-means_clustering
 [k_means_clustering_youtube]: https://www.youtube.com/watch?v=NfPGFSUM-nI
 [implement]: https://www.youtube.com/watch?v=maJIRFeQBVI&list=PLG8XxYPkVOUvVzz1ZKcGAJpIBK7GRrFYR
-[api_design_and_numerical_stability]: https://www.youtube.com/watch?v=BOoTX0hkO6k
-[the_reproducible_mnist]: https://github.com/oniani/mnist
-[mnist_cnn]: models/mnist_cnn.py
-[deep_learning_setup]: https://www.youtube.com/watch?v=2JkJZQP9dHg
+[api_design_and_numerical_stability_youtube]: https://www.youtube.com/watch?v=BOoTX0hkO6k
+[the_reproducible_mnist]: https://github.com/davidoniani/mnist
+[mnist_cnn]: model/dl/mnist_cnn.py
+[deep_learning_setup_youtube]: https://www.youtube.com/watch?v=2JkJZQP9dHg
+[vgg]: model/dl/vgg.py
+[vgg_youtube]: https://www.youtube.com/watch?v=0Ak4i2j_diM

activations/gelu.py activation/gelu.py

@@ -0,0 +1,54 @@
+import torch
+import torch.nn as nn
+
+
+VGG_VARIANT: dict[int, list[int | str]] = {
+    11: [64, "M", 128, "M", 256, 256, "M", 512, 512, "M", 512, 512, "M"],
+    13: [64, 64, "M", 128, 128, "M", 256, 256, "M", 512, 512, "M", 512, 512, "M"],
+    16: [64, 64, "M", 128, 128, "M", 256, 256, 256, "M", 512, 512, 512, "M", 512, 512, 512, "M"],
+    # 19: [64, 64, "M", 128, 128, "M", 256, 256, 256, 256, "M", 512, 512, 512, 512, "M", 512, 512, 512, 512, "M"],
+}
+
+
+class VGG(nn.Module):
+    """A generic VGG model representation."""
+
+    def __init__(self, variant: int) -> None:
+        """Performs inheritance and defines model blocks."""
+
+        super().__init__()
+
+        self.features = self.make_layers(VGG_VARIANT[variant])
+        self.classifier = nn.Linear(512, 10)
+
+    def forward(self, data: torch.Tensor) -> torch.Tensor:
+        """Performs a forward pass."""
+
+        out = self.features(data)
+        out = out.flatten(1)
+        out = self.classifier(out)
+
+        return out
+
+    def make_layers(self, model_config: list[int | str]) -> nn.Sequential:
+        """Uses the configuration and builds up a VGG model."""
+
+        layers = []
+        in_channels = 3
+        for val in model_config:
+            if isinstance(val, int):
+                layer = [
+                    nn.Conv2d(in_channels=in_channels, out_channels=val, kernel_size=3, padding=1),
+                    nn.BatchNorm2d(num_features=val),
+                    nn.ReLU(),
+                ]
+                layers.extend(layer)
+                in_channels = val
+            elif val == "M":
+                layers.append(nn.MaxPool2d(kernel_size=2, stride=2))
+            else:
+                raise ValueError(f"Unknown value: {val}")
+
+        layers.append(nn.AvgPool2d(kernel_size=1, stride=1))
+
+        return nn.Sequential(*layers)