Universidade Federal do Rio Grande do Norte

Instituto Metrópole Digital

IMD1114 - Deep Learning

References

• 📚 Adrian Rosebrock. Deep Learning for Computer Vision with Python [Link]
• 📚 Jason Brownlee. Deep Learning for Computer Vision: Image Classification, Object Detection, and Face Recognition in Python [Link]
• 📚 Jason Brownlee. Better Deep Learning [Link]
• 📚 Aurélien Géron. Hands on Machine Learning with Scikit-Learn, Keras and TensorFlow. [Link]
• 📚 François Chollet. Deep Learning with Python [Link]

Lessons

Week 01 - Introduction [Slide]

• Course Outline & Presentation
• Google Colab Introduction [Video]
• Google Colab Cont. [Video]
• Functions, Context Managers and Decorators
• Object-Oriented, Functional Programming
• Git and Version Control
  • You'll learn how to: a) organize your code using version control, b) resolve conflicts in version control, c) employ Git and Github to collaborate with others.
  • 👊 U1T1: guided project + getting a git repository.

Weeks 02 & 03 - Fundamentals of Deep Learning [Slide]

• Outline [Video]
• The perceptron [Video]
• Building Neural Networks [Video]
• Matrix Dimension [Video]
• Applying Neural Networks [Video]
• Training a Neural Networks [Video]
• Backpropagation with Pencil & Paper [Video]
• Learning rate & Batch Size [Video]
• Exponentially Weighted Average [Video]
• Adam, Momentum, RMSProp, Learning Rate Decay [Video]
• TensorFlow Crash Course
• Fundamentals of TensorFlow
• Optimization Methods
• Better Learning - Part I

Weeks 04 & 05 Better Generalization vs Better Learning [Slide]

• Better Generalization
  • Outline [Video]
  • Spliting Data [Video]
  • Bias vs Variance [Video]
  • Weight Regularization [Video]
  • Weight Constraint [Video]
  • Dropout [Video]
  • Promote Robustness with Noise [Video]
  • Early Stopping [Video]
  • Better Generalization - Part I
  • Better Generalization - Part II
• Better Learning
  • Data scaling [Video]
  • Vanishing/Exploding Gradient [Video]
  • Fix Vanishing Gradient with Relu [Video]
  • Fix Exploding Gradient with Gradient Clipping [Video]
  • Better Learning - Part II
  • Better Learning - Part III

Week 06 - Hyperparameter Tuning & Batch Normalization [Slide]

• Outline [Video]
• Hyperparameter Tuning Fundamentals [Video]
• Keras Tuner, and Weight and Biases [Video]
• Wandb - Part 01 [Video]
• Wandb - Part 02 [Video]
• Batch Normalization Fundamentals [Video]
• Batch Normalization Math Details [Video]
• Batch Normalization Case Study [Video]
• Hyperparameter tuning using keras tuner
• Hyperparameter tuning using weights and biases
• Batch Normalization

Week 07 - Fundamentals of Convolutional Neural Networks (CNN) [Slide]

• Outline [Video]
• CNN: Introduction and motivation [Video]
• Convolutional layer [Video]
• Case study of convolutional layer [Video]
• Pooling layer [Video]
• Fully connected layer [Video]
• Case study - signs dataset [Video]
• Foundations of DL for CV
• CNN case study

Week 08 - Convolutional Neural Networks (CNN) Architecture I [Slide]

• Outline [Video]
• Typical CNN Architecture [Video]
• Best practices when building your own CNN [Video]
• LeNet-5 [Video]
• Training LeNet-5 using MNIST dataset [Video]
• ImageNet & ILSVRC [Video]
• AlexNet [Video]
• Dogs vs Cats Challenge + HDF5 [Video]
• CNN Architectures - Hands on Part #01 [Video]
• CNN Architectures - Hands on Part #02 [Video]
• LeNet-5 and AlexNet applied to Cat & Dogs problem and using HDFS files

Week 09 - Convolutional Neural Networks (CNN) Architecture II [Slide]

• Outline [Video]
• VGG [Video]
• Case study using CIFAR-10 [Video]
• How to use 1x1 convolutions [Video]
• GoogLeNet [Video]
• MiniGoogleLeNet using CIFAR-10 [Video]
• DeeperGoogLeNet and Tiny ImageNet Challenge [Video]
• VGG and GoogLeNet hands on

Week 10 - Convolutional Neural Networks (CNN) Architecture III - ResNet

• ResNet hands on

Week 11 - Transfer Learning [Slide]

• Outline [Video]
• Feature extractors [Video]
• Feature extractors: case study [Video]
• Fine-tuning [Video]
• Fine-tuning: case study I - flowers 17 [Video]
• Fine-tuning: case study II - cats and dogs [Video]
• Transfer Learning hands on
• Case Study: cats and dogs

---

🎓 ML & Deep Learning Course Notes

Sharing course notes on all topics related to machine learning, NLP, and AI.

MIT 6.S191 Introduction to Deep Learning (2022)

Website | Lectures by: Alexander Amini and Ava Soleimany

Lecture	Description	Video	Notes	Author
Introduction to Deep Learning	Basic fundamentals of neural networks and deep learning.	Video	Notes	Elvis
RNNs and Transformers	Introduction to recurrent neural networks and transformers.	Video	Notes	Elvis
Deep Computer Vision	Deep Neural Networks for Computer Vision.	Video	Notes	Elvis
Deep Generative Modeling	Autoencoders and GANs.	Video	Notes	Elvis
Deep Reinforcement Learning	Deep RL key concepts and DQNs.	Video	Notes	Elvis
Limitations and New Frontiers	Limitations and New Frontiers in Deep Learning.	Video	WIP	Elvis
Autonomous Driving with LiDAR	Autonomous Driving with LiDAR.	Video	WIP	Elvis

CS224N: Natural Language Processing with Deep Learning (2022)

Website

Lecture	Description	Video	Notes	Author
Introduction and Word Vectors	Introduction to NLP and Word Vectors.	Video	Notes 🆕	Elvis
Neural Classifiers	Neural Classifiers for NLP.	Video	WIP	Elvis

CS25: Transformers United

Website | Instructors: Div Garg, Chetanya Rastogi, Advay Pal

Lecture	Description	Video	Notes	Author
Introduction to Transformers	A short summary of attention and Transformers.	Video	Notes 🆕	Elvis
Transformers in Language: GPT-3, Codex	The development of GPT Models including GPT3.	Video	WIP	Elvis