Studies

Courses

• Stanford - Machine Learning [Website]
• MIT 6.S191 - Deep Learning (2021) [Website]
• David Silver - Reinforcement Learning [Website]
• Stanford - CS330: Multi-Task and Meta-Learning [Website]
• Imperial College London - Probabilistic Deep Learning [Website]
• ‪Stephen Boyd‬ - Convex Optimization Short Course [Website]
• Kaggle - Time Series [Website]
• Kaggle - Machine Learning Explainability [Website]
• Stanford - CS224n: Natural Language Processing with Deep Learning (2021) [Website]

Notable General Machine Learning Papers

• A Unified Approach to Interpreting Model Predictions (2017)

Notable Deep Learning Papers

General

• Deep Learning (2015)
• Explaining And Harnessing Adversarial Examples (2015)
• One Pixel Attack For Fooling Deep Neural Networks (2017)
• Visualizing the Loss Landscape of Neural Nets (2017)
• Batch Normalization (2015) + Yannic Kilcher's Review
• Understanding Deep Learning Requires Rethinking Generalization (2016)
• Inductive Biases for Deep Learning of Higher-Level Cognition (2021)
• Do Wide and Deep Networks Learn the Same Things? (2020)

NLP

• Word2Vec (2013) + Yannic Kilcher's Review
• Attention Is All You Need (2017) + Yannic Kilcher's Review

Vision

• Deep Residual Learning for Image Recognition (2015) + Yannic Kilcher's Review
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2020) + Yannic Kilcher's Review
• MLP-Mixer: An all-MLP Architecture for Vision (2021) + Yannic Kilcher's Review
• A Neural Algorithm of Artistic Style (2015)
• U-Net: Convolutional Networks for Biomedical Image Segmentation (2015)
• You Only Look Once: Unified, Real-Time Object Detection (2016)

Continual Learning

• Overcoming Catastrophic Forgetting In Neural Networks (2017)
• Layerwise Optimization by Gradient Decomposition for Continual Learning (2021)
• Training Networks in Null Space of Feature Covariance for Continual Learning (2021)

Other

• NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis (2020) + Yannic Kilcher's Review
• Implicit Neural Representations with Periodic Activation Functions (2020)
• Backpropagation and The Brain (2020) + Yannic Kilcher's Review
• WaveNet: A Generative Model For Raw Audio (2016) + Blog Post
• Supervised Contrastive Learning (2020) + Yannic Kilcher's Review
• Single Cortical Neurons as Deep Artificial Neural Networks (2021)

Notable Reinforcement Learning Papers

General

• Playing Atari with Deep Reinforcement Learning (2013)
• Meta-Learning through Hebbian Plasticity in Random Networks (2020) + Yannic Kilcher's Review
• Learning to Explore with Meta-Policy Gradient (2018)
• Overcoming Exploration in Reinforcement Learning with Demonstrations (2018)
• Decision Transformer: Reinforcement Learning via Sequence Modeling (2021)
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning (2017)

Imitation Learning & Inverse RL

• Generative Adversarial Imitation Learning (2016)
• End-to-end Driving via Conditional Imitation Learning (2018)
• Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments (2021)
• Playing Hard Exploration Games by Watching YouTube (2018)

Other

• Practical Deep Reinforcement Learning Approach for Stock Trading (2018)

Articles

• Understanding Variational Autoencoders (VAEs)
• Understanding Generative Adversarial Networks (GANs)
• Christopher Olah - Understanding LSTM Networks
• Distill - Attention and Augmented Recurrent Neural Networks
• Distill - Feature Visualization
• Why Uncertainty Matters in Deep Learning and How to Estimate It
• Alien Dreams: An Emerging Art Scene
• Andrej Karpathy - Software 2.0
• Sebastian Ruder - An Overview of Gradient Descent Optimization Algorithms
• Distill - A Gentle Introduction to Graph Neural Networks

Videos

• Ari Seff - What is Automatic Differentiation?
• Elliot Waite - PyTorch Autograd Explained
• Yann LeCun - Optimisation for Deep Learning
• zedstatistics - Statistical Inference