ELL 881 2018: Fundamentals of Deep Learning
Instructors: Raghavendra Singh @raghavsi & Vineet Kumar @vineetm
Teaching Assistants (TAs): Aravind @Maxaravind & Vinay @VinayKyatham
Master the fundamentals of Deep Learning and implement the ideas learnt in Tensorflow.
-
We will closely follow the Deep Learning Book
-
Relevant concepts would be implemented using Tensorflow
-
Relevant low-level and important details such as data handling and manipulation using tf.data would be taught
-
Individual Project: Implement two selected publications pertaining to NLP and Computer Vision. Further details TBD later in the course
| Lecture | Contents | Readings |
|---|---|---|
| Lec 01 (27 July) | Introduction; Logistics; Linear Algebra; Probability & Information Theory | Ch 01, 02, 03 |
| Lec 02 (03 Aug) | Numerical Computation; Machine Learning Basics | Ch 04, 05 |
| Lec 03a (08 Aug) | Deep Feedforward Networks | Ch 06 |
| Lec 03b (10 Aug) | Computational Graphs; Backprop | Ch 06 |
| Lec 04a (29 Aug) | Tensorflow, Regularization | |
| Lec 04b (31 Aug) | Regularization | |
| Lec 05a (05 Sep) | NLP Basics, NLP data pipeline | Stanford CS230 Blog |
| Lec 05b (07 Sep) | Word2Vec | Paper,Blog |
| Lec 06a (12 Sep) | Word2Vec (Contd.) RNN Basics, Language Modeling | Ch 10 |
| Lec 06b (14 Sep) | RNN (Contd.), Perplexity, Gradient Clipping, Checkpointing, Project 1 | Ch 10 |
| Project 1 START | 14 Sep 2018, 05 pm | Project1 |
| Project 1 END | 12 Oct 2018, 01 PM | |
| 02 & 09 Nov | Project 1 In-Class Viva | |
| 11 Nov | Mandatory submission of current version of Project 2 | |
| 14 Nov | Lecture on GANs | |
| 18 Nov | Project 2 End. Try to submit by 16th night (before majors) | |
| 26-28 Nov | Project2 Viva on one of these days (TBA) |
| Content | Weightage |
|---|---|
| Project | 45 % |
| End Term Exam | 20 % |
| Assignments | 15 % |
| In-class Quiz; Class Participation | 10 % |
| Minor-I,II | 10 % |
- 13 Sep 2018: lec06a notebooks have been updated with checkpoint, perplexity, dropout. Project details added!
- 13 Sep 2018: lec06a notebook has been updated with links to LSTM blog and data tx image. Added PDF!
- 12 Sep 2018: Project1 initial description is up!
- 11 Sep 2018: lec06a notebook is up! Please download data as listed in P4 section of the notebook!
- 08 Sep 2018: lec05b filled notebook is up for tf.data Exercises done in the class. Bonus Assignment is up!
- 08 Sep 2018: lec05b notebooks are up. Please sync repo before class!
- 05 Sep 2018: Assignment 2 is up! Lec05a notebooks are updated. Read note on OrderedDict
- 03 Sep 2018: Lec05a starter notebooks and data is up! Please sync your repo before class!
- 10 Aug 2018: Lec03b materials are up! Next class is on Aug 17.
- 08 Aug 2018: Lec03a materials are up!
- 08 Aug 2018: Posted readings for Week 03.
- 05 Aug 2018: Lec02 materials are up.
- 27 July 2018: Lec 01 materials are up. Refer to notes for more resources on Python & Linear Algebra.
- 26 July 2018: Course site is up!