Fine-tune and analyze a deep neural network for visual recognition (AlexNet) with TensorFlow.
This is a beginner-level project that starts with a pre-trained neural network and then fine-tunes it for a task of your choosing. It also includes code for analyzing the network, such as: visualizing the weights and layer activations, occlusion analysis, t-SNE and image with highest probability in class.
Code and weights for AlexNet in TensorFlow are from here (Thank you!)
- Install Python (+ numpy, matplotlib, scipy, sklearn...), Tensorflow and Jupyter Notebook. I've found that no matter what I did, Tensorflow did not work on Windows (as of Oct. 2016), therefore I used Linux (Ubuntu). I also recommend looking at other libraries for deep learning (such as Caffe, Torch, Theano...), because Tensorflow is not the only option and it's not perfect for every use. That said, TensorFlow is great for beginners who use Python.
- Run AlexNet pre-trained on ImageNet (weights and biases can be found here), and test out the classification (link).
- Visualize the behaviour of the network: look at the weights and activations of the different layers (link).
- Define the next task you would like the network to work on. As an example, I defined a task for classifying between dogs, cats and flowers. This code will download images based on a Google search. Then you'll need to crop or resize them to match the size expected by AlexNet (227x227x3) using this code. This project distinguishes between three classes, and you can change the number of classes in your task by updating the size of the fc8 layer in the network (search for n_classes).
- Fine-tune AlexNet. You can define or loop over different values of learning rates, drop-out, tuning depth (which layers will be fine-tuned), etc (link).
- Save your network (link).
- Find the images with the highest probability in each class (link).
- Visualize your network's behavior by plotting your images on a scatter plot using t-SNE (link). This was done in collaboration with Tanya Glozman.
- Occlusion analysis is a nice way of visializing which regions in your image are significant for your network's decision (link).
Probability map for correctly classifying the cat, as function of the location of an occluding square in the image. The probability for classifying the cat correctly reduces when the cat's face is occluded.
A 2-dimensional representation of the output of the network, showing the separation between images of different classes (cats, dogs and flowers).
- Deep-learning Stanford class, CS231n
- [AlexNet paper] (https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf)
- ["Visualizing and Understanding Convolutional Networks" paper] (https://arxiv.org/pdf/1311.2901v3.pdf)