Notebook 3 - CNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convolutional Neural Network - MNIST\n",
    "\n",
    "For those of you unfamiliar with Jupyter notebooks, this is a concise explanation.\n",
    "Each \"cell\" of the notebook (a rectangular area) contains either explanatory text (\"Markdown,\" like this cell) or Python/Keras commands (\"Code\"). By clicking on this cell, you select it (and the contours are highlighted). If you press \"__Run__\" in the menu, Jupyter processes the contents of this cell and moves on to the next. Scroll to the next cell, read the command and press __Run__ again. The result of the command (if any) will become visible. Just proceed through the notebook in this fashion, and return to previous cells, whenever necessary (either to re-read an explanation or command, or to change parameters). Please note that if you want to restart the entire notebook, you have to start at the top.\n",
    "\n",
    "Let's start with importing the required plugins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import cv2\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'\n",
    "import keras\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Convolutional Neural Networks (CNNs) are a type of neural network which is particularly effective in classification and pattern recognition of images and signals. One of the first CNN architectures was used for the recognition of digital images of handwritten digits and characters. A digital image can be represented as a matrix of values, with each cell in the matrix representing a single pixel value. When provided with this input, a CNN performs convolution and subsampling operations, which are explained below. Eventually, the CNN assigns a probability to each class, in the case of handwritten digit recognition the numbers in the range 0-9.\n",
    "\n",
    "A CNN architecture differs from other types of neural network architectures, due to the presence of convolution layers. The goal of these convolutions is to extract useful visual characteristics, commonly referred to as __features__, from the data. A convolution layer extracts task-relevant features by making use of a __filter__ that is defined by the training procedure. Convolution preserves the spatial relationship between pixels by learning image features using small squares of input data. We will not go into the mathematical details of convolution here, but will try to understand how it works over images.\n",
    "\n",
    "As we discussed above, every image can be considered as a matrix of pixel values. Consider a 5 x 5 image whose pixel values are only 0 and 1 (note that for a grayscale image, pixel values range from 0 to 255, the green matrix below is a special case where pixel values are only 0 and 1):\n",
    "\n",
    "![](images/cnn1.png)\n",
    "\n",
    "Also, consider another 3 x 3 matrix as shown below:\n",
    "\n",
    "![](images/cnn2.png)\n",
    "\n",
    "Then, the convolution of the 5 x 5 image and the 3 x 3 matrix can be computed as shown in the animation in Figure 5 below:\n",
    "\n",
    "![](images/cnn3.gif)\n",
    ">_The Convolution operation. The output matrix is called Convolved Feature or __Feature Map__._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a practical look at a very simple CNN example. We will use our CNN to classify MNIST digits, a task that you've already been through in the previous notebook. Even though our CNN will be very basic, its accuracy will still blow out of the water that of the MLP model from the previous notebook.\n",
    "\n",
    "The lines of code below show you what a basic CNN looks like. It's a stack of `Conv2D` and `MaxPooling2D` layers. Importantly, a CNN takes as input tensors of shape `(image_height, image_width, image_channels)` (not including the batch dimension). \n",
    "\n",
    "In our case, we will configure our CNN to process inputs of size `(28, 28, 1)`, which is the format of MNIST images. We do this via passing the argument `input_shape=(28, 28, 1)` to our first layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import layers\n",
    "from keras import models\n",
    "\n",
    "model = models.Sequential()\n",
    "model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(layers.MaxPooling2D((2, 2)))\n",
    "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(layers.Flatten())\n",
    "model.add(layers.Dense(64, activation='relu'))\n",
    "model.add(layers.Dense(10, activation='softmax'))\n",
    "\n",
    "print('Model configured.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's display the architecture of our CNN so far:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's train our CNN on the MNIST digits. We will reuse a lot of the code we have already covered in the MNIST example from the previous notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.datasets import mnist\n",
    "from keras.utils import to_categorical\n",
    "\n",
    "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n",
    "\n",
    "train_images = train_images.reshape((60000, 28, 28, 1))\n",
    "train_images = train_images.astype('float32') / 255\n",
    "\n",
    "test_images = test_images.reshape((10000, 28, 28, 1))\n",
    "test_images = test_images.astype('float32') / 255\n",
    "\n",
    "train_labels = to_categorical(train_labels)\n",
    "test_labels = to_categorical(test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='rmsprop',\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "model.fit(train_images, train_labels, epochs=5, batch_size=64)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's evaluate the model on the test data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_loss, test_acc = model.evaluate(test_images, test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_acc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While our MLP network from the previous notebook had a test accuracy of approximately 92%, our basic CNN has a test accuracy of 99% !\n",
    "\n",
    "Now that we have trained our CNN, we're able to make predictions:\n",
    "\n",
    "- Write down a single number on a sheet of paper, and take a picture of it with your smartphone.\n",
    "- Using your smartphone, generate a square cut-out of the picture containing the number.\n",
    "- Upload your image to the folder in which this notebook is located.\n",
    "\n",
    "Once you've uploaded your photo, please change the following variable `file` so that it contains the same filename as your photo. If you are not able to take a picture yourself, you can use an example photo by typing `images/img.jpg`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file = 'your_filename_here.jpg'\n",
    "print('Filename updated.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we can visualize our uploaded photo:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "img = cv2.imread(file,0)\n",
    "img = cv2.resize(img, (28, 28))\n",
    "img = (255-img)\n",
    "img = np.reshape(img, [1, 28, 28, 1])\n",
    "plt.imshow(img[0].reshape(28,28), cmap=plt.get_cmap('gray'))\n",
    "plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This should show significant resemblance to the visualizations of our training dataset, shown in the previous notebook. Now we provide the network with our input image, and let it generate a prediction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(np.argmax(model.predict(img)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Did the network correctly predict your written number?__\n",
    "\n",
    "This interactive Python Notebook is based on https://github.com/fchollet/deep-learning-with-python-notebooks and https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}