Copy_of_C3W4_Assignment_final.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "private_outputs": true,
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.4"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vNQiSujBfjWj"
      },
      "source": [
        "# **Week 4 Assignment: Saliency Maps**\n",
        "\n",
        "Welcome to the final programming exercise of this course! For this week, your task is to adapt the [Cats vs Dogs](https://www.tensorflow.org/datasets/catalog/cats_vs_dogs) Class Activation Map ungraded lab (the second ungraded lab of this week) and make it generate saliency maps instead.\n",
        "\n",
        "As discussed in the lectures, a saliency map shows the pixels which greatly impacts the classification of an image. \n",
        "- This is done by getting the gradient of the loss with respect to changes in the pixel values, then plotting the results. \n",
        "- From there, you can see if your model is looking at the correct features when classifying an image. \n",
        "  - For example, if you're building a dog breed classifier, you should be wary if your saliency map shows strong pixels outside the dog itself (e.g. sky, grass, dog house, etc...).\n",
        "\n",
        "In this assignment you will be given prompts but less starter code to fill in in. \n",
        "- It's good practice for you to try and write as much of this code as you can from memory and from searching the web.\n",
        "- **Whenever you feel stuck**, please refer back to the labs of this week to see how to write the code. In particular, look at:\n",
        "  - **Ungraded Lab 2: Cats vs Dogs CAM**\n",
        "  - **Ungraded Lab 3: Saliency**\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wDHISSfBq40T"
      },
      "source": [
        "### Download test files and weights\n",
        "\n",
        "Let's begin by first downloading files we will be using for this lab."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Laatr1c6lr1w"
      },
      "source": [
        "# Download the same test files from the Cats vs Dogs ungraded lab\n",
        "!wget -O cat1.jpg https://storage.googleapis.com/tensorflow-1-public/tensorflow-3-temp/MLColabImages/cat1.jpeg\n",
        "!wget -O cat2.jpg https://storage.googleapis.com/tensorflow-1-public/tensorflow-3-temp/MLColabImages/cat2.jpeg\n",
        "!wget -O catanddog.jpg https://storage.googleapis.com/tensorflow-1-public/tensorflow-3-temp/MLColabImages/catanddog.jpeg\n",
        "!wget -O dog1.jpg https://storage.googleapis.com/tensorflow-1-public/tensorflow-3-temp/MLColabImages/dog1.jpeg\n",
        "!wget -O dog2.jpg https://storage.googleapis.com/tensorflow-1-public/tensorflow-3-temp/MLColabImages/dog2.jpeg\n",
        "\n",
        "# Download prepared weights\n",
        "!wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=1kipXTxesGJKGY1B8uSPRvxROgOH90fih' -O 0_epochs.h5\n",
        "!wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=1oiV6tjy5k7h9OHGTQaf0Ohn3FmF-uOs1' -O 15_epochs.h5\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g24L3lKwqb3E"
      },
      "source": [
        "### Import the required packages\n",
        "\n",
        "Please import:\n",
        "\n",
        "  * Tensorflow\n",
        "  * Tensorflow Datasets\n",
        "  * Numpy\n",
        "  * Matplotlib's PyPlot\n",
        "  * Keras Models API classes you will be using\n",
        "  * Keras layers you will be using\n",
        "  * OpenCV (cv2)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X86LKLvpBO2S"
      },
      "source": [
        "%tensorflow_version 2.x\n",
        "\n",
        "\n",
        "import warnings \n",
        "warnings.filterwarnings(\"ignore\")\n",
        "\n",
        "import os\n",
        "import glob\n",
        "import cv2\n",
        "from pathlib import Path\n",
        "\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "\n",
        "from skimage.io import imread, imsave\n",
        "from skimage.transform import resize \n",
        "from sklearn.model_selection import train_test_split\n",
        "from tensorflow.keras.models import Model\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras.applications import vgg16\n",
        "from tensorflow.keras.utils import to_categorical\n",
        "from tensorflow.keras.optimizers import SGD, Adam, RMSprop\n",
        "\n",
        "import tensorflow as tf\n",
        "import tensorflow.keras.backend as K\n",
        "import tensorflow_datasets as tfds\n",
        "import tensorflow_hub as hub"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "th4dA3I8-9Ue"
      },
      "source": [
        "### Download and prepare the dataset.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y1hujOK9rDyU"
      },
      "source": [
        "#### Load Cats vs Dogs \n",
        "\n",
        "* Required: Use Tensorflow Datasets to fetch the `cats_vs_dogs` dataset. \n",
        "  * Use the first 80% of the *train* split of the said dataset to create your training set.\n",
        "  * Set the `as_supervised` flag to create `(image, label)` pairs.\n",
        "    \n",
        "* Optional: You can create validation and test sets from the remaining 20% of the *train* split of `cats_vs_dogs` (i.e. you already used 80% for the train set). This is if you intend to train the model beyond what is required for submission."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7w5HNdoHBQv_"
      },
      "source": [
        "tfds.disable_progress_bar()\n",
        "\n",
        "splits = ['train[:80%]', 'train[80%:90%]', 'train[90%:]']\n",
        "\n",
        "# load the dataset given the splits defined above\n",
        "splits, info = tfds.load('cats_vs_dogs', with_info=True, as_supervised=True, split = splits)\n",
        "\n",
        "(train_examples, validation_examples, test_examples) = splits\n",
        "\n",
        "num_examples = info.splits['train'].num_examples\n",
        "num_classes = info.features['label'].num_classes"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tXp0mV5Rbo76"
      },
      "source": [
        "#### Create preprocessing function\n",
        "\n",
        "Define a function that takes in an image and label. This will:\n",
        "  * cast the image to float32\n",
        "  * normalize the pixel values to [0, 1]\n",
        "  * resize the image to 300 x 300\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pRkrL2aK2_UZ"
      },
      "source": [
        "num_examples, num_classes\n",
        "IMG_SIZE = (300,300)\n",
        "def augmentimages(image, label):\n",
        "  image = tf.cast(image, tf.float32)\n",
        "  image /= 255.0\n",
        "  image = tf.image.resize(image, IMG_SIZE)\n",
        "\n",
        "  return image, label"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pzvF61GV32k_"
      },
      "source": [
        "#### Preprocess the training set\n",
        "\n",
        "Use the `map()` and pass in the method that you just defined to preprocess the training set.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vpNEfDKM353a"
      },
      "source": [
        "BATCH_SIZE = 32\n",
        "IMAGE_SIZE = (300,300)\n",
        "augmented_training_data = train_examples.map(augmentimages)\n",
        "\n",
        "augmented_validation_batches = validation_examples.map(augmentimages)\n",
        "augmented_test_batches = test_examples.map(augmentimages).batch(1)\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y4nFaMIMbrvA"
      },
      "source": [
        "#### Create batches of the training set. \n",
        "\n",
        "This is already provided for you. Normally, you will want to shuffle the training set. But for predictability in the grading, we will simply create the batches.\n",
        "\n",
        "```Python\n",
        "# Shuffle the data if you're working on your own personal project \n",
        "train_batches = augmented_training_data.shuffle(1024).batch(32)\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "POhDDPBY3vnL"
      },
      "source": [
        "train_batches = augmented_training_data.shuffle(1024).batch(32)\n",
        "val_batches = augmented_validation_batches.shuffle(1024).batch(32)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "za5HxgT1_Cw6"
      },
      "source": [
        "### Build the Cats vs Dogs classifier \n",
        "\n",
        "You'll define a model that is nearly the same as the one in the Cats vs. Dogs CAM lab.\n",
        "* Please preserve the architecture of the model in the Cats vs Dogs CAM lab (this week's second lab) except for the final `Dense` layer.\n",
        "* You should modify the Cats vs Dogs model at the last dense layer to output 2 neurons instead of 1. \n",
        "  - This is because you will adapt the `do_salience()` function from the lab and that works with one-hot encoded labels. \n",
        "  - You can do this by changing the `units` argument of the output Dense layer from 1 to 2, with one for each of the classes (i.e. cats and dogs).\n",
        "  - You should choose an activation that outputs a probability for each of the 2 classes (i.e. categories), where the sum of the probabilities adds up to 1."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IoyCA80GBSlG"
      },
      "source": [
        "from keras import layers, models, Sequential,initializers\n",
        "model = Sequential()\n",
        "model.add(layers.Conv2D(16, (3, 3), padding=\"same\", activation=\"relu\",\n",
        "                                    kernel_initializer = 'he_normal', input_shape=(300, 300, 3)))\n",
        "model.add(tf.keras.layers.MaxPool2D((2, 2), strides=2, padding=\"same\"))\n",
        "\n",
        "\n",
        "model.add(layers.Conv2D(32, (3, 3), padding=\"same\", activation=\"relu\", \n",
        "                        kernel_initializer = 'he_normal'))#karpathy trick\n",
        "model.add(layers.MaxPool2D((2, 2), strides=2, padding=\"same\"))\n",
        "\n",
        "model.add(layers.Conv2D(64, (3, 3), padding=\"same\", activation=\"relu\",\n",
        "                        kernel_initializer = 'he_normal'))\n",
        "model.add(layers.MaxPool2D((2, 2), strides=2, padding=\"same\"))\n",
        "\n",
        "model.add(layers.Conv2D(128, (3, 3), padding=\"same\", activation=\"relu\",\n",
        "                        kernel_initializer = 'he_normal'))\n",
        "model.add(layers.GlobalAveragePooling2D())\n",
        "\n",
        "model.add(layers.Dense(units=2, activation=\"softmax\", \n",
        "                       bias_initializer = initializers.Constant(-np.log(1/2))))\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ktnATyllHXC4"
      },
      "source": [
        "**Expected Output:**\n",
        "\n",
        "```txt\n",
        "Model: \"sequential\"\n",
        "_________________________________________________________________\n",
        "Layer (type)                 Output Shape              Param #   \n",
        "=================================================================\n",
        "conv2d (Conv2D)              (None, 300, 300, 16)      448       \n",
        "_________________________________________________________________\n",
        "max_pooling2d (MaxPooling2D) (None, 150, 150, 16)      0         \n",
        "_________________________________________________________________\n",
        "conv2d_1 (Conv2D)            (None, 150, 150, 32)      4640      \n",
        "_________________________________________________________________\n",
        "max_pooling2d_1 (MaxPooling2 (None, 75, 75, 32)        0         \n",
        "_________________________________________________________________\n",
        "conv2d_2 (Conv2D)            (None, 75, 75, 64)        18496     \n",
        "_________________________________________________________________\n",
        "max_pooling2d_2 (MaxPooling2 (None, 37, 37, 64)        0         \n",
        "_________________________________________________________________\n",
        "conv2d_3 (Conv2D)            (None, 37, 37, 128)       73856     \n",
        "_________________________________________________________________\n",
        "global_average_pooling2d (Gl (None, 128)               0         \n",
        "_________________________________________________________________\n",
        "dense (Dense)                (None, 2)                 258       \n",
        "=================================================================\n",
        "Total params: 97,698\n",
        "Trainable params: 97,698\n",
        "Non-trainable params: 0\n",
        "_________________________________________________________________\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J6nou82P_b5d"
      },
      "source": [
        "### Create a function to generate the saliency map\n",
        "\n",
        "Complete the `do_salience()` function below to save the **normalized_tensor** image. \n",
        "- The major steps are listed as comments below.\n",
        "  - Each section may involve multiple lines of code.\n",
        "- Try your best to write the code from memory or by performing web searches.\n",
        "  - Whenever you get stuck, you can review the \"saliency\" lab (the third lab of this week) to help remind you of what code to write"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sKbvh3bl9vnG"
      },
      "source": [
        "def do_salience(image, model, label, prefix):\n",
        "  '''\n",
        "  Generates the saliency map of a given image.\n",
        "\n",
        "  Args:\n",
        "    image (file) -- picture that the model will classify\n",
        "    model (keras Model) -- your cats and dogs classifier\n",
        "    label (int) -- ground truth label of the image\n",
        "    prefix (string) -- prefix to add to the filename of the saliency map\n",
        "  '''\n",
        "\n",
        "  # Read the image and convert channel order from BGR to RGB\n",
        "  img_read = cv2.imread(image)\n",
        "  img = cv2.cvtColor(img_read, cv2.COLOR_BGR2RGB)\n",
        "  \n",
        "  # Resize the image to 300 x 300 and normalize pixel values to the range [0, 1]\n",
        "  img = cv2.resize(img, (300,300))/255.0\n",
        "  kek = img\n",
        "\n",
        "  # Add an additional dimension (for the batch), and save this in a new variable\n",
        "  img = np.expand_dims(img, axis=0)\n",
        "\n",
        "\n",
        "  # Declare the number of classes\n",
        "  # YOUR CODE HERE\n",
        "  num_classes = 2\n",
        "  expected_output = tf.one_hot([label] * img.shape[0], num_classes)\n",
        "  # Define the expected output array by one-hot encoding the label\n",
        "  # The length of the array is equal to the number of classes\n",
        "  # YOUR CODE HERE\n",
        "  print(expected_output)\n",
        "\n",
        "  # Witin the GradientTape block:\n",
        "  # Cast the image as a tf.float32\n",
        "  # Use the tape to watch the float32 image\n",
        "  # Get the model's prediction by passing in the float32 image\n",
        "  # Compute an appropriate loss\n",
        "  # between the expected output and model predictions.\n",
        "  # you may want to print the predictions to see if the probabilities adds up to 1\n",
        "  # YOUR CODE HERE\n",
        "  with tf.GradientTape() as tape:\n",
        "    image_tf = tf.cast(img, tf.float32)\n",
        "    tape.watch(image_tf)\n",
        "    predictions = model(image_tf)\n",
        "    loss = tf.keras.losses.categorical_crossentropy(\n",
        "        expected_output, predictions\n",
        "    )   \n",
        "\n",
        "  # get the gradients of the loss with respect to the model's input image\n",
        "  gradients = tape.gradient(loss, image_tf)\n",
        "\n",
        "    \n",
        "  # generate the grayscale tensor\n",
        "  #destory the filter later\n",
        "  print(gradients.shape)\n",
        "  grayscale_tensor = tf.reduce_sum(tf.abs(gradients), axis=-1)\n",
        "  print(grayscale_tensor.shape)\n",
        "\n",
        "  # normalize the pixel values to be in the range [0, 255].\n",
        "\n",
        "  # the max value in the grayscale tensor will be pushed to 255.\n",
        "  # the min value will be pushed to 0.\n",
        "  # Use the formula: 255 * (x - min) / (max - min)\n",
        "  # Use tf.reduce_max, tf.reduce_min\n",
        "  # Cast the tensor as a tf.uint8\n",
        "  # YOUR CODE HERE\n",
        "  normalized_tensor = 255 * (grayscale_tensor - tf.reduce_min(grayscale_tensor)) / (tf.reduce_max(grayscale_tensor) - tf.reduce_min(grayscale_tensor))\n",
        "  normalized_tensor = tf.cast(normalized_tensor, dtype=tf.uint8)  \n",
        "    \n",
        "  # Remove dimensions that are size 1\n",
        "  normalized_tensor = tf.squeeze(normalized_tensor)\n",
        "\n",
        "    \n",
        "  # plot the normalized tensor\n",
        "  # Set the figure size to 8 by 8\n",
        "  # do not display the axis\n",
        "  # use the 'gray' colormap\n",
        "  # This code is provided for you.\n",
        "  plt.figure(figsize=(8, 8))\n",
        "  plt.axis('off')\n",
        "  plt.imshow(normalized_tensor, cmap='gray')\n",
        "  plt.show()\n",
        "\n",
        "  # optional: superimpose the saliency map with the original image, then display it.\n",
        "  # we encourage you to do this to visualize your results better\n",
        "  gradient_color = cv2.applyColorMap(normalized_tensor.numpy(), cv2.COLORMAP_HOT)\n",
        "  gradient_color = gradient_color / 255.0\n",
        "  print(type(kek))\n",
        "  print(type(gradient_color))\n",
        "  super_imposed = cv2.addWeighted(kek, 0.5, gradient_color, 0.5, 0.0)\n",
        "\n",
        "  plt.figure(figsize=(8, 8))\n",
        "  plt.imshow(super_imposed)\n",
        "  plt.axis('off')\n",
        "  plt.show()\n",
        "\n",
        "\n",
        "  # save the normalized tensor image to a file. this is already provided for you.\n",
        "  salient_image_name = prefix + image\n",
        "  normalized_tensor = tf.expand_dims(normalized_tensor, -1)\n",
        "  normalized_tensor = tf.io.encode_jpeg(normalized_tensor, quality=100, format='grayscale')\n",
        "  writer = tf.io.write_file(salient_image_name, normalized_tensor)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "li1idRy-parp"
      },
      "source": [
        "### Generate saliency maps with untrained model\n",
        "\n",
        "As a sanity check, you will load initialized (i.e. untrained) weights and use the function you just implemented. \n",
        "- This will check if you built the model correctly and are able to create a saliency map. \n",
        "\n",
        "If an error pops up when loading the weights or the function does not run, please check your implementation for bugs.\n",
        "- You can check the ungraded labs of this week.\n",
        "\n",
        "Please apply your `do_salience()` function on the following image files:\n",
        "\n",
        "* `cat1.jpg`\n",
        "* `cat2.jpg`\n",
        "* `catanddog.jpg`\n",
        "* `dog1.jpg`\n",
        "* `dog2.jpg`\n",
        "\n",
        "Cats will have the label `0` while dogs will have the label `1`. \n",
        "- For the catanddog, please use `0`. \n",
        "- For the prefix of the salience images that will be generated, please use the prefix `epoch0_salient`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k39fF4n8fgG0"
      },
      "source": [
        "# load initial weights\n",
        "model.load_weights('0_epochs.h5')\n",
        "from tensorflow.python.framework.importer import import_graph_def\n",
        "# generate the saliency maps for the 5 test images\n",
        "# YOUR CODE HERE\n",
        "do_salience(\"cat1.jpg\", model, 0, \"epoch0_salient\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8kcdyut5E2Tk"
      },
      "source": [
        "With untrained weights, you will see something like this in the output. \n",
        "- You will see strong pixels outside the cat that the model uses that when classifying the image. \n",
        "- After training that these will slowly start to localize to features inside the pet.\n",
        "\n",
        "<img src='https://drive.google.com/uc?export=view&id=1h5wP52lwbBUMVLlsgyb-tQl_I9eu42X7' alt='saliency'>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ZhZgd0x_JvN"
      },
      "source": [
        "### Configure the model for training\n",
        "\n",
        "Use `model.compile()` to define the loss, metrics and optimizer. \n",
        "\n",
        "* Choose a loss function for the model to use when training. \n",
        "  - For `model.compile()` the ground truth labels from the training set are passed to the model as **integers** (i.e. 0 or 1) as opposed to one-hot encoded vectors.\n",
        "  - The model predictions are class probabilities. \n",
        "  - You can browse the [tf.keras.losses](https://www.tensorflow.org/api_docs/python/tf/keras/losses) and determine which one is best used for this case. \n",
        "  - Remember that you can pass the function as a string (e.g. `loss = 'loss_function_a'`). \n",
        "\n",
        "* For metrics, you can measure `accuracy`. \n",
        "* For the optimizer, please use [RMSProp](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/RMSprop).\n",
        "  - Please use the default learning rate of `0.001`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DkyWZ5KdBo-z"
      },
      "source": [
        "rmsprop = RMSprop(learning_rate=0.001)\n",
        "model.compile(\n",
        "    loss=tf.keras.losses.SparseCategoricalCrossentropy(),\n",
        "    optimizer = rmsprop,\n",
        "    metrics=[\"accuracy\"]\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "otIoJJw7_ZFN"
      },
      "source": [
        "### Train your model\n",
        "\n",
        "Please pass in the training batches and train your model for just **3** epochs. \n",
        "- **Note:** Please do not exceed 3 epochs because the grader will expect 3 epochs when grading your output.\n",
        "  - After submitting your zipped folder for grading, feel free to continue training to improve your model.\n",
        "\n",
        "We have loaded pre-trained weights for 15 epochs so you can get a better output when you visualize the saliency maps."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5YSNp7k7BqfL"
      },
      "source": [
        "# load pre-trained weights\n",
        "model.load_weights('15_epochs.h5')\n",
        "\n",
        "# train the model for just 3 epochs\n",
        "model.fit(train_batches,\n",
        "          epochs=3,\n",
        "          validation_data=val_batches,\n",
        "          shuffle=True,\n",
        "          batch_size = 32)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2tTqtLN3tQJx"
      },
      "source": [
        "### Generate saliency maps at 18 epochs\n",
        "\n",
        "You will now use your `do_salience()` function again on the same test images. Please use the same parameters as before but this time, use the prefix `salient`."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "img_read = cv2.imread('cat1.jpg')\n",
        "img = cv2.cvtColor(img_read, cv2.COLOR_BGR2RGB)\n",
        "\n",
        "#Resize the image to 300 x 300 and normalize pixel values to the range [0, 1]\n",
        "img=cv2.resize(img, (300,300))/255.0\n",
        "tensor_image = np.expand_dims(img, axis=0)\n",
        "\n",
        "  # get the features and prediction\n",
        "print(tensor_image.shape)\n",
        "model.predict(tensor_image)"
      ],
      "metadata": {
        "id": "GQJEX-dAz-6p"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bXFtabyVhIKN"
      },
      "source": [
        "# YOUR CODE HERE\n",
        "do_salience('cat1.jpg', model, 0, 'salient') \n",
        "do_salience('cat2.jpg', model, 0, 'salient') \n",
        "do_salience('catanddog.jpg', model, 0, 'salient')  \n",
        "do_salience('dog1.jpg', model, 1, 'salient')  \n",
        "do_salience('dog2.jpg', model, 1, 'salient')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wGTFcfEgM6aV"
      },
      "source": [
        "You should see that the strong pixels are now very less than the ones you generated earlier. Moreover, most of them are now found on features within the pet."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rPtx-u4u_jL5"
      },
      "source": [
        "### Zip the images for grading\n",
        "\n",
        "Please run the cell below to zip the normalized tensor images you generated at 18 epochs. If you get an error, please check that you have files named:\n",
        "\n",
        "* salientcat1.jpg\n",
        "* salientcat2.jpg\n",
        "* salientcatanddog.jpg\n",
        "* salientdog1.jpg\n",
        "* salientdog2.jpg\n",
        "\n",
        "Afterwards, please download the **images.zip** from the Files bar on the left."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b-MhcA8Uh8H_"
      },
      "source": [
        "from zipfile import ZipFile\n",
        "\n",
        "!rm images.zip\n",
        "\n",
        "filenames = ['cat1.jpg', 'cat2.jpg', 'catanddog.jpg', 'dog1.jpg', 'dog2.jpg']\n",
        "\n",
        "# writing files to a zipfile \n",
        "with ZipFile('images.zip','w') as zip:\n",
        "  for file in filenames:\n",
        "    zip.write('salient' + file)\n",
        "\n",
        "print(\"images.zip generated!\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SMOgx-N55A6p"
      },
      "source": [
        "### Optional: Saliency Maps at 95 epochs\n",
        "\n",
        "We have pre-trained weights generated at 95 epochs and you can see the difference between the maps you generated at 18 epochs."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "elUfhSmMvJZh"
      },
      "source": [
        "!wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=14vFpBJsL_TNQeugX8vUTv8dYZxn__fQY' -O 95_epochs.h5\n",
        "\n",
        "model.load_weights('95_epochs.h5')\n",
        "\n",
        "do_salience('cat1.jpg', model, 0, \"epoch95_salient\")\n",
        "do_salience('cat2.jpg', model, 0, \"epoch95_salient\")\n",
        "do_salience('catanddog.jpg', model, 0, \"epoch95_salient\")\n",
        "do_salience('dog1.jpg', model, 1, \"epoch95_salient\")\n",
        "do_salience('dog2.jpg', model, 1, \"epoch95_salient\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HuKLdQhvAaTd"
      },
      "source": [
        "**Congratulations on completing this week's assignment! Please go back to the Coursera classroom and upload the zipped folder to be graded.**"
      ]
    }
  ]
}