POLA_Project_3_-_ML_Model_3.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "POLA_ML_Model_3.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ronaldmaj/data-analysis-notebooks/blob/master/POLA_Project_3_-_ML_Model_3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "E8QdwlijA-ZU",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# POLA Ship Detection Model 3 - Google Colab Notebook"
      ]
    },
    {
      "metadata": {
        "id": "awjFUSzfBM-8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "In this notebook I will be looking at running an algoirthm to determine the presence of a container ship in an 80x80 sized image.\n",
        "\n",
        "This is in support of the project to identify container ships in the Port of Los Angeles (POLA) satellite images taken from the Planet (Planet Labs) servers for dates in 2018.\n",
        "\n",
        "This is the third notebook, which differs from the second in that additional images were added to the training and test datasets. Hopefully the larger dataset will increase the accuracy of the final model."
      ]
    },
    {
      "metadata": {
        "id": "bEb8n253A7xW",
        "colab_type": "code",
        "outputId": "6d082b2a-db3e-4c0f-a250-a76a367e3f83",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "import json, sys, random\n",
        "import numpy as np\n",
        "#! pip install tensorflow-gpu\n",
        "!pip3 install https://github.com/OlafenwaMoses/ImageAI/releases/download/2.0.2/imageai-2.0.2-py3-none-any.whl"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting imageai==2.0.2 from https://github.com/OlafenwaMoses/ImageAI/releases/download/2.0.2/imageai-2.0.2-py3-none-any.whl\n",
            "\u001b[?25l  Downloading https://github.com/OlafenwaMoses/ImageAI/releases/download/2.0.2/imageai-2.0.2-py3-none-any.whl (151kB)\n",
            "\u001b[K    100% |████████████████████████████████| 153kB 549kB/s \n",
            "\u001b[?25hInstalling collected packages: imageai\n",
            "Successfully installed imageai-2.0.2\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "hWtrNquQGwvD",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from imageai.Prediction.Custom import ModelTraining"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "_uRCwu9n_13B",
        "colab_type": "code",
        "outputId": "02bcd455-3973-48eb-d76a-e31c5b3736d4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        }
      },
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/gdrive')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code\n",
            "\n",
            "Enter your authorization code:\n",
            "··········\n",
            "Mounted at /content/gdrive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "c5Do8PuEAXqr",
        "colab_type": "code",
        "outputId": "057c055e-cef6-4060-d484-85088831fc38",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "!ls 'gdrive/My Drive/ships_sat3'"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "test  train\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "9Rdy4h9eHsnO",
        "colab_type": "code",
        "outputId": "c1c2bbf9-2e6f-4e30-f310-7edd9605dbbf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 680
        }
      },
      "cell_type": "code",
      "source": [
        "from tensorflow.python.client import device_lib\n",
        "\n",
        "print(\"Show System RAM Memory:\\n\\n\")\n",
        "!cat /proc/meminfo | egrep \"MemTotal*\"\n",
        "\n",
        "\n",
        "print(\"\\n\\nShow Devices:\\n\\n\"+str(device_lib.list_local_devices()))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Show System RAM Memory:\n",
            "\n",
            "\n",
            "MemTotal:       13335276 kB\n",
            "\n",
            "\n",
            "Show Devices:\n",
            "\n",
            "[name: \"/device:CPU:0\"\n",
            "device_type: \"CPU\"\n",
            "memory_limit: 268435456\n",
            "locality {\n",
            "}\n",
            "incarnation: 15290051623476468038\n",
            ", name: \"/device:XLA_CPU:0\"\n",
            "device_type: \"XLA_CPU\"\n",
            "memory_limit: 17179869184\n",
            "locality {\n",
            "}\n",
            "incarnation: 2239164157054362402\n",
            "physical_device_desc: \"device: XLA_CPU device\"\n",
            ", name: \"/device:XLA_GPU:0\"\n",
            "device_type: \"XLA_GPU\"\n",
            "memory_limit: 17179869184\n",
            "locality {\n",
            "}\n",
            "incarnation: 18017595014950864608\n",
            "physical_device_desc: \"device: XLA_GPU device\"\n",
            ", name: \"/device:GPU:0\"\n",
            "device_type: \"GPU\"\n",
            "memory_limit: 11276946637\n",
            "locality {\n",
            "  bus_id: 1\n",
            "  links {\n",
            "  }\n",
            "}\n",
            "incarnation: 14021411447661680770\n",
            "physical_device_desc: \"device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7\"\n",
            "]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "8wybm8YlNo_h",
        "colab_type": "code",
        "outputId": "cf457029-8ed6-4c28-fb9c-1c60c6bdfd5f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 8299
        }
      },
      "cell_type": "code",
      "source": [
        "model_trainer = ModelTraining()\n",
        "model_trainer.setModelTypeAsResNet()\n",
        "model_trainer.setDataDirectory('gdrive/My Drive/ships_sat3')\n",
        "model_trainer.trainModel(num_objects=3, num_experiments=20, enhance_data=True, batch_size=32, show_network_summary=True)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_1 (InputLayer)            (None, 224, 224, 3)  0                                            \n",
            "__________________________________________________________________________________________________\n",
            "conv2d (Conv2D)                 (None, 112, 112, 64) 9472        input_1[0][0]                    \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1 (BatchNo (None, 112, 112, 64) 256         conv2d[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "activation (Activation)         (None, 112, 112, 64) 0           batch_normalization_v1[0][0]     \n",
            "__________________________________________________________________________________________________\n",
            "max_pooling2d (MaxPooling2D)    (None, 55, 55, 64)   0           activation[0][0]                 \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_2 (Conv2D)               (None, 55, 55, 64)   4160        max_pooling2d[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_2 (Batch (None, 55, 55, 64)   256         conv2d_2[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "activation_1 (Activation)       (None, 55, 55, 64)   0           batch_normalization_v1_2[0][0]   \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_3 (Conv2D)               (None, 55, 55, 64)   36928       activation_1[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_3 (Batch (None, 55, 55, 64)   256         conv2d_3[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "activation_2 (Activation)       (None, 55, 55, 64)   0           batch_normalization_v1_3[0][0]   \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_4 (Conv2D)               (None, 55, 55, 256)  16640       activation_2[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_1 (Conv2D)               (None, 55, 55, 256)  16640       max_pooling2d[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_4 (Batch (None, 55, 55, 256)  1024        conv2d_4[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_1 (Batch (None, 55, 55, 256)  1024        conv2d_1[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "add (Add)                       (None, 55, 55, 256)  0           batch_normalization_v1_4[0][0]   \n",
            "                                                                 batch_normalization_v1_1[0][0]   \n",
            "__________________________________________________________________________________________________\n",
            "activation_3 (Activation)       (None, 55, 55, 256)  0           add[0][0]                        \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_5 (Conv2D)               (None, 55, 55, 64)   16448       activation_3[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_5 (Batch (None, 55, 55, 64)   256         conv2d_5[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "activation_4 (Activation)       (None, 55, 55, 64)   0           batch_normalization_v1_5[0][0]   \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_6 (Conv2D)               (None, 55, 55, 64)   36928       activation_4[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_6 (Batch (None, 55, 55, 64)   256         conv2d_6[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "activation_5 (Activation)       (None, 55, 55, 64)   0           batch_normalization_v1_6[0][0]   \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_7 (Conv2D)               (None, 55, 55, 256)  16640       activation_5[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_7 (Batch (None, 55, 55, 256)  1024        conv2d_7[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "add_1 (Add)                     (None, 55, 55, 256)  0           batch_normalization_v1_7[0][0]   \n",
            "                                                                 activation_3[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "activation_6 (Activation)       (None, 55, 55, 256)  0           add_1[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_8 (Conv2D)               (None, 55, 55, 64)   16448       activation_6[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_8 (Batch (None, 55, 55, 64)   256         conv2d_8[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "activation_7 (Activation)       (None, 55, 55, 64)   0           batch_normalization_v1_8[0][0]   \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_9 (Conv2D)               (None, 55, 55, 64)   36928       activation_7[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_9 (Batch (None, 55, 55, 64)   256         conv2d_9[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "activation_8 (Activation)       (None, 55, 55, 64)   0           batch_normalization_v1_9[0][0]   \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_10 (Conv2D)              (None, 55, 55, 256)  16640       activation_8[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_10 (Batc (None, 55, 55, 256)  1024        conv2d_10[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_2 (Add)                     (None, 55, 55, 256)  0           batch_normalization_v1_10[0][0]  \n",
            "                                                                 activation_6[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "activation_9 (Activation)       (None, 55, 55, 256)  0           add_2[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_12 (Conv2D)              (None, 28, 28, 128)  32896       activation_9[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_12 (Batc (None, 28, 28, 128)  512         conv2d_12[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_10 (Activation)      (None, 28, 28, 128)  0           batch_normalization_v1_12[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_13 (Conv2D)              (None, 28, 28, 128)  147584      activation_10[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_13 (Batc (None, 28, 28, 128)  512         conv2d_13[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_11 (Activation)      (None, 28, 28, 128)  0           batch_normalization_v1_13[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_14 (Conv2D)              (None, 28, 28, 512)  66048       activation_11[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_11 (Conv2D)              (None, 28, 28, 512)  131584      activation_9[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_14 (Batc (None, 28, 28, 512)  2048        conv2d_14[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_11 (Batc (None, 28, 28, 512)  2048        conv2d_11[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_3 (Add)                     (None, 28, 28, 512)  0           batch_normalization_v1_14[0][0]  \n",
            "                                                                 batch_normalization_v1_11[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "activation_12 (Activation)      (None, 28, 28, 512)  0           add_3[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_15 (Conv2D)              (None, 28, 28, 128)  65664       activation_12[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_15 (Batc (None, 28, 28, 128)  512         conv2d_15[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_13 (Activation)      (None, 28, 28, 128)  0           batch_normalization_v1_15[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_16 (Conv2D)              (None, 28, 28, 128)  147584      activation_13[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_16 (Batc (None, 28, 28, 128)  512         conv2d_16[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_14 (Activation)      (None, 28, 28, 128)  0           batch_normalization_v1_16[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_17 (Conv2D)              (None, 28, 28, 512)  66048       activation_14[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_17 (Batc (None, 28, 28, 512)  2048        conv2d_17[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_4 (Add)                     (None, 28, 28, 512)  0           batch_normalization_v1_17[0][0]  \n",
            "                                                                 activation_12[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_15 (Activation)      (None, 28, 28, 512)  0           add_4[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_18 (Conv2D)              (None, 28, 28, 128)  65664       activation_15[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_18 (Batc (None, 28, 28, 128)  512         conv2d_18[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_16 (Activation)      (None, 28, 28, 128)  0           batch_normalization_v1_18[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_19 (Conv2D)              (None, 28, 28, 128)  147584      activation_16[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_19 (Batc (None, 28, 28, 128)  512         conv2d_19[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_17 (Activation)      (None, 28, 28, 128)  0           batch_normalization_v1_19[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_20 (Conv2D)              (None, 28, 28, 512)  66048       activation_17[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_20 (Batc (None, 28, 28, 512)  2048        conv2d_20[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_5 (Add)                     (None, 28, 28, 512)  0           batch_normalization_v1_20[0][0]  \n",
            "                                                                 activation_15[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_18 (Activation)      (None, 28, 28, 512)  0           add_5[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_21 (Conv2D)              (None, 28, 28, 128)  65664       activation_18[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_21 (Batc (None, 28, 28, 128)  512         conv2d_21[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_19 (Activation)      (None, 28, 28, 128)  0           batch_normalization_v1_21[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_22 (Conv2D)              (None, 28, 28, 128)  147584      activation_19[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_22 (Batc (None, 28, 28, 128)  512         conv2d_22[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_20 (Activation)      (None, 28, 28, 128)  0           batch_normalization_v1_22[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_23 (Conv2D)              (None, 28, 28, 512)  66048       activation_20[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_23 (Batc (None, 28, 28, 512)  2048        conv2d_23[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_6 (Add)                     (None, 28, 28, 512)  0           batch_normalization_v1_23[0][0]  \n",
            "                                                                 activation_18[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_21 (Activation)      (None, 28, 28, 512)  0           add_6[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_25 (Conv2D)              (None, 14, 14, 256)  131328      activation_21[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_25 (Batc (None, 14, 14, 256)  1024        conv2d_25[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_22 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_25[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_26 (Conv2D)              (None, 14, 14, 256)  590080      activation_22[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_26 (Batc (None, 14, 14, 256)  1024        conv2d_26[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_23 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_26[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_27 (Conv2D)              (None, 14, 14, 1024) 263168      activation_23[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_24 (Conv2D)              (None, 14, 14, 1024) 525312      activation_21[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_27 (Batc (None, 14, 14, 1024) 4096        conv2d_27[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_24 (Batc (None, 14, 14, 1024) 4096        conv2d_24[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_7 (Add)                     (None, 14, 14, 1024) 0           batch_normalization_v1_27[0][0]  \n",
            "                                                                 batch_normalization_v1_24[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "activation_24 (Activation)      (None, 14, 14, 1024) 0           add_7[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_28 (Conv2D)              (None, 14, 14, 256)  262400      activation_24[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_28 (Batc (None, 14, 14, 256)  1024        conv2d_28[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_25 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_28[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_29 (Conv2D)              (None, 14, 14, 256)  590080      activation_25[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_29 (Batc (None, 14, 14, 256)  1024        conv2d_29[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_26 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_29[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_30 (Conv2D)              (None, 14, 14, 1024) 263168      activation_26[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_30 (Batc (None, 14, 14, 1024) 4096        conv2d_30[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_8 (Add)                     (None, 14, 14, 1024) 0           batch_normalization_v1_30[0][0]  \n",
            "                                                                 activation_24[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_27 (Activation)      (None, 14, 14, 1024) 0           add_8[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_31 (Conv2D)              (None, 14, 14, 256)  262400      activation_27[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_31 (Batc (None, 14, 14, 256)  1024        conv2d_31[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_28 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_31[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_32 (Conv2D)              (None, 14, 14, 256)  590080      activation_28[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_32 (Batc (None, 14, 14, 256)  1024        conv2d_32[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_29 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_32[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_33 (Conv2D)              (None, 14, 14, 1024) 263168      activation_29[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_33 (Batc (None, 14, 14, 1024) 4096        conv2d_33[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_9 (Add)                     (None, 14, 14, 1024) 0           batch_normalization_v1_33[0][0]  \n",
            "                                                                 activation_27[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_30 (Activation)      (None, 14, 14, 1024) 0           add_9[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_34 (Conv2D)              (None, 14, 14, 256)  262400      activation_30[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_34 (Batc (None, 14, 14, 256)  1024        conv2d_34[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_31 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_34[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_35 (Conv2D)              (None, 14, 14, 256)  590080      activation_31[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_35 (Batc (None, 14, 14, 256)  1024        conv2d_35[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_32 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_35[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_36 (Conv2D)              (None, 14, 14, 1024) 263168      activation_32[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_36 (Batc (None, 14, 14, 1024) 4096        conv2d_36[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_10 (Add)                    (None, 14, 14, 1024) 0           batch_normalization_v1_36[0][0]  \n",
            "                                                                 activation_30[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_33 (Activation)      (None, 14, 14, 1024) 0           add_10[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_37 (Conv2D)              (None, 14, 14, 256)  262400      activation_33[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_37 (Batc (None, 14, 14, 256)  1024        conv2d_37[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_34 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_37[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_38 (Conv2D)              (None, 14, 14, 256)  590080      activation_34[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_38 (Batc (None, 14, 14, 256)  1024        conv2d_38[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_35 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_38[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_39 (Conv2D)              (None, 14, 14, 1024) 263168      activation_35[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_39 (Batc (None, 14, 14, 1024) 4096        conv2d_39[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_11 (Add)                    (None, 14, 14, 1024) 0           batch_normalization_v1_39[0][0]  \n",
            "                                                                 activation_33[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_36 (Activation)      (None, 14, 14, 1024) 0           add_11[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_40 (Conv2D)              (None, 14, 14, 256)  262400      activation_36[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_40 (Batc (None, 14, 14, 256)  1024        conv2d_40[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_37 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_40[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_41 (Conv2D)              (None, 14, 14, 256)  590080      activation_37[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_41 (Batc (None, 14, 14, 256)  1024        conv2d_41[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_38 (Activation)      (None, 14, 14, 256)  0           batch_normalization_v1_41[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_42 (Conv2D)              (None, 14, 14, 1024) 263168      activation_38[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_42 (Batc (None, 14, 14, 1024) 4096        conv2d_42[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_12 (Add)                    (None, 14, 14, 1024) 0           batch_normalization_v1_42[0][0]  \n",
            "                                                                 activation_36[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_39 (Activation)      (None, 14, 14, 1024) 0           add_12[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_44 (Conv2D)              (None, 7, 7, 512)    524800      activation_39[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_44 (Batc (None, 7, 7, 512)    2048        conv2d_44[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_40 (Activation)      (None, 7, 7, 512)    0           batch_normalization_v1_44[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_45 (Conv2D)              (None, 7, 7, 512)    2359808     activation_40[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_45 (Batc (None, 7, 7, 512)    2048        conv2d_45[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_41 (Activation)      (None, 7, 7, 512)    0           batch_normalization_v1_45[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_46 (Conv2D)              (None, 7, 7, 2048)   1050624     activation_41[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_43 (Conv2D)              (None, 7, 7, 2048)   2099200     activation_39[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_46 (Batc (None, 7, 7, 2048)   8192        conv2d_46[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_43 (Batc (None, 7, 7, 2048)   8192        conv2d_43[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_13 (Add)                    (None, 7, 7, 2048)   0           batch_normalization_v1_46[0][0]  \n",
            "                                                                 batch_normalization_v1_43[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "activation_42 (Activation)      (None, 7, 7, 2048)   0           add_13[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_47 (Conv2D)              (None, 7, 7, 512)    1049088     activation_42[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_47 (Batc (None, 7, 7, 512)    2048        conv2d_47[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_43 (Activation)      (None, 7, 7, 512)    0           batch_normalization_v1_47[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_48 (Conv2D)              (None, 7, 7, 512)    2359808     activation_43[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_48 (Batc (None, 7, 7, 512)    2048        conv2d_48[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_44 (Activation)      (None, 7, 7, 512)    0           batch_normalization_v1_48[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_49 (Conv2D)              (None, 7, 7, 2048)   1050624     activation_44[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_49 (Batc (None, 7, 7, 2048)   8192        conv2d_49[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_14 (Add)                    (None, 7, 7, 2048)   0           batch_normalization_v1_49[0][0]  \n",
            "                                                                 activation_42[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_45 (Activation)      (None, 7, 7, 2048)   0           add_14[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_50 (Conv2D)              (None, 7, 7, 512)    1049088     activation_45[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_50 (Batc (None, 7, 7, 512)    2048        conv2d_50[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_46 (Activation)      (None, 7, 7, 512)    0           batch_normalization_v1_50[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_51 (Conv2D)              (None, 7, 7, 512)    2359808     activation_46[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_51 (Batc (None, 7, 7, 512)    2048        conv2d_51[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "activation_47 (Activation)      (None, 7, 7, 512)    0           batch_normalization_v1_51[0][0]  \n",
            "__________________________________________________________________________________________________\n",
            "conv2d_52 (Conv2D)              (None, 7, 7, 2048)   1050624     activation_47[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "batch_normalization_v1_52 (Batc (None, 7, 7, 2048)   8192        conv2d_52[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "add_15 (Add)                    (None, 7, 7, 2048)   0           batch_normalization_v1_52[0][0]  \n",
            "                                                                 activation_45[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_48 (Activation)      (None, 7, 7, 2048)   0           add_15[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "global_avg_pooling (GlobalAvera (None, 2048)         0           activation_48[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "dense (Dense)                   (None, 3)            6147        global_avg_pooling[0][0]         \n",
            "__________________________________________________________________________________________________\n",
            "activation_49 (Activation)      (None, 3)            0           dense[0][0]                      \n",
            "==================================================================================================\n",
            "Total params: 23,593,859\n",
            "Trainable params: 23,540,739\n",
            "Non-trainable params: 53,120\n",
            "__________________________________________________________________________________________________\n",
            "Using Enhanced Data Generation\n",
            "Found 7056 images belonging to 3 classes.\n",
            "Found 3384 images belonging to 3 classes.\n",
            "JSON Mapping for the model classes saved to  gdrive/My Drive/ships_sat3/json/model_class.json\n",
            "Number of experiments (Epochs) :  20\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.cast instead.\n",
            "Epoch 1/20\n",
            "106/106 [==============================] - 1469s 14s/step - loss: 1.1456 - acc: 0.3777\n",
            "\n",
            "Epoch 00001: saving model to gdrive/My Drive/ships_sat3/models/model_ex-001_acc-0.377660.h5\n",
            "221/221 [==============================] - 4660s 21s/step - loss: 1.1811 - acc: 0.5974 - val_loss: 1.1456 - val_acc: 0.3777\n",
            "Epoch 2/20\n",
            "106/106 [==============================] - 29s 270ms/step - loss: 0.8409 - acc: 0.6445\n",
            "\n",
            "Epoch 00002: saving model to gdrive/My Drive/ships_sat3/models/model_ex-002_acc-0.644504.h5\n",
            "221/221 [==============================] - 232s 1s/step - loss: 0.5715 - acc: 0.7697 - val_loss: 0.8409 - val_acc: 0.6445\n",
            "Epoch 3/20\n",
            "106/106 [==============================] - 28s 269ms/step - loss: 6.8452 - acc: 0.4022\n",
            "\n",
            "Epoch 00003: saving model to gdrive/My Drive/ships_sat3/models/model_ex-003_acc-0.402187.h5\n",
            "221/221 [==============================] - 231s 1s/step - loss: 0.4114 - acc: 0.8407 - val_loss: 6.8452 - val_acc: 0.4022\n",
            "Epoch 4/20\n",
            "106/106 [==============================] - 28s 266ms/step - loss: 0.5799 - acc: 0.7843\n",
            "\n",
            "Epoch 00004: saving model to gdrive/My Drive/ships_sat3/models/model_ex-004_acc-0.784279.h5\n",
            "221/221 [==============================] - 231s 1s/step - loss: 0.3657 - acc: 0.8654 - val_loss: 0.5799 - val_acc: 0.7843\n",
            "Epoch 5/20\n",
            "106/106 [==============================] - 28s 266ms/step - loss: 2.3088 - acc: 0.7497\n",
            "\n",
            "Epoch 00005: saving model to gdrive/My Drive/ships_sat3/models/model_ex-005_acc-0.749704.h5\n",
            "221/221 [==============================] - 231s 1s/step - loss: 0.3497 - acc: 0.8727 - val_loss: 2.3088 - val_acc: 0.7497\n",
            "Epoch 6/20\n",
            "106/106 [==============================] - 29s 269ms/step - loss: 5.7225 - acc: 0.5694\n",
            "\n",
            "Epoch 00006: saving model to gdrive/My Drive/ships_sat3/models/model_ex-006_acc-0.569444.h5\n",
            "221/221 [==============================] - 231s 1s/step - loss: 0.3283 - acc: 0.8831 - val_loss: 5.7225 - val_acc: 0.5694\n",
            "Epoch 7/20\n",
            "106/106 [==============================] - 28s 269ms/step - loss: 0.3908 - acc: 0.8664\n",
            "\n",
            "Epoch 00007: saving model to gdrive/My Drive/ships_sat3/models/model_ex-007_acc-0.866430.h5\n",
            "221/221 [==============================] - 231s 1s/step - loss: 0.2919 - acc: 0.8965 - val_loss: 0.3908 - val_acc: 0.8664\n",
            "Epoch 8/20\n",
            "106/106 [==============================] - 28s 268ms/step - loss: 0.5174 - acc: 0.8191\n",
            "\n",
            "Epoch 00008: saving model to gdrive/My Drive/ships_sat3/models/model_ex-008_acc-0.819149.h5\n",
            "221/221 [==============================] - 231s 1s/step - loss: 0.3250 - acc: 0.8961 - val_loss: 0.5174 - val_acc: 0.8191\n",
            "Epoch 9/20\n",
            "106/106 [==============================] - 29s 270ms/step - loss: 0.3307 - acc: 0.8800\n",
            "\n",
            "Epoch 00009: saving model to gdrive/My Drive/ships_sat3/models/model_ex-009_acc-0.880024.h5\n",
            "221/221 [==============================] - 231s 1s/step - loss: 0.2668 - acc: 0.9097 - val_loss: 0.3307 - val_acc: 0.8800\n",
            "Epoch 10/20\n",
            "106/106 [==============================] - 29s 271ms/step - loss: 0.2255 - acc: 0.9184\n",
            "\n",
            "Epoch 00010: saving model to gdrive/My Drive/ships_sat3/models/model_ex-010_acc-0.918440.h5\n",
            "221/221 [==============================] - 232s 1s/step - loss: 0.2014 - acc: 0.9297 - val_loss: 0.2255 - val_acc: 0.9184\n",
            "Epoch 11/20\n",
            "106/106 [==============================] - 29s 270ms/step - loss: 0.2074 - acc: 0.9317\n",
            "\n",
            "Epoch 00011: saving model to gdrive/My Drive/ships_sat3/models/model_ex-011_acc-0.931738.h5\n",
            "221/221 [==============================] - 232s 1s/step - loss: 0.1844 - acc: 0.9338 - val_loss: 0.2074 - val_acc: 0.9317\n",
            "Epoch 12/20\n",
            "106/106 [==============================] - 28s 267ms/step - loss: 0.2717 - acc: 0.9220\n",
            "\n",
            "Epoch 00012: saving model to gdrive/My Drive/ships_sat3/models/model_ex-012_acc-0.921986.h5\n",
            "221/221 [==============================] - 232s 1s/step - loss: 0.1696 - acc: 0.9375 - val_loss: 0.2717 - val_acc: 0.9220\n",
            "Epoch 13/20\n",
            "106/106 [==============================] - 28s 268ms/step - loss: 0.2260 - acc: 0.9202\n",
            "\n",
            "Epoch 00013: saving model to gdrive/My Drive/ships_sat3/models/model_ex-013_acc-0.920213.h5\n",
            "221/221 [==============================] - 232s 1s/step - loss: 0.1790 - acc: 0.9382 - val_loss: 0.2260 - val_acc: 0.9202\n",
            "Epoch 14/20\n",
            "106/106 [==============================] - 29s 269ms/step - loss: 0.2041 - acc: 0.9335\n",
            "\n",
            "Epoch 00014: saving model to gdrive/My Drive/ships_sat3/models/model_ex-014_acc-0.933511.h5\n",
            "221/221 [==============================] - 231s 1s/step - loss: 0.1653 - acc: 0.9396 - val_loss: 0.2041 - val_acc: 0.9335\n",
            "Epoch 15/20\n",
            "106/106 [==============================] - 29s 270ms/step - loss: 0.2034 - acc: 0.9335\n",
            "\n",
            "Epoch 00015: saving model to gdrive/My Drive/ships_sat3/models/model_ex-015_acc-0.933511.h5\n",
            "221/221 [==============================] - 231s 1s/step - loss: 0.1574 - acc: 0.9453 - val_loss: 0.2034 - val_acc: 0.9335\n",
            "Epoch 16/20\n",
            "106/106 [==============================] - 29s 269ms/step - loss: 0.2023 - acc: 0.9329\n",
            "\n",
            "Epoch 00016: saving model to gdrive/My Drive/ships_sat3/models/model_ex-016_acc-0.932920.h5\n",
            "221/221 [==============================] - 232s 1s/step - loss: 0.1624 - acc: 0.9435 - val_loss: 0.2023 - val_acc: 0.9329\n",
            "Epoch 17/20\n",
            "106/106 [==============================] - 29s 270ms/step - loss: 0.2014 - acc: 0.9338\n",
            "\n",
            "Epoch 00017: saving model to gdrive/My Drive/ships_sat3/models/model_ex-017_acc-0.933806.h5\n",
            "221/221 [==============================] - 232s 1s/step - loss: 0.1585 - acc: 0.9436 - val_loss: 0.2014 - val_acc: 0.9338\n",
            "Epoch 18/20\n",
            "106/106 [==============================] - 29s 270ms/step - loss: 0.2021 - acc: 0.9344\n",
            "\n",
            "Epoch 00018: saving model to gdrive/My Drive/ships_sat3/models/model_ex-018_acc-0.934397.h5\n",
            "221/221 [==============================] - 232s 1s/step - loss: 0.1585 - acc: 0.9425 - val_loss: 0.2021 - val_acc: 0.9344\n",
            "Epoch 19/20\n",
            "106/106 [==============================] - 29s 269ms/step - loss: 0.2017 - acc: 0.9338\n",
            "\n",
            "Epoch 00019: saving model to gdrive/My Drive/ships_sat3/models/model_ex-019_acc-0.933806.h5\n",
            "221/221 [==============================] - 232s 1s/step - loss: 0.1600 - acc: 0.9423 - val_loss: 0.2017 - val_acc: 0.9338\n",
            "Epoch 20/20\n",
            "106/106 [==============================] - 28s 268ms/step - loss: 0.2022 - acc: 0.9338\n",
            "\n",
            "Epoch 00020: saving model to gdrive/My Drive/ships_sat3/models/model_ex-020_acc-0.933806.h5\n",
            "221/221 [==============================] - 231s 1s/step - loss: 0.1580 - acc: 0.9443 - val_loss: 0.2022 - val_acc: 0.9338\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "TihQIRVUVDJt",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Based on these results, we can see that the model accurate has increased from the second notebook and is averaging around 93%."
      ]
    },
    {
      "metadata": {
        "id": "VCEVYboXVC7G",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}