05_NLP/03. basic_imdb_classifier_exercise.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.5.2"
    },
    "colab": {
      "provenance": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P2f8N4D-4NmU"
      },
      "source": [
        "# NLP Notebook 1: Naive IMDB Classifier"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dqsrt-Je4NmX"
      },
      "source": [
        "%matplotlib inline\n",
        "import keras"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gh2OvKQj4Nma"
      },
      "source": [
        "## The IMDB dataset\n",
        "\n",
        "\n",
        "We'll be working with \"IMDB dataset\", a set of 50,000 highly-polarized reviews from the Internet Movie Database. They are split into 25,000\n",
        "reviews for training and 25,000 reviews for testing, each set consisting in 50% negative and 50% positive reviews.\n",
        "\n",
        "Why do we have these two separate training and test sets? You should never test a machine learning model on the same data that you used to\n",
        "train it! Just because a model performs well on its training data doesn't mean that it will perform well on data it has never seen, and\n",
        "what you actually care about is your model's performance on new data (since you already know the labels of your training data -- obviously\n",
        "you don't need your model to predict those). For instance, it is possible that your model could end up merely _memorizing_ a mapping between\n",
        "your training samples and their targets -- which would be completely useless for the task of predicting targets for data never seen before.\n",
        "\n",
        "Just like the MNIST dataset, the IMDB dataset comes packaged with Keras. It has already been preprocessed: the reviews (sequences of words)\n",
        "have been turned into sequences of integers, where each integer stands for a specific word in a dictionary.\n",
        "\n",
        "The following code will load the dataset (when you run it for the first time, about 80MB of data will be downloaded to your machine):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OHd724ya4Nma",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5d827f25-0954-486d-df61-206716c2d2b2"
      },
      "source": [
        "from keras.datasets import imdb\n",
        "\n",
        "vocab_size = 10000\n",
        "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=vocab_size)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz\n",
            "17465344/17464789 [==============================] - 0s 0us/step\n",
            "17473536/17464789 [==============================] - 0s 0us/step\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lbkwpy0qsp4T",
        "outputId": "8c9d2288-61a9-4c39-edea-4231b2145c38"
      },
      "source": [
        "train_data.shape, len(train_data[0]), len(train_data[1])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((25000,), 218, 189)"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6aUSIfy84Nmb"
      },
      "source": [
        "\n",
        "The argument `num_words=10000` means that we will only keep the top 10,000 most frequently occurring words in the training data. Rare words\n",
        "will be discarded. This allows us to work with vector data of manageable size.\n",
        "\n",
        "The variables `train_data` and `test_data` are lists of reviews, each review being a list of word indices (encoding a sequence of words).\n",
        "`train_labels` and `test_labels` are lists of 0s and 1s, where 0 stands for \"negative\" and 1 stands for \"positive\":"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": false,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fL4RFOUV4Nmb",
        "outputId": "92e87465-2b6d-4261-bf2d-ed0d16069b90"
      },
      "source": [
        "train_data[0]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1,\n",
              " 14,\n",
              " 22,\n",
              " 16,\n",
              " 43,\n",
              " 530,\n",
              " 973,\n",
              " 1622,\n",
              " 1385,\n",
              " 65,\n",
              " 458,\n",
              " 4468,\n",
              " 66,\n",
              " 3941,\n",
              " 4,\n",
              " 173,\n",
              " 36,\n",
              " 256,\n",
              " 5,\n",
              " 25,\n",
              " 100,\n",
              " 43,\n",
              " 838,\n",
              " 112,\n",
              " 50,\n",
              " 670,\n",
              " 2,\n",
              " 9,\n",
              " 35,\n",
              " 480,\n",
              " 284,\n",
              " 5,\n",
              " 150,\n",
              " 4,\n",
              " 172,\n",
              " 112,\n",
              " 167,\n",
              " 2,\n",
              " 336,\n",
              " 385,\n",
              " 39,\n",
              " 4,\n",
              " 172,\n",
              " 4536,\n",
              " 1111,\n",
              " 17,\n",
              " 546,\n",
              " 38,\n",
              " 13,\n",
              " 447,\n",
              " 4,\n",
              " 192,\n",
              " 50,\n",
              " 16,\n",
              " 6,\n",
              " 147,\n",
              " 2025,\n",
              " 19,\n",
              " 14,\n",
              " 22,\n",
              " 4,\n",
              " 1920,\n",
              " 4613,\n",
              " 469,\n",
              " 4,\n",
              " 22,\n",
              " 71,\n",
              " 87,\n",
              " 12,\n",
              " 16,\n",
              " 43,\n",
              " 530,\n",
              " 38,\n",
              " 76,\n",
              " 15,\n",
              " 13,\n",
              " 1247,\n",
              " 4,\n",
              " 22,\n",
              " 17,\n",
              " 515,\n",
              " 17,\n",
              " 12,\n",
              " 16,\n",
              " 626,\n",
              " 18,\n",
              " 2,\n",
              " 5,\n",
              " 62,\n",
              " 386,\n",
              " 12,\n",
              " 8,\n",
              " 316,\n",
              " 8,\n",
              " 106,\n",
              " 5,\n",
              " 4,\n",
              " 2223,\n",
              " 5244,\n",
              " 16,\n",
              " 480,\n",
              " 66,\n",
              " 3785,\n",
              " 33,\n",
              " 4,\n",
              " 130,\n",
              " 12,\n",
              " 16,\n",
              " 38,\n",
              " 619,\n",
              " 5,\n",
              " 25,\n",
              " 124,\n",
              " 51,\n",
              " 36,\n",
              " 135,\n",
              " 48,\n",
              " 25,\n",
              " 1415,\n",
              " 33,\n",
              " 6,\n",
              " 22,\n",
              " 12,\n",
              " 215,\n",
              " 28,\n",
              " 77,\n",
              " 52,\n",
              " 5,\n",
              " 14,\n",
              " 407,\n",
              " 16,\n",
              " 82,\n",
              " 2,\n",
              " 8,\n",
              " 4,\n",
              " 107,\n",
              " 117,\n",
              " 5952,\n",
              " 15,\n",
              " 256,\n",
              " 4,\n",
              " 2,\n",
              " 7,\n",
              " 3766,\n",
              " 5,\n",
              " 723,\n",
              " 36,\n",
              " 71,\n",
              " 43,\n",
              " 530,\n",
              " 476,\n",
              " 26,\n",
              " 400,\n",
              " 317,\n",
              " 46,\n",
              " 7,\n",
              " 4,\n",
              " 2,\n",
              " 1029,\n",
              " 13,\n",
              " 104,\n",
              " 88,\n",
              " 4,\n",
              " 381,\n",
              " 15,\n",
              " 297,\n",
              " 98,\n",
              " 32,\n",
              " 2071,\n",
              " 56,\n",
              " 26,\n",
              " 141,\n",
              " 6,\n",
              " 194,\n",
              " 7486,\n",
              " 18,\n",
              " 4,\n",
              " 226,\n",
              " 22,\n",
              " 21,\n",
              " 134,\n",
              " 476,\n",
              " 26,\n",
              " 480,\n",
              " 5,\n",
              " 144,\n",
              " 30,\n",
              " 5535,\n",
              " 18,\n",
              " 51,\n",
              " 36,\n",
              " 28,\n",
              " 224,\n",
              " 92,\n",
              " 25,\n",
              " 104,\n",
              " 4,\n",
              " 226,\n",
              " 65,\n",
              " 16,\n",
              " 38,\n",
              " 1334,\n",
              " 88,\n",
              " 12,\n",
              " 16,\n",
              " 283,\n",
              " 5,\n",
              " 16,\n",
              " 4472,\n",
              " 113,\n",
              " 103,\n",
              " 32,\n",
              " 15,\n",
              " 16,\n",
              " 5345,\n",
              " 19,\n",
              " 178,\n",
              " 32]"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "u3GoRa104Nmb",
        "outputId": "48ce1a2b-b8a8-47b5-d406-935d14c0ef3d"
      },
      "source": [
        "train_labels[0]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E9QmNu8I4Nmc"
      },
      "source": [
        "Since we restricted ourselves to the top 10,000 most frequent words, no word index will exceed 10,000:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wbCg9RU-4Nmc",
        "outputId": "63634a18-4856-461f-8ede-490bcd365296"
      },
      "source": [
        "max([max(sequence) for sequence in train_data])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "9999"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3ffEqy014Nmc"
      },
      "source": [
        "For kicks, here's how you can quickly decode one of these reviews back to English words. First we create a function to take the first n elements of a dictionary (for display purposes)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GVE49bwTw6Hv"
      },
      "source": [
        "from more_itertools import take\n",
        "\n",
        "def dict_head(dict, n):\n",
        "  return take(n, dict.items())"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0Xofa_5T4Nmd",
        "outputId": "45e65b5d-cae0-4f4a-9b18-1e491e0474f7"
      },
      "source": [
        "# word_index is a dictionary mapping words to an integer index\n",
        "word_index = imdb.get_word_index()\n",
        "dict_head(word_index, 5), len(word_index.keys())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb_word_index.json\n",
            "1646592/1641221 [==============================] - 0s 0us/step\n",
            "1654784/1641221 [==============================] - 0s 0us/step\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "([('fawn', 34701),\n",
              "  ('tsukino', 52006),\n",
              "  ('nunnery', 52007),\n",
              "  ('sonja', 16816),\n",
              "  ('vani', 63951)],\n",
              " 88584)"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kySNMm04SCEy"
      },
      "source": [
        "#### Exercise 1\n",
        "**TODO**: Reverse the dictionary content (switch keys and values) and display a review in English (instead of in numbers). Use 'xxunk' for an unknown token in the vocab.\n",
        "\n",
        "Note: the first 3 indices in the vocab are special tokens. You need to offset your indexing by 3."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pAjzgxlxugMo",
        "outputId": "8ae8a566-ae48-4867-f6b7-c49b72382f7a"
      },
      "source": [
        "reverse_word_index = #\n",
        "dict_head(reverse_word_index, 5)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(34701, 'fawn'),\n",
              " (52006, 'tsukino'),\n",
              " (52007, 'nunnery'),\n",
              " (16816, 'sonja'),\n",
              " (63951, 'vani')]"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 120
        },
        "id": "ZfF2SSmqw1wD",
        "outputId": "ef27001b-756d-4d0c-fb7c-1001c6c6e16f"
      },
      "source": [
        "decoded_review = #\n",
        "decoded_review"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "\"xxunk this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert xxunk is an amazing actor and now the same being director xxunk father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for xxunk and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also xxunk to the two little boy's that played the xxunk of norman and paul they were just brilliant children are often left out of the xxunk list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\""
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V7zVvPrn4Nmd"
      },
      "source": [
        "## Preparing the data\n",
        "\n",
        "\n",
        "Next we turn our lists of indices into tensors. There are two ways we could do that - we'll cover more options in more detail later on:\n",
        "\n",
        "* We could pad our lists so that they all have the same length, and turn them into an integer tensor of shape `(samples, word_indices)`,\n",
        "then use as first layer in our network a layer capable of handling such integer tensors (the `Embedding` layer, which we will cover in\n",
        "detail later).\n",
        "* We could one-hot-encode our lists to turn them into vectors of 0s and 1s. Concretely, this would mean for instance turning the sequence\n",
        "`[3, 5]` into a 10,000-dimensional vector that would be all-zeros except for indices 3 and 5, which would be ones. Then we could use as\n",
        "first layer in our network a `Dense` layer, capable of handling floating point vector data.\n",
        "\n",
        "We will go with the latter solution. Let's vectorize our data, which we will do manually for maximum clarity."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LTne7X2FWxbr"
      },
      "source": [
        "#### Exercise 2\n",
        "**TODO**: Create a one-hot encoded representation for each review in the train- and test data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TSPABNPg4Nme"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "def vectorize_sequences(sequences, dimension=vocab_size):\n",
        "    #\n",
        "\n",
        "# Our vectorized training data\n",
        "x_train = vectorize_sequences(train_data)\n",
        "# Our vectorized test data\n",
        "x_test = vectorize_sequences(test_data)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n3MgKJ1j4Nme"
      },
      "source": [
        "Here's what our samples look like now:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NMl01vUx4Nme",
        "outputId": "2b801a3d-edd6-4bd7-e3d1-3e2cf294996a"
      },
      "source": [
        "x_train[0]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0., 1., 1., ..., 0., 0., 0.])"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vhePODWgXgqF",
        "outputId": "2336acfa-1452-4b18-a98b-e1c7384724c2"
      },
      "source": [
        "x_train.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(25000, 10000)"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wWHe2rJd4Nme"
      },
      "source": [
        "We should also vectorize our labels, which is straightforward:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M1xqdPdv4Nmf"
      },
      "source": [
        "# Our vectorized labels\n",
        "y_train = np.asarray(train_labels).astype('float32')\n",
        "y_test = np.asarray(test_labels).astype('float32')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z-ys23DY4Nmf"
      },
      "source": [
        "Now our data is ready to be fed into a neural network."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tjglxCCL4X05"
      },
      "source": [
        "## Building our networks"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rvlzMsIU4Nmf"
      },
      "source": [
        "### Feedforward\n",
        "\n",
        "\n",
        "Our input data is simply vectors, and our labels are scalars (1s and 0s): this is the easiest setup you will ever encounter. A type of\n",
        "network that performs well on such a problem would be a simple stack of fully-connected (`Dense`) layers with `relu` activations: `Dense(16,\n",
        "activation='relu')`\n",
        "\n",
        "The argument being passed to each `Dense` layer (16) is the number of \"hidden units\" of the layer. What's a hidden unit? It's a dimension\n",
        "in the representation space of the layer. You may remember from the previous exercise that each such `Dense` layer with a `relu` activation implements\n",
        "the following chain of tensor operations:\n",
        "\n",
        "`output = relu(dot(W, input) + b)`\n",
        "\n",
        "Having 16 hidden units means that the weight matrix `W` will have shape `(input_dimension, 16)`, i.e. the dot product with `W` will project the\n",
        "input data onto a 16-dimensional representation space (and then we would add the bias vector `b` and apply the `relu` operation). You can\n",
        "intuitively understand the dimensionality of your representation space as \"how much freedom you are allowing the network to have when\n",
        "learning internal representations\". Having more hidden units (a higher-dimensional representation space) allows your network to learn more\n",
        "complex representations, but it makes your network more computationally expensive and may lead to learning unwanted patterns (patterns that\n",
        "will improve performance on the training data but not on the test data).\n",
        "\n",
        "There are two key architecture decisions to be made about such stack of dense layers:\n",
        "\n",
        "* How many layers to use.\n",
        "* How many \"hidden units\" to choose for each layer.\n",
        "\n",
        "Later on, you will learn formal principles to guide you in making these choices.\n",
        "For the time being, you will have to trust us with the following architecture choice:\n",
        "two intermediate layers with 16 hidden units each,\n",
        "and a third layer which will output the scalar prediction regarding the sentiment of the current review.\n",
        "The intermediate layers will use `relu` as their \"activation function\",\n",
        "and the final layer will use a sigmoid activation so as to output a probability\n",
        "(a score between 0 and 1, indicating how likely the sample is to have the target \"1\", i.e. how likely the review is to be positive).\n",
        "A `relu` (rectified linear unit) is a function meant to zero-out negative values,\n",
        "while a sigmoid \"squashes\" arbitrary values into the `[0, 1]` interval, thus outputting something that can be interpreted as a probability."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rvavO2wg4Nmf"
      },
      "source": [
        "Here's what our network looks like:\n",
        "\n",
        "![3-layer network](https://s3.amazonaws.com/book.keras.io/img/ch3/3_layer_network.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YEaDR8o74Nmf"
      },
      "source": [
        "And here's the Keras implementation, very similar to the first MNIST example you saw previously:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "53VO1zJ84Nmg"
      },
      "source": [
        "from keras import models\n",
        "from keras import layers\n",
        "\n",
        "model = models.Sequential()\n",
        "model.add(layers.Dense(16, activation='relu', input_shape=(vocab_size,)))\n",
        "model.add(layers.Dense(16, activation='relu'))\n",
        "model.add(layers.Dense(1, activation='sigmoid'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RXnw5oov4Nmg"
      },
      "source": [
        "\n",
        "Lastly, we need to pick a loss function and an optimizer. Since we are facing a binary classification problem and the output of our network\n",
        "is a probability (we end our network with a single-unit layer with a sigmoid activation), is it best to use the `binary_crossentropy` loss.\n",
        "It isn't the only viable choice: you could use, for instance, `mean_squared_error`. But crossentropy is usually the best choice when you\n",
        "are dealing with models that output probabilities. Crossentropy is a quantity from the field of Information Theory, that measures the \"distance\"\n",
        "between probability distributions, or in our case, between the ground-truth distribution and our predictions.\n",
        "\n",
        "Here's the step where we configure our model with the `rmsprop` optimizer and the `binary_crossentropy` loss function. Note that we will\n",
        "also monitor accuracy during training."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jHolpBiO4Nmg",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "a98e29af-809b-4ce0-811e-1530067f389e"
      },
      "source": [
        "model.compile(optimizer='rmsprop',\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "'''\n",
        "or\n",
        "model.compile(optimizer=tf.keras.optimizers.RMSprop(learning_rate=0.001),\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "'''"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "\"\\nor\\nmodel.compile(optimizer=tf.keras.optimizers.RMSprop(learning_rate=0.001),\\n              loss='binary_crossentropy',\\n              metrics=['accuracy'])      \\n\""
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KOA2x-Lb4Nmh"
      },
      "source": [
        "### Training the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "nwuVl96r4Nmi"
      },
      "source": [
        "We will now train our model for 20 epochs (20 iterations over all samples in the `x_train` and `y_train` tensors), in mini-batches of 512\n",
        "samples. At the same time we will monitor loss and accuracy on the 10,000 samples that we set apart. This is done by passing a percentage as the `validation_split` argument:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "k9fg-zTd4Nmi",
        "outputId": "2d30f8d8-38bd-4afa-d8cf-3797ee104845"
      },
      "source": [
        "history = model.fit(x_train, y_train,\n",
        "                    epochs=20,\n",
        "                    batch_size=512,\n",
        "                    validation_split=0.2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "40/40 [==============================] - 3s 37ms/step - loss: 0.4810 - accuracy: 0.8128 - val_loss: 0.3452 - val_accuracy: 0.8840\n",
            "Epoch 2/20\n",
            "40/40 [==============================] - 1s 31ms/step - loss: 0.2767 - accuracy: 0.9097 - val_loss: 0.3265 - val_accuracy: 0.8670\n",
            "Epoch 3/20\n",
            "40/40 [==============================] - 1s 31ms/step - loss: 0.2101 - accuracy: 0.9276 - val_loss: 0.2685 - val_accuracy: 0.8948\n",
            "Epoch 4/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.1724 - accuracy: 0.9413 - val_loss: 0.2945 - val_accuracy: 0.8844\n",
            "Epoch 5/20\n",
            "40/40 [==============================] - 1s 31ms/step - loss: 0.1445 - accuracy: 0.9505 - val_loss: 0.2975 - val_accuracy: 0.8846\n",
            "Epoch 6/20\n",
            "40/40 [==============================] - 1s 31ms/step - loss: 0.1237 - accuracy: 0.9590 - val_loss: 0.3107 - val_accuracy: 0.8860\n",
            "Epoch 7/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.1063 - accuracy: 0.9654 - val_loss: 0.3204 - val_accuracy: 0.8846\n",
            "Epoch 8/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0919 - accuracy: 0.9710 - val_loss: 0.3401 - val_accuracy: 0.8858\n",
            "Epoch 9/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0786 - accuracy: 0.9762 - val_loss: 0.3667 - val_accuracy: 0.8822\n",
            "Epoch 10/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0644 - accuracy: 0.9815 - val_loss: 0.3920 - val_accuracy: 0.8800\n",
            "Epoch 11/20\n",
            "40/40 [==============================] - 1s 31ms/step - loss: 0.0552 - accuracy: 0.9848 - val_loss: 0.4221 - val_accuracy: 0.8768\n",
            "Epoch 12/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0473 - accuracy: 0.9871 - val_loss: 0.4666 - val_accuracy: 0.8710\n",
            "Epoch 13/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0404 - accuracy: 0.9896 - val_loss: 0.4737 - val_accuracy: 0.8744\n",
            "Epoch 14/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0336 - accuracy: 0.9921 - val_loss: 0.5070 - val_accuracy: 0.8750\n",
            "Epoch 15/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0253 - accuracy: 0.9949 - val_loss: 0.5479 - val_accuracy: 0.8690\n",
            "Epoch 16/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0227 - accuracy: 0.9951 - val_loss: 0.5694 - val_accuracy: 0.8706\n",
            "Epoch 17/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0180 - accuracy: 0.9963 - val_loss: 0.5983 - val_accuracy: 0.8686\n",
            "Epoch 18/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0143 - accuracy: 0.9975 - val_loss: 0.6325 - val_accuracy: 0.8702\n",
            "Epoch 19/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0105 - accuracy: 0.9987 - val_loss: 0.6697 - val_accuracy: 0.8702\n",
            "Epoch 20/20\n",
            "40/40 [==============================] - 1s 30ms/step - loss: 0.0104 - accuracy: 0.9977 - val_loss: 0.7076 - val_accuracy: 0.8702\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xz6Alp2W4Nmi"
      },
      "source": [
        "On CPU, this will take less than two seconds per epoch -- training is over in 20 seconds. At the end of every epoch, there is a slight pause\n",
        "as the model computes its loss and accuracy on the 10,000 samples of the validation data.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RT7wUuOl4Nmj",
        "outputId": "267b0d63-d6c2-4fa5-dde9-7ea997213203"
      },
      "source": [
        "history_dict = history.history\n",
        "history_dict.keys()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0wYcoVcu4Nmj"
      },
      "source": [
        "It contains 4 entries: one per metric that was being monitored, during training and during validation. Let's use Matplotlib to plot the\n",
        "training and validation loss side by side, as well as the training and validation accuracy:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": false,
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "L4zMbuc94Nmj",
        "outputId": "1fd93a63-2b7b-464f-8db0-c76bd5edd641"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "epochs = range(1, len(loss) + 1)\n",
        "\n",
        "plt.plot(epochs, loss, label='Training loss')\n",
        "plt.plot(epochs, val_loss, label='Validation loss')\n",
        "plt.title('Training and validation loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1fn48c+TfSUhC2tYgiQgyB5ARBBEFFxAERf0q/J116pVa9XWVqmtv29d2loUrbi31aJ1QVQUFVRARAmIyL4GCbKEACEr2Z7fH/cSh5CEAJmZJPO8X6+8Zubec+88uZncZ845954jqooxxpjAFeTvAIwxxviXJQJjjAlwlgiMMSbAWSIwxpgAZ4nAGGMCnCUCY4wJcJYITIMSkY9E5JqGLutPIpIlImd5Yb8qIl3d5/8Qkd/Xp+xxvM+VIvLJ8cZZx35HiEh2Q+/X+F6IvwMw/iciBR4vo4CDQIX7+iZVfa2++1LVsd4o29yp6s0NsR8R6QxsAUJVtdzd92tAvf+GJvBYIjCoasyh5yKSBVyvqp9VLyciIYdOLsaY5sOahkytDlX9ReQ+EdkJvCwiLUXkAxHJEZF97vMUj22+EJHr3eeTRWShiDzhlt0iImOPs2yqiMwXkXwR+UxEponIv2uJuz4x/lFEvnL394mIJHmsv0pEtopIrog8UMfxGSwiO0Uk2GPZRSKywn0+SES+FpH9IrJDRJ4WkbBa9vWKiPzJ4/Wv3W1+EpFrq5U9T0S+E5EDIrJNRKZ4rJ7vPu4XkQIRGXLo2Hpsf5qILBGRPPfxtPoem7qIyMnu9vtFZJWIjPNYd66IrHb3uV1E7nGXJ7l/n/0isldEFoiInZd8zA64OZo2QALQCbgR5zPzsvu6I1AMPF3H9oOBdUAS8BjwoojIcZR9HfgWSASmAFfV8Z71ifEK4H+BVkAYcOjE1AN41t1/O/f9UqiBqn4DFAJnVtvv6+7zCuAu9/cZAowCbq0jbtwYxrjxjAbSgOr9E4XA1UA8cB5wi4hc6K4b7j7Gq2qMqn5dbd8JwIfAVPd3+yvwoYgkVvsdjjg2R4k5FHgf+MTd7nbgNRHp5hZ5EaeZMRY4BZjnLv8VkA0kA62B3wI27o2PWSIwR1MJPKSqB1W1WFVzVfVtVS1S1XzgEeCMOrbfqqrPq2oF8CrQFucfvt5lRaQjMBB4UFVLVXUhMKu2N6xnjC+r6npVLQbeBPq6yycCH6jqfFU9CPzePQa1+Q8wCUBEYoFz3WWo6lJVXayq5aqaBTxXQxw1udSNb6WqFuIkPs/f7wtV/UFVK1V1hft+9dkvOIljg6r+y43rP8Ba4AKPMrUdm7qcCsQAf3b/RvOAD3CPDVAG9BCRFqq6T1WXeSxvC3RS1TJVXaA2AJrPWSIwR5OjqiWHXohIlIg85zadHMBpioj3bB6pZuehJ6pa5D6NOcay7YC9HssAttUWcD1j3OnxvMgjpnae+3ZPxLm1vRfOt/8JIhIOTACWqepWN450t9ljpxvH/8OpHRzNYTEAW6v9foNF5HO36SsPuLme+z20763Vlm0F2nu8ru3YHDVmVfVMmp77vRgnSW4VkS9FZIi7/HFgI/CJiGwWkfvr92uYhmSJwBxN9W9nvwK6AYNVtQU/N0XU1tzTEHYACSIS5bGsQx3lTyTGHZ77dt8zsbbCqroa54Q3lsObhcBpYloLpLlx/PZ4YsBp3vL0Ok6NqIOqxgH/8Njv0b5N/4TTZOapI7C9HnEdbb8dqrXvV+1XVZeo6nicZqOZODUNVDVfVX+lql2AccDdIjLqBGMxx8gSgTlWsTht7vvd9uaHvP2G7jfsTGCKiIS53yYvqGOTE4nxLeB8ETnd7dh9mKP/n7wO/BIn4fy3WhwHgAIR6Q7cUs8Y3gQmi0gPNxFVjz8Wp4ZUIiKDcBLQITk4TVldatn3bCBdRK4QkRARuQzogdOMcyK+wak93CsioSIyAudvNMP9m10pInGqWoZzTCoBROR8Eenq9gXl4fSr1NUUZ7zAEoE5Vk8CkcAeYDHwsY/e90qcDtdc4E/AGzj3O9TkuGNU1VXAL3BO7juAfTidmXU51EY/T1X3eCy/B+cknQ8878Zcnxg+cn+HeTjNJvOqFbkVeFhE8oEHcb9du9sW4fSJfOVeiXNqtX3nAufj1JpygXuB86vFfcxUtRTnxD8W57g/A1ytqmvdIlcBWW4T2c04f09wOsM/AwqAr4FnVPXzE4nFHDuxfhnTFInIG8BaVfV6jcSY5s5qBKZJEJGBInKSiAS5l1eOx2lrNsacILuz2DQVbYB3cDpus4FbVPU7/4ZkTPNgTUPGGBPgrGnIGGMCXJNrGkpKStLOnTv7OwxjjGlSli5dukdVk2ta1+QSQefOncnMzPR3GMYY06SISPU7yqtY05AxxgQ4SwTGGBPgLBEYY0yAa3J9BDUpKysjOzubkpKSoxc2fhUREUFKSgqhoaH+DsUY42oWiSA7O5vY2Fg6d+5M7XOeGH9TVXJzc8nOziY1NdXf4RhjXM2iaaikpITExERLAo2ciJCYmGg1N2MamWaRCABLAk2E/Z2MaXy8mghEZIyIrBORjTXNPCQifxOR5e7PehHZ7814jDGmSSotgk9+D/trnZjvhHgtEbjTAk7DGZ+8BzDJnRi8iqrepap9VbUv8BTOoGJNTm5uLn379qVv3760adOG9u3bV70uLS2tc9vMzEzuuOOOo77Haaed1iCxfvHFF5x//vkNsi9jjA/8+A3843RYNBU2zPHKW3izs3gQsFFVNwOIyAycoYNX11J+Ej6Y7cobEhMTWb58OQBTpkwhJiaGe+65p2p9eXk5ISE1H+qMjAwyMjKO+h6LFi1qmGCNMU1DWTHM+xN8PQ3iOsDVs6DLGV55K282DbXn8Am4szl8guwqItIJSOXImZgOrb9RRDJFJDMnJ6fBA/WGyZMnc/PNNzN48GDuvfdevv32W4YMGUK/fv047bTTWLduHXD4N/QpU6Zw7bXXMmLECLp06cLUqVOr9hcTE1NVfsSIEUycOJHu3btz5ZVXcmgE2dmzZ9O9e3cGDBjAHXfccdRv/nv37uXCCy+kd+/enHrqqaxYsQKAL7/8sqpG069fP/Lz89mxYwfDhw+nb9++nHLKKSxYsKDBj5kxxpWdCf8YBl8/DQMmw62LvJYEoPFcPno58JaqVtS0UlWnA9MBMjIy6hw3+w/vr2L1TwcaNLge7Vrw0AU9j3m77OxsFi1aRHBwMAcOHGDBggWEhITw2Wef8dvf/pa33377iG3Wrl3L559/Tn5+Pt26deOWW2454pr77777jlWrVtGuXTuGDh3KV199RUZGBjfddBPz588nNTWVSZMmHTW+hx56iH79+jFz5kzmzZvH1VdfzfLly3niiSeYNm0aQ4cOpaCggIiICKZPn84555zDAw88QEVFBUVFRcd8PIwxR1FWAl/8n9MMFNsO/ucd6DrK62/rzUSwHejg8TrFXVaTy3HmiW1WLrnkEoKDgwHIy8vjmmuuYcOGDYgIZWVlNW5z3nnnER4eTnh4OK1atWLXrl2kpKQcVmbQoEFVy/r27UtWVhYxMTF06dKl6vr8SZMmMX369DrjW7hwYVUyOvPMM8nNzeXAgQMMHTqUu+++myuvvJIJEyaQkpLCwIEDufbaaykrK+PCCy+kb9++J3RsjDHVbF8GM2+BnLXQ7yo45xGIiPPJW3szESwB0kQkFScBXI4zkfdhRKQ70BJn4uoTdjzf3L0lOjq66vnvf/97Ro4cybvvvktWVhYjRoyocZvw8PCq58HBwZSXlx9XmRNx//33c9555zF79myGDh3KnDlzGD58OPPnz+fDDz9k8uTJ3H333Vx99dUN+r7GBKTyg/DlY7DwbxDTGq58C9JG+zQEr/URqGo5cBswB1gDvKmqq0TkYREZ51H0cmCGNvOp0vLy8mjf3ukieeWVVxp8/926dWPz5s1kZWUB8MYbbxx1m2HDhvHaa68BTt9DUlISLVq0YNOmTfTq1Yv77ruPgQMHsnbtWrZu3Urr1q254YYbuP7661m2bFmD/w7GBJyflsP0kbDgCehzOdz6tc+TAHi5j0BVZwOzqy17sNrrKd6MobG49957ueaaa/jTn/7Eeeed1+D7j4yM5JlnnmHMmDFER0czcODAo25zqHO6d+/eREVF8eqrrwLw5JNP8vnnnxMUFETPnj0ZO3YsM2bM4PHHHyc0NJSYmBj++c9/NvjvYEzAKC+FBX9xEkBUIkx6A7qN8Vs4TW7O4oyMDK0+Mc2aNWs4+eST/RRR41FQUEBMTAyqyi9+8QvS0tK46667/B3WEezvZQLazh+cvoCdP0Dvy2DMnyEqwetvKyJLVbXGa9WbzRATBp5//nn69u1Lz549ycvL46abbvJ3SMaYQyrKnL6A6SMhfydc9hpMmO6TJHA0jeXyUdMA7rrrrkZZAzAm4O1aBTNvhR3L4ZSLYezjEJ3o76iqWCIwxhhvKS91rgaa/7hzKeglr0LPC/0d1REsERhjjDf8tBze+wXsWgmnTISxjzWqWoAnSwTGGNOQyg/Cl4/CwichOhkufx26N/yVgg3JEoExxjSU7EynFpCzFvpcAWP+H0S29HdUR2VXDTWAkSNHMmfO4cPDPvnkk9xyyy21bjNixAgOXQZ77rnnsn//kVMxTJkyhSeeeKLO9545cyarV/88oOuDDz7IZ599dizh18iGqzbmGJQVwye/gxdHw8F85+7gi55tEkkALBE0iEmTJjFjxozDls2YMaNeA7+BM2pofHz8cb139UTw8MMPc9ZZZx3Xvowxx+HHxe58AU85YwT56e7gE2GJoAFMnDiRDz/8sGoSmqysLH766SeGDRvGLbfcQkZGBj179uShh2qebqFz587s2bMHgEceeYT09HROP/30qqGqwblHYODAgfTp04eLL76YoqIiFi1axKxZs/j1r39N37592bRpE5MnT+att94CYO7cufTr149evXpx7bXXcvDgwar3e+ihh+jfvz+9evVi7dq1df5+Nly1MTUoLYSP7oOXxkBFKVw1E8ZN9dlAcQ2p+fURfHS/c8deQ2rTC8b+udbVCQkJDBo0iI8++ojx48czY8YMLr30UkSERx55hISEBCoqKhg1ahQrVqygd+/eNe5n6dKlzJgxg+XLl1NeXk7//v0ZMGAAABMmTOCGG24A4He/+x0vvvgit99+O+PGjeP8889n4sSJh+2rpKSEyZMnM3fuXNLT07n66qt59tlnufPOOwFISkpi2bJlPPPMMzzxxBO88MILtf5+Nly1MdVsmQ/v3Qb7t8KgG2HUQxAe4++ojpvVCBqIZ/OQZ7PQm2++Sf/+/enXrx+rVq06rBmnugULFnDRRRcRFRVFixYtGDfu57H5Vq5cybBhw+jVqxevvfYaq1atqjOedevWkZqaSnp6OgDXXHMN8+fPr1o/YcIEAAYMGFA1UF1tFi5cyFVXXQXUPFz11KlT2b9/PyEhIQwcOJCXX36ZKVOm8MMPPxAbG1vnvo1pUg7mwwd3wasXgATB5Nlw7uNNOglAc6wR1PHN3ZvGjx/PXXfdxbJlyygqKmLAgAFs2bKFJ554giVLltCyZUsmT55MSUnJce1/8uTJzJw5kz59+vDKK6/wxRdfnFC8h4ayPpFhrG24ahNQNs6F938Jedkw5DYY+QCERfk7qgZhNYIGEhMTw8iRI7n22muragMHDhwgOjqauLg4du3axUcffVTnPoYPH87MmTMpLi4mPz+f999/v2pdfn4+bdu2paysrGroaIDY2Fjy8/OP2Fe3bt3Iyspi48aNAPzrX//ijDOOb6o7G67aBLQDO+Cta+HfEyA0Eq77xJk0ppkkAWiONQI/mjRpEhdddFFVE1GfPn3o168f3bt3p0OHDgwdOrTO7fv3789ll11Gnz59aNWq1WFDSf/xj39k8ODBJCcnM3jw4KqT/+WXX84NN9zA1KlTqzqJASIiInj55Ze55JJLKC8vZ+DAgdx8883H9XvZcNUmIFWUwTfPOVNHVpTBGffD6XdBaIS/I2twNgy18Tn7e5lGb+si+PAe2L0K0s6GsY9CQhd/R3VC6hqG2moExhhzSMFu+PRB+P4/ENfBGSq6+3kg4u/IvMoSgTHGVFbAkhdh3p+grAiG/cr5CYs++rbNQLNJBKqKNPOs3Rw0taZIEwC2LYEP74adK6DLCDj3CUhK83dUPtUsEkFERAS5ubkkJiZaMmjEVJXc3FwiIppfZ5tpggpz4bOH4Lt/QWw7uOQV6HFhs28GqolXE4GIjAH+DgQDL6jqERf5i8ilwBRAge9V9YpjfZ+UlBSys7PJyck5wYiNt0VERJCSkuLvMEwgq6yEZa/C3D84N4iddjuccR+EB+7Nj15LBCISDEwDRgPZwBIRmaWqqz3KpAG/AYaq6j4RaXU87xUaGkpqampDhG2Mac62L4MPfwU/LYNOp8N5T0Aru4LNmzWCQcBGVd0MICIzgPGA5xgLNwDTVHUfgKru9mI8xphAlb8LvvwzZL4MMa1gwgvQa2JANgPVxJuJoD2wzeN1NjC4Wpl0ABH5Cqf5aIqqflx9RyJyI3AjQMeOHb0SrDGmGSrMha+ehG+fd0YIHXwzjPxNkxwh1Jv83VkcAqQBI4AUYL6I9FLVw2ZpUdXpwHRwbijzdZDGmCameB8sehq++YczXHTvS51+gMST/B1Zo+TNRLAd6ODxOsVd5ikb+EZVy4AtIrIeJzEs8WJcxpjmquQALH4Wvp4GB/Og50Uw4jeQ3M3fkTVq3kwES4A0EUnFSQCXA9WvCJoJTAJeFpEknKaizV6MyRjTHB0sgG+nw6KpTm2g+/lOAmhzir8jaxK8lghUtVxEbgPm4LT/v6Sqq0TkYSBTVWe5684WkdVABfBrVc31VkzGmGamrNi5I3jh36BojzMu0MjfQrt+/o6sSWkWg84ZYwJM+UFY+ios+AsU7HTuCB75AHQY5O/IGi0bdM4Y0zxUlMF3/4b5T8CBbOg0FCa+BJ3rHuLd1M0SgTGm8SsrgZVvwZePOfMEpwyE8U87NQG7F+CEWSIwxjROqrBjuVMD+OG/UJIHbfs6g8KljbYE0IAsERhjGpfCPbDiDfjuNWdimJAIOPkC6Hul1QC8xBKBMcb/Ksph42fOSKDrP4bKcmjXH877K5xyMUTG+zvCZs0SgTHGf3LWw/J/w/czoGAXRCU5w0D0vRJa9/B3dAHDEoExxrdKDsCqd5ymn+xvQYIh/Rzn5J9+DgSH+jvCgGOJwBjjG1lfOU0/q2ZCeTEkdYPRf4Tel0Fsa39HF9AsERhjvOtgPsz+tTMhfHgL6HMZ9LsK2g+wjt9GwhKBMcZ7ti+Ft6+HfVkw/F44/S4Ii/J3VKYaSwTGmIZXWekMADfvjxDTBiZ/CJ1O83dUphaWCIwxDSt/J7x7E2z+Ak4eB+OmQmRLf0dl6mCJwBjTcNZ9DO/dCqVFcMFU6H+19QM0AZYIjDEnrqwEPn0Qvn0OWveCiS/aZDBNiCUCY8yJ2b0W3r4Odq2EU2+FUQ9BaIS/ozLHwBKBMeb4qMLSl+Hj30JYNFzxX0g/299RmeNgicAYc+yK9sL7d8Ca96HLSLjoObsprAmzRGCMOTZZC+GdG6Fgt3Nn8JDbICjI31GZE2CJwBhTPxXl8OWjsOAJaJkK139qcwM3E15N4yIyRkTWichGEbm/hvWTRSRHRJa7P9d7Mx5jzHHavRZeORfmPwZ9roCb5lsSaEa8ViMQkWBgGjAayAaWiMgsVV1dregbqnqbt+IwxpyAkjz44s/wzXMQHgsXvwi9Jvo7KtPAvNk0NAjYqKqbAURkBjAeqJ4IjDGNTWUlfP86fDbFmTFswGQ48/cQnejvyIwXeDMRtAe2ebzOBgbXUO5iERkOrAfuUtVt1QuIyI3AjQAdO3b0QqjGmCrbl8Lse2F7JnQYDFe+Be36+jsq40X+7up/H+isqr2BT4FXayqkqtNVNUNVM5KTk30aoDEBoyAH3rsNnh8FeducS0KvnWNJIAB4s0awHejg8TrFXVZFVXM9Xr4APObFeIwxNakohyXPw+f/B2WFcNptzpDRES38HZnxEW8mgiVAmoik4iSAy4ErPAuISFtV3eG+HAes8WI8xpjqtsx3moFy1sBJZ8KYRyE53d9RGR/zWiJQ1XIRuQ2YAwQDL6nqKhF5GMhU1VnAHSIyDigH9gKTvRXPvsJSlmfvZ2S3Vt56C2Oajv3b4JPfweqZEN8RLnsNup9nI4UGKFFVf8dwTDIyMjQzM/OYt5s6dwN/+2w9mQ+cRWJMuBciM6YJKCuBRU/Bgr8ACqffDUPvgNBIf0dmvExElqpqRk3r/N1Z7DPD05NRhYUb9/g7FGN8S9UZG2j1LHhmMHz+J0gbDbctgRH3WRIwgTPERK/2ccRHhTJ//R7G923v73CMaVjF+2H/j7B/q/v4I+zzeF6a75RL6gZXvwddRvgzWtPIBEwiCA4STu+axIINOagqYm2hpqnZs8H5OeyEvxX2/QgH8w4vGxYD8Z2gZSdIHeb0AyR0ga5nQXCof+I3jVbAJAJwmoc+WLGDtTvzObmtXRpnmojyUvj4fsh88edloVHOiT6+I3Q41Tnhx3d0fzo5cwTblx1TTwGVCIalJQEwf32OJQLTNBz4Cd68GrKXOMM9nzLBOdFHJdqJ3jSYgEoEbeMiSW8dw4INe7jpjJP8HY4xdcv6Cv57DZQVwyWvQs8L/R2RaaYC5qqhQ4anJfNt1l6KSyv8HYoxNVOFr5+BVy+AiDi4fq4lAeNVgZcI0pMpLa9k8Zbcoxc2xtdKC+Ht62HObyB9DNzwObTq7u+oTDMXcIlgUGoC4SFBzF+f4+9QjDlc7iZ4YTSsfNsZ8vmyf9t4P8YnAqqPACAiNJjBXRItEZjGZf0cePsGZ+7f/3nLuczTGB8JuBoBwPC0JDblFLJ9f7G/QzGBrrLSmQHs9UuhZUe48QtLAsbnAjMRpDtzGiywWoHxp+L9MGMSfPF/0GcSXPcptOzs76hMAArIRJDWKoY2LSKYv8ESgfGTXatg+gjY+Bmc+wRc+KyN+WP8JuD6CABEhOHpSXy8ciflFZWEBAdkPjT+8sNbMOt2CG8Bkz+Ejqf6OyIT4AL2DDg8PZkDJeV8n5139MLGNISKMvj4t/D2ddC2D9z0pSUB0ygEbCIYelISItjVQ8Y3cjfBP8fD4mkw+Ga45n2IbePvqIwBAjgRtIwOo3dKPAusn8B404Ed8P6d8PRA+Gk5XDQdxj5qI4CaRiUg+wgOOSMtiac/30heURlxUfaPaRpQ8T5Y+CR88w+orICB18PweyDGpko1jU/A1gjA6SeoVPhqk81aZhpIaREs+Cv8vQ989XfocSHcngnnPmZJwDRaXk0EIjJGRNaJyEYRub+OcheLiIpIjfNpekvfDvHEhodYP4E5cRVlsORFmNoP5v4BOg6BmxfChOfs3gDT6HmtaUhEgoFpwGggG1giIrNUdXW1crHAL4FvvBVLbUKCgzitqzPchM1aZo5LZSWsegfm/Qn2bXEmibnkFeg0xN+RGVNv3qwRDAI2qupmVS0FZgDjayj3R+BRoMSLsdRqeHoyP+WVsCmn0B9vb5oqVdjwGUwf7lwOGhoFV7wJ135sScA0Od5MBO2BbR6vs91lVUSkP9BBVT+sa0cicqOIZIpIZk5OwzbjDE9zhpuw5iFTb9uWwCvnw2sXQ8kBmPC80wyUfo7NGmaaJL91FotIEPBX4FdHK6uq01U1Q1UzkpOTGzSODglRdEmKtuEmzNHtXgMzroQXz4I965yhIW7LhN6XOqOGGtNEefPy0e1AB4/XKe6yQ2KBU4Av3Lb5NsAsERmnqplejOsIw9OTmbHkR0rKKogIDfblW5vGThW2fQtfPwVrPoDwWBj5Ozj1FgiP8Xd0xjQIbyaCJUCaiKTiJIDLgSsOrVTVPCDp0GsR+QK4x9dJAJxJ7V9ZlEVm1j5OT0s6+gam+asoh7Xvw6KnYXsmRMTD6Xc5E8hHJ/o7OmMaVL0SgYhEA8WqWiki6UB34CNVLattG1UtF5HbgDlAMPCSqq4SkYeBTFWd1QDxN4hTuyQSGiws2JBjiSDQHcyHZf+Cb56F/T9Cy1SnCajvFRAW7e/ojPEKUdWjFxJZCgwDWgJf4XzbL1XVK70b3pEyMjI0M7PhKw2Tpi9mX1EpH985vMH3bZqAvGznLuClr8LBA9DxNBjyC+g2FoKsudA0fSKyVFVrvFervk1DoqpFInId8IyqPiYiyxsuRP8bnp7Mox+vZfeBElq1iPB3OMZXfvoOvp4Gq951+gN6jHeaf1IG+DsyY3ym3olARIYAVwLXucua1dek4elJPPoxzN+wh4kDUvwdjvGmykpY/7GTALYuhLBYZ0TQwTdBfEd/R2eMz9U3EdwJ/AZ4123n7wJ87r2wfO/kNi1Iigln/vocSwTNVWkRfP86fP0M7N0EcR3g7Eeg/9UQ0cLf0RnjN/VKBKr6JfAlVF3/v0dV7/BmYL4WFCQMS0viy/U5VFYqQUF2Y1CzUbQXvp0O3zwHxXuhXT+4+EVnQLjggB6A1xig/lcNvQ7cDFTgdBS3EJG/q+rj3gzO14anJ/Hud9tZ9dMBeqXE+Tscc6LytjvNP0tfgbJCSB8LQ+9wBoSzO4CNqVLfr0M9VPWAiFwJfATcDywFmlUiGHZouIkNOZYImrI9G+CrJ+H7N0ArodclcPqd0Opkf0dmTKNU30QQKiKhwIXA06paJiJHv+60iUmKCadnuxZ8uT6HX4zs6u9wzLH66TtnLoA170NIOGT8r3MFUMtO/o7MmEatvongOSAL+B6YLyKdgAPeCsqfhqcn8/z8zeSXlBEbYbOWNXqqsGU+LPwrbP4CwuNg2K+cq4BiGnZcKmOaq3qNlKWqU1W1vaqeq46twEgvx+YXw9KSKK9Uvt6U6+9QTF0qK51v/i+Mgn+OcwaEG/0w3LUSRv3ekoAxx6C+ncVxwEPAodtuvwQeBvK8FFfDW/M+ZL4MPS+EbufVOl5MRqcEosKCWbBhD2f3bOPjIM1RVZTBD/915gPes86Z/ev8v0GfKyDUbgQ05njUt2noJb+Nf0EAABzlSURBVGAlcKn7+irgZWCCN4LyirIS59rxWbeD3Ampw527SE++AKJ/Hl8oLCSIIV0SbVjqxqZ4n9P5u+gpOJANrXvZJaDGNJD6/gedpKoXe7z+Q5MbYqL3JdBrIuz4Hla/B6tnwgd3wod3Q+fT3aQwDmJaMTw9mblrd7M1t5BOiTbQmN+UFjl3AP/wFmz8FCpKnTGALngSup5ll4Aa00DqmwiKReR0VV0IICJDgWLvheUlItCur/Mz6kHYtdJJCqtmwoe/gg/vgU5DOa/TWJ4mnvnrc7hqiCUCn6ooczp9f/gvrP0QSgsgpg0MvMFJ5O37+ztCY5qd+o4+2gf4J3Do4vp9wDWqusKLsdXIK6OPqjqdjatnOokhZy2VCBvDTyF95P9Aj3HQol3Dvqf5WWUlbFvsfPNf9a5z929EnFNL63UJdBpqI4Aac4LqGn20XonAY0ctANyby+5U1ScbKMZ689Yw1IfZvZbP3p5Ox52fkC7utMsdBkPvyyDjWmuSaAiqsPMH55v/ynecdv+QSOh+LpwyEbqOcu4FMMY0iIYYhhpwEoDHy7sBnycCn2jVnfJh93L2v0fx3mXJ9Mn/0m0+utu5a3XM/1kyOF65m2Dl204C2LMegkLgpFFw1hRn7H+b/tEYnzuRyy2a9ZnwtK6JBAcJn+xuQZ9zfg3D7oE5v4XFz0BwqHPNuiWDupUcgJy1sHu10/S27Rvn7l9wmntOvQVOHm9TPxrjZyeSCJrdEBOeWkSE0r9jPPPX7+HX5+Cc9M/5f86VK4umOs0WZ/7O32E2DmXFkLPOOdnnrHEed6+BvG0/lwmNhtY9YfQf4ZQJEGdDfRvTWNSZCEQkn5pP+AJEeiWiRmRYWjJ/+2w9uQUHSYwJd5LB2MedZDD/cQgOgzPu9XeYvlNR5jTtHPqGf+hx3xZncDdwjklSOnQ8FVr9L7Tq4Qz2FtcRgup1I7sxxsfqTASqGuurQBqj4enJ/PXT9SzcuIfxfds7C4OC4Py/OyfFzx9xTnyn3+nfQL2lMNe5mufHr+HHxc49GBWlzjoJgoSTnG/5vS6BVt2dk35CF6fpzBjTZHj1lkwRGQP8HWdayxdU9c/V1t8M/AJnnoMC4EZVXe3NmI5Fr/ZxxEeFMn+9RyIAJxmMn+acFD97yEkGQ271X6ANQRX2bnZO+D9+7bTn71nvrAsOcyZzGXyTc0dvq5Odb/02pIMxzYLXEoGIBAPTgNFANrBERGZVO9G/rqr/cMuPA/4KjPFWTMcqOEg4vWsSCzbkoKqIZ+dwUDBc9JyTDOb8xvkWPOgG/wV7rCrKYOeKn0/8P34DhbuddRHxzuWyfSY5k7i062cnfWOaMW/WCAYBG1V1M4CIzADGA1WJoNrlqNE0wg7o4enJfLBiB2t35nNy22rz2gaHwsUvwZtXw+x7nG/OA67xT6A1KS917sw9mO/8FOyEbd86J/7sTCgrcsrFd4KTRjrt+h2HQFI3a883JoB4MxG0BzwuGyEbGFy9kIj8AueehDDgzJp2JCI3AjcCdOzYscEDrcuwNGdAuvnrc45MBAAhYXDpqzDjCnj/l04y6Dup4QPJ3wkbPoWSvMNP7lXP3cdSj+cVB4/cjwRBm17Q7yr3xH+q3TVtTIDz+7CNqjoNmCYiVwC/A474Sq2q04Hp4NxZ7Mv42sZFkt46hvkbcrjpjJNqLhQSDpf9G16/DN671akp9JrYMAHkrHcuV13xxs8dteBcjhkeA+GxEOY+xndwnx9aHnt4mahEZ5yl8IC+BsAYU403E8F2oIPH6xR3WW1mAM96MZ7jNjwtmX9+vZXi0goiw2oZ8yY0EibNgNcmwjs3Osmgx/jjf9MfF8NXU2HdhxASAf2vhoHXO9/ew2Js7B1jTIPxZkPwEiBNRFJFJAy4HJjlWUBE0jxengds8GI8x214ejKlFZUs3nKUWcvCouCKN6D9AHjrWlj30bG9UWWlM+Lmi2fDS+fAj4vgjPvgrlVw3l+cq3Ui4iwJGGMalNdqBKpaLiK3AXNwLh99SVVXicjDQKaqzgJuE5GzgDLcEU29Fc+JGJSaQHhIEPPX5zCyW6u6C4fHwv+8Bf+80OlEvvw/kHZW3duUlThNP4uegtwNEN/RuXGt35UQZsNgG2O8y6t9BKo6G5hdbdmDHs9/6c33bygRocEM7pLI/PX1nLUsIg6uegdeHed0Il/xhnNVTnXF+yHzJfjmH1CwC9r2gYkvOePv2KxbxhgfsWsE62lU91ZsyinkzcxtRy8MENkSrpoJiV3hP5Mga+HP6/KyYc4D8LeeMPcP0PoUuPo9uPFLOOViSwLGGJ+yM049XTG4I5+u3sUD7/5Ax4QoTu1SjxEzoxOdE/wr58FrlzpTLG6a5wzBrOqc9Ife4VzOaYwxfnJME9M0Bj6ZmKYWecVlTHjmK3ILS3n31qGkJtWz/T5/J7x8Luzd5Fz2OeAaZwjmeN/eE2GMCVwNNkNZY+DPRACwNbeQC6d9RXxUGO/eehrxUWH12zB/J6ybDT0uhKgE7wZpjDHV1JUIrI/gGHVKjGb61Rls31fMzf9eSml5Zf02jG3jTHNpScAY08hYIjgOAzsn8OjEXizevJffzfyBplarMsYYT9ZZfJwu6pfC5pxCnpq3kS7JMdxc2/ATxhjTyFkiOAF3nZXO5j2FPPrxWjonRjPmlDb+DskYY46ZNQ2dgKAg4S+X9KFPSjx3vvEdP2Tn+TskY4w5ZpYITlBEaDDPX51BYnQ41726hB15xf4OyRhjjoklggaQHBvOi5MzKCqt4LpXMik8WO7vkIwxpt4sETSQ7m1a8NQV/Vi78wC/nLGcikq7ksgY0zRYImhAI7u14sHze/DZml38+aM1/g7HGGPqxa4aamCTh6ayeU8hzy/YQpfkGCYNsmEkjDGNmyUCL3jw/B5szS3i9zNX0jEhiqFdk/wdkjHG1MqahrwgJDiIp67oR5fkaG7+91I27i7wd0jGGFMrSwRe0iIilBevGUh4SBDXvrKEvYWlR9/IGGP8wBKBF3VIiOK5qzLYeaCEm/6VycHyCn+HZIwxR7BE4GUDOrXkiUv6sCRrH/e9tYKyinqOVmqMMT7i1UQgImNEZJ2IbBSR+2tYf7eIrBaRFSIyV0Q6eTMefxnXpx2/Gp3OzOU/cdEzX7FmxwF/h2SMMVW8lghEJBiYBowFegCTRKRHtWLfARmq2ht4C3jMW/H42+2j0nj2yv7s2F/CuKcXMnXuBqsdGGMaBW/WCAYBG1V1s6qWAjOA8Z4FVPVzVS1yXy4GUrwYj9+N7dWWT+8+g3N6tuGvn67nome+Yu1Oqx0YY/zLm4mgPbDN43W2u6w21wEf1bRCRG4UkUwRyczJyWnAEH0vITqMp6/oX1U7uOCphTxltQNjjB81is5iEfkfIAN4vKb1qjpdVTNUNSM5Odm3wXmJZ+3gL1Y7MMb4kTcTwXagg8frFHfZYUTkLOABYJyqHvRiPI2O1Q6MMY2BNxPBEiBNRFJFJAy4HJjlWUBE+gHP4SSB3V6MpVGz2oExxp+8lghUtRy4DZgDrAHeVNVVIvKwiIxziz0OxAD/FZHlIjKrlt01ezXVDp6eZ7UDY4z3iWrTGjc/IyNDMzMz/R2GV+0tLOXB91bywYodnNK+BX+5pC/d2sT6OyxjTBMmIktVNaOmdY2is9gc7lDt4Bm3dnD+Uwt4et4Gyq12YIzxAksEjdi5vdryyV3DObtnG574ZD3jp33FF+t209RqccaYxs0SQSOXGBPONLd2sK+wlMkvL+GiZxbx+VpLCMaYhmF9BE1IaXklby/LZtrnG8neV0zvlDjuODONUSe3QkT8HZ4xphGrq4/AEkETVFZRybvLtvP05xv5cW8RPdu14I5RaYw+uTVBQZYQjDFHskTQTJVVVPLe8p94et4GsnKL6N4mll+OSuOcnm0sIRhjDmOJoJkrr6jk/RU/8dS8jWzOKaRb61huH9WVsae0JdgSgjEGSwQBo6JS+cBNCBt3F9C1VQy3n9mV83u3s4RgTICzRBBgKiuV2St38NTcjazblU+X5GhuP7MrF/RuR0iwXShmTCCyRBCgKiuVOat28ve5G1i7M59OiVFcmtGBC/u1p318pL/DM8b4kCWCAFdZqXy6ZhcvLtzCt1v2AjCkSyIT+rdnbK+2xISH+DlCY4y3WSIwVbbtLWLmd9t557vtbNlTSERoEGN6tmFC/xSGdk2yvgRjmilLBOYIqsp32/bzzrJs3v9+B3nFZbSKDeeifu2Z0D/FBrkzppmxRGDqdLC8gnlrdvP2su18sW435ZVKz3YtmNA/hXF92pEcG+7vEI0xJ8gSgam33IKDvP/9T7zz3XZWZOcRHCSckZ7MhP7tOevk1kSEBvs7RGPMcbBEYI7Lhl35vPPddmZ+t50deSXEhIcwolsyo3u0ZkS3VsRFhvo7RGNMPVkiMCekolJZvDmXWct/Yu7aXewpKCUkSDi1SyKje7TmrB6t7XJUYxo5SwSmwVRUKsu37eOT1bv4dPUuNucUAtCzXQtG92jN6B6t6dG2hY2GakwjY4nAeM2mnAI+dZPCsh/3oQrt4yOrksKg1ARC7W5mY/zOb4lARMYAfweCgRdU9c/V1g8HngR6A5er6ltH26clgsYrJ/8g89Y6SWHBhj0cLK+kRUQII7u3YnSP1pyRnkxshPUrGOMPfkkEIhIMrAdGA9nAEmCSqq72KNMZaAHcA8yyRNB8FJWWs2DDHj5dvYt5a3ezt9DpV+jTIZ4hXRI57aRE+ndqaVchGeMjdSUCb44tMAjYqKqb3SBmAOOBqkSgqlnuOpuVvZmJCgvhnJ5tOKdnGyoqlWU/7mPe2t0s2pTLM19s5OnPNxIWHES/jvEMOSmRIV0S6dsxnvAQSwzG+Jo3E0F7YJvH62xgsBffzzRSwUHCwM4JDOycAEB+SRlLsvby9aZcvt6cy9/nbuDJzzYQERpERqcEhpyUyKldEumdEmf9C8b4QJMYbUxEbgRuBOjYsaOfozEnKjYilDO7t+bM7q0ByCsqY/GWXL7elMvizbk8PmcdANFhwQxMTWBIl0SGnJRIz3ZxNhaSMV7gzUSwHejg8TrFXXbMVHU6MB2cPoITD800JnFRoVXNSODc3fzNlr0s2rSHrzfl8sW6HABiI0IY0KllVe2id0qc9TEY0wC8mQiWAGkikoqTAC4HrvDi+5lmIjEmnHN7teXcXm0B2H2ghK8357J4814ys/byxTqnxhAWHETvlDgyOicwKLUlAzomEBdlVyUZc6y8ffnouTiXhwYDL6nqIyLyMJCpqrNEZCDwLtASKAF2qmrPuvZpVw2ZvYWlLN26j8ysvXybtZeV2/Moq3A+x91axzIw9edaQzu749kYwG4oM81ccWkF32fvZ8mWvSzZuo9lW/dRcLAccG5uy+j8c2JIaxVDkPUzmADkr8tHjfGJyLBgTu3iXGkEUF5Rydqd+WRm7WVJ1j4WbcrlveU/AZAQHcbg1ISq8pYYjLEagQkAqsq2vcV8syWXb7Y4l61u318MWGIwgcNqBCagiQgdE6PomBjFJRnOhWzb9hax2O2AXrw5l49W7gScxHBql8MTgw2gZ5o7SwQmIHVIiKJDQu2JYfYPTmJIjA5jsJsYBqUmcFJyjN3kZpodSwTGUP/EEBospCZF07VVDF1bxZLWKoa01jGkJkXb8BimybJEYEwNPBODqpK9r5jMrXvZsKuADbsLWLMjn49X7qTS7WILDhI6JUTR1U0Maa1i6doqhpOSY4gMswRhGjdLBMYchYhUJQZPJWUVbNlTyIbdBWzclc+G3U6SmLd2N+VuhhCBDi3dBNEqhk6J0XR2+yvaxkXakBmmUbBEYMxxiggN5uS2LTi5bYvDlpeWV7I110kQTg0in427C1i4YQ+lFT8PtBsWHERKQiSdE6PpmBBF58QoOiVG0ykxipSWUYSFWF+E8Q1LBMY0sLCQINJax5LWOhZ6/by8olLZkVfMj7lFZOUWsXVvYdXzbzbnUlhaUVU2SKBdfCSdDiWHhKiqBJHSMpK4yFC7msk0GEsExvhIcJC4J/IoTut6+DpVZU9BKT/uLSRrTxFb9xbxY24hWblFfLxyJ3sLSw8rHx0WTPuWkaS0jKJ9fKT7PJL28c6ypJgwSxSm3iwRGNMIiAjJseEkx4YzoFPCEesPlJTxY24R2fuKyd5XxPb9xWzfV0z2vmKWbt1HXnHZYeXDQ4KOSBDtW0bSPj6K9i0jadMiwvonTBVLBMY0AS0iQjmlfRyntI+rcX1+SdlhyeHn50V8uuMAewoOr1EEBwltWkQ4iaIqSfz82C4+0ob4DiCWCIxpBmIjQuneJpTubVrUuL64tMJJDm6C2L6/yH0sZvHmXHYeKKm6FPaQpJjwIxJF56RoUhOjad/SrnhqTiwRGBMAIsOC3ZvgYmpcX1ZRyc68Eo9E8fPj6h0H+HTNLkrLD7/iqUNCJKlJMaQmRZGaFEPnpCi6JMXQukW49U80MZYIjDGEBgfVeK/EIZWVyp6Cg2TlFrFlTwFb9jiPWXuKmL8h57AkERka7NQckqJITYqmc2I0XZKj6ZQYTVxkqA3R0QhZIjDGHFVQkNCqRQStWkQwKPXwzuzKSmXHgRK25BSyJbeQLTmFZOUWsmZHPp+s2lV1c90hYcFBRIUHExUaTFR4CNFhwUSFhRAdfvhjVPXlYcHER4WRFBNGQnQY8VFh1jzVQCwRGGNOSFCQOB3N8ZGcnpZ02Lqyikqy9xWTtaeQrbmFFBwsp7C0gqJDj6XlFJVWUHSwgh15JRSVVlB40FlWWFpOXaPki0DLKCcpJESHkVjtMSEmnKToMBLcxJEQFUaI1UZqZInAGOM1ocFBpCZFk5oUfczbqiolZZVVySK/pJz9RaXsKSxlb8FB9haWkltYWvW4YXcBewtL2VdUWmsCiQkPIS4y9MifKOexRWQo8TWsbxEZ2qxrH5YIjDGNkogQGRZMZFgwicewXUWlsq/ITRAFzuPewoPkFpaSV1xGXlGZ81hcxqacgqrnBz36OWoSGx7iJIqo2hOJ5098ZBhxkaHERoQ0+smOLBEYY5qV4CAhKSacpJhwaF3/7UrKKqqSQvWEsb+4jAOe64rL2LC7oKqc5xhS1Yk4SSQuKpSQoMObpqSWF9XTxqGrsH45Ko0L+rSr/y9VT15NBCIyBvg7EAy8oKp/rrY+HPgnMADIBS5T1SxvxmSMMTWJCA0mIjSY1i0ijmm7Q01YedUSRV5xGfuLSqsSyIGScircjvPqLVeeUwYf0arlsSAuMvSYYqsvryUCEQkGpgGjgWxgiYjMUtXVHsWuA/apalcRuRx4FLjMWzEZY0xD82zCahN3bEmksfBmF/ogYKOqblbVUmAGML5amfHAq+7zt4BRYneiGGOMT3kzEbQHtnm8znaX1VhGVcuBPDiyX0hEbhSRTBHJzMnJ8VK4xhgTmJrERbWqOl1VM1Q1Izk52d/hGGNMs+LNRLAd6ODxOsVdVmMZEQkB4nA6jY0xxviINxPBEiBNRFJFJAy4HJhVrcws4Br3+URgnmpd9xIaY4xpaF67akhVy0XkNmAOzuWjL6nqKhF5GMhU1VnAi8C/RGQjsBcnWRhjjPEhr95HoKqzgdnVlj3o8bwEuMSbMRhjjKlbk+gsNsYY4z3S1JrkRSQH2OrvOGqRBOzxdxB1sPhOTGOPDxp/jBbfiTmR+Dqpao2XXTa5RNCYiUimqmb4O47aWHwnprHHB40/RovvxHgrPmsaMsaYAGeJwBhjApwlgoY13d8BHIXFd2Iae3zQ+GO0+E6MV+KzPgJjjAlwViMwxpgAZ4nAGGMCnCWCYyQiHUTkcxFZLSKrROSXNZQZISJ5IrLc/Xmwpn15McYsEfnBfe/MGtaLiEwVkY0iskJE+vswtm4ex2W5iBwQkTurlfH58RORl0Rkt4is9FiWICKfisgG97FlLdte45bZICLX1FTGC7E9LiJr3b/fuyISX8u2dX4WvBzjFBHZ7vF3PLeWbceIyDr383i/D+N7wyO2LBFZXsu2Xj2GtZ1TfPr5U1X7OYYfoC3Q330eC6wHelQrMwL4wI8xZgFJdaw/F/gIZ2rUU4Fv/BRnMLAT50YXvx4/YDjQH1jpsewx4H73+f3AozVslwBsdh9bus9b+iC2s4EQ9/mjNcVWn8+Cl2OcAtxTj8/AJqALEAZ8X/3/yVvxVVv/F+BBfxzD2s4pvvz8WY3gGKnqDlVd5j7PB9Zw5IQ7jd144J/qWAzEi0hbP8QxCtikqn6/U1xV5+MMfOjJcwa9V4ELa9j0HOBTVd2rqvuAT4Ex3o5NVT9RZzIngMU4w7z7TS3Hrz7qM5PhCasrPndWxEuB/zT0+9ZHHecUn33+LBGcABHpDPQDvqlh9RAR+V5EPhKRnj4NzJnu+hMRWSoiN9awvj6zx/nC5dT+z+fP43dIa1Xd4T7fCbSuoUxjOJbX4tTwanK0z4K33eY2X71US9NGYzh+w4BdqrqhlvU+O4bVzik++/xZIjhOIhIDvA3cqaoHqq1ehtPc0Qd4Cpjp4/BOV9X+wFjgFyIy3Mfvf1TizFExDvhvDav9ffyOoE49vNFday0iDwDlwGu1FPHnZ+FZ4CSgL7ADp/mlMZpE3bUBnxzDus4p3v78WSI4DiISivMHe01V36m+XlUPqGqB+3w2ECoiSb6KT1W3u4+7gXdxqt+e6jN7nLeNBZap6q7qK/x9/DzsOtRk5j7urqGM346liEwGzgeudE8UR6jHZ8FrVHWXqlaoaiXwfC3v7dfPojgzI04A3qitjC+OYS3nFJ99/iwRHCO3PfFFYI2q/rWWMm3ccojIIJzj7JMpOEUkWkRiDz3H6VRcWa3YLOBq9+qhU4E8jyqor9T6Lcyfx68azxn0rgHeq6HMHOBsEWnpNn2c7S7zKhEZA9wLjFPVolrK1Oez4M0YPfudLqrlveszk6E3nQWsVdXsmlb64hjWcU7x3efPWz3hzfUHOB2nirYCWO7+nAvcDNzslrkNWIVzBcRi4DQfxtfFfd/v3RgecJd7xifANJyrNX4AMnx8DKNxTuxxHsv8evxwktIOoAynnfU6IBGYC2wAPgMS3LIZwAse214LbHR//tdHsW3EaRs+9Bn8h1u2HTC7rs+CD4/fv9zP1wqck1rb6jG6r8/FuVJmk7dirCk+d/krhz53HmV9egzrOKf47PNnQ0wYY0yAs6YhY4wJcJYIjDEmwFkiMMaYAGeJwBhjApwlAmOMCXCWCIxxiUiFHD4yaoONhCkinT1HvjSmMQnxdwDGNCLFqtrX30EY42tWIzDmKNzx6B9zx6T/VkS6uss7i8g8d1C1uSLS0V3eWpw5Ar53f05zdxUsIs+7Y85/IiKRbvk73LHoV4jIDD/9miaAWSIw5meR1ZqGLvNYl6eqvYCngSfdZU8Br6pqb5xB36a6y6cCX6ozaF5/nDtSAdKAaaraE9gPXOwuvx/o5+7nZm/9csbUxu4sNsYlIgWqGlPD8izgTFXd7A4OtlNVE0VkD86wCWXu8h2qmiQiOUCKqh702EdnnHHj09zX9wGhqvonEfkYKMAZZXWmugPuGeMrViMwpn60lufH4qDH8wp+7qM7D2fsp/7AEndETGN8xhKBMfVzmcfj1+7zRTijZQJcCSxwn88FbgEQkWARiattpyISBHRQ1c+B+4A44IhaiTHeZN88jPlZpBw+gfnHqnroEtKWIrIC51v9JHfZ7cDLIvJrIAf4X3f5L4HpInIdzjf/W3BGvqxJMPBvN1kIMFVV9zfYb2RMPVgfgTFH4fYRZKjqHn/HYow3WNOQMcYEOKsRGGNMgLMagTHGBDhLBMYYE+AsERhjTICzRGCMMQHOEoExxgS4/w/QWqi6jo2X/QAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "shCB995u4Nmj",
        "outputId": "fcc91464-986b-41da-af6c-c4393bc14cc7"
      },
      "source": [
        "plt.clf()   # clear figure\n",
        "acc = history.history['accuracy']\n",
        "val_acc = history.history['val_accuracy']\n",
        "\n",
        "plt.plot(epochs, acc, label='Training acc')\n",
        "plt.plot(epochs, val_acc, label='Validation acc')\n",
        "plt.title('Training and validation accuracy')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xV9f348dc7C7IYWYyELaAgsiJuxY1oRXBBrUJt6261fmuL1ipf1Kr9WfVrHa0TcJQ6EQduqbYu9hYhIULCCit75/3743MSLuEmJCE3N+P9fDzO4557zuec+7knN/d9P+N8PqKqGGOMMfUVEuwMGGOMaV0scBhjjGkQCxzGGGMaxAKHMcaYBrHAYYwxpkEscBhjjGkQCxzmsInIAhGZ2tRpg0lEMkTkrACcV0XkCG/97yLyp/qkbcTrXCEiHzU2n8bURew+jvZJRPJ9nkYBJUCF9/xaVX25+XPVcohIBvBLVf2kic+rwEBV3dhUaUWkL7AJCFfV8qbIpzF1CQt2BkxwqGpM1XpdX5IiEmZfRqalsM9jy2BVVeYAIjJWRDJF5A8ish14QUS6isi7IpItInu99RSfYxaKyC+99Wki8h8RechLu0lEzmtk2n4i8oWI5InIJyLyhIi8VEu+65PHe0Tkv975PhKRBJ/9V4rIjyKyW0T+WMf1OU5EtotIqM+2iSKy0lsfIyJfi8g+EdkmIo+LSEQt55olIvf6PL/NO2ariFxdI+35IrJMRHJFZIuIzPDZ/YX3uE9E8kXkhKpr63P8iSKySERyvMcT63ttGnid40TkBe897BWReT77JojIcu89pInIOG/7AdWCIjKj6u8sIn29KrtfiMhm4DNv+2ve3yHH+4wM9Tk+UkT+6v09c7zPWKSIvCciv67xflaKyER/79XUzgKH8ac7EAf0Aa7BfU5e8J73BoqAx+s4/jhgPZAA/AV4TkSkEWlfAb4D4oEZwJV1vGZ98vhT4OdAEhAB/A5ARIYAT3nn7+m9Xgp+qOq3QAFwRo3zvuKtVwC/9d7PCcCZwA115BsvD+O8/JwNDARqtq8UAFcBXYDzgetF5CJv36neYxdVjVHVr2ucOw54D3jMe28PA++JSHyN93DQtfHjUNf5RVzV51DvXI94eRgDzAFu897DqUBGbdfDj9OAo4BzvecLcNcpCVgK+FatPgSMBk7EfY5/D1QCs4GfVSUSkeFAMu7amIZQVVva+YL7Bz7LWx8LlAId60g/Atjr83whrqoLYBqw0WdfFKBA94akxX0plQNRPvtfAl6q53vyl8c7fZ7fAHzgrd8FzPXZF+1dg7NqOfe9wPPeeizuS71PLWlvAd7yea7AEd76LOBeb/154AGfdIN80/o576PAI956Xy9tmM/+acB/vPUrge9qHP81MO1Q16Yh1xnogfuC7uon3T+q8lvX5897PqPq7+zz3vrXkYcuXprOuMBWBAz3k64jsBfXbgQuwDzZ3P9vbWGxEofxJ1tVi6ueiEiUiPzDK/rn4qpGuvhW19SwvWpFVQu91ZgGpu0J7PHZBrCltgzXM4/bfdYLffLU0/fcqloA7K7ttXCli0ki0gGYBCxV1R+9fAzyqm+2e/n4M670cSgH5AH4scb7O05EPveqiHKA6+p53qpz/1hj24+4X9tVars2BzjEde6F+5vt9XNoLyCtnvn1p/raiEioiDzgVXflsr/kkuAtHf29lveZ/hfwMxEJAabgSkimgSxwGH9qdrX7H2AwcJyqdmJ/1Uht1U9NYRsQJyJRPtt61ZH+cPK4zffc3mvG15ZYVdfivnjP48BqKnBVXt/jftV2Au5oTB5wJS5frwDzgV6q2hn4u895D9U1ciuuaslXbyCrHvmqqa7rvAX3N+vi57gtwIBazlmAK21W6e4nje97/CkwAVed1xlXKqnKwy6guI7Xmg1cgatCLNQa1XqmfixwmPqIxRX/93n15XcH+gW9X/CLgRkiEiEiJwA/CVAeXwcuEJGTvYbsmRz6f+MV4GbcF+drNfKRC+SLyJHA9fXMw6vANBEZ4gWumvmPxf2aL/baC37qsy8bV0XUv5Zzvw8MEpGfikiYiFwODAHerWfeaubD73VW1W24tocnvUb0cBGpCizPAT8XkTNFJEREkr3rA7AcmOylTwUuqUceSnClwihcqa4qD5W4ar+HRaSnVzo5wSsd4gWKSuCvWGmj0SxwmPp4FIjE/Zr7BvigmV73ClwD825cu8K/cF8Y/jQ6j6q6BrgRFwy24erBMw9x2D9xDbafqeoun+2/w32p5wHPeHmuTx4WeO/hM2Cj9+jrBmCmiOTh2mRe9Tm2ELgP+K+43lzH1zj3buACXGlhN66x+IIa+a6vQ13nK4EyXKlrJ66NB1X9Dtf4/giQA/yb/aWgP+FKCHuB/+XAEpw/c3AlvixgrZcPX78DVgGLgD3Agxz4XTcHGIZrMzONYDcAmlZDRP4FfK+qAS/xmLZLRK4CrlHVk4Odl9bKShymxRKRY0VkgFe1MQ5Xrz3vUMcZUxuvGvAG4Olg56U1s8BhWrLuuK6i+bh7EK5X1WVBzZFptUTkXFx70A4OXR1m6mBVVcYYYxrEShzGGGMapF0McpiQkKB9+/YNdjaMMaZVWbJkyS5VTay5vV0Ejr59+7J48eJgZ8MYY1oVEak54gBgVVXGGGMayAKHMcaYBrHAYYwxpkHaRRuHP2VlZWRmZlJcXHzoxCYoOnbsSEpKCuHh4cHOijHGR0ADh4g8jxsjZ6eqHu1nvwD/B4zHDeU8TVWXevumAnd6Se9V1dne9tG4eQwicYO33ayNuBklMzOT2NhY+vbtS+1zDJlgUVV2795NZmYm/fr1C3Z2jDE+Al1VNQsYV8f+83CzeA3EzTT3FFTPWHY3bna4McDdItLVO+Yp4Fc+x9V1/loVFxcTHx9vQaOFEhHi4+OtRGhMCxTQwKGqX+BGp6zNBGCOOt/gJoTpgZse8mNVrZoU5mNgnLevk6p+45Uy5gAX1Xr2Q7Cg0bLZ38eYlinYbRzJHDjrWaa3ra7tmX62H0RErsGVYujdu+acOMYY03KUlFewp6CU3fml7Clwy+6CUvKLy4ntGEaXqHC6RIXTOTLCrUeG0ykynPDQ4PRvCnbgCBhVfRpvBMzU1NQWNyDX7t27OfPMMwHYvn07oaGhJCa6GzS/++47IiIiaj128eLFzJkzh8cee6zO1zjxxBP56quvmi7Txph6KSqtYE9hKXu9ALCnoITd+d6697i7oMQFifxS8krKG/U6MR3C6BwZ7hNYDgwuXaLCOXdod7pE1f590hjBDhxZHDhdZoq3LQsYW2P7Qm97ip/0rU58fDzLly8HYMaMGcTExPC73/2uen95eTlhYf7/PKmpqaSmph7yNSxoGHN4VJXCUlca2FdYVh0M9lY/um37CkvZU1DmPZZSUl7p93xhIUJcdARx0RHEx0TQq2sXtx4dQVyMe4yP6VC9LaZDGPkl5ewrLGNfkTt/TlEZ+wrLqh/3FZWS4+3fnpNHTlE5OUWllFW438tj+sW3ucAxH7hJRObiGsJzVHWbiHwI/NmnQfwc4HZV3SMiud4MZ98CVwF/C0rOA2DatGl07NiRZcuWcdJJJzF58mRuvvlmiouLiYyM5IUXXmDw4MEsXLiQhx56iHfffZcZM2awefNm0tPT2bx5M7fccgu/+c1vAIiJiSE/P5+FCxcyY8YMEhISWL16NaNHj+all15CRHj//fe59dZbiY6O5qSTTiI9PZ133z1wRtGMjAyuvPJKCgoKAHj88cc58cQTAXjwwQd56aWXCAkJ4bzzzuOBBx5g48aNXHfddWRnZxMaGsprr73GgAG1TQFtTPNRVXKLy73qIJ9SQHU1UYkrDVRVGRWWUlpLEBCBzpHhxEW5X/jJXToytGcn4qIj6BoVQdeocLpERZAQ4wWK6A50igxrcNtdl6iIBn/xVwW8fUVlJMV2aNCx9RHo7rj/xJUcEkQkE9dTKhxAVf+O6047HjdVZiFuakm8AHEPbupHgJmqWtXIfgP7u+Mu8JbD8r/vrGHt1tzDPc0BhvTsxN0/Gdrg4zIzM/nqq68IDQ0lNzeXL7/8krCwMD755BPuuOMO3njjjYOO+f777/n888/Jy8tj8ODBXH/99Qfd+7Bs2TLWrFlDz549Oemkk/jvf/9Lamoq1157LV988QX9+vVjypQpfvOUlJTExx9/TMeOHdmwYQNTpkxh8eLFLFiwgLfffptvv/2WqKgo9uxxf6IrrriC6dOnM3HiRIqLi6ms9P+PZ4w/lZXK5j2FrN2Wy4+7C6morKRSoaJSqVS3VFS6L0e3DZ/tXppKqFCltLySvYWl7PKCwp6C/b/Ea4qOCCUuJoK46A706Lw/CFQHgmgXDLp6zztHhhMa0jI7cIgI0R3CiO4QmK/4gAYOVfX/TbR/v+Lmeva373ncpPM1ty8GDronpK249NJLCQ0NBSAnJ4epU6eyYcMGRISysjK/x5x//vl06NCBDh06kJSUxI4dO0hJSTkgzZgxY6q3jRgxgoyMDGJiYujfv3/1fRJTpkzh6acPnhitrKyMm266ieXLlxMaGsoPP/wAwCeffMLPf/5zoqKiAIiLiyMvL4+srCwmTpwIuJv4jKlNSXkFG3bks3ZrLmu35bJmaw7rtuWRX0udvwiEiBAqggiEhhy4HiJCSIgQIhAqQnhYCF2jIkju0pFhyZ2Ii+7gVQftLwVUVRF1DA9t5nffegW7qqpFaEzJIFCio6Or1//0pz9x+umn89Zbb5GRkcHYsWP9HtOhw/6iaGhoKOXlB//T1SdNbR555BG6devGihUrqKystGBgGiWnqIx123JZszW3OlBs2JFHeaUrAURFhHJUj05MGpXMkB6dGNKzEwMSYwgPDfGCgnXRbikscLRgOTk5JCe73sazZs1q8vMPHjyY9PR0MjIy6Nu3L//6179qzUdKSgohISHMnj2biooKAM4++2xmzpzJFVdcUV1VFRcXR0pKCvPmzeOiiy6ipKSEioqK6lKJafvKKyrZvKeQH3bk88OOPNZszWHttly27CmqTpMY24EhPToxdnAiQ3t2YkiPTvSNjyakhVb9mANZ4GjBfv/73zN16lTuvfdezj///CY/f2RkJE8++STjxo0jOjqaY4891m+6G264gYsvvpg5c+ZUpwUYN24cy5cvJzU1lYiICMaPH8+f//xnXnzxRa699lruuusuwsPDee211+jfv3+T598EV3lFJRm7C9m4M48fduSzYWc+G3bkkZ5dQGmFa9cSgX7x0RyT0oXJx/Z2QaJnJ5JirdTamrWLOcdTU1O15kRO69at46ijjgpSjlqO/Px8YmJiUFVuvPFGBg4cyG9/+9tgZ6ua/Z2Cr6yikh93F7BhR74XIPLYsCOfTbv2BwiAlK6RDOoWy8CkGAZ6j0ckxQSsgdYEnogsUdWD+v7bX7Sde+aZZ5g9ezalpaWMHDmSa6+9NthZMkFSVlFJxq4CNux0VUxVJYhNuwoO6InUKy6SQUmxjD0ykUFJsQzs5gJEVIR9nbQX9pdu537729+2qBKGCbzS8koydhe44LAjn41eoNi0q6C6oVoEenWNYmBSDGcc2Y1B3WIYmBTLgKRoCxDGAocxbVVJeQUZuwoPKD1s2JlPRo0A0ScuiiOSYjl7SDcGVgWIxBgiI6x7qvHPAocxrZiqsjOvhLTsfNKzC9yyy61n7i3Eiw+ECPSJj2ZgUgznDu3GQK+KaUBijN2/YBrMAocxrUBRaQWbdu0PCunZ+aTvcoHC92a5yPBQ+iVEc0xKZy4amcyAxGgGJsXSPzHaAoRpMhY4jGlhtucUs3zLXpZt2cfarbmkZxeQta/ogDTJXSLpnxjNxaOS6Z8YQ//EaAYkxtC9U0e7F8IEnAWOIDn99NOZPn065557bvW2Rx99lPXr1/PUU0/5PWbs2LE89NBDpKamMn78eF555RW6dOlyQBp/I+3WNG/ePAYNGsSQIUMAuOuuuzj11FM566yzmuCdmYYoLC1nVWYOy7fsq1625bhZD8NDhcHdY0nt25XLEnoxICma/gkx9EuItvYHE1QWOIJkypQpzJ0794DAMXfuXP7yl7/U6/j333+/0a89b948LrjggurAMXPmzEafy9RfZaWSviufZZv3sWzLPpZv3sf6HXlUeA0RveIiSe0bx8heXRjRuwtDenSy6iXTIgVn+ijDJZdcwnvvvUdpaSnghi7funUrp5xyCtdffz2pqakMHTqUu+++2+/xffv2ZdeuXQDcd999DBo0iJNPPpn169dXp3nmmWc49thjGT58OBdffDGFhYV89dVXzJ8/n9tuu40RI0aQlpbGtGnTeP311wH49NNPGTlyJMOGDePqq6+mpKSk+vXuvvtuRo0axbBhw/j+++8PylNGRgannHIKo0aNYtSoUQfMB/Lggw8ybNgwhg8fzvTp0wHYuHEjZ511FsOHD2fUqFGkpaU1wZVtOfJLyvl03Q4e+nA9P3v2W4bP/IizHv6C215fyTvLtxIXHcENYwfw3NRUFt95Fl/+/gz+NmUkV5/cj1G9u1rQMC2WlTgAFkyH7aua9pzdh8F5D9S6Oy4ujjFjxrBgwQImTJjA3LlzueyyyxAR7rvvPuLi4qioqODMM89k5cqVHHPMMX7Ps2TJEubOncvy5cspLy9n1KhRjB49GoBJkybxq1/9CoA777yT5557jl//+tdceOGFXHDBBVxyySUHnKu4uJhp06bx6aefMmjQIK666iqeeuopbrnlFgASEhJYunQpTz75JA899BDPPvvsAcfb8OuuC+zC9dnMX76VT9btoKS8ktAQYXC3WH4yvCcje3VhZO8u9E+IsbYI02pZ4AiiquqqqsDx3HPPAfDqq6/y9NNPU15ezrZt21i7dm2tgePLL79k4sSJ1YMIXnjhhdX7Vq9ezZ133sm+ffvIz88/oFrMn/Xr19OvXz8GDRoEwNSpU3niiSeqA8ekSZMAGD16NG+++eZBx7fX4dcrKpVvN+3m7WVbWbB6G7nF5cRFR3D5sb0Yd3R3RvTqYjfNmTbFPs1QZ8kgkCZMmMBvf/tbli5dSmFhIaNHj2bTpk089NBDLFq0iK5duzJt2jSKi4sbdf5p06Yxb948hg8fzqxZs1i4cOFh5bdqaPbahmVvT8Ovqyqrs3J5e3kW76zcyo7cEqIjQjl3aHcuHNGTk45IIDzUaoJN22Sf7CCKiYnh9NNP5+qrr66efS83N5fo6Gg6d+7Mjh07WLCg7gkOTz31VObNm0dRURF5eXm888471fvy8vLo0aMHZWVlvPzyy9XbY2NjycvLO+hcgwcPJiMjg40bNwLw4osvctppp9X7/eTk5NCjRw9CQkJ48cUXDxh+/YUXXqCwsBCAPXv2EBsbWz38OkBJSUn1/pZs064CHv3kB87867/5yeP/YfbXGRyT0oXHfzqSxXeezcOXj2Ds4CQLGqZNC/TUseOA/wNCgWdV9YEa+/vgZvlLBPYAP1PVTBE5HXjEJ+mRwGRVnScis4DTgBxv3zRVXR7I9xFIU6ZMYeLEicydOxeA4cOHM3LkSI488kh69erFSSedVOfxo0aN4vLLL2f48OEkJSUdMDT6Pffcw3HHHUdiYiLHHXdcdbCYPHkyv/rVr3jssceqG8XBVRe98MILXHrppZSXl3Psscdy3XXX1fu9tNXh13fmFjN/xVbmr9jKyswcROD4fvFcc2p/zju6B52jwg99EmPakIANqy4iocAPwNlAJm7+8CmqutYnzWvAu6o6W0TOAH6uqlfWOE8cbk7yFFUt9ALHu6r6OvVkw6q3XsH6O+UUlfHh6u28vSKLr9J2owrDkjszYURPLjimJ907t91qOGOqBGNY9THARlVN9zIwF5gArPVJMwS41Vv/HJjn5zyXAAtUteXXY5hWrbisgs++38nby7P4/PtsSisq6RsfxW/OGMiFI3oyIDEm2Fk0pkUIZOBIBrb4PM8EjquRZgUwCVedNRGIFZF4Vd3tk2Yy8HCN4+4TkbuAT4HpqlpS88VF5BrgGoDevXsfzvswbVh5RSVfpe3m7eVb+XDNdvJLykmM7cDPju/DhBE9OSals81zbUwNwe5V9TvgcRGZBnwBZAEVVTtFpAcwDPjQ55jbge1ABPA08AfgoFufVfVpbz+pqal+6+NU1b4UWrBAVaOqKsu27GP+8q28u3Iru/JLie0Yxvhh3ZkwIpnj+8cTavdYGFOrQAaOLKCXz/MUb1s1Vd2KK3EgIjHAxaq6zyfJZcBbqlrmc8w2b7VERF7ABZ8G69ixI7t37yY+Pt6CRwukquzevbtJu/Ru2JHH28tdI/fmPYVEhIVw1lFJXDg8mbGDE+1ObWPqKZCBYxEwUET64QLGZOCnvglEJAHYo6qVuJLE8zXOMcXb7ntMD1XdJu7b/iJgdWMyl5KSQmZmJtnZ2Y053DSDjh07kpKScljn2LqviHdWbGXe8q2s25ZLiMBJRyTwmzMHcs7QbnTqaD2ijGmogAUOVS0XkZtw1UyhwPOqukZEZgKLVXU+MBa4X0QUV1V1Y9XxItIXV2L5d41TvywiiYAAy4H69xf1ER4eTr9+/RpzqGnhcovLWLBqG28uzeLbTW54kxG9unD3T4Zw/jE9SIq1HlHGHI6AdcdtSfx1xzVtS3lFJV9u2MUbSzP5eK0bI6pfQjQTRyYzYURP+sRHBzuLxrQ6weiOa0xAqSprtubyxtJM3lnhGrm7RIVz+bG9mDgymRG9ulj7lTEBYIHDtDrbcoqYt2wrby7NZMPOfCJCQzjjyCQmjUpm7OAkIsJsuA9jAskCh2kV8kvK+WD1dt5alll9J/foPl2596KjueCYHnSJigh2Fo1pNyxwmBarolL578ZdvLk0kw/X7KCorILece5O7okjk+mbYO0WxgSDBQ7T4uwtKGXuoi289M2PZO0rolPHMCaOSmbSyGRG9+lq7RbGBJkFDtNirNmaw+yvMnh7+VZKyis5vn8cd4w/ijOPSrKb84xpQSxwmKAqq6jkg9Xbmf1VBot/3EtkeCiTRqUw9cQ+HNm9U7CzZ4zxwwKHCYqdecX889stvPztj+zMK6F3XBR3nn8Ul47uZfNbGNPCWeAwzWrZ5r3M/iqD91Zto6xCOWVgAvdPGsbYwUk2sKAxrYQFDhNwJeUVvLtiG3O+zmBFZg4xHcK44rg+XHlCH5vjwphWyAKHCZidecXM+epH/vndZnYXlNI/MZr/vXAok0YlE2uDCxrTalngME2uuKyC5/6ziSc+30hRWQVnHpnE1BP7cvIRCdaV1pg2wAKHaTKqyvurtvPn99eRta+Is4d04/bzjqS/VUcZ06ZY4DBNYlVmDjPfXcOijL0c2T2Wl395HCcdkRDsbBljAsAChzksO3KL+X8frueNpZnERUXw54nDuPzYXtZDypg2zAKHaZTisgqe/TKdJxemUVZRyTWn9OfGM46wGfWMaQcscJgGUVXeW7WN+9//nqx9RZwzpBt3jD/KBhw0ph2xwGHqbWXmPma+s5bFP+7lqB6d+H+XHsOJA6wdw5j2JqCBQ0TGAf+Hm3P8WVV9oMb+PsDzQCKwB/iZqmZ6+yqAVV7Szap6obe9HzAXiAeWAFeqamkg30d7V9WO8fqSTOKjI7h/0jAuS7V2DGPaq4AFDhEJBZ4AzgYygUUiMl9V1/okewiYo6qzReQM4H7gSm9fkaqO8HPqB4FHVHWuiPwd+AXwVKDeR3vm245RXqFce1p/bjzd2jGMae8CWeIYA2xU1XQAEZkLTAB8A8cQ4FZv/XNgXl0nFHf32BnAT71Ns4EZWOBocp99v4O7569hy54ixg3tzu3jj6RPvLVjGGMCGziSgS0+zzOB42qkWQFMwlVnTQRiRSReVXcDHUVkMVAOPKCq83DVU/tUtdznnMn+XlxErgGuAejdu3fTvKN2IGtfETPfWcOHa3YwIDGaV355HCfa/RjGGB/Bbhz/HfC4iEwDvgCygApvXx9VzRKR/sBnIrIKyKnviVX1aeBpgNTUVG3SXLdBZRWVPP+fTTz6yQYU5ffjBvPLk/sTERYS7KwZY1qYQAaOLKCXz/MUb1s1Vd2KK3EgIjHAxaq6z9uX5T2mi8hCYCTwBtBFRMK8UsdB5zQN9236bv709mp+2JHP2UO6cdcFQ+gVFxXsbBljWqhABo5FwECvF1QWMJn9bRMAiEgCsEdVK4HbcT2sEJGuQKGqlnhpTgL+oqoqIp8Dl+B6Vk0F3g7ge2jTduWXcP/73/PG0kySu0Ty7FWpnDWkW7CzZYxp4QIWOFS1XERuAj7Edcd9XlXXiMhMYLGqzgfGAveLiOKqqm70Dj8K+IeIVAIhuDaOqkb1PwBzReReYBnwXKDeQ1tVWan8c9Fm/vLBegpLy7lh7AB+fcZAIiNsXm9jzKGJatuv/k9NTdXFixcHOxstwuqsHP44bzUrtuzjhP7x3HPRUI5Iig12towxLZCILFHV1Jrbg904bppJbnEZD3/0A3O+ziAuugP/N3kEFw7vafNjGGMazAJHG6eqzF+xlXveXceeghKuPL4Pt54zmM6RdhOfMaZxLHC0YRm7CrjjrVV8lbab4SmdeWHasQxL6RzsbBljWjkLHG3U0s17uXrWIiorlXsvOpopY3rb2FLGmCZhgaMN+nz9Tm54aSndOnVgztXH0Tve7skwxjQdCxxtzFvLMrnttZUc2SOWF6aNITG2Q7CzZIxpYyxwtCHPfpnOve+t48QB8fzjytHE2ii2xpgAsMDRBqgqDyz4nn98kc75w3rw8OXD6RBmN/MZYwLDAkcrV1ZRyfQ3VvHG0kyuOqEPd/9kqDWCG2MCygJHK1ZUWsGNryzls+93cuvZg/j1GUfYDX3GmICzwNFK7Sss5epZi1i+ZR/3TTyaK47rE+wsGWPaCQscrdC2nCKueu47ftxdyJNXjGLc0T2CnSVjTDtigaOV2bgzj6ue+4684nJmXz2GEwbEBztLxph2xgJHK7Js815+PmsRYSEhzL32eIb2tOFDjDHNzwJHK1F1N3hSpw68aHeDG2OCyAJHK1B1N/jg7rHM+rndDW6MCS4LHC2c3Q1ujGlpLHC0YI9+8ndQ8DEAACAASURBVAOPfrKB8cO688jlI+xucGNMixASyJOLyDgRWS8iG0Vkup/9fUTkUxFZKSILRSTF2z5CRL4WkTXevst9jpklIptEZLm3jAjkewiWN5Zk8ugnG7hkdAp/mzLKgoYxpsUIWOAQkVDgCeA8YAgwRUSG1Ej2EDBHVY8BZgL3e9sLgatUdSgwDnhURLr4HHebqo7wluWBeg/BsihjD7e/uYoTB8Rz/6RhNoSIMaZFCWSJYwywUVXTVbUUmAtMqJFmCPCZt/551X5V/UFVN3jrW4GdQGIA89pibN5dyLUvLiG5ayRPXjGK8NCAFgqNMabBAvmtlAxs8Xme6W3ztQKY5K1PBGJF5IA72kRkDBABpPlsvs+rwnpERPx2MRKRa0RksYgszs7OPpz30Wxyi8v4xexFVFQqz01NpUtURLCzZIwxBwn2z9nfAaeJyDLgNCALqKjaKSI9gBeBn6tqpbf5duBI4FggDviDvxOr6tOqmqqqqYmJLb+wUl5RyU2vLGPTrgKe+tko+ifGBDtLxhjjVyB7VWUBvXyep3jbqnnVUJMARCQGuFhV93nPOwHvAX9U1W98jtnmrZaIyAu44NPq3fveOr74IZsHJg3jxAEJwc6OMcbUKpAljkXAQBHpJyIRwGRgvm8CEUkQkao83A48722PAN7CNZy/XuOYHt6jABcBqwP4HprFi19nMOurDH55cj8mj+kd7OwYY0ydAhY4VLUcuAn4EFgHvKqqa0Rkpohc6CUbC6wXkR+AbsB93vbLgFOBaX663b4sIquAVUACcG+g3kNz+OKHbGa8s5Yzj0zi9vFHBTs7xhhzSKKqwc5DwKWmpurixYuDnY2DbNyZx8QnviK5aySvX38iMR3sfkxjTMshIktUNbXm9mA3jrdbewpKuXrWYjqEh/Ls1FQLGsaYVsO+rYKgpLyC615cwvbcYuZeczwpXW2kW2NM62EljmamqvzxrdV8l7GHhy4dzqjeXYOdJWOMaRALHM3sH1+k8/qSTG4+cyAXDu8Z7OwYY0yDWeBoRh+u2c6DH3zPT4b35JazBgY7O8YY0ygWOJrJ6qwcbpm7nOEpXfh/lxyDuw3FGGNan3oFDhGJrrpRT0QGiciFImIzCtXTjtxifjl7MV2jwnn6qtF0DLch0o0xrVd9SxxfAB1FJBn4CLgSmBWoTLUlRaUV/GrOYnKLy3hu2rEkxXYMdpaMMeaw1DdwiKoW4saVelJVLwWGBi5bbUNlpfI/ry1nVVYOj00eyVE9OgU7S8YYc9jqHThE5ATgCtzAgwBW33IIT3+ZzvurtnPHeUdx1pBuwc6OMcY0ifoGjltwgxC+5Y031R838ZKpwwertzOqdxd+eUq/YGfFGGOaTL3uHFfVfwP/BvAayXep6m8CmbHWTlVJz85nwohk60FljGlT6tur6hUR6SQi0bhhzNeKyG2BzVrrtiu/lNzicgYkRgc7K8YY06TqW1U1RFVzcfNfLAD64XpWmVqkZecDMCDJZvIzxrQt9Q0c4d59GxcB81W1DGj747EfhqrAYVPAGmPamvoGjn8AGUA08IWI9AFyA5WptiBtZwGR4aH06GT3bRhj2pb6No4/Bjzms+lHETk9MFlqG9Ky8+mfGE1IiDWMG2Palvo2jncWkYdFZLG3/BVX+jjUceNEZL2IbBSR6X729xGRT0VkpYgsFJEUn31TRWSDt0z12T5aRFZ553xMWmiXpfRd+QywaipjTBtU36qq54E83Fzgl+GqqV6o6wARCQWeAM4DhgBTRGRIjWQPAXNU9RhgJnC/d2wccDdwHDAGuFtEqiaueAr4FTDQW8bV8z00m+KyCjL3FlngMMa0SfUNHANU9W5VTfeW/wX6H+KYMcBGL30pMBeYUCPNEOAzb/1zn/3nAh+r6h5V3Qt8DIwTkR5AJ1X9Rt1k6XNwDfYtyqZdBajCgCTrimuMaXvqGziKROTkqicichJQdIhjkoEtPs8zvW2+VuDGvwKYCMSKSHwdxyZ763WdsyqP11RVrWVnZx8iq02rukdVgpU4jDFtT30Dx3XAEyKSISIZwOPAtU3w+r8DThORZcBpQBZQ0QTnRVWfVtVUVU1NTExsilPWW9rOAkSgX4KVOIwxbU99e1WtAIaLSCfvea6I3AKsrOOwLKCXz/MUb5vvebfilThEJAa4WFX3iUgWMLbGsQu941NqbD/gnC1BWnY+yV0iiYywcSCNMW1Pg2YAVNVc7w5ygFsPkXwRMFBE+olIBDAZmO+bQEQSqiaIwg2i+Ly3/iFwjoh09RrFzwE+VNVtQK6IHO/1proKeLsh76E5pGVbjypjTNt1OFPH1tkNVlXLgZtwQWAd8Ko3su5MEbnQSzYWWC8iPwDdgPu8Y/cA9+CCzyJgprcN4AbgWWAjkIYbAqXFqKxU0rMLLHAYY9qselVV1eKQQ46o6vvA+zW23eWz/jrwei3HPs/+Eojv9sXA0Q3NbHPZnltMUVlF0/So+uAO2PIt/ORR6D7s8M9njDFNoM7AISJ5+A8QAkQGJEetXJP1qNq+Gr55EiQEnj4dTr8DTroZQqzdxBgTXHVWValqrKp28rPEqurhlFbarLSdVaPiHmaJ45MZ0LET3LQIjhwPn/4vvDAe9qQffiaNMeYwHE4bh/EjLbuA2I5hJMZ0aPxJNn0BGz+GU/4H4gfApbNh0jOwcx08dTIsmQVqgxMbY4LDAkcTq+pR1eghtFTh47uhUzKMucZtE4FjLoMbvoKUVHjnZnjlcsjb0XQZN8aYerLA0cQOu0fVmrdg61I4/Y8QXqMZqXMKXDkPxj0Im/4NTx4Pa1tcb2RjTBtngaMJ5ZeUsz23uPHtGxVl8OlMSBoKwyf7TxMSAsdfB9d+CV37wKtXwZvXQtG+xmfcGGMawAJHE0qvmi62sSWOJbNg7yY4a8ahe08lDoJffAynTYdVr8FTJ0H6wsa9rjHGNIAFjiZUPc94YiNKHCV5sPAB6HMyDDy7fseEhsPpt7sAEt4R5kyABdOh7FDjTxpjTONZl9omlLazgNAQoXdcIwLHV3+Dwl1w9kzXGN4QKaNd1dUnM+DbpyDtM5j0D+g5suH5aIyKcpf3/B2Qv9PncSdUlEJMkrd085YkiE5ywa4plRVBwS4o3O3yU7jHldx6Hefah4wxTcICRxNKy86nT1wUEWENLMjl7YCvHochF7kg0BgRUTD+LzB4HMy7EZ49C079vevSG+rnz6wKlRVQWQ7qPVZWuEUr9u8rLYCCnT4BYQfkZx8YJAp34/c+0YhYVyoq2nPwPoAOnX0CSpL/ABPW0QsEu32CQi3Pywprvz5dekPvE6HPCe4xYWDDA7QxBrDAUbcNH7svpNoaqmtIzy6gf2PaN/79IFSUwJl3HTrtoQw4w3Xbff82WPhnV5KRkBrBoZx6jBjjX2iH/V/qXftCrzHe88SDSxQRUe6YijIoyK4RdGqUTravdI8luXW+PAARMRAVB1EJEJ0IiUdCdML+bVHxbolOgNJ82PwN/Phf2PgJrJzrzhGVsD+I9DkBug3zH2CNMQex/5TaqMK3/4D0z92X0xFn1pm8olLZtKuAsUc2cO6PXRtdo3jqz93Nfk0hsitc/KwrwaQvdNU1EuoeQ0IhJMx7HuZ6aVWvV+0L2f88PPLAgNChU8N/qYeGQ6eebjmU0sIDSzjlJQcHhIZWcfUcCcdf7/6muzfCj1/B5q/d47p3XJqIGBcEqwJJ8uiDu0MbYwALHLUTgUueh1njXZfXae9BzxG1Js/cW0hpRWXDe1R9NtN9QZ32h8PMsB9HXeCW1iQiCiL6utJMUxNxVVQJA2H0VLctJ2t/ENn8NXx+r9seGgE9R7kbLhMGQtwAF9hje1gVl2n3LHDUpWMn+Olr8NzZ8MplrvdS1z5+kzaqR1XmYncD39jb3a950/w6J8OwS9wCrkF9y7f7A8l3z7hqxCrh0RDXH+L7Q/wR+wNK/BGuNGRBxbQDFjgOpVMP+NkbLni8dDH84iNXdVJD2s4CoAGj4qrCx3e5arATbmzKHJvDERUHg89zC7g2oZxM2JMGu71lTxpsXwXr3nVtR1U6dK4RUI5w99skDXHVdca0ERY46iNxMEyZC3Mugn9OhqvePqj+Oy07n/joCLpGR9TvnD986Bpsxz8EHWIDkGnTJEJCXSmzax/X8cBXRRns2+zaTaoCyu6NsPlbWPU61R0QwjpCj+Gu3aRq6drXSiem1bLAUV99ToRJT8Nr0+CNX8Jlcw64u7tBY1RVVrh7LuL6w+hpgcitaQ6h4V41lZ9ODWXFbhSAHWsgaylkLYHFz7s5VsBVa/kGkuTRfkuyxrREFjgaYuhFkHc/fDDdLef9pfpXY1p2PucM7Va/86z4J2Svg0tnWRVGWxXeEZKOcktV+0lFGexc64JI5hL3uOFjqksmXfu5xvjk0ZCc6mZ9bOqbJI1pAgENHCIyDvg/IBR4VlUfqLG/NzAb6OKlma6q74vIFcBtPkmPAUap6nIRWQj0AKrG1ThHVXcG8n0c4PjrXZ3314+7oc9PvoW9BaXsLiitX4mjrAg+/7P7chhyUeDza1qO0HBXZdVjOKRe7baV5MHWZS6IZC1xjfKrXnP7QsLc5+SYy13w6dg5eHk3xkfAAoeIhAJPAGcDmcAiEZmvqmt9kt0JvKqqT4nIENz85H1V9WXgZe88w4B5qrrc57grvLnHg+PseyBvG3zi5s1I7+zu8ehfnx5V3/4DcrNctZfVcZsOsdDvVLdUyd22P5D88AG8dyt8+EcYMgFG/gz6nmyfHRNUgSxxjAE2qmo6gIjMBSYAvoFDgU7eemdgq5/zTAHmBjCfDRcSAhc95YYKmXc9ecf+HYg8dImjcA/852EYeK775zfGn049oJN3D86Zd7n5WZa95BrcV851VVojr4ARV9TvpsrGUIV9P0L2D1BW4G7MLCt0JeayIp/1Aj/bCr30Ra4TybBLYdSVNl5YGyIaoClIReQSYJyq/tJ7fiVwnKre5JOmB/AR0BWIBs5S1SU1zpMGTFDV1d7zhUA8UAG8Adyrft6EiFwDXAPQu3fv0T/++GOTv0eK9sHz4yjes5lLi+9i3j3XEhpSxy/BD/8IXz8B1/8Xug1t+vyYtq20ENbNd0Ek40t3h/+AM92X8qDzIKyePfr8KSuGbSsg8zt3H8uW79yd+7UJ7eCCQniUe4yI2r/u+5iT6UYvEIEjznadQQaeY8O7tBIiskRVUw/aHuTAcauXh7+KyAnAc8DRqlrp7T8O1zYyzOeYZFXNEpFYXOB4SVXn1JWX1NRUXbw4QDVbOZnseew0KiqVxFu+qP1X1b7N8LfR7tfXRU8GJi+m/didBstfhuWvuGrTqHg4ZrKryuo25NDH523fHyC2fAfblruRjMGVaHqNcUv3Y1x1Ws2AcKj5YnztzYClL7qAl7/d3X0/8koX8Lr0btTbN80jGIHjBGCGqp7rPb8dQFXv90mzBhdctnjP04Hjqxq7ReQRIFtV/1zLa0wDUn2DkT8BDRzALx6cxeMldxAZ3xuu/gAiuxyc6K3rYPWb8JulVmQ3TaeyAjZ+CstehPULoLLMNaiPvBKOvtiNflBRDjtWe0HiW1eq2LfZHR/awY3l1WuMG36+15jAjWJQUebuX1oyyw04CXDEWa4UMuhc62HYAgUjcIQBPwBnAlnAIuCnqrrGJ80C4F+qOktEjgI+BZJVVUUkBNgCnOLTThIGdFHVXSISDvwT+ERV/15XXgIZOErLKznqrg+4f+ReLlt3s/vnu/JNCOuwP9H2VfD3U+DEX8M59wQkH8ZQsAtW/sv9us9eB2GRrkvvjtX7h5yP7eETJI5zJYrDqeJqrH2bvVLIi67EFNPdlZZGXVXrsD4NUpzrzluSB517uWBoHQoarNkDh/ei44FHcV1tn1fV+0RkJrBYVed7PameAWJwDeW/V9WPvGPHAg+o6vE+54sGvgDCvXN+Atyq6jvuw8ECGTg27szjrIe/4JHLhzMx9Gt485cwdCJc/LxrRAc3VEnmYrh5uRu51phAUnU3HS6bAzvWHlii6JzSsr5AK8phw0ewdLZ7VHV36I+e5oZ9qVkKqaxwQ/TnbnVL3jafxyzXIy1vmxtO31dYpLtb39/Spff+KQAaqrLSy09Wjbxsg7ytUFDLXDXNafLLjR40tLbAEdAWKlV9H9fF1nfbXT7ra4GTajl2IXB8jW0FQCNnOgqMjd4YVQMSYyDlUvcB8rrpcu59kP5vVyw/+x4LGqZ5iLgJwRo7KVhzCg2DI8e7JSfTtYMsnQOvXunmdBl0LhTn7P9Cztt+4Phg4O53ienueqMlHeWmQOjUE2J7QkS0O+/eDLfs+xE2feF6g/mK6e4Fkj4HBpWoBDfMf1UgyN3mEyS2uTabyvIDzyWhENvdle46pzSsPSgQQpq+CtC6NhymqlFxqydwOulm98H6+nH34V31GnRKgTHXBDGXxrQCnVNg7HQ49Tb3Y2vJLPj+XTcQaGwP6HeaCw6xPbzA0MP9QItOaNiXs6qboK0qmOzd5D3+6G7AXPkqtZYSImK81+0B/U6pkZeq/CQGP1gEmAWOw5SeXUD3Th2J6eBdShEY94D7hfThHW7bRU/Z0BHG1FdIqCtpDDo3MOcXccEmOsEN8VJTeYlXStnkqppiEl1AiO3hOhsYCxyHKy07nwFJNe4YDwl1M/C9fCmUF7shI4wxrUNYh9oHrzSABY7DoqqkZeczcWTywTvDI2HqO64xr40XW40x7UtIsDPQmmXnl5BXXF77UCMidoesMabNscBxGKpn/WvIdLHGGNPKWeA4DPvnGa/nBE7GGNMGWOA4DOnZBURFhNK9k/WYMsa0HxY4DkNadj79E6MJqWtEXGOMaWMscByGtOx8q6YyxrQ7Fjgaqai0gqx9RRY4jDHtjgWORtq0qwBV61FljGl/LHA0kvWoMsa0VxY4GiktOx8R6JdgJQ5jTPtigaOR0rMLSOkaScdwG07EGNO+WOBoJOtRZYxpryxwNEJlpZKeXWCBwxjTLgU0cIjIOBFZLyIbRWS6n/29ReRzEVkmIiu9qWYRkb4iUiQiy73l7z7HjBaRVd45HxNp/nkwt+UWU1RWYT2qjDHtUsACh4iEAk8A5wFDgCneHOO+7gReVdWRwGTgSZ99aao6wluu89n+FPArYKC3jAvUe6hN2k7rUWWMab8CWeIYA2xU1XRVLQXmAhNqpFGgakqtzsDWuk4oIj2ATqr6jaoqMAe4qGmzfWjWFdcY054FMnAkA1t8nmd623zNAH4mIpnA+8Cvffb186qw/i0ip/icM/MQ5wRARK4RkcUisjg7O/sw3sbB0rML6NQxjISYiCY9rzHGtAbBbhyfAsxS1RRgPPCiiIQA24DeXhXWrcArItKgyX5V9WlVTVXV1MTExCbNtJsuNoYgNK8YY0zQBTJwZAG9fJ6neNt8/QJ4FUBVvwY6AgmqWqKqu73tS4A0YJB3fMohzhlw1hXXGNOeBTJwLAIGikg/EYnANX7Pr5FmM3AmgIgchQsc2SKS6DWuIyL9cY3g6aq6DcgVkeO93lRXAW8H8D0cJK+4jB25JdajyhjTbgVsQmxVLReRm4APgVDgeVVdIyIzgcWqOh/4H+AZEfktrqF8mqqqiJwKzBSRMqASuE5V93invgGYBUQCC7yl2aRnu+lircRhjGmvAhY4AFT1fVyjt++2u3zW1wIn+TnuDeCNWs65GDi6aXNaf9ajyhjT3gW7cbzVSc8uICxE6BMfFeysGGNMUFjgaKC07Hx6x0cRHmqXzhjTPtm3XwNZjypjTHtngaMByisqydhVaD2qjDHtmgWOBsjcW0RpRaWVOIwx7ZoFjgawHlXGGGOBo0H238NhVVXGmPbLAkcDpGXnkxATQZcoG9zQGNN+WeBogLTsfPpbNZUxpp2zwNEAaTZdrDHGWOCorz0FpewpKLX2DWNMu2eBo57SrUeVMcYAFjjqzUbFNcYYxwJHPaVl5xMRFkJy18hgZ8UYY4LKAkc9pWXn0z8hmtAQmy7WGNO+WeCoJ+tRZYwxjgWOeigpr2DzHhvc0BhjwAJHvWzeXUhFpVqJwxhjCHDgEJFxIrJeRDaKyHQ/+3uLyOciskxEVorIeG/72SKyRERWeY9n+Byz0Dvncm9JCuR7AFdNBdajyhhjIIBzjotIKPAEcDaQCSwSkfnePONV7gReVdWnRGQIbn7yvsAu4CequlVEjgY+BJJ9jrvCm3u8WVSNimtVVcYYE9gSxxhgo6qmq2opMBeYUCONAp289c7AVgBVXaaqW73ta4BIEekQwLzWKS07nx6dOxLdIWBx1hhjWo1ABo5kYIvP80wOLDUAzAB+JiKZuNLGr/2c52JgqaqW+Gx7waum+pOI+O0fKyLXiMhiEVmcnZ3d6DcB1qPKGGN8BbtxfAowS1VTgPHAiyJSnScRGQo8CFzrc8wVqjoMOMVbrvR3YlV9WlVTVTU1MTGx0RlUVdJ35ls1lTHGeAIZOLKAXj7PU7xtvn4BvAqgql8DHYEEABFJAd4CrlLVtKoDVDXLe8wDXsFViQVMdl4JeSXlVuIwxhhPIAPHImCgiPQTkQhgMjC/RprNwJkAInIULnBki0gX4D1guqr+tyqxiISJSFVgCQcuAFYH8D1YjypjjKkhYIFDVcuBm3A9otbhek+tEZGZInKhl+x/gF+JyArgn8A0VVXvuCOAu2p0u+0AfCgiK4HluBLMM4F6D+Azz3iSVVUZYwwEsDsugKq+j2v09t12l8/6WuAkP8fdC9xby2lHN2UeDyUtO5+oiFC6d+rYnC9rjDEtVrAbx1u8qh5VtXTeMsaYdscCxyGkWY8qY4w5gAWOOhSVVpC1r8gaxo0xxocFjjqk77LpYo0xpiYLHHWoni7WelQZY0w1Cxx1SMvORwT6xlvgMMaYKhY46pCWXUCvrlF0DA8NdlaMMabFsOFe63Bk91hSukYGOxvGGNOiWOCow42nHxHsLBhjTItjVVXGGGMaxAKHMcaYBrHAYYwxpkEscBhjjGkQCxzGGGMaxAKHMcaYBrHAYYwxpkEscBhjjGkQcTO1tm0ikg38GOx81CIB2BXsTNTB8nd4LH+Hx/J3eA43f31UNbHmxnYROFoyEVmsqqnBzkdtLH+Hx/J3eCx/hydQ+bOqKmOMMQ1igcMYY0yDWOAIvqeDnYFDsPwdHsvf4bH8HZ6A5M/aOIwxxjSIlTiMMcY0iAUOY4wxDWKBoxmISC8R+VxE1orIGhG52U+asSKSIyLLveWuZs5jhois8l57sZ/9IiKPichGEVkpIqOaMW+Dfa7LchHJFZFbaqRp1usnIs+LyE4RWe2zLU5EPhaRDd5j11qOneql2SAiU5sxf/9PRL73/n5viUiXWo6t87MQwPzNEJEsn7/h+FqOHSci673P4vRmzN+/fPKWISLLazm2Oa6f3++UZvsMqqotAV6AHsAobz0W+AEYUiPNWODdIOYxA0ioY/94YAEgwPHAt0HKZyiwHXdjUtCuH3AqMApY7bPtL8B0b3068KCf4+KAdO+xq7fetZnydw4Q5q0/6C9/9fksBDB/M4Df1ePvnwb0ByKAFTX/lwKVvxr7/wrcFcTr5/c7pbk+g1biaAaquk1Vl3rrecA6IDm4uWqwCcAcdb4BuohIjyDk40wgTVWDOhKAqn4B7KmxeQIw21ufDVzk59BzgY9VdY+q7gU+BsY1R/5U9SNVLfeefgOkNPXr1lct168+xgAbVTVdVUuBubjr3qTqyp+ICHAZ8M+mft36quM7pVk+gxY4mpmI9AVGAt/62X2CiKwQkQUiMrRZMwYKfCQiS0TkGj/7k4EtPs8zCU7wm0zt/7DBvH4A3VR1m7e+HejmJ01LuY5X40qQ/hzqsxBIN3lVac/XUs3SEq7fKcAOVd1Qy/5mvX41vlOa5TNogaMZiUgM8AZwi6rm1ti9FFf9Mhz4GzCvmbN3sqqOAs4DbhSRU5v59Q9JRCKAC4HX/OwO9vU7gLo6gRbZ111E/giUAy/XkiRYn4WngAHACGAbrjqoJZpC3aWNZrt+dX2nBPIzaIGjmYhIOO4P/LKqvllzv6rmqmq+t/4+EC4iCc2VP1XN8h53Am/hqgR8ZQG9fJ6neNua03nAUlXdUXNHsK+fZ0dV9Z33uNNPmqBeRxGZBlwAXOF9sRykHp+FgFDVHapaoaqVwDO1vG6wr18YMAn4V21pmuv61fKd0iyfQQsczcCrE30OWKeqD9eSpruXDhEZg/vb7G6m/EWLSGzVOq4RdXWNZPOBq7zeVccDOT5F4uZS6y+9YF4/H/OBqh4qU4G3/aT5EDhHRLp6VTHneNsCTkTGAb8HLlTVwlrS1OezEKj8+baZTazldRcBA0Wkn1cCnYy77s3lLOB7Vc30t7O5rl8d3ynN8xkMZMu/LdW9GE7GFRlXAsu9ZTxwHXCdl+YmYA2ul8g3wInNmL/+3uuu8PLwR2+7b/4EeALXo2UVkNrM1zAaFwg6+2wL2vXDBbBtQBmujvgXQDzwKbAB+ASI89KmAs/6HHs1sNFbft6M+duIq9uu+gz+3UvbE3i/rs9CM+XvRe+ztRL3BdijZv685+NxvYjSmjN/3vZZVZ85n7TBuH61fac0y2fQhhwxxhjTIFZVZYwxpkEscBhjjGkQCxzGGGMaxAKHMcaYBrHAYYwxpkEscBjTSCJSIQeO2ttkI7WKSF/fkVmNaUnCgp0BY1qxIlUdEexMGNPcrMRhTBPz5mP4izcnw3cicoS3va+IfOYN4vepiPT2tncTNz/GCm850TtVqIg848238JGIRHrpf+PNw7BSROYG6W2adswChzGNF1mjqupyn305qjoMeBx41Nv2N2C2qh6DG2DwMW/7Y8C/1Q3QOAp3xzHAQOAJVR0K7AMu9rZPB0Z657kuUG/OmNrYnePGNJKI5KtqjJ/tGcAZqpruDUS3drnNQgAAATFJREFUXVXjRWQXbhiNMm/7NlVNEJFsIEVVS3zO0Rc3Z8JA7/kfgHBVvVdEPgDycSMAz1NvcEdjmouVOIwJDK1lvSFKfNYr2N8meT5u3LBRwCJvxFZjmo0FDmMC43Kfx6+99a9wo7kCXAF86a1/Cv+/nTu2TRiIozD+vbhKZWUAtsgyKSMqCkSF2IOSJk2GSMcKEWuQIhv8KexIkUIkDhnRfL/OV5x81bvns44FQJIuSf/fpEkegFlV7YEN0AN/Wo90S+5UpOs9Jvn89fxRVT+/5D4lOTC0hpdxbAm8JVkDR+B1HF8BuyRzhmaxYLiZ9ZwOeB/DJcC2qr4nW5F0Ac84pImNZxzPVfV173eRbsFPVZKkJjYOSVITG4ckqYnBIUlqYnBIkpoYHJKkJgaHJKnJCfWemsBnZ6VdAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Mr9MKYWd4Nml"
      },
      "source": [
        "Our fairly naive approach achieves an accuracy of around 87%. With state-of-the-art approaches, one should be able to get close to 95%."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PiIZ7hA24Nml"
      },
      "source": [
        "### Using a trained network to generate predictions on new data\n",
        "\n",
        "After having trained a network, you will want to use it in a practical setting. You can generate the likelihood of reviews being positive\n",
        "by using the `predict` method:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "snx9JWg14Nml",
        "outputId": "257cf7a7-5271-40bb-a570-c97405f54620"
      },
      "source": [
        "model.predict(x_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.04039168],\n",
              "       [1.        ],\n",
              "       [0.7644507 ],\n",
              "       ...,\n",
              "       [0.02527058],\n",
              "       [0.00225222],\n",
              "       [0.07414556]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dQE8AYgZ4Nmm"
      },
      "source": [
        "As you can see, the network is very confident for some samples (0.99 or more, or 0.01 or less) but less confident for others (0.6, 0.4).\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "la19phRI4rW7"
      },
      "source": [
        "### Embeddings"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hus_meajAFVP"
      },
      "source": [
        "#### Exercise 3\n",
        "\n",
        "**TODO**: So far, we've been using one-hot encoded vectors to represent our text. Try to use an embedding layer instead. Check the keras documentation for the specifics of an embedding layer."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e44tG5GVa43x"
      },
      "source": [
        "All of our reviews have different lengths, and the longest of the bunch is 2494 words long. In order to not have to wait forever, we will limit our maximum review size to 500."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GSkWzqWMZzdl",
        "outputId": "9e3ee947-e8c3-40b5-ca5b-5785eccf2a7f"
      },
      "source": [
        "max([len(x) for x in train_data])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2494"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gu2MBIin643w",
        "outputId": "72c10b77-45be-4680-9745-f7fdc0bed0b1"
      },
      "source": [
        "from keras.preprocessing import sequence\n",
        "\n",
        "maxlen = 500\n",
        "\n",
        "print('Pad sequences (samples x time)')\n",
        "input_train = sequence.pad_sequences(train_data, maxlen=maxlen)\n",
        "input_test = sequence.pad_sequences(test_data, maxlen=maxlen)\n",
        "print('input_train shape:', input_train.shape)\n",
        "print('input_test shape:', input_test.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Pad sequences (samples x time)\n",
            "input_train shape: (25000, 500)\n",
            "input_test shape: (25000, 500)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hhAssq3YaGyp"
      },
      "source": [
        "from keras.layers import Dense, Embedding\n",
        "from keras.models import Sequential\n",
        "\n",
        "# start with a small embedding size if you don't want to wait 17 years\n",
        "embedding_size = 32\n",
        "\n",
        "model = models.Sequential()\n",
        "model.add()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T6W7MKNzaYQ9"
      },
      "source": [
        "model.compile(optimizer='rmsprop',\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['accuracy'])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BGBCUH5hadOr",
        "outputId": "d07cd971-7fde-494c-e3d5-b294558ab3b6"
      },
      "source": [
        "history = model.fit(input_train, y_train,\n",
        "                    epochs=20,\n",
        "                    batch_size=512,\n",
        "                    validation_split=0.2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "40/40 [==============================] - 6s 130ms/step - loss: 0.6921 - accuracy: 0.5144 - val_loss: 0.6909 - val_accuracy: 0.5199\n",
            "Epoch 2/20\n",
            "40/40 [==============================] - 5s 126ms/step - loss: 0.6898 - accuracy: 0.5252 - val_loss: 0.6890 - val_accuracy: 0.5237\n",
            "Epoch 3/20\n",
            "40/40 [==============================] - 5s 128ms/step - loss: 0.6882 - accuracy: 0.5238 - val_loss: 0.6891 - val_accuracy: 0.5183\n",
            "Epoch 4/20\n",
            "40/40 [==============================] - 5s 129ms/step - loss: 0.6874 - accuracy: 0.5213 - val_loss: 0.6875 - val_accuracy: 0.5237\n",
            "Epoch 5/20\n",
            "40/40 [==============================] - 5s 127ms/step - loss: 0.6868 - accuracy: 0.5251 - val_loss: 0.6877 - val_accuracy: 0.5227\n",
            "Epoch 6/20\n",
            "40/40 [==============================] - 5s 126ms/step - loss: 0.6867 - accuracy: 0.5260 - val_loss: 0.6871 - val_accuracy: 0.5249\n",
            "Epoch 7/20\n",
            "40/40 [==============================] - 5s 126ms/step - loss: 0.6864 - accuracy: 0.5239 - val_loss: 0.6873 - val_accuracy: 0.5257\n",
            "Epoch 8/20\n",
            "40/40 [==============================] - 5s 128ms/step - loss: 0.6864 - accuracy: 0.5239 - val_loss: 0.6875 - val_accuracy: 0.5214\n",
            "Epoch 9/20\n",
            "40/40 [==============================] - 5s 125ms/step - loss: 0.6863 - accuracy: 0.5214 - val_loss: 0.6870 - val_accuracy: 0.5248\n",
            "Epoch 10/20\n",
            "40/40 [==============================] - 5s 126ms/step - loss: 0.6863 - accuracy: 0.5232 - val_loss: 0.6870 - val_accuracy: 0.5253\n",
            "Epoch 11/20\n",
            "40/40 [==============================] - 5s 125ms/step - loss: 0.6862 - accuracy: 0.5267 - val_loss: 0.6872 - val_accuracy: 0.5243\n",
            "Epoch 12/20\n",
            "40/40 [==============================] - 5s 128ms/step - loss: 0.6862 - accuracy: 0.5239 - val_loss: 0.6873 - val_accuracy: 0.5223\n",
            "Epoch 13/20\n",
            "40/40 [==============================] - 5s 127ms/step - loss: 0.6862 - accuracy: 0.5237 - val_loss: 0.6873 - val_accuracy: 0.5243\n",
            "Epoch 14/20\n",
            "40/40 [==============================] - 5s 126ms/step - loss: 0.6861 - accuracy: 0.5253 - val_loss: 0.6873 - val_accuracy: 0.5221\n",
            "Epoch 15/20\n",
            "40/40 [==============================] - 5s 124ms/step - loss: 0.6862 - accuracy: 0.5241 - val_loss: 0.6878 - val_accuracy: 0.5205\n",
            "Epoch 16/20\n",
            "40/40 [==============================] - 5s 126ms/step - loss: 0.6862 - accuracy: 0.5243 - val_loss: 0.6872 - val_accuracy: 0.5225\n",
            "Epoch 17/20\n",
            "40/40 [==============================] - 5s 127ms/step - loss: 0.6861 - accuracy: 0.5225 - val_loss: 0.6878 - val_accuracy: 0.5227\n",
            "Epoch 18/20\n",
            "40/40 [==============================] - 5s 127ms/step - loss: 0.6861 - accuracy: 0.5254 - val_loss: 0.6872 - val_accuracy: 0.5226\n",
            "Epoch 19/20\n",
            "40/40 [==============================] - 5s 128ms/step - loss: 0.6861 - accuracy: 0.5232 - val_loss: 0.6870 - val_accuracy: 0.5256\n",
            "Epoch 20/20\n",
            "40/40 [==============================] - 5s 128ms/step - loss: 0.6861 - accuracy: 0.5272 - val_loss: 0.6871 - val_accuracy: 0.5250\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "95y4Omho4gLZ"
      },
      "source": [
        "### Recurrent Neural Network"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-Q-vc8RObbOg"
      },
      "source": [
        "Now we're going to use a recurrent neural network (RNN) for this task. An RNN can handle input sequences of arbitrary length and is good for tasks where the history of a sequence needs to be remembered. For this \"simple\" classification task it will most probably fall short of the performance of our simple linear neural network above, but when we will be using language models, some kind of recurrent neural network will be our best choice.\n",
        "\n",
        "RNNs will be further developed in the coming classes."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Exercise 4\n",
        "\n",
        "**TODO**: use the SimpleRNN layer to add a recurring neural network to our model. As in the previous model, start with an embedding layer. Then train the network."
      ],
      "metadata": {
        "id": "-cD5q3cMwqIJ"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Npy7SIto4e0f"
      },
      "source": [
        "from keras.layers import Dense, Embedding, SimpleRNN\n",
        "from keras.models import Sequential\n",
        "\n",
        "embedding_size = 32\n",
        "\n",
        "model = Sequential()\n",
        "model.add(...)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LT-kpgKP4rj7"
      },
      "source": [
        "model.compile(optimizer='rmsprop',\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['accuracy'])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0hBa33Le4tXD",
        "outputId": "5e633361-64d2-406f-853e-a748419dcd07"
      },
      "source": [
        "history = model.fit(input_train, y_train,\n",
        "                    epochs=10,\n",
        "                    batch_size=512,\n",
        "                    validation_split=0.2)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "40/40 [==============================] - 15s 349ms/step - loss: 0.6842 - accuracy: 0.5548 - val_loss: 0.6660 - val_accuracy: 0.5922\n",
            "Epoch 2/10\n",
            "40/40 [==============================] - 14s 346ms/step - loss: 0.5610 - accuracy: 0.7552 - val_loss: 0.5856 - val_accuracy: 0.6890\n",
            "Epoch 3/10\n",
            "40/40 [==============================] - 14s 341ms/step - loss: 0.4312 - accuracy: 0.8288 - val_loss: 0.5285 - val_accuracy: 0.7382\n",
            "Epoch 4/10\n",
            "40/40 [==============================] - 13s 336ms/step - loss: 0.3673 - accuracy: 0.8505 - val_loss: 0.3765 - val_accuracy: 0.8456\n",
            "Epoch 5/10\n",
            "40/40 [==============================] - 14s 344ms/step - loss: 0.2907 - accuracy: 0.8905 - val_loss: 0.4854 - val_accuracy: 0.7864\n",
            "Epoch 6/10\n",
            "40/40 [==============================] - 14s 339ms/step - loss: 0.2495 - accuracy: 0.9056 - val_loss: 0.4130 - val_accuracy: 0.8146\n",
            "Epoch 7/10\n",
            "40/40 [==============================] - 14s 344ms/step - loss: 0.1990 - accuracy: 0.9302 - val_loss: 0.3825 - val_accuracy: 0.8408\n",
            "Epoch 8/10\n",
            "40/40 [==============================] - 14s 346ms/step - loss: 0.1632 - accuracy: 0.9440 - val_loss: 0.3781 - val_accuracy: 0.8558\n",
            "Epoch 9/10\n",
            "40/40 [==============================] - 14s 340ms/step - loss: 0.1317 - accuracy: 0.9579 - val_loss: 0.4344 - val_accuracy: 0.8596\n",
            "Epoch 10/10\n",
            "40/40 [==============================] - 14s 338ms/step - loss: 0.1372 - accuracy: 0.9561 - val_loss: 0.4120 - val_accuracy: 0.8362\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SxUw4hFO7zG7",
        "outputId": "9609ecaa-ce03-420b-be21-f2fee759a72e"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "epochs = range(1, len(loss) + 1)\n",
        "\n",
        "plt.plot(epochs, loss, label='Training loss')\n",
        "plt.plot(epochs, val_loss, label='Validation loss')\n",
        "plt.title('Training and validation loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hU1dbA4d9KI0ASeg8YeocEAggRpVhoglKkfSJYQbBgQfRaUK9eC15RwQIieBVEQEUQEZVeBAkt9F4SauiBQOr+/jgDBEwjmcmZZNb7PDzMnDlnn5UJzJq99zlrizEGpZRSnsvL7gCUUkrZSxOBUkp5OE0ESinl4TQRKKWUh9NEoJRSHk4TgVJKeThNBMqpRGSeiDzg7H3tJCL7ReR2F7RrRKSG4/HnIvJKdvbNwXn6i8jvOY0zk3bbiEiMs9tVec/H7gCU/UTkfJqnRYAEIMXx/DFjzJTstmWM6eiKfQs6Y8xgZ7QjIiHAPsDXGJPsaHsKkO3fofI8mggUxpiAy49FZD/wsDHmz+v3ExGfyx8uSqmCQ4eGVIYud/1F5AUROQpMEpESIvKLiMSKyGnH4+A0xywWkYcdjweKyHIRGe3Yd5+IdMzhvlVFZKmIxInInyIyTkS+zSDu7MT4poiscLT3u4iUTvP6/SJyQEROisi/Mnl/WojIURHxTrPtXhGJcjxuLiJ/icgZETkiImNFxC+DtiaLyL/TPH/eccxhEXnwun07i8h6ETknItEiMirNy0sdf58RkfMi0vLye5vm+FYiskZEzjr+bpXd9yYzIlLXcfwZEdkiIl3TvNZJRLY62jwkIs85tpd2/H7OiMgpEVkmIvq5lMf0DVdZKQ+UBG4CHsX6NzPJ8bwKcBEYm8nxLYAdQGngPWCiiEgO9p0K/A2UAkYB92dyzuzE2A8YBJQF/IDLH0z1gM8c7Vd0nC+YdBhjVgMXgHbXtTvV8TgFGO74eVoC7YHHM4kbRwwdHPHcAdQErp+fuAAMAIoDnYEhInKP47VbHX8XN8YEGGP+uq7tksBc4GPHz/ZfYK6IlLruZ/jHe5NFzL7AHOB3x3FPAFNEpLZjl4lYw4yBQANgoWP7s0AMUAYoB7wEaN2bPKaJQGUlFXjNGJNgjLlojDlpjPnBGBNvjIkD3gJuy+T4A8aYCcaYFOBroALWf/hs7ysiVYBmwKvGmERjzHJgdkYnzGaMk4wxO40xF4HpQKhje0/gF2PMUmNMAvCK4z3IyHdAXwARCQQ6ObZhjFlrjFlljEk2xuwHvkgnjvTc54hvszHmAlbiS/vzLTbGbDLGpBpjohzny067YCWOXcaYbxxxfQdsB+5Os09G701mbgYCgHccv6OFwC843hsgCagnIkHGmNPGmHVptlcAbjLGJBljlhktgJbnNBGorMQaYy5dfiIiRUTkC8fQyTmsoYjiaYdHrnP08gNjTLzjYcAN7lsROJVmG0B0RgFnM8ajaR7Hp4mpYtq2HR/EJzM6F9a3/+4iUgjoDqwzxhxwxFHLMexx1BHH21i9g6xcEwNw4Lqfr4WILHIMfZ0FBmez3cttH7hu2wGgUprnGb03WcZsjEmbNNO22wMrSR4QkSUi0tKx/X1gN/C7iOwVkZHZ+zGUM2kiUFm5/tvZs0BtoIUxJoirQxEZDfc4wxGgpIgUSbOtcib75ybGI2nbdpyzVEY7G2O2Yn3gdeTaYSGwhpi2AzUdcbyUkxiwhrfSmorVI6psjCkGfJ6m3ay+TR/GGjJLqwpwKBtxZdVu5evG96+0a4xZY4zphjVsNAurp4ExJs4Y86wxphrQFXhGRNrnMhZ1gzQRqBsViDXmfsYx3vyaq0/o+IYdCYwSET/Ht8m7MzkkNzHOBLqIyC2Oid03yPr/yVTgKayEM+O6OM4B50WkDjAkmzFMBwaKSD1HIro+/kCsHtIlEWmOlYAui8UayqqWQdu/ArVEpJ+I+IhIb6Ae1jBObqzG6j2MEBFfEWmD9Tua5vid9ReRYsaYJKz3JBVARLqISA3HXNBZrHmVzIbilAtoIlA3agxQGDgBrAJ+y6Pz9seacD0J/Bv4Hut+h/TkOEZjzBZgKNaH+xHgNNZkZmYuj9EvNMacSLP9OawP6ThggiPm7MQwz/EzLMQaNll43S6PA2+ISBzwKo5v145j47HmRFY4rsS5+bq2TwJdsHpNJ4ERQJfr4r5hxphErA/+jljv+6fAAGPMdscu9wP7HUNkg7F+n2BNhv8JnAf+Aj41xizKTSzqxonOy6j8SES+B7YbY1zeI1GqoNMegcoXRKSZiFQXES/H5ZXdsMaalVK5pHcWq/yiPPAj1sRtDDDEGLPe3pCUKhh0aEgppTycDg0ppZSHy3dDQ6VLlzYhISF2h6GUUvnK2rVrTxhjyqT3mksTgWNS7yPAG/jSGPPOda9/CLR1PC0ClDXGFM+szZCQECIjI10RrlJKFVgicv0d5Ve4LBE4bucfh1U4KwZYIyKzHXdiAmCMGZ5m/yeAMFfFo5RSKn2unCNoDuw2xux13GwyDeuSv4z0xVGsSymlVN5xZSKoxLWFs2K4trDVFSJyE1CVf95BqZRSysXcZbK4DzDTUX74H0TkUaxa+FSpcn39LaWUqyUlJRETE8OlS5ey3lnZyt/fn+DgYHx9fbN9jCsTwSGuraAYTMYVDvtg1XdJlzFmPDAeIDw8XG98UCqPxcTEEBgYSEhICBmvK6TsZozh5MmTxMTEULVq1Wwf58qhoTVATbGWGPTD+rD/x2IijqqMJbAKTiml3NClS5coVaqUJgE3JyKUKlXqhntuLksEjkXOhwHzgW3AdGPMFhF5I+1aplgJYpquSqSUe9MkkD/k5Pfk0jkCY8yvWPXP02579brno1wZw2V7Y88zPTKGFzrU1n/QSimVhseUmFi4/TifL9nDVyv22x2KUuoGnTx5ktDQUEJDQylfvjyVKlW68jwxMTHTYyMjI3nyySezPEerVq2cEuvixYvp0qWLU9rKK+5y1ZDLPXRLVVbvO8V/ft1GaOViNL2ppN0hKaWyqVSpUmzYsAGAUaNGERAQwHPPPXfl9eTkZHx80v84Cw8PJzw8PMtzrFy50jnB5kMe0yMQEUb3akzF4oUZOmU9J89ntLiVUio/GDhwIIMHD6ZFixaMGDGCv//+m5YtWxIWFkarVq3YsWMHcO039FGjRvHggw/Spk0bqlWrxscff3ylvYCAgCv7t2nThp49e1KnTh369+/P5SnMX3/9lTp16tC0aVOefPLJLL/5nzp1invuuYdGjRpx8803ExUVBcCSJUuu9GjCwsKIi4vjyJEj3HrrrYSGhtKgQQOWLVvm9PcsIx7TIwAoVtiXT/s3oftnK3n6+w1MHtQcby+dL1DqRrw+ZwtbD59zapv1Kgbx2t31b/i4mJgYVq5cibe3N+fOnWPZsmX4+Pjw559/8tJLL/HDDz/845jt27ezaNEi4uLiqF27NkOGDPnHNffr169ny5YtVKxYkYiICFasWEF4eDiPPfYYS5cupWrVqvTt2zfL+F577TXCwsKYNWsWCxcuZMCAAWzYsIHRo0czbtw4IiIiOH/+PP7+/owfP5677rqLf/3rX6SkpBAfH3/D70dOeUyP4LIGlYrxRtf6LNt1go8W7LI7HKVULvTq1Qtvb28Azp49S69evWjQoAHDhw9ny5Yt6R7TuXNnChUqROnSpSlbtizHjh37xz7NmzcnODgYLy8vQkND2b9/P9u3b6datWpXrs/PTiJYvnw5999/PwDt2rXj5MmTnDt3joiICJ555hk+/vhjzpw5g4+PD82aNWPSpEmMGjWKTZs2ERgYmNO35YZ5To/AGIg/BUVL0btZZdbsP80nC3fRpEpx2tQua3d0SuUbOfnm7ipFixa98viVV16hbdu2/PTTT+zfv582bdqke0yhQoWuPPb29iY5OTlH++TGyJEj6dy5M7/++isRERHMnz+fW2+9laVLlzJ37lwGDhzIM888w4ABA5x63ox4To9g1WfwaQuIiURE+Pc9DahdLpDh32/g0JmLdkenlMqls2fPUqmSVc5s8uTJTm+/du3a7N27l/379wPw/fffZ3lM69atmTJlCmDNPZQuXZqgoCD27NlDw4YNeeGFF2jWrBnbt2/nwIEDlCtXjkceeYSHH36YdevWOf1nyIjnJIIa7cGvKEzuDJt/pLCfN5/2b0JSimHolHUkJqfaHaFSKhdGjBjBiy++SFhYmNO/wQMULlyYTz/9lA4dOtC0aVMCAwMpVqxYpseMGjWKtWvX0qhRI0aOHMnXX38NwJgxY2jQoAGNGjXC19eXjh07snjxYho3bkxYWBjff/89Tz31lNN/hozkuzWLw8PDTY4XprlwAqb1h+hV0O5laP0cv24+yuNT1jGwVQijurpPl1cpd7Jt2zbq1q1rdxi2O3/+PAEBARhjGDp0KDVr1mT48OFZH5jH0vt9ichaY0y619F6To8AoGhpGPAzNOwFC/8Ns4bQqW5JHrqlKpNX7mfOxsN2R6iUcmMTJkwgNDSU+vXrc/bsWR577DG7Q3IKz5ksvszXH7pPgFI1YfHbcPoAI3t9w4boM4z8IYq6FYKoUTbA7iiVUm5o+PDhbtkDyC3P6hFcJgJtXoDuX8KhSHwn3cFnHQIp5OvNkG/XEp/o/PFFpZRyV56ZCC5r1Ase+AUunaPs9134um0Cu2PP89KPm8hvcydKKZVTnp0IAKq0gEcWQEA5Gi58gAkNtjFrw2GmrD5od2RKKZUnNBEAlAiBh36HkNbcvutNxpb9mTfnbCYq5ozdkSmllMtpIriscHHoPwOaDqLLue/5vNDHPP3NX5yJz7zErVLK9dq2bcv8+fOv2TZmzBiGDBmS4TFt2rTh8qXmnTp14syZf36xGzVqFKNHj8703LNmzWLr1q1Xnr/66qv8+eefNxJ+utypXLUmgrS8faHLh3DnW7RJXc2Yiy/x+tSFpKbqfIFSdurbty/Tpk27Ztu0adOyVe8HrKqhxYsXz9G5r08Eb7zxBrfffnuO2nJXmgiuJwKthiF9plLX9wjPRT/O9Lnz7I5KKY/Ws2dP5s6de2URmv3793P48GFat27NkCFDCA8Pp379+rz22mvpHh8SEsKJEycAeOutt6hVqxa33HLLlVLVYN0j0KxZMxo3bkyPHj2Ij49n5cqVzJ49m+eff57Q0FD27NnDwIEDmTlzJgALFiwgLCyMhg0b8uCDD5KQkHDlfK+99hpNmjShYcOGbN++PdOfz+5y1Z53H0F21emEz0O/UeSrHnSJHMS2oLHUva2X3VEpZb95I+HoJue2Wb4hdHwnw5dLlixJ8+bNmTdvHt26dWPatGncd999iAhvvfUWJUuWJCUlhfbt2xMVFUWjRo3SbWft2rVMmzaNDRs2kJycTJMmTWjatCkA3bt355FHHgHg5ZdfZuLEiTzxxBN07dqVLl260LNnz2vaunTpEgMHDmTBggXUqlWLAQMG8Nlnn/H0008DULp0adatW8enn37K6NGj+fLLLzP8+ewuV609gkxIxVAKDV7EEe+K1Fr0KOcWf2JVMVVK5bm0w0Nph4WmT59OkyZNCAsLY8uWLdcM41xv2bJl3HvvvRQpUoSgoCC6du165bXNmzfTunVrGjZsyJQpUzIsY33Zjh07qFq1KrVq1QLggQceYOnSpVde7969OwBNmza9UqguI3aXq9YeQRaKlK6CPPgbiyb04/bFL5Nyfh/eHd8Db33rlIfK5Ju7K3Xr1o3hw4ezbt064uPjadq0Kfv27WP06NGsWbOGEiVKMHDgQC5dupSj9gcOHMisWbNo3LgxkydPZvHixbmK93Ip69yUsc6rctXaI8iGGsHluHDPJL5I7ox35ET4rjdccu4KTUqpzAUEBNC2bVsefPDBK72Bc+fOUbRoUYoVK8axY8eYNy/z+bxbb72VWbNmcfHiReLi4pgzZ86V1+Li4qhQoQJJSUlXSkcDBAYGEhcX94+2ateuzf79+9m9ezcA33zzDbfddluOfja7y1VrIsimbmFViA5/kZFJD5O6ZzF8dRec0ZvOlMpLffv2ZePGjVcSweWyzXXq1KFfv35ERERkenyTJk3o3bs3jRs3pmPHjjRr1uzKa2+++SYtWrQgIiKCOnXqXNnep08f3n//fcLCwtizZ8+V7f7+/kyaNIlevXrRsGFDvLy8GDx4cI5+LrvLVXtWGepcSkhOodfnf1H2xCrG+32El28h6DsNgtOt7KpUgaFlqPMXLUPtQoV8vBnXrwlrpBFD/N8l1ffqQjdKKZVfaSK4QZVLFuHD3o2Zf7wYb1X4GCqEwsxBsPR9vaJIKZUvaSLIgXZ1yjGsbQ0mrj/PDw3GXbPQDckJdoenlEvkt2FkT5WT35NLE4GIdBCRHSKyW0RGZrDPfSKyVUS2iMhUV8bjTMPvqEWr6qV4ac4utt78AbR5CTZ+B/+7B+JP2R2eUk7l7+/PyZMnNRm4OWMMJ0+exN/f/4aOc9lksYh4AzuBO4AYYA3Q1xizNc0+NYHpQDtjzGkRKWuMOZ5Zu3ZOFl8vNi6Bzh8vo4ifN7OfuIWgXT/DrMehWCXoNx1K17Q7RKWcIikpiZiYmBxfo6/yjr+/P8HBwfj6+l6zPbPJYlfeFdUc2G2M2esIYhrQDUh7298jwDhjzGmArJKAuykTWIhx/ZvQZ/wqRsyI4rP/64EUqwzT+sGXt0Pvb6Fqa7vDVCrXfH19qVq1qt1hKBdx5dBQJSA6zfMYx7a0agG1RGSFiKwSkQ7pNSQij4pIpIhExsbGuijcnGkWUpKRHerw25ajTFy+75qFbvjmHlj3jd0hKqVUpuyeLPYBagJtgL7ABBH5R61YY8x4Y0y4MSa8TJkyeRxi1h5uXZW76pfjnXnbidx/6pqFbpg9DP54DVJT7Q5TKaXS5cpEcAionOZ5sGNbWjHAbGNMkjFmH9acQr4bWBcR3u/VmEolCjN06jpOnE+4ZqEbVoyBGQMgMfdVApVSytlcmQjWADVFpKqI+AF9gNnX7TMLqzeAiJTGGira68KYXCbI35dP+zfhTHwST01bT0qqubrQzV1vw7ZfYHIniDtqd6hKKXUNlyUCY0wyMAyYD2wDphtjtojIGyJyufbrfOCkiGwFFgHPG2NOuiomV6tfsRhvdmvAit0n+ejPndZGEWg5FPpMhdidMKG982u5K6VULmitIRcYMXMj0yNjmDSoGW1rl736wpGNMLUPxJ+AVk9C62fBr4h9gSqlPIbWGspjb3RrQJ3ygQz/fgMxp9PMC1RoDI8tgfr3wrLR8GkL2P6rfYEqpRSaCFzC39ebz/+vKSkphqFT15OQnHL1xYCy0H08DJwLvkVhWl+Y2htO7bMvYKWUR9NE4CIhpYvyfq/GbIw+w9tzt6Wzwy0weBnc8SbsWwaf3gyL34UkvXNTKZW3NBG4UIcG5XmkdVW+/usAszce/ucO3r4Q8SQMWwO1O8Lit62EsOuPvA9WKeWxNBG42IgOdWgWUoKRP0Sx+/g/l7sDrNpEvSbD/bPAyxum9IRp/eFMdPr7K6WUE2kicDFfby/G9mtCET9vBn+7jgsJmSxiXb0tDFkJ7V+F3QtgXHNY9l9ITsy7gJVSHkcTQR4oF+TPR33C2Bt7npd+2pR5KV+fQtZlpcP+hurtYMHr8Fkr2Ls4z+JVSnkWTQR5JKJGaZ65oxY/bzjMt6sOZH1A8SrQZwr0nwmpyfC/bjBjIJxLZ65BKaVyQRNBHnq8TQ3a1i7Dm79sY+vhc9k7qOYd8PgqaPOidc/B2Gaw8hNISXJtsEopj6GJIA95eQkf3BdK8SK+DPtuHfGJmcwXpOXrD21GwtDVcFME/P4yfN4a9q9wbcBKKY+giSCPlSzqx5g+oew7cYHXZ2/N+oBrDq4K/b636hYlXrCK2P34KMQdc02wSimPoInABq2ql2Zomxp8HxnNnPTuL8iMCNTpbPUOWj8HW36CseGw+gtIyWYPQyml0tBEYJOnbq9JkyrFeenHTUSfysE6BX5FoP0rMOQvqNQU5o2ACW0g+m+nx6qUKtg0EdjE19uLj/qEgcAT360nKSWHK5iVrgH3/wS9voYLJ2HiHfDzULhwwrkBK6UKLE0ENqpcsgjvdG/EhugzfPjHzpw3JAL177FKVUQ8BRunwSdNYc1ESE3J+nillEfTRGCzzo0q0Ld5ZT5bsocVu3P5Lb5QANzxBgxeAeUbwtxn4Mv2cGitc4JVShVImgjcwKtd6lO9TABPf7+Bk+cTct9g2TrwwBzo/qV1A9qE9jDnaYg/lfu2lVIFjiYCN1DYz5tP+oZx9mISz83YSGqqE1aNE4FGvWBYJNw8BNb9z7q6aN03kJrD+QilVIGkicBN1K0QxMud67JoRyyTVu53XsP+QdDhP9bKaKVqwuxh8NVdcCTKeedQSuVrmgjcyP0338Qd9crxzrxtbD501rmNl28Ig+bBPZ/Bqb0w/jZYOda551BK5UuaCNyIiPBej0aUDijEE9+tz7xkdU54eUFoP3giEmreBX++Bsdu8O5mpVSBo4nAzZQo6seHvUM5cPICr/68xTUnKVwCuo2DQoHwy3CdM1DKw2kicEM3VyvFsHY1+WFdDLPWH3LNSYqWgjv/DdGrYP03rjmHUipf0ETgpp5sV4NmISV4edZmDpy84JqThPa3qpn+8Sqcj3XNOZRSbk8TgZvy8fZiTJ8wvL2EJ79bT2KyC4ZvRKDLh1Yl099fdn77Sql8QROBG6tUvDDv9mjIxpizfPDHDtecpExtqyxF1DTYu8Q151BKuTVNBG6uQ4MK9G9RhS+W7GXpThcN39z6HJSoak0cJ11yzTmUUm7LpYlARDqIyA4R2S0iI9N5faCIxIrIBsefh10ZT371Spd61CoXwDPTNxIb54QSFNfzLQydP4BTe2D5h85vXynl1lyWCETEGxgHdATqAX1FpF46u35vjAl1/PnSVfHkZ/6+3nzStwlxl5J41lklKK5Xoz006AnL/wsndjm/faWU23Jlj6A5sNsYs9cYkwhMA7q58HwFWu3ygbzSpR5Ld8Yycfk+15zkrrfBp7BVtdS4INkopdySKxNBJSA6zfMYx7br9RCRKBGZKSKV02tIRB4VkUgRiYyN9dzLHPu3qEKH+uV5b/52omLOOP8EgeXg9tdg31KI+t757Sul3JLdk8VzgBBjTCPgD+Dr9HYyxow3xoQbY8LLlCmTpwG6ExHhnR4NKeMoQXHe2SUoAJoOguBmMP8lLVutlIdwZSI4BKT9hh/s2HaFMeakMeby7OeXQFMXxlMgFC/ix0d9w4g+Fc8rszY7/wReXtBlDFw8Y91oppQq8FyZCNYANUWkqoj4AX2A2Wl3EJEKaZ52Bba5MJ4Co1lISZ5qX4uf1h/ix3Uxzj9B+QbQcqhVeuLASue3r5RyKy5LBMaYZGAYMB/rA366MWaLiLwhIl0duz0pIltEZCPwJDDQVfEUNMPa1aB51ZK8PGsz+064oARFm5FQrIp1b0FyovPbdyepqbD7T0iMtzsSpWwhJp9dHRIeHm4iIyPtDsMtHDl7kY4fLSO4RGF+HBKBn4+T8/qO3+C73tD+VWj9rHPbdie/vwIrP4ay9eG+/0HpGnZHpJTTichaY0x4eq/ZPVmscqFCscK816MRmw+d473ftjv/BLU7QN27Ycl71mI2BdGaL60kULszxB2G8W1gyyy7o1IqT2kiyOfurF+eAS1v4svl+1i047jzT9DxPfDyhbnPFbx7C3b+Dr8+D7U6WD2Bx5ZZtZdmPAC/vVjwh8SUctBEUAC81KkudcoH8tz0jRw/5+RaQUEVod3LsGcBbPnRuW3b6chGmDHQWsKzx0Tw9oHila3lPFsMhlWfwuTOcNZF60Eo5UY0ERQA/r7ejO0XxoXEZJ6Z7oISFM0fgQqh1rfkiy64kS2vnY2BKfdBkZLQbzoUCrj6mo8fdHwXek6C41vhi9awZ6F9sSqVBzQRFBA1ygYy6u76LN99gi+WOnk838sb7h4DF2Jh4ZvObTuvXToLU3pBUjz0nwGB5dPfr0F3eGQRFC0L33SHxe/qkp6qwNJEUID0blaZzg0r8MHvO1h/8LRzG68YBs0fhTUTISafXrWVkgTTH4ATO6H3N1C2bub7l6kFjyyARvfB4rdhSk+4cDJvYlUqD2kiKEBEhLe7N6RckD9PTlvPuUtJzj1B239BYAWY8zSkuKC8hSsZA788DXsXwd0fQ7U22TvOryjc+4V1t/X+ZdZQUfQaV0aqVJ7TRFDAFCvsy8d9wzh85hIv/7QZp94n4h9kjZ8f2wSrP3Neu3lh2WhY/y3c9gKE9b+xY0UgfBA89Ls1TDapI6z6vOBdRaU8liaCAqjpTSV45o5azN54mJlrnVyCou7d1uWWi96GM9FZ7+8OoqbDwn9Doz7Q5sWct1MxDB5bCjVuh99esK46unTOaWEqZRdNBAXU4Nuq07JaKV79eQt7Ys87r2ER6PS+9fjX593/W/H+5fDzUAhpDV0/seLPjcIloM9UuP112DYbJrSFY1ucE6tSNtFEUEB5ewlj+oTi7+vFE1PXk5Cc4rzGi1exahHtnAfbf3Feu84WuxOm9bfWY+79jXVpqDN4ecEtT8MDc6wewYT2sHGac9pWygaaCAqwckH+jO7VmK1HzvHOPCeXoLj5cSjXAH4dAQlxzm3bGc4ft67y8fa1LhMtXML55wi5BQYvg0pN4afHYM5TkOTkG/qUygOaCAq49nXLMSgihEkr9rNg2zHnNezta11JE3fEmi9wJ4nx8F0fKxn0+x5K3OS6cwWWhwE/wy3DYe1kmHgHnHLRUqJKuYgmAg8wsmMd6lUI4vmZURxzZgmKys2sq2lWfw6HNziv3dxITYEfH4FD66DnROvbuqt5+8Dto6DvNDhzAMbfBtt/df15lXISTQQeoJCPN5/0C+NiYgpPT9tAijNLULR/DYqUtq7RT3XiPERO/f6KNW/R4R2o0zlvz127Izy6BEqEwLS+8Mdr+e9+C+WRNBF4iOplAni9W33+2nuSz5fscV7DhYtDh//A4fVWSWc7rf4CVo2DFkPg5sH2xFCyKjz4u7X284ox8L+uEHfUnliUyiZNBB6kV+Q4EUIAAB4zSURBVNNg7m5ckf/+sZO1B5xYgqJBD6jeDha8CecOO6/dG7H9V/htJNTpAne9ZU8Ml/n6W7WZ7v3CSpCft4Z9y+yNSeVcYjwkJ2S9Xz6micCDiAhv3duAisX9efK79Zy96KQSFCLQ+QNITbI+jPPaoXXww0NWhdTuE6y7f91B4z7w8ALwL2b1DJZ/qIXr8pPkROt39n4N+E9l+KojLHgDdv1hFS8sQHSpSg+0/uBpen3+F6GVi/Nh71AqlyzinIaXvm/dwdtvBtS60zltZuX0Afjydutb+MMLIKBs3pz3RiTEwewnYMtPUKsj3PuZay5nVc6zewHMGwEnd1u/s1LV4eBf1joWqckgXlCuPlRpBTe1hCotM65k6yYyW6oyW4lARIoCF40xqSJSC6gDzDPGOLmqWdY0ETjH7I2HeenHTRhjeKlzXfo1r4Lk9q7b5ET4/BZIughDV1kF21zp4hmYeCecPwoP/WGtLuaujIG/J8D8lyCogrUiWsUwu6NS1ztz0PodbZsDJatBh3ev/VKTeMGqvnvwLziwEmLWWCXNwbpx8aZWVlK4qZV1fG7/TzmRMxLBWqA1UAJYAawBEo0xN1i9K/c0ETjPoTMXeWFmFMt3n6B1zdK826MRFYsXzl2j+1fA5E4Q8RTc8YZzAk1PciJ82x0OroL7f4KqrV13LmeKXmPVKLpw3Crg13SQW31YeKykS7DyE1j2gfX81ueg5TCrp5mZlCQ4EmUlhst/4h2lyouWhSo3X00O5RvaOmzpjESwzhjTRESeAAobY94TkQ3GmFBnB5sVTQTOZYzh29UH+c+v2/AW4dW769GzaXDuegc/D7VKLjy21Oo+O5sxMGsIbPwO7h0PjXs7/xyudOGkda/DngVWIbwu/3V970llbOd8mPcCnN4HdbvCXW9by5bmhDHWehcHVjp6DX/B2YPWa36BULm5o8fQ0rrHxTeXX7xugDMSwXrgceBD4CFjzBYR2WSMaejcULOmicA1Dp6M57mZG/l73ylur1uWt+9tSNmgLL4NZST+FIwNh5LV4cH5Vm0eZ1r8Diz+j7U+wm0jnNt2XklNtUpjL3obytSxhorK1LI7Ks9yap+1/OrOeVCqJnR6z7r6zdnOxlg918vJ4fhWa7u3nzU8eHkoqXJzl84dOSMR3AY8C6wwxrwrItWAp40xTzo31KxpInCd1FTDpJX7ee+37RT28+b1rvXp2rhiznoHG6Za39q7jLHuPnaWy+2G/h90G5v/h1X2LIQfHrYuT+z6sXUprnKtpIvW1UDLx4CXD7R5wbr3xFlFCbMSfwqiVzsSwyrrEuPUJECgbL2rk883tYKgik47ba4TwXWNeQEBxhhbCrFrInC9PbHneW7GRtYfPEOnhuV5s1sDSgUUurFGjIGv74ajUTAs0jlX8+xdYs0LhNwC/Wda9Y4KgrOHYOYg68Mh/CFoOdS6SkU5lzGwfS7Mf9GaFG7QE+5806kftjmSGA+H1l47AZ3oKB1fvEqaK5NaQemaOf7y44wewVRgMJCCNVEcBHxkjHk/RxHlgiaCvJGcksr4ZXsZ88cuAv19eOvehnRocIOXx8XuhM8joN490GNC7gI6vg0m3mX9p31ovnVtfkGSkmSVpFg1znpetr61CFC9rta3xPze87HbyT3W5aC7/4Qyda01Ndz1AoOUZGsVwAN/wcGV1t/xJ6zX7nwLWg3LUbPOSAQbjDGhItIfaAKMBNYaYxrlKKJc0ESQt3YcjePZGRvYfOgc94RWZFTX+hQvcgNd6EVvw5J3rSt7cjr+GncMvmwPKYnWvQI5ncjLD84chG2/WIveHFwFGOsyxLp3Q91uUKmJJoUbkXgBlo6Gv8aCj7+1Ql3zR/JXb9IY636Gg39ZQ0ala+aoGWckgi1AKDAVGGuMWSIiG40xjbM4rgPwEeANfGmMeSeD/XoAM4FmxphMP+U1EeS9pJRUPl20h08W7qJkUT/e7dGItnWyOdSTdAk+awUYGPJX1pfjXS/xAkzqBCd2waBfoWKeX6hmn7hjsGMubJ0N+5dZNzIFVXIkhbutDwV3uYva3RgDW2fB/H/BuUPQuK+1qlxgObsjs40zEsGTwAvARqAzUAX41hiTYd9KRLyBncAdQAzWkFJfY8zW6/YLBOYCfsAwTQTua/Ohszw7fSM7jsVxX3gwL3epR5B/Nr5Z7VkE39wDt46Adv/K/glTU6wVxnbNhz7fQe0OOQ8+v4s/BTt/s2502r0AUhKsqq91OlvDRyG35t1kp7uL3WEto7pvCZRrCJ1HW9fzezinThanadTHGJNhjV0RaQmMMsbc5Xj+IoAx5j/X7TcG+AN4HnhOE4F7S0hO4aM/d/H5kj2UD/LnvZ6NuaVm6awP/OERq8TCkJXZu0zSGGtM9+/x0Gm01Z1XloQ4q97Ntjmw63drYtG/mFUKoe7dUKN9nl6f7jYS4qxhyFWfWfdltHvFumHP28fuyNxCZokgWxd4i0gxEfmviEQ6/nwAZHUHTCUgOs3zGMe2tO02ASobY+Zmcf5HL587NjY2OyErFynk482IDnX4YUgr/P28+b+Jq3l51iYuJGRRd/+ut63/nL8Mz96C96s+s5JAqyc0CVyvUCA06A69JsHze6wFcWp3tnoM3/eH96rB9AGwaaa1pnJBZwxEzYBPwq27gxv3hSfWOeYCNAlkR3aHhn4ANgNfOzbdDzQ2xnTP5JieQAdjzMOO5/cDLYwxwxzPvYCFwEBjzH4RWYz2CPKVS0kpjJ6/g4kr9lG5RBHe79mIFtVKZXzA2snWur7dPoWwTKqTbJ1tfZDVvRt6fe38G9IKqpQkay5h2xxrwvnCceumpWptreGj2p2gSEm7o3SuY1usYaADK6ybszp9AMF5sCpdPuS0q4ay2nbd65kODYlIMWAP4LhglvLAKaBrZslAE4H7+XvfKZ6fuZGDp+IZ1KoqIzrUxt83nUnM1FSY1NG6BX9YJBRNJ2nERMLkzlZdlgfmeOYQhzOkpkD0346kMBvORoN4W/dg1L3bWrchqILdUebcpbOw6D9Wr9E/yFopr8kAnTzPhDMSwV/A88aY5Y7nEcBoY0zLTI7xwZosbg8cwpos7meM2ZLB/ovRHkG+FZ+YzDvztvO/vw5QrXRRRt/XmCZV0rld/thW+KK1VWPnnnHXvnZqn1VSulCAdZlo0WzMPaisGQNHNlg9rW2zrUsREaukweUrkEqE2B1l9qSmQtQ0+ONVuHDCumu93SsFr6fjAs5IBI2B/wGX7+I5DTxgjInK4rhOwBisy0e/Msa8JSJvAJHGmNnX7bsYTQT53ordJxgxM4ojZy/y2G3Vefr2mhTyue5b2h+vWcs4DvwVQiKsbfGnrJLS8SfgoT+hdI28D94TGAOx26/2FI5usraXb2QNH9Xt6r7lvI9stIaBoldDcDPrIgJPupw4l5x21ZCIBAEYY86JyNPGmDFOijHbNBG4v7hLSbw1dxvT1kRTq1wAH/QKpWFwmjuBE+Ph0xbWDT6Dl1vbvrnXurV+wM9WjRWVN07tvXoDW8waa1vpWlD1Nms9ar+i4Bfg+LtoOs8dj32LuO5Gt/hTsOgtiPwKCpeEO16Hxv107ugGuery0YPGmCq5iiwHNBHkH4t2HGfkD1GcOJ/IsLY1GNauBr7ejv+8u/6AKT2tCqIndsKmGdBjIjTsaW/QnuzcYasWz9af4fAGR72b7H4+yHXJoqhVdjnd5JFBMrn+sW9hq8jggtfh4mlo9gi0fclKUOqGuSoRRBtj8vxef00E+cvZ+CRen7OFH9cfon7FID64rzF1ygdZL05/wLr7E6zJvtbP2Beo+idjrEqdiRespJB4IYvH2XntvHWH9I2o0sqqDVS+gWt+Tg+hPQJlu/lbjvKvnzZx9mIST99ei8durYbPhWPwxa3W2HSn0VpDx1MkJ6aTJOL+mUASzkPZulCvm/7bcIIcJwIRiSP9vqFgrVSW53draCLIv05dSOSVWZuZu+kIjSsX54NejalRqlD+KgCmVD6V4zuLjTGBxpigdP4E2pEEVP5Wsqgf4/o3YWy/MA6evEDnj5fx5cpoUlJz1itVSjmHTrurPNelUUXmD7+V1jXL8O+52xg0eQ1n45PsDkspj6WJQNmibKA/EwY05e17G/LXnhN0G7ec3cfj7A5LKY+kiUDZRkTo16IK3z1yM+cTUrhn3Er+3HrM7rCU8jiaCJTtwkNKMueJCKqVKcoj30TyyYJd5PRqNqXUjdNEoNxChWKFmf5YS+4JrcQHf+zk8Snrsi5trZRyCk0Eym34+3rz3/sa83LnuszfcpQen60k+lS83WEpVeBpIlBuRUR4uHU1vn6wOUfOXuLusctZsfuE3WEpVaBpIlBuqXXNMvw8NIIyAYUY8NXfTFqxT+cNlHIRTQTKbYWULspPQyNoX6csr8/ZyvMzo7iUlGJ3WEoVOJoIlFsLKOTD5//XlKfa12Tm2hj6jF/FsXOX7A5LqQJFE4Fye15ewvA7avH5/zVl57E47v5kOesOnrY7LKUKDE0EKt/o0KA8Pz0egb+vN32+WMX0yGi7Q1KqQNBEoPKV2uUDmT0sguZVSzJiZhSjZm8hKSXV7rCUytc0Eah8p3gRPyYPasZDt1Rl8sr9DJj4N6cuJNodllL5liYClS/5eHvxSpd6fNCrMWsPnqbr2OVsO3LO7rCUypc0Eah8rUfTYGY81pLkFEP3T1cyN+qI3SEple9oIlD5XuPKxZn9RAR1KwQydOo6Rs/fQaoudqNUtmkiUAVC2UB/vnv0ZnqHV2bsot088r9Izl3SxW6Uyg5NBKrAKOTjzTs9GvJGt/os2RnLveNWsDf2vN1hKeX2NBGoAkVEGNAyhG8easHp+CS6jVvB4h3H7Q5LKbemiUAVSC2rl2L2sAgqlyjCoMlr+HzJHi1ap1QGNBGoAiu4RBFmDmlJp4YVeGfedp6atoGLiVq0TqnruTQRiEgHEdkhIrtFZGQ6rw8WkU0iskFElotIPVfGozxPET8fxvYN4/m7ajMn6jA9P1/JoTMX7Q5LKbfiskQgIt7AOKAjUA/om84H/VRjTENjTCjwHvBfV8WjPJeIMLRtDSY+EM7Bk/F0/WQ5q/eetDsspdyGK3sEzYHdxpi9xphEYBrQLe0Oxpi0t4IWBXQQV7lMuzrlmDUsgmJFfOn/5Wq+XXXA7pCUcguuTASVgLTlIWMc264hIkNFZA9Wj+DJ9BoSkUdFJFJEImNjY10SrPIM1csEMGtoBLfWKsPLszbz4o+bSEzWonXKs9k+WWyMGWeMqQ68ALycwT7jjTHhxpjwMmXK5G2AqsAJ8vdlwoBwHm9Tne/+Pki/CauIjUuwOyylbOPKRHAIqJzmebBjW0amAfe4MB6lrvD2EkZ0qMMnfcPYfPgsXccuZ+baGBKS9aoi5XlcmQjWADVFpKqI+AF9gNlpdxCRmmmedgZ2uTAepf7h7sYV+WFIK4oV9uW5GRuJeGchH/6xk+Nxuhym8hziyptsRKQTMAbwBr4yxrwlIm8AkcaY2SLyEXA7kAScBoYZY7Zk1mZ4eLiJjIx0WczKMxljWLH7JJNW7GPB9uP4eXvRpXEFHoyoSoNKxewOT6lcE5G1xpjwdF/Lb3dbaiJQrrbvxAW+XrmfGZHRXEhMoXlISQZFhHBHvXL4eNs+raZUjmgiUCoHzl1KYvqaaCav3E/M6YtUKl6YB1rdRO9mVShW2Nfu8JS6IZoIlMqFlFTDn9uO8dXyfazed4oift70aBLMwIgQqpcJsDs8pbJFE4FSTrLl8FkmrdjP7A2HSUxJpU3tMjwYUZXWNUsjInaHp1SGNBEo5WSxcQlMXX2Qb1cfIDYugRplAxgUEUL3sGAK+3nbHZ5S/6CJQCkXSUhOYW7UEb5asY/Nh85RrLAvfZtXYUDLm6hYvLDd4Sl1hSYCpVzMGEPkgdNMWrGP3zYfRUTo0KA8D0aE0KRKCR02UrbLLBH45HUwShVEIkKzkJI0CylJzOl4vvnrAN/9fZC5UUdoHFyMQRFV6dSwAn4+evmpcj/aI1DKReITk/lh3SEmrdjH3tgLlA0sxP0330S/FlUoFVDI7vCUh9GhIaVslJpqWLorlq9W7Gfpzlj8fLy4J7QigyKqUrdCkN3hKQ+hQ0NK2cjLS2hTuyxtapdl9/E4Jq3Yz4/rDjE9MoaW1UoxKCKE9nXL4e2l8wjKHtojUMoGZ+ITmbYmmv+t3M/hs5eoUrIIA1uF0Cs8mEB/vWtZOZ8ODSnlppJTUpm/5RiTVuwj8sBpAgv58Mit1XjolqoULaQdduU8mgiUygeiYs4wduFuft96jNIBhXiyfQ36NKuiVxopp9BEoFQ+su7gad6dt53V+05RpWQRnr2zFnc3qoiXziGoXMgsEehXDaXcTJMqJZj26M1MGtSMooV8eGraBu4eu5wlO2PJb1/cVP6giUApNyQitK1dlrlP3MKY3qGcu5TEA1/9Td8Jq1h/8LTd4akCRhOBUm7My0u4J6wSC55pw+td67P7+Hnu/XQlg79Zy+7j5+0OTxUQOkegVD5yPiGZicv2MX7pHi4mpXBfeGWeur0mFYppgTuVOZ0sVqqAOXk+gbGLdvPtqgN4iTCwVQhD2lSneBE/u0NTbkoTgVIFVPSpeD78Yyc/bThEYCEfBrepzqBWVXVNBPUPmgiUKuC2HTnH6Pk7WLD9OOWCCvFU+1r0Cg/G11unAZVFLx9VqoCrWyGIiQObMf2xlgSXKMJLP23irg+XMjfqiF5yqrKkiUCpAqR51ZLMHNySCQPC8fEWhk5dR7dxK1ix+4TdoSk3polAqQJGRLijXjnmPXUr7/dsxIm4BPp/uZr7J65mU8xZu8NTbkjnCJQq4C4lpfDtqgOMW7Sb0/FJdG5UgefurE3V0kXtDk3lIZ0sVkpx7lISE5bu5ctl+0hKSaV3s8o81b4mZYP87Q5N5QHbJotFpIOI7BCR3SIyMp3XnxGRrSISJSILROQmV8ajlCcL8vfl2Ttrs2REG/q1qML3a6K57f3FvD9/O2cvJtkdnrKRy3oEIuIN7ATuAGKANUBfY8zWNPu0BVYbY+JFZAjQxhjTO7N2tUeglHMcOHmBD37fyeyNhylexJfH21RnQMsQ/H31HoSCyK4eQXNgtzFmrzEmEZgGdEu7gzFmkTEm3vF0FRDswniUUmncVKooH/cN45cnbqFRcHHe/nU7bUcvZvqaaJJTUu0OT+UhVyaCSkB0mucxjm0ZeQiYl94LIvKoiESKSGRsbKwTQ1RKNahUjP892Jypj7SgbJA/I36I4vb/LuHTxbs5du6S3eGpPOAWl4+KyP8B4cD76b1ujBlvjAk3xoSXKVMmb4NTykO0ql6aWY+34rP+TSgTWIj3fttBy/8sYOCkv5kbdYSE5BS7Q1Qu4spFUQ8BldM8D3Zsu4aI3A78C7jNGJPgwniUUlkQETo2rEDHhhXYd+ICM9dG8+O6Qwyduo5ihX3pFlqRXk0r06BSECK6YlpB4crJYh+syeL2WAlgDdDPGLMlzT5hwEyggzFmV3ba1clipfJWSqphxe4TzFgbw/wtR0lMTqVO+UB6Ng3m3rBKlAooZHeIKhtsu49ARDoBYwBv4CtjzFsi8gYQaYyZLSJ/Ag2BI45DDhpjumbWpiYCpexzNj6J2VGHmRkZzcaYs/h4Ce3qlKVXeGXa1C6jRe7cmN5QppRyup3H4pi5NoYf1x3ixPkESgf4cW9YJXo2rUzt8oF2h6euo4lAKeUySSmpLNkRy4y10SzYdpzkVEOj4GL0ahpM18aVKFbE1+4QFZoIlFJ55OT5BGZtOMyMyGi2H43Dz8eLO+uVo1d4ZW6pURpvL51gtosmAqVUnjLGsOXwOWZERvPzxsOciU+ifJA/3ZtUold4ZS14ZwNNBEop2yQkp7Bg23FmREazZGcsqQbCbypBr/BgOjeqSEAhV17Fri7TRKCUcgvHzl3ix3WHmLE2mr2xFyjs603HhuXp1bQyLaqWxEuHjlxGE4FSyq0YY1h38Awz10YzZ+MRzickU7lkYXo2qUyPppUILlHE7hALHE0ESim3dTExhd+2HGFGZAwr95xEBFpVL0XPpsF0qF+Bwn5aDdUZNBEopfKFmNPx/LD2EDPXRRN96iJF/LxpV6csXRpVoE3tsloiOxc0ESil8pXUVMPqfaeYE3WY3zYf5dSFRE0KuaSJQCmVbyWnpLJ63ynmbjqiSSEXNBEopQoETQo5p4lAKVXgaFK4MZoIlFIFmiaFrGkiUEp5DE0K6dNEoJTySJoUrtJEoJTyeJ6eFDQRKKVUGnYmBWMMCcmpnLuURNylZMefJM47Hv9je8LVx0PaVKdDgwo5Om9miUDL/imlPI6PtxcRNUoTUaM0b3Stf01S+CXqSIZJwRjDhcSUKx/c5xwf0Gk/uOMuJXM+Ie0H+tXXrQ/1JJJSsv4CHlDIh4BCPgT6W3+KF/GjkI9rkpP2CJRSyiGjnkKJIn5Xvp2nZvGR6SU4PsB9CfT3Icjxd6C/DwH+V7cH+vsSdHl7oav7BPr7ElDIx+mL+OjQkFJK3aDLSWH+lqOcT0i+5gP98of1tR/o1utF/LwRcb9y2jo0pJRSNyjt8FFB52V3AEoppeyliUAppTycJgKllPJwmgiUUsrDaSJQSikPp4lAKaU8nCYCpZTycJoIlFLKw+W7O4tFJBY4YHccuVQaOGF3EG5E34+r9L24lr4f18rN+3GTMaZMei/ku0RQEIhIZEa3ensifT+u0vfiWvp+XMtV74cODSmllIfTRKCUUh5OE4E9xtsdgJvR9+MqfS+upe/HtVzyfugcgVJKeTjtESillIfTRKCUUh5OE0EeEpHKIrJIRLaKyBYRecrumOwmIt4isl5EfrE7FruJSHERmSki20Vkm4i0tDsmO4nIcMf/k80i8p2I+NsdU14Rka9E5LiIbE6zraSI/CEiuxx/l3DW+TQR5K1k4FljTD3gZmCoiNSzOSa7PQVsszsIN/ER8Jsxpg7QGA9+X0SkEvAkEG6MaQB4A33sjSpPTQY6XLdtJLDAGFMTWOB47hSaCPKQMeaIMWad43Ec1n/0SvZGZR8RCQY6A1/aHYvdRKQYcCswEcAYk2iMOWNvVLbzAQqLiA9QBDhsczx5xhizFDh13eZuwNeOx18D9zjrfJoIbCIiIUAYsNreSGw1BhgBpNodiBuoCsQCkxxDZV+KSFG7g7KLMeYQMBo4CBwBzhpjfrc3KtuVM8YccTw+CpRzVsOaCGwgIgHAD8DTxphzdsdjBxHpAhw3xqy1OxY34QM0AT4zxoQBF3Bi1z+/cYx/d8NKkBWBoiLyf/ZG5T6Mdd2/067910SQx0TEFysJTDHG/Gh3PDaKALqKyH5gGtBORL61NyRbxQAxxpjLPcSZWInBU90O7DPGxBpjkoAfgVY2x2S3YyJSAcDx93FnNayJIA+JiGCNAW8zxvzX7njsZIx50RgTbIwJwZoEXGiM8dhvfMaYo0C0iNR2bGoPbLUxJLsdBG4WkSKO/zft8eDJc4fZwAOOxw8APzurYU0EeSsCuB/r2+8Gx59Odgel3MYTwBQRiQJCgbdtjsc2jp7RTGAdsAnrs8pjyk2IyHfAX0BtEYkRkYeAd4A7RGQXVo/pHaedT0tMKKWUZ9MegVJKeThNBEop5eE0ESillIfTRKCUUh5OE4FSSnk4TQRKOYhISprLejeIiNPu7BWRkLSVJJVyJz52B6CUG7lojAm1Owil8pr2CJTKgojsF5H3RGSTiPwtIjUc20NEZKGIRInIAhGp4theTkR+EpGNjj+XSyN4i8gER43930WksGP/Jx1rVESJyDSbfkzlwTQRKHVV4euGhnqnee2sMaYhMBarairAJ8DXxphGwBTgY8f2j4ElxpjGWPWCtji21wTGGWPqA2eAHo7tI4EwRzuDXfXDKZURvbNYKQcROW+MCUhn+36gnTFmr6No4FFjTCkROQFUMMYkObYfMcaUFpFYINgYk5CmjRDgD8eiIojIC4CvMebfIvIbcB6YBcwyxpx38Y+q1DW0R6BU9pgMHt+IhDSPU7g6R9cZGIfVe1jjWIhFqTyjiUCp7Omd5u+/HI9XcnX5xP7AMsfjBcAQuLImc7GMGhURL6CyMWYR8AJQDPhHr0QpV9JvHkpdVVhENqR5/psx5vIlpCUcVUETgL6ObU9grSj2PNbqYoMc258CxjsqRqZgJYUjpM8b+NaRLAT4WJeoVHlN5wiUyoJjjiDcGHPC7liUcgUdGlJKKQ+nPQKllPJw2iNQSikPp4lAKaU8nCYCpZTycJoIlFLKw2kiUEopD/f/mRLEb7xmgXcAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "goEmXPml7zmc",
        "outputId": "f42607cc-9241-43dd-c9cf-452b66de739c"
      },
      "source": [
        "plt.clf()   # clear figure\n",
        "acc = history.history['accuracy']\n",
        "val_acc = history.history['val_accuracy']\n",
        "\n",
        "plt.plot(epochs, acc, label='Training acc')\n",
        "plt.plot(epochs, val_acc, label='Validation acc')\n",
        "plt.title('Training and validation accuracy')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xUZfb48c9JBwKht4TeaygBVCwoVUAQLAs2sCz2guvuqusqoq66PyzrV11FAbEt6q4iVZoidgjSA0gxQBJKSEiBkH5+f9wbGEKAAJlMynm/XvPKzK1nbpI585T7PKKqGGOMMYX5+ToAY4wxZZMlCGOMMUWyBGGMMaZIliCMMcYUyRKEMcaYIlmCMMYYUyRLEKbYRGShiIwr6W19SURiRWSAF46rItLaff6WiPy9ONuew3luFJHF5xqnMacjdh9ExSYihz1eVgWygDz39Z2q+lHpR1V2iEgscIeqLi3h4yrQRlW3l9S2ItIc+B0IVNXckojTmNMJ8HUAxrtUNbTg+ek+DEUkwD50TFlhf49lg1UxVVIi0k9E4kTkryKyD5ghIrVEZJ6IJIrIIfd5hMc+y0XkDvf5eBH5XkSmuNv+LiJXnuO2LURkhYiki8hSEXlDRD48RdzFifEZEfnBPd5iEanrsf5mEdklIkki8rfTXJ8+IrJPRPw9lo0SkfXu894i8pOIpIjIXhF5XUSCTnGs90TkWY/Xf3b3SRCR2wptO0xE1ohImojsEZFJHqtXuD9TROSwiFxYcG099r9IRFaJSKr786LiXpuzvM61RWSG+x4Oichsj3UjRWSt+x52iMgQd/kJ1XkiMqng9ywizd2qtttFZDfwtbv8M/f3kOr+jXTy2L+KiLzk/j5T3b+xKiIyX0TuL/R+1ovIqKLeqzk1SxCVW0OgNtAMmIDz9zDDfd0UOAq8fpr9+wBbgbrAP4FpIiLnsO3HwEqgDjAJuPk05yxOjDcAtwL1gSDgEQAR6Qj82z1+Y/d8ERRBVX8BjgBXFDrux+7zPGCi+34uBPoD95wmbtwYhrjxDATaAIXbP44AtwA1gWHA3SJytbvuUvdnTVUNVdWfCh27NjAfeM19by8D80WkTqH3cNK1KcKZrvMHOFWWndxjveLG0Bt4H/iz+x4uBWJPdT2KcBnQARjsvl6Ic53qA78CnlWiU4CewEU4f8d/AfKBmcBNBRuJSCQQjnNtzNlQVXtUkgfOP+oA93k/IBsIOc323YBDHq+X41RRAYwHtnusqwoo0PBstsX58MkFqnqs/xD4sJjvqagYn/B4fQ/wlfv8SWCWx7pq7jUYcIpjPwtMd59Xx/nwbnaKbR8CvvB4rUBr9/l7wLPu8+nACx7btfXctojjvgq84j5v7m4b4LF+PPC9+/xmYGWh/X8Cxp/p2pzNdQYa4XwQ1ypiu7cL4j3d35/7elLB79njvbU8TQw13W3CcBLYUSCyiO1CgEM47TrgJJI3S/v/rSI8rARRuSWqambBCxGpKiJvu0X2NJwqjZqe1SyF7Ct4oqoZ7tPQs9y2MZDssQxgz6kCLmaM+zyeZ3jE1Njz2Kp6BEg61blwSgujRSQYGA38qqq73DjautUu+9w4/oFTmjiTE2IAdhV6f31E5Bu3aicVuKuYxy049q5Cy3bhfHsucKprc4IzXOcmOL+zQ0Xs2gTYUcx4i3Ls2oiIv4i84FZTpXG8JFLXfYQUdS73b/oT4CYR8QPG4pR4zFmyBFG5Fe7C9iegHdBHVWtwvErjVNVGJWEvUFtEqnosa3Ka7c8nxr2ex3bPWedUG6tqDM4H7JWcWL0ETlXVFpxvqTWAx88lBpwSlKePgTlAE1UNA97yOO6Zuhwm4FQJeWoKxBcjrsJOd5334PzOahax3x6g1SmOeQSn9FigYRHbeL7HG4CRONVwYTiljIIYDgKZpznXTOBGnKq/DC1UHWeKxxKE8VQdp9ie4tZnP+XtE7rfyKOBSSISJCIXAld5Kcb/AsNF5GK3QXkyZ/4f+Bh4EOcD8rNCcaQBh0WkPXB3MWP4FBgvIh3dBFU4/uo4384z3fr8GzzWJeJU7bQ8xbEXAG1F5AYRCRCRPwAdgXnFjK1wHEVeZ1Xdi9M28KbbmB0oIgUJZBpwq4j0FxE/EQl3rw/AWmCMu30UcG0xYsjCKeVVxSmlFcSQj1Nd97KINHZLGxe6pT3chJAPvISVHs6ZJQjj6VWgCs63s5+Br0rpvDfiNPQm4dT7f4LzwVCUc45RVTcB9+J86O/FqaeOO8Nu/8FpOP1aVQ96LH8E58M7HXjHjbk4MSx038PXwHb3p6d7gMkiko7TZvKpx74ZwHPAD+L0nrqg0LGTgOE43/6TcBpthxeKu7jOdJ1vBnJwSlEHcNpgUNWVOI3grwCpwLccL9X8Hecb/yHgaU4skRXlfZwSXDwQ48bh6RFgA7AKSAZe5MTPtPeBLjhtWuYc2I1ypswRkU+ALarq9RKMqbhE5BZggqpe7OtYyisrQRifE5FeItLKrZIYglPvPPtM+xlzKm713T3AVF/HUp5ZgjBlQUOcLpiHcfrw362qa3wakSm3RGQwTnvNfs5cjWVOw6qYjDHGFMlKEMYYY4pUYQbrq1u3rjZv3tzXYRhjTLmyevXqg6par6h1FSZBNG/enOjoaF+HYYwx5YqIFL77/hirYjLGGFMkSxDGGGOKZAnCGGNMkSpMG0RRcnJyiIuLIzMz88wbG58ICQkhIiKCwMBAX4dijCmkQieIuLg4qlevTvPmzTn1PDbGV1SVpKQk4uLiaNGiha/DMcYUUqGrmDIzM6lTp44lhzJKRKhTp46V8Iwpoyp0ggAsOZRx9vsxpuyq0FVMxhhT0nLz8tl24DAb4lOJP3SUoAA/gvz9CArwI9D9WbAsOODE1wXbBBdaFhTgR4CflLkvTJYgvCgpKYn+/fsDsG/fPvz9/alXz7lhceXKlQQFBZ1y3+joaN5//31ee+21057joosu4scffyy5oI0xx+TlKzsSD7MhLpUN8amsj0shZm8amTn5JX4uEZyE4ZlkCiWRY88LLWtRrxr39Gtd4jF5NUG4Qzf/C/AH3lXVFwqtb4YzK1Q9nAk/blLVOHddHs5kIAC7VXWEN2P1hjp16rB27VoAJk2aRGhoKI888six9bm5uQQEFP0riIqKIioq6oznsORgTMnIz1d2HjzChvgUNsSlsSE+hU0JaWRk5wFQNcifzo3DuLFPM7qEh9ElIozmdaqRm59Pdq77yDv+PCs3n5y8k5dn5znrsj3Xe2yT5fG88Pqs3HzSM3OPvS5Yf/BwFvQr+WvitQThTm7+BjAQZ9auVSIyx53nt8AU4H1VnSkiVwDP48xUBXBUVbt5Kz5fGT9+PCEhIaxZs4a+ffsyZswYHnzwQTIzM6lSpQozZsygXbt2LF++nClTpjBv3jwmTZrE7t272blzJ7t37+ahhx7igQceACA0NJTDhw+zfPlyJk2aRN26ddm4cSM9e/bkww8/RERYsGABDz/8MNWqVaNv377s3LmTefNOnIUyNjaWm2++mSNHjgDw+uuvc9FFFwHw4osv8uGHH+Ln58eVV17JCy+8wPbt27nrrrtITEzE39+fzz77jFatTjU9sDFlS36+sis5g/VxKcdKB5sS0jiclQtASKAfnRqHcX1UE7pGhNElPIyW9ULx9zu5Csjfz5/gAP/SfgulwpsliN7AdlXdCSAis3AmgvFMEB2Bh93n3+DFSWKenruJmIS0Ej1mx8Y1eOqqTme9X1xcHD/++CP+/v6kpaXx3XffERAQwNKlS3n88cf53//+d9I+W7Zs4ZtvviE9PZ127dpx9913n3TvwJo1a9i0aRONGzemb9++/PDDD0RFRXHnnXeyYsUKWrRowdixY4uMqX79+ixZsoSQkBC2bdvG2LFjiY6OZuHChXz55Zf88ssvVK1aleTkZABuvPFGHn30UUaNGkVmZib5+SVf5DamJKgqe5KPsj7+eDLYEJ9KeqaTDIIC/OjYqAaje4QfKxm0rhdKgH+F78NzRt5MEOHAHo/XcUCfQtusA0bjVEONAqqLSB13bt0QEYkGcoEXVPWk5CEiE4AJAE2bNi35d+Al1113Hf7+zjeO1NRUxo0bx7Zt2xARcnJyitxn2LBhBAcHExwcTP369dm/fz8REREnbNO7d+9jy7p160ZsbCyhoaG0bNny2H0GY8eOZerUkyfZysnJ4b777mPt2rX4+/vz22+/AbB06VJuvfVWqlatCkDt2rVJT08nPj6eUaNGAc7NbsaUBapKfMpRNsSlsj4+9VhCSD3q/F8F+gsdGtVgRGRjukaE0Tk8jLYNqhNoyaBIvm6kfgR4XUTGAytwJifPc9c1U9V4EWkJfC0iG1R1h+fOqjoVd0rBqKio0858dC7f9L2lWrVqx57//e9/5/LLL+eLL74gNjaWfv36FblPcHDwsef+/v7k5uae0zan8sorr9CgQQPWrVtHfn6+feibMk9V2ZeWyfq4VI+EkMKhDCcZBPgJ7RpWZ2iXhnQJr0mX8DDaNgytsNVB3uDNBBEPNPF4HeEuO0ZVE3BKEIhIKHCNqqa46+LdnztFZDnQHTghQVQEqamphIeHA/Dee++V+PHbtWvHzp07iY2NpXnz5nzyySenjCMiIgI/Pz9mzpxJXp6TpwcOHMjkyZO58cYbj1Ux1a5dm4iICGbPns3VV19NVlYWeXl5x0oZxpQ0VWV/WpZTPRSXcqya6ODhbAD8/YQ29UMZ2LEBXSJq0jU8jHYNqxMSaMngfHgzQawC2ohIC5zEMAa4wXMDEakLJKtqPvAYTo8mRKQWkKGqWe42fYF/ejFWn/nLX/7CuHHjePbZZxk2bFiJH79KlSq8+eabDBkyhGrVqtGrV68it7vnnnu45ppreP/9949tCzBkyBDWrl1LVFQUQUFBDB06lH/84x988MEH3HnnnTz55JMEBgby2Wef0bJlyxKP31Q+RSeDNKenDuAn0Lp+KJe1re80IEeE0bFRDUsGXuDVOalFZCjwKk431+mq+pyITAaiVXWOiFyL03NJcaqY7nWTwkXA20A+zt3er6rqtNOdKyoqSgtPGLR582Y6dOhQ4u+rvDl8+DChoaGoKvfeey9t2rRh4sSJvg7rGPs9VV7FTQadw52eRF3Cw+jYuAZVg3xdO15xiMhqVS2yT71Xr7KqLgAWFFr2pMfz/wL/LWK/H4Eu3oytMnnnnXeYOXMm2dnZdO/enTvvvNPXIZlK6IRkcCwhnJwMLm1b15JBGWFXvhKYOHFimSoxmIrPkkHFYL8NY8x5KZwMNsansj4u1ZJBBWC/IWNMsVkyqFzst2aMOS1VZX1cKl+sieerjfvYl+bM32HJoOKz36QxpkixB48we208X65N4PeDRwgK8OOKdvXp07K2JYNKwu4v96LLL7+cRYsWnbDs1Vdf5e677z7lPv369aOgu+7QoUNJSUk5aZtJkyYxZcqU05579uzZxMQcH/bqySefZOnSpWcTvqmEDh7O4r0ffufqN36g35Tl/GvZNhrWCOHFa7qw6m8DeOvmntzatwVRzWtbcqgE7DfsRWPHjmXWrFkMHjz42LJZs2bxz38W756/BQsWnHmjU5g9ezbDhw+nY8eOAEyePPmcj2UqtozsXBZv2s/stfF8t+0geflKh0Y1eHxoe66KbEyjsCq+DtH4iJUgvOjaa69l/vz5ZGc7wwHExsaSkJDAJZdcwt13301UVBSdOnXiqaeeKnL/5s2bc/DgQQCee+452rZty8UXX8zWrVuPbfPOO+/Qq1cvIiMjueaaa8jIyODHH39kzpw5/PnPf6Zbt27s2LGD8ePH89//OrecLFu2jO7du9OlSxduu+02srKyjp3vqaeeokePHnTp0oUtW7acFFNsbCyXXHIJPXr0oEePHifMR/Hiiy/SpUsXIiMjefTRRwHYvn07AwYMIDIykh49erBjR4UbLaVcys3L55utB3ho1hp6PrOUhz5Zy7b9h7nz0pYsnngpCx+8hAmXtrLkUMlVnhLEwkdh34Yzb3c2GnaBK1845eratWvTu3dvFi5cyMiRI5k1axbXX389IsJzzz1H7dq1ycvLo3///qxfv56uXbsWeZzVq1cza9Ys1q5dS25uLj169KBnz54AjB49mj/+8Y8APPHEE0ybNo3777+fESNGMHz4cK699toTjpWZmcn48eNZtmwZbdu25ZZbbuHf//43Dz30EAB169bl119/5c0332TKlCm8++67J+xvw4KXX6rK2j0pfLk2gbnrEkg6kk1YlUBG9Qjn6m7hRDWrhV8R8x2YyqvyJAgfKahmKkgQ06Y5I4Z8+umnTJ06ldzcXPbu3UtMTMwpE8R3333HqFGjjg2GN2LE8cn1Nm7cyBNPPEFKSgqHDx8+oTqrKFu3bqVFixa0bdsWgHHjxvHGG28cSxCjR48GoGfPnnz++ecn7W/Dgpc/OxMPM3ttAnPWxhOblEFQgB8DOzRgZLfGXNauno1uak6p8iSI03zT96aRI0cyceJEfv31VzIyMujZsye///47U6ZMYdWqVdSqVYvx48eTmZl5TscfP348s2fPJjIykvfee4/ly5efV7wFQ4afarhwGxa8fEhMz2Le+gRmr4lnXVwqInBRqzrcc3lrhnRuSI2QwDMfxFR61gbhZaGhoVx++eXcdtttx2ZzS0tLo1q1aoSFhbF//34WLlx42mNceumlzJ49m6NHj5Kens7cuXOPrUtPT6dRo0bk5OTw0UcfHVtevXp10tPTTzpWu3btiI2NZfv27QB88MEHXHbZZcV+P6mpqTRq1Ag/Pz8++OCDE4YFnzFjBhkZGQAkJydTvXr1Y8OCA2RlZR1bb0rekaxcvlgTxy3TV3LB88t4em4MufnK34Z24KdH+/PRHRdwfVQTSw6m2CpPCcKHxo4dy6hRo5g1axYAkZGRdO/enfbt29OkSRP69u172v179OjBH/7wByIjI6lfv/4JQ3Y/88wz9OnTh3r16tGnT59jSWHMmDH88Y9/5LXXXjvWOA1ONc+MGTO47rrryM3NpVevXtx1113Ffi82LHjZkpOXz3fbEpm9JoElMfs5mpNHRK0q3HVZS67uFk6bBtV9HaIpx7w63HdpsuG+yy/7PZ0dVWXNnhRmr4ln3vq9JB/JpmbVQIZ1acSo7uH0bFYLEWtsNsXjs+G+jTElZ0fiYb5cE8+X6xLYlZRBcIAfAzo2YFS3cC5tW4+gAKsxNiXLqwlCRIYA/8KZMOhdVX2h0PpmOLPI1QOSgZtUNc5dNw54wt30WVWd6c1YjSlLVJXYpAzW7D7Emt0prIpNZsu+dPwE+rauy/1XtGFwpwZUt/YE40VeSxAi4g+8AQwE4oBVIjJHVWM8NpsCvK+qM0XkCpzZ5W4WkdrAU0AUzmxzq919D51tHKpqxe0yrKJUcZ6vtMwc1u1JYc3uFNbsPsTaPSkcysgBIDQ4gK4RYTwxrAMjIhtTv4b1HDOlw5sliN7AdlXdCSAis4CRgGeC6Ag87D7/BpjtPh8MLFHVZHffJcAQ4D9nE0BISAhJSUnUqVPHkkQZpKokJSVVuq6yefnK9gOHWbP7EL+6JYTtiYdRBRFoUz+UQR0b0r1pTbo3rUXr+qH42w1sxge8mSDCgT0er+OAPoW2WQeMxqmGGgVUF5E6p9g3/GwDiIiIIC4ujsTExLPd1ZSSkJAQIiIifB2GVyUdzmJtQelgzyHW7UnlcJZzj0mtqoF0b1qLEZGN6d60Fl2bhFk3VFNm+LqR+hHgdREZD6wA4oG84u4sIhOACQBNmzY9aX1gYCAtWrQokUCNKY7s3Hy27Es7VlW0Zk8Ku5Kcez/8/YQOjaozuke4UzpoUotmdapa6daUWd5MEPFAE4/XEe6yY1Q1AacEgYiEAteoaoqIxAP9Cu27vPAJVHUqMBWcbq4lGLsxxbI39ejxZLA7hQ3xqWTlOuNN1a8eTI+mtbihd1O6N61Fl/AwqgTZsBam/PBmglgFtBGRFjiJYQxwg+cGIlIXSFbVfOAxnB5NAIuAf4hILff1IHe9MT6TmZPHhvjUY8lgze6UY7OrBQX40SU8jJsvaEb3prXo3rQmjcJCrHRgyjWvJQhVzRWR+3A+7P2B6aq6SUQmA9GqOgenlPC8iChOFdO97r7JIvIMTpIBmFzQYG1MaVFVVmw7yLLN+1mzO4XNe9PIzXcKqk1rV6VPy9p0b+I0JHdoVMPuQzAVToW+k9qYc7X9QDqT521mxW+JVAvyJ7JJzWPtBt2a1qRuaLCvQzSmRNid1MYUU+rRHP61dBvv/xRLlSB/nhzekZsvbEagv5UOTOVjCcIYnHsTZq3azUuLf+NQRjZjezflTwPbUsdKCqYSswRhKr2fdybx9NwYNu9No3eL2jx1VUc6NQ7zdVjG+JwlCFNpxR3K4PkFW5i/YS/hNavwxg09GNqlofU8MsZlCcJUOhnZubz17U7e/nYHIvDwwLZMuLQlIYF2j4IxnixBmEpDVZmzLoEXFm5hb2omIyIb8+iV7Wlcs4qvQzOmTLIEYSqFDXGpPD13E9G7DtE5vAavje1Or+a1fR2WMWWaJQhToSWmZzFl0VY+Xb2HOtWCePGaLlzbs4mNjmpMMViCMBVSdm4+M3+M5bVl2ziak8cdF7fg/v5tbKRUY86CJQhT4Xy9ZT/PzNvM7wePcEX7+vxtWAda1Qv1dVjGlDuWIEyFsf3AYZ6dH8PyrYm0rFeNGbf24vJ29X0dljHlliUIU+6lHs3htWXbmPljLFUC/XliWAduubC5DZ5nzHmyBGHKrbx85dPoPUxZtJXkjGzG9GrCnwa1s4H0jCkhliBMufSLOzxGzN40ejevzcyrOtI53IbHMF6UmwVZ6ZCV5v5Mh8Aq0Kg7+FXM0qolCFOuxKcc5R8LNjN//V4ah4Xwf2O7M7xrIxsewxRNFXIzj3+ge364Z6VDZtrJy4rcNg3ysos+R40I6HQ1dBoN4T2gAv0tWoIw5cLR7Dze+nYHb7nDYzw0oA13XtrKpvCsbLKPQMIa2B8Dmaln+HB31+Xnnvm4/sEQXN3jUQNqhBdaVh1Cwk58nbYXNn0Bv7wNP70ONZtBp1HOo1FkuU8WXk0QIjIE+BfOjHLvquoLhdY3BWYCNd1tHlXVBSLSHNgMbHU3/VlV7/JmrKZsUlXmrt/LCws2k5CayfCujXhsaAfCbXiMii8/H5K2Q9wq9xENB2JA845vE1Dl5A/xmk09PtBrnPihX3jbgmUB59FuFfkHOJoCW+Y7yeKn1+GHV6F2KydRdB4N9TuWy2ThtRnlRMQf+A0YCMThTB86VlVjPLaZCqxR1X+LSEdggao2dxPEPFXtXNzz2YxyFc/GeGd4jFWxh+jYqAaTRnSidwsbHqPCykh2kkB8tJsQVkNWqrMuOAwiekJ4FET0cr6dV60N/mXwxseMZNg8FzZ9Dr+vAM2Huu2cRNFpFNRr5+sIT+CrGeV6A9tVdacbxCxgJBDjsY0CNdznYUCCF+Mx5cChI9ks3byfRZv2s2zLfmpVDeL50V24PsqGx6hQ8nJg/0YnIRSUDpJ3OOvED+p3cj5QI3pBRBTUaVN+GoKr1oae45zH4UTY/CVsmg3LX4Dlz7vvbZTTZlGnla+jPS1vliCuBYao6h3u65uBPqp6n8c2jYDFQC2gGjBAVVe7JYhNOCWQNOAJVf2uiHNMACYANG3atOeuXbu88l6Md8WnHGXxpn0s2rSPVbGHyMtXGoeFMKJbOHf3a0VYlTL4LdEUnyqkxR9PBHHRsHet03gMENrgeCKI6AWNukFwBbzzPX0fxHwJGz+HPT87yxpFOomi0yio1cwnYZ2uBOHrBPGwG8NLInIhMA3oDAQCoaqaJCI9gdlAJ1VNO9X5rIqp/FBVftt/2EkKMfvYGO/8Wts2CGVQx4YM7tSQztUOIWl7odmFPo7WnLXsI5Cw9njbQfxqSN/rrPMPhsbdnEQQ3tP5GRZRLuvnz0tqnFOq2PS5c33AuR4FySIsvNRC8VUVUzzQxON1hLvM0+3AEABV/UlEQoC6qnoAyHKXrxaRHUBbwDJAOZWfr6zZc4hFm/azeNM+YpMyAOjRtCaPXdmeQZ0a0qJuNWfjnEz492inyuGKv8Mlf6p8HyDlRUFD8rF2g1VOD6OChuRaLaD5JW4JoSc06AIBQb6NuSwIi4CL7nMeh2KPJ4vFf3MeTS5wqtg6joTqDX0WpjdLEAE4VUT9cRLDKuAGVd3ksc1C4BNVfU9EOgDLgHCgLpCsqnki0hL4DuiiqsmnOp+VIMqerNw8ftyRxOJN+1kSs5+Dh7MI9BcubFWXwZ0aMLBDA+rXCDl5x6VPw/cvOx8ssd9Bt5tg+Cv2wVIWZCQ733gLqovio53upuD0CArvebyqKLwnVKvr23jLm6QdTqLY+AUc2AQINL/Yuc+iw0gIrVfip/RJFZN74qHAqzhdWKer6nMiMhmIVtU5bs+ld4BQnAbrv6jqYhG5BpgM5AD5wFOqOvd057IEUTakZ+awfGsiizbtY/nWRA5n5VItyJ9+7eszqGMDLm9f//RDbu/bAG9fBpFjYOQbTsPety9Ai0vh+g+gSs3SezPGkZsFWxfAmo9gxzKnV474OV03jyWDKKjbtvw0JJcHiVud9opNn8PB35xr3uJSpxqqw1VOY3gJ8FmCKE2WIHwnMT2LJTH7WRyzjx+3J5Gdl0+dakEM7NiAwZ0acmGrOsWb7zkvF6YNcOpn7115/B9g3Sz48j6o3RJu/BRqNffq+zGuveucpLDhUzh6yLlxLHIMtLwcGnevmA3JZZGqc/9HQbJI3gl+Ac7vodMoaD/svL44WYIwJS724BEWx+xj0ab9/Lr7EKrQtHZVBndqwKBODenRtNbZd0v98f9g8RNw7Qyn/vWEE34Ps250/jHGzoImvUruzZjjjiQ5CWHNR7B/g9Oo3H4YdL/R+UDyszvXfUrVSdybPnduykvZDf5BTqIYPfWcDmkJwpw3VWVTQhqLNu1j8ab9bN2fDkCnxjWcnkedG9CuQfVzHxMp+Xd480Jo2Q/G/qfoRumD2+Cj65weMaPecv4pzPnLy3WqjtZ8CFsXQn6O09W0+03Q+ZoSq0gSBl0AACAASURBVMowJUwV4n91koX4waBnzukwvurFZMq53Lx8VsYmH2tkjk85ip9Ar+a1eXJ4RwZ2bECT2lXP/0SqMO8hp3Qw7KVT91iq2wbuWAazboDPxju9P/o+ZD2cztXBbU5SWDcLDu+DqnWg9wSntNCgk6+jM2ci4vQMi+jptVNYgjAnyM3L5+stB1gcs59lm/dzKCOH4AA/LmlTjwcHtGFAhwbUrlbCvYnWfgw7lzvJ4Uz9v6vVgVu+hC/vhaWTnF4fw18pm0MulEWZac43zjUfQdxKEH9oM8hJCm0GW08xcwJLEOaY3Lx87v/PGhZu3EeNkAD6d2jAoI4NuLRtPaoFe+lP5fABWPQ4NL0Qet5WvH0CQ+Cad51G6xX/dOphr3/fejidSn4+7PreSQoxX0LuUWdsoIHPQNc/QPUGvo7QlFGWIAzg3Mj22OcbWLhxH38d0p47LmlBoH8pdFlc+BfIyYCrXju7LpIicMXfoHYLmPMATB8MN3zqs+EKyqSU3U7pbO3HkLLLuU8hcozTthDe06rmzBlZgjCoKs/O38xnq+N4sH8b7u5XSgOIbVng9MS4/Amo1/bcjtHtBghrAp/cCO/2h7GfeLVOtszLzoAt85y2hd9XAAotLoMrnoD2wyGoBNqMTKVhCcLw2rLtTP/hd27t25yHBrQpnZNmpsL8PzkjW/Z98PyO1eISuH0pfHwdvDfU6e7XcWTJxFkeqDp3Na/90Okrn5XmzInQ7zHoNtZ5bsw5sARRyU3//ndeWfob1/aM4O/DOpbe1J1Ln3a6q/7hw5JpGK3X1unh9J+x8OktMHAyXPRAxa5GSd8P62c5bQsHtzqT53Qc6TQ4N7vY7mo2580SRCX2WfQeJs+LYUinhrwwugt+pTXfwq6fIHoaXHBPyVYHVasL4+bC7LthyZPOHadDp1SsHk652bBtkVOFtG2JMyheRG+46l/OEAwhNc58DGOKyRJEJfXVxr389X/ruaRNXf41thsBpdEgDc5IrXPud6o9rnii5I8fGALXTHMar797CQ7tgutnOnMJl2f7NzlJYf0nkJHkzKFw0X3OQIbn2n5jzBlYgqiEvtuWyAP/WUu3JjV5++aeBAeU4vAJ302BpG1w0+cQVM075/Dzg/5POt1g5z4I0wY7YziVt7r4nEynW2r0NNjzC/gFQrsrnV5IrfqDv/37Gu+yv7BKZvWuQ0x4fzUt61VjxvjeVA0qxT+BfRvh+1cgciy07u/983W/ye3hdDO80x9umOV07yzrknbA6hlO28LRZCfRDXoWIm9wbhQ0ppRYgqhENu9N49YZK2lQI5gPbu9DWNVSrJvPz3OqlkJqwuB/lN55W14GdyyBj66FGcPgmnecoZLLmrxc+G0hrJoGO79x7nBuPwyibnO6qVqDs/EBSxCVxO8Hj3DztJVUCw7gwzv6UK96cOkG8MtbkPCr0z5Q2oO/1WsHd3wNs8Y6pYlBz8CF95WNHk5pCbB6Jvw60+nVVb0x9HscetwCNRr5OjpTyVmCqAQSUo5y07u/kK/KB7dfQEStUr5Z6lAsfP2sM9ZP52tK99wFQus5PZy+uMsZUjxph9vDyQf/Avn5TikherozeqrmOW0Kw15yrpG1LZgywqt/iSIyBPgXzoxy76rqC4XWNwVmAjXdbR5V1QXuusdw5qzOAx5Q1UXejLWiSjqcxU3TfiHtaA7/mXABreuX8iQvqjBvojMc8elGai0NgVWcuSa+bulMaZqyG657r/S6hmYkOz2RVs9wuuBWreP0ROo53mlnMKaM8VqCEBF/4A1gIBAHrBKROaoa47HZE8Cnqvpvd/rRBUBz9/kYoBPQGFgqIm1VC2ZCN8WRlpnDLdNXEn/oKB/c3ofO4T7o6rluFuz42vm2XrNJ6Z+/MD8/GPCU0w123kSYPgRu+MR7sanCnpVOT6RNsyEvyxmYsN/j0HEEBJRyVZ8xZ8GbJYjewHZV3QkgIrOAkYBnglCg4OtbGJDgPh8JzFLVLOB3EdnuHu8nL8ZboRzNzuOO96LZui+dd8ZF0buFDyZ9OZwIix5zbuSKur30z386PW5xur1+coszhtMNnzjTaJaUrHTnnoXoGbB/IwRVd84ZdavNtWDKDW8miHBgj8frOKBPoW0mAYtF5H6gGjDAY9+fC+170kQBIjIBmADQtGk56+PuRdm5+dz90WpW7Urm/8Z25/J29X0TyFd/hewjMOL/ymYvnJb94PbFzhhOM4Y6Q4i3H3Z+x9y30SktrP8Usg9Dwy4w/FXocp3N4WzKHV//144F3lPVCGAo8IGIFDsmVZ2qqlGqGlWvXj2vBVme5OUrEz9dy/KtiTw/qgvDuzb2TSBbv4KN/4NLHoH67X0TQ3HUb++M4VS/gzPn9U9vONVCZyMn06lKe3cgvNXXGV67wwjnuHd+55QaLDmYcsibJYh4wLNiN8Jd5ul2YAiAqv4kIiFA3WLuawpRVf72xQbmr9/L40PbM6a3j0pVWekw/2Go1wEunuibGM5GaH0YNw++uNOZvCh5Jwx58cy9iQrf0FantXOPR+RYm8fZVAjeTBCrgDYi0gLnw30McEOhbXYD/YH3RKQDEAIkAnOAj0XkZZxG6jbASi/GWu6pKs8v3MKsVXu47/LWTLi0lOZ0KMrSp53+/be/V36msAyqCtfNhGVPww+vOmM4XTcDgqufuF3hG9r8Atwb2m6HFpeWjXsrjCkhXksQqporIvcBi3C6sE5X1U0iMhmIVtU5wJ+Ad0RkIk6D9XhVVWCTiHyK06CdC9xrPZhO783lO5i6Yie3XNiMPw3y4eBtu3+GVe9CnzuhSW/fxXEu/Pxg4NNuD6eHj/dwCos4+Ya2GhHOREc9bobqDX0duTFeIXq29a1lVFRUlEZHR/s6DJ94/6dYnvxyE6O6h/PSdZGlN2x3YblZ8NYlzhSi9/xcvuvdd3wNn46DwKrO+E2/fQWaD60HOMNftBlkN7SZCkFEVqtqVFHr7C+8nPtiTRxPfrmJAR0a8M9ru/ouOYAzvPbBrXDjf8t3cgBodQXctgg+/oMzkmrfB6DHOKd0YUwlYQmiHFsSs59HPlvPhS3r8PoN3QksrTkdirI/Br57GbpcD20G+i6OktSgI9wf7dwFXpEmHTKmmCxBlFM/bj/IvR//SufwMN4ZF0VIYCnO6VBYwUitwdVhyPO+i8Mb7E5nU4lZgiiH1uw+xB3vR9O8TlVm3tqL0GAf/xpXvgPx0TD6HWfaT2NMheDrG+XMWdq6L53xM1ZRN9SZ06FmVR93I03ZDcsmQ+uBzt3CxpgKwxJEObIr6Qg3T/uFkEA/PrqjDw1qhPg2IFWY+5DzfPjLdg+AMRWMVTGVE/tSM7lp2i/k5OXz6Z0X0qR2Kc/pUJQNn8GOZc5dx+VtvmdjzBkVqwQhItUKxkgSkbYiMkJErFtHKUk+ks3N034h+XA2793amzYNqp95J287chAW/hXCo6D3H30djTHGC4pbxbQCCBGRcGAxcDPwnreCMselZ+YwfsZKdiVn8O64XkQ2qenrkBxfPeaMuTTydfDzYQ8qY4zXFDdBiKpmAKOBN1X1OpzJfIwXZebkccfMaGIS0vj3jT24sFUdX4fk2LYENnwKlzzsjIJqjKmQip0gRORC4EZgvrvMvjZ6UU5ePvd+9CsrY5N56fpI+ndo4OuQHFnpTsN03XZwyZ98HY0xxouK20j9EPAY8IU74F5L4BvvhVW55eUrf/p0Hcu2HODZqzszsttJcyX5zrJnIC3eGYbCbiIzpkIrVoJQ1W+BbwHcxuqDqvqANwOrrFSVJ7/cyJx1CfxlSDtuuqCZr0M6bs9KWDkVet0BTQtPDmiMqWiK24vpYxGpISLVgI1AjIj82buhVU7/b9FWPvplN3dd1op7+rX2dTjH5WY7w2nUaAwDnvJ1NMaYUlDcNoiOqpoGXA0sBFrg9GQyJejfy3fw5vId3NCnKX8d0s7X4Zzo+5chcQsMe/nkSXSMMRVScRNEoHvfw9XAHFXNwZng57REZIiIbBWR7SLyaBHrXxGRte7jNxFJ8ViX57FuTnHfUHm1KjaZF7/awlWRjXlmZGekLN2VfGALrJgCna+BdkN8HY0xppQUt5H6bSAWWAesEJFmQNrpdhARf+ANYCAQB6wSkTmqGlOwjapO9Nj+fqC7xyGOqmq3YsZX7n3+axxVg/z55zVd8fflnA6FHRupNdS5Y9oYU2kUqwShqq+pariqDlXHLuDyM+zWG9iuqjtVNRuYBYw8zfZjgf8UK+oKJjs3n4Ub9zGwYwOqBJWx3sOrpkHcShj8PITW83U0xphSVNxG6jAReVlEot3HS0C1M+wWDuzxeB3nLivq+M1w2jW+9lgc4p7rZxG5+hT7TSiIKTExsThvpUz6YftBUjJyuKprY1+HcqKUPbDsaWd2tcgxvo7GGFPKitsGMR1IB653H2nAjBKMYwzwX1XN81jWzJ0n9QbgVRFpVXgnVZ2qqlGqGlWvXvn9djt3XQI1QgK4pG0ZmktBFeY/7MzDPPwVG6nVmEqouG0QrVT1Go/XT4vI2jPsEw808Xgd4S4ryhjgXs8Fqhrv/twpIstx2id2FDPeciMzJ4/FMfsZ2qUhwQFlqHpp4/9g22IY/A+o1dzX0RhjfKC4JYijInJxwQsR6QscPcM+q4A2ItJCRIJwksBJvZFEpD1QC/jJY1ktEQl2n9cF+gIxhfetCJZvPcDhrFxGRJahu6WPJMHCv0DjHtDnLl9HY4zxkeKWIO4C3heRMPf1IWDc6XZQ1VwRuQ9YhDNu03R3mI7JQLSqFiSLMcAsVfXsNtsBeFtE8nGS2AuevZ8qkrnr9lI3NIgLWtb2dSjHLXocMlNhxP/ZSK3GVGLFHWpjHRApIjXc12ki8hCw/gz7LQAWFFr2ZKHXk4rY70egS3FiK88OZ+WybMt+ro9qQoB/GZjcTxXWfADrZ8Glf4aGnX0dkTHGh87qU0lV09w7qgEe9kI8lcqyzfvJzMnnqsgy0HspMxX+d4dzz0Ozi+GSR3wdkTHGx85nylHr1nKe5qxNoFFYCD2b1vJtILt+gs8nOKO0Xv6EM8+DVS0ZU+mdT4I441Ab5tRSMrJZsS2R8Rc1x89Xd07n5cK3L8J3U5w5pW9fDBFRvonFGFPmnDZBiEg6RScCAap4JaJKYtGmfeTkqe+ql5J3OqWGuFUQeQMM/acNwmeMOcFpE4Sq2ieGl8xdt5dmdarSJTzszBuXJFVYNwsWPALiD9dOdwbhM8aYQs6nismco8T0LH7ccZB7+rUu3VFbj6bAvImw6XNo1hdGvQ01m5x5P2NMpWQJwgcWbtxLvsKIbqVYvRT7A3xxJ6TvhSv+DhdPtIZoY8xpWYLwgbnrEmjXoDptG5RCDV5eDix/wZnwp1ZzuG0xRPT0/nmNMeWeJYhSlpBylFWxh3hkUFvvnyx5p3NvQ/xq6HYTXPmiM6+DMcYUgyWIUjZ//V4AhntzaG9VWPuxM56Snz9c9x50GuW98xljKiRLEKVs7voEukaE0bzumabTOEdHD7kN0V84d0SPfhvCIrxzLmNMhWYJohT9fvAI6+NS+dvQDt45Qez38PmdcHgf9H8K+j5oDdHGmHNmCaIUzVuXAMCwro1K9sB5ObD8efjuZajd0rkjOtwaoo0x58cSRCmauz6BXs1r0bhmCd6EnrTDaYhO+BW63wxDXrCGaGNMibAEUUq27kvnt/2HmTyyU8kcUBXWfAgL/wr+gXD9+9BxZMkc2xhjsARRauauS8BPYGiXEqheykiGeQ9BzJfQ/BLnjuiwMjQjnTGmQvDqLDUiMkREtorIdhF5tIj1r4jIWvfxm4ikeKwbJyLb3MdpZ68r61SVuesT6Nu6LnVDg8/vYL9/B29dDFvmw4BJcMuXlhyMMV7htRKEiPgDbwADgThglYjM8Zw6VFUnemx/P9DdfV4beAqIwhlNdrW77yFvxetNG+JT2ZWUwb39Wp/7QXKzYfk/4PtXoU4ruGMpNO5eckEaY0wh3ixB9Aa2q+pOVc0GZgGnqyQfC/zHfT4YWKKqyW5SWAIM8WKsXjV3XQKB/sLgTg3P7QAHt8P0QfD9K9DjZrhzhSUHY4zXebMNIhzY4/E6DuhT1IYi0gxoAXx9mn1PqkcRkQnABICmTZuef8RekJ+vzFu/l8va1iOsauDZ7VwwR/TCv0JAMFz/AXQc4Z1AjTGmEK+2QZyFMcB/VTXvbHZS1amqGqWqUfXq1fNSaOcnetch9qZmnv3EQBnJ8OktzhzREVFw94+WHIwxpcqbJYh4wHOygQh3WVHGAPcW2rdfoX2Xl2BspWbuugRCAv0Y0KFB8Xfa+S18cRccSYSBk+HC+8GvrORyY0xl4c1PnVVAGxFpISJBOElgTuGNRKQ9UAv4yWPxImCQiNQSkVrAIHdZuZKbl8+CDXvp374B1YKLkYtzs2HJk/D+SAiq5jRE933QkoMxxie8VoJQ1VwRuQ/ng90fmK6qm0RkMhCtqgXJYgwwS1XVY99kEXkGJ8kATFbVZG/F6i0/7Uwi6Ug2V0UW496Hw4nw0bWwdy30HA+D/+EkCWOM8RGv3iinqguABYWWPVno9aRT7DsdmO614ErB3HUJhAYH0K9d/dNvqAqz74bELfCHD6HDVaUToDHGnIbdSe0lWbl5fLVxH4M6NSAk8Awjqq6cCtuXwNAplhyMMWWGVW57yXe/HSQtM/fMvZf2x8Div0ObwdDrjtIJzhhjisEShJfMXZ9AzaqBXNy67qk3ysmE/90OITVg5BsgUnoBGmPMGViC8IKj2XksidnPlZ0bEeh/mku8dBIciIGr/w2hZfM+DmNM5WUJwgu+3nKAjOy80/de2rYUfvk39L4T2gwsveCMMaaYLEF4wZx18dSrHkyfFnWK3uBwotNrqX5H50Y4Y4wpgyxBlLC0zBy+2ZrIsC6N8Pcrok1BFebcB5mpcM27EBhS+kEaY0wxWDfXErZk036yc/NP3Xtp1bvw21cw5EVoUEKzyxljjBdYCaKEzV2fQHjNKvRoWvPklQe2wOInoPUA6HNn6QdnjDFnwRJECUo+ks332w5yVWRjpHCX1dws+N8dEBQKI9+0Lq3GmDLPqphK0Fcb95Gbr0X3Xlo2GfZvgLGfQPWzGNnVGGN8xEoQJWjuugRa1qtGx0Y1Tlyx42v46XXnTul25XZiPGNMJWMJooQcSMvk59+TuKproeqlI0nwxd1Qtx0MetZ3ARpjzFmyKqYSMn/DXlQ5sXpJ1ZkR7mgy3PgZBFbxXYDGGHOWLEGUkDnrEujQqAat61c/vnD1e7B1Pgx6Dhp19VlsxhhzLrxaxSQiQ0Rkq4hsF5FHT7HN9SISIyKbRORjj+V5IrLWfZw0E11Zsic5gzW7U04sPST+Bl89Bi0vhwvu8V1wxhhzjrxWghARf+ANYCAQB6wSkTmqGuOxTRvgMaCvqh4SEc+ZdY6qajdvxVeS5q3fC8BVXd2b43KznVFaA6s4A/HZlKHGmHLIm59cvYHtqrpTVbOBWcDIQtv8EXhDVQ8BqOoBL8bjNXPXJdCtSU2a1K7qLPj6Gdi3Hka+DjWKMd2oMcaUQd5MEOHAHo/Xce4yT22BtiLyg4j8LCKefUBDRCTaXX51UScQkQnuNtGJiYklG30xbT9wmJi9aYwoGFpj53L48TXoeSu0H+aTmIwxpiT4upE6AGgD9AMigBUi0kVVU4BmqhovIi2Br0Vkg6ru8NxZVacCUwGioqK0dEN3zFufgAgM69oIMpKdLq112sDg53wRjjHGlBhvliDigSYeryPcZZ7igDmqmqOqvwO/4SQMVDXe/bkTWA5092Ks50RVmbsugT4tatOgejDMfQCOJDqjtAZV83V4xhhzXryZIFYBbUSkhYgEAWOAwr2RZuOUHhCRujhVTjtFpJaIBHss7wvEUMZs3pvOjsQjzsitaz6AzXOh/9+hcbloWzfGmNPyWoJQ1VzgPmARsBn4VFU3ichkERnhbrYISBKRGOAb4M+qmgR0AKJFZJ27/AXP3k9lxdz1Cfj7CcMaZ8DCv0KLS+HC+30dljHGlAhR9UnVfYmLiorS6OjoUjufqnLJP7+hbd1gpuc+Dodi4Z6foMYp5oEwxpgySERWq2pUUet83Uhdbq3Zk0LcoaO8G74A9qyF6z+w5GCMqVDsDq5zNHddAhcHbKbd9nehxy3QccSZdzLGmHLEShDnIC9fWbFuG58Fv4WEtYTBz/s6JGOMKXGWIM7Byp1JTMx6k5oBh+CaTyE41NchGWNMibMqpnMQ/+00hvv/Qu5lj0F4D1+HY4wxXmEJ4izlJG7nyj0vs61KJEGXTvR1OMYY4zWWIM5GXg5HZ91Krvqxt/9r4Ofv64iMMcZrLEGcjW9fpEbSeibLnfTp1sXX0RhjjFdZgiiuXT+i373EF9oPv06jCA6w0oMxpmKzBFEcR1Pg8wlkVI3giaybnbGXjDGmgrNurmeiCvMfhrQE3ox4jeDMMC5qVcfXURljjNdZCeJM1n8CG/9H9iV/ZVpsHYZ2aUiAv102Y0zFZ590p5P8O8x/BJpexMJaY8nMyWdEZOFJ8YwxpmKyBHEqebnw+QQQPxj9NnPXH6BhjRCimtXydWTGGFMqLEGcyor/B3ErYfjLpAY14tvfDjC8ayP8/MTXkRljTKmwBFGU3T/Din9C1zHQ5VoWxewjJ0+t95IxplLxaoIQkSEislVEtovIo6fY5noRiRGRTSLyscfycSKyzX2M82acJ8hMhc//CGFNYOj/A5yhvZvWrkrXiLBSC8MYY3zNa91cRcQfeAMYCMQBq0RkjufUoSLSBngM6Kuqh0Skvru8NvAUEAUosNrd95C34j1mwZ8hNR5u+wpCanDwcBY/7kjirstaImLVS8aYysObJYjewHZV3amq2cAsYGShbf4IvFHwwa+qB9zlg4ElqprsrlsCDPFirI71nzndWi/7KzTpDcDCjfvIy7fqJWNM5ePNBBEO7PF4Hecu89QWaCsiP4jIzyIy5Cz2RUQmiEi0iEQnJiaeX7SHdjk3xDXpA5f86djiuWsTaFM/lHYNqp/f8Y0xppzxdSN1ANAG6AeMBd4RkZrF3VlVp6pqlKpG1atX79yjKOjSCjB6Kvg7NW97U4+yMjaZqyIbW/WSMabS8WaCiAeaeLyOcJd5igPmqGqOqv4O/IaTMIqzb8n5/mXY8zMMewlqNT+2eP76vQBWvWSMqZS8mSBWAW1EpIWIBAFjgDmFtpmNU3pAROriVDntBBYBg0SklojUAga5y0rewW2w/AXoch10vf6EVXPXJdAlPIwWdat55dTGGFOWea0Xk6rmish9OB/s/sB0Vd0kIpOBaFWdw/FEEAPkAX9W1SQAEXkGJ8kATFbVZK8EWqc1jHoL2g4+YfGupCOsi0vl8aHtvXJaY4wp67w6mquqLgAWFFr2pMdzBR52H4X3nQ5M92Z8AIicVHIAmOdWLw3ratVLxpjKydeN1GXW3HUJRDWrRXjNKr4OxRhjfMISRBF+25/Oln3p1jhtjKnULEEUYd66BPwEruzS0NehGGOMz1iCKERVmbMugQtb1aF+9RBfh2OMMT5jCaKQjfFpxCZlcJU1ThtjKjlLEIXMXZ9AoL8wpLNVLxljKjdLEB7y85V56xK4tE09alYN8nU4xhjjU5YgPPy6+xAJqZnWe8kYY7AEcYK56xIIDvBjQMcGvg7FGGN8zhKEKzcvn/kb9tK/Q31Cg716g7kxxpQLliBcv/yezMHD2dZ7yRhjXJYgXHPXJVAtyJ/L29f3dSjGGFMmWIIAsnPzWbhxH4M6NSQk0N/X4RhjTJlgCQL4blsiqUdzuCqyka9DMcaYMsMSBE71Us2qgVzc+jymLTXGmArGqwlCRIaIyFYR2S4ijxaxfryIJIrIWvdxh8e6PI/lhWeiKzFHs/NYErOfKzs3JCjA8qUxxhTwWn9OEfEH3gAG4sw9vUpE5qhqTKFNP1HV+4o4xFFV7eat+AqkZeZwRYcGXN0t3NunMsaYcsWbHf57A9tVdSeAiMwCRgKFE4RPNagRwv+N7e7rMIwxpszxZp1KOLDH43Wcu6ywa0RkvYj8V0SaeCwPEZFoEflZRK4u6gQiMsHdJjoxMbEEQzfGGOPrSve5QHNV7QosAWZ6rGumqlHADcCrItKq8M6qOlVVo1Q1ql49a2A2xpiS5M0EEQ94lggi3GXHqGqSqma5L98Fenqsi3d/7gSWA1YPZIwxpcibCWIV0EZEWohIEDAGOKE3koh43ngwAtjsLq8lIsHu87pAX8pY24UxxlR0XmukVtVcEbkPWAT4A9NVdZOITAaiVXUO8ICIjABygWRgvLt7B+BtEcnHSWIvFNH7yRhjjBeJqvo6hhIRFRWl0dHRvg7DGGPKFRFZ7bb3nsTXjdTGGGPKKEsQxhhjilRhqphEJBHY5es4zlNd4KCvgyhD7HqcyK7HcXYtTnQ+16OZqhZ5n0CFSRAVgYhEn6ousDKy63Eiux7H2bU4kbeuh1UxGWOMKZIlCGOMMUWyBFG2TPV1AGWMXY8T2fU4zq7FibxyPawNwhhjTJGsBGGMMaZIliCMMcYUyRJEGSAiTUTkGxGJEZFNIvKgr2PyNRHxF5E1IjLP17H4mojUdOdL2SIim0XkQl/H5EsiMtH9P9koIv8RkRBfx1SaRGS6iBwQkY0ey2qLyBIR2eb+rFUS57IEUTbkAn9S1Y7ABcC9ItLRxzH52oO4o/sa/gV8partgUgq8XURkXDgASBKVTvjDAQ6xrdRlbr3gCGFlj0KLFPVNsAy9/V5swRRBqjqXlX91X2ejvMBUGknyRaRCGAYzhwhlZqIhAGXAtMAVDVbVVN8G5XPBQBVRCQAqAok+DieUqWqK3BGv/Y0kuMTrs0EipyF82xZgihjRKQ5zuRIv/g2Ep96FfgLkO/rQMqAFkAiMMOtcntXRKr5vTDoRwAAA25JREFUOihfcScSmwLsBvYCqaq62LdRlQkNVHWv+3wf0KAkDmoJogwRkVDg/7d3PyFWVnEYx79P6mJSkCgIQ2UWhQspNVpE7rQ2KrZwIVEh0qYWVpvQ9hLSwsWoBLUKHHLh/5UkGhEkFMY4Ii2CGmrAv4uKCRlkeFq8Z/RW7zCkd+Zcuc8HLvfMmeHM713M/N5zzn3P7xjwvu0/a8dTg6QtwA3bF2vH0iMWAs8Dn9heB/xFl5YPHkZlbf1VmsT5FLBY0ht1o+otbp5d6MrzC0kQPULSIprkMGz7eO14KloPbJU0BhwBNkg6XDekqsaBcdvTM8qjNAmjX70M/GL7pu07wHHgpcox9YLr0xU6y/uNbgyaBNEDJIlmjflH2/trx1OT7Q9tL7c9SLP5eN52394h2r4G/CZpVenaSH+X3/0VeFHSo+XvZiN9vGnf4TSwo7R3AKe6MWgSRG9YD7xJc7c8Ul6bagcVPWMXMCxpFFgLfFQ5nmrKTOoo8ANwmeZ/WF8duyHpC+ACsErSuKS3gH3AK5J+opll7evK78pRGxER0SYziIiIaJUEERERrZIgIiKiVRJERES0SoKIiIhWSRARs5A01fHx4xFJXXuSWdJg56mcEb1kYe0AIh4Ct22vrR1ExHzLDCLiPkkak/SxpMuSvpP0dOkflHRe0qikc5JWlv4nJZ2QdKm8po+IWCDps1Lj4EtJA+Xn3y01QkYlHal0mdHHkiAiZjfwryWm7R3f+8P2s8BBmlNoAQ4An9t+DhgGhkr/EPC17TU05yldKf3PAIdsrwZ+B7aV/j3AujLO23N1cREzyZPUEbOQNGF7SUv/GLDB9s/lsMVrth+XdAtYZvtO6b9q+wlJN4Hltic7xhgEzpZCL0jaDSyyvVfSGWACOAmctD0xx5ca8Q+ZQUQ8GM/Q/j8mO9pT3Nsb3AwcopltfF8K5ETMmySIiAezveP9Qml/y70ymK8D35T2OeAduFtze+lMg0p6BFhh+ytgN7AU+M8sJmIu5Y4kYnYDkkY6vj5je/qjro+VU1YngddK3y6aCnAf0FSD21n63wM+LadvTtEki6u0WwAcLklEwFBKjcZ8yx5ExH0qexAv2L5VO5aIuZAlpoiIaJUZREREtMoMIiIiWiVBREREqySIiIholQQRERGtkiAiIqLV32MPeBqNh5DRAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vGnF8sTU1s9x"
      },
      "source": [
        "As a reminder, our very first naive approach to this very dataset got us to 88% test accuracy. Unfortunately, our small\n",
        "recurrent network doesn't perform very well at all compared to this baseline (only up to 85% validation accuracy). Part of the problem is\n",
        "that our inputs only consider the first 500 words rather the full sequences --\n",
        "hence our RNN has access to less information than our earlier baseline model. The remainder of the problem is simply that `SimpleRNN` isn't very good at processing long sequences, like text."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Exercise 5\n",
        "**TODO**: Use pretrained word embeddings to optimize your model even further."
      ],
      "metadata": {
        "id": "GWh0PPb8wZnw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Exercise 6\n",
        "\n",
        "Come up with other ways of solving the problem. You are free to use whatever you like:\n",
        "* Other optimizations or embeddings\n",
        "* LSTMs\n",
        "* Transfer Learning with HuggingFace Transformers\n",
        "* LLMs"
      ],
      "metadata": {
        "id": "L7WWGEbWywK3"
      }
    }
  ]
}