电影评论分类.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 电影评论分类： 二分类问题"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 14, 22, 16, 43, 530, 973, 1622, 1385, 65, 458, 4468, 66, 3941, 4, 173, 36, 256, 5, 25, 100, 43, 838, 112, 50, 670, 2, 9, 35, 480, 284, 5, 150, 4, 172, 112, 167, 2, 336, 385, 39, 4, 172, 4536, 1111, 17, 546, 38, 13, 447, 4, 192, 50, 16, 6, 147, 2025, 19, 14, 22, 4, 1920, 4613, 469, 4, 22, 71, 87, 12, 16, 43, 530, 38, 76, 15, 13, 1247, 4, 22, 17, 515, 17, 12, 16, 626, 18, 2, 5, 62, 386, 12, 8, 316, 8, 106, 5, 4, 2223, 5244, 16, 480, 66, 3785, 33, 4, 130, 12, 16, 38, 619, 5, 25, 124, 51, 36, 135, 48, 25, 1415, 33, 6, 22, 12, 215, 28, 77, 52, 5, 14, 407, 16, 82, 2, 8, 4, 107, 117, 5952, 15, 256, 4, 2, 7, 3766, 5, 723, 36, 71, 43, 530, 476, 26, 400, 317, 46, 7, 4, 2, 1029, 13, 104, 88, 4, 381, 15, 297, 98, 32, 2071, 56, 26, 141, 6, 194, 7486, 18, 4, 226, 22, 21, 134, 476, 26, 480, 5, 144, 30, 5535, 18, 51, 36, 28, 224, 92, 25, 104, 4, 226, 65, 16, 38, 1334, 88, 12, 16, 283, 5, 16, 4472, 113, 103, 32, 15, 16, 5345, 19, 178, 32]\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "# load的文件默认放在~/.keras/datasets下\n",
    "# 此处加载时报错，修改了miniconda2/envs/keras2.0.8/lib/python3.6/site-packages/keras/datasets/imdb.py 52行\n",
    "#\n",
    "# 原: with np.load(path) as f:\n",
    "#        x_train, labels_train = f['x_train'], f['y_train']\n",
    "#        x_test, labels_test = f['x_test'], f['y_test']\n",
    "#\n",
    "# 改: with np.load(path, allow_picle=True) as f:\n",
    "#        x_train, labels_train = f['x_train'], f['y_train']\n",
    "#        x_test, labels_test = f['x_test'], f['y_test']\n",
    "\n",
    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)\n",
    "\n",
    "print(train_data[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 解码数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://s3.amazonaws.com/text-datasets/imdb_word_index.json\n",
      "1646592/1641221 [==============================] - 2s     \n"
     ]
    }
   ],
   "source": [
    "word_index = imdb.get_word_index()  # 获取字符索引文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "reverse_word_index = dict(\n",
    "    [(value, key) for (key, value) in word_index.items()]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 索引-3，因为0、1、2是为了填充（padding）\n",
    "decoded_review = ' '.join([reverse_word_index.get(i - 3, '?') for i in train_data[0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"? this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert ? is an amazing actor and now the same being director ? father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for ? and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also ? to the two little boy's that played the ? of norman and paul they were just brilliant children are often left out of the ? list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all\""
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "decoded_review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 对数据进行矩阵转换，将数据转为10000维向量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "def vectorize_sequences(sequences, dimension=10000):\n",
    "    results = np.zeros((len(sequences), dimension))\n",
    "    for i, sequence in enumerate(sequences):\n",
    "        results[i, sequence] = 1.\n",
    "    return results\n",
    "\n",
    "x_train = vectorize_sequences(train_data)\n",
    "x_test = vectorize_sequences(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 1., 1., ..., 0., 0., 0.])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 对标签也进行向量化\n",
    "y_train = np.asarray(train_labels).astype('float32')\n",
    "y_test = np.asarray(test_labels).astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 构建网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import models\n",
    "from keras import layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = models.Sequential()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
    "model.add(layers.Dense(16, activation='relu'))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 编译模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_val = x_train[:10000]\n",
    "partial_x_train = x_train[10000:]\n",
    "\n",
    "y_val = y_train[:10000]\n",
    "partial_y_train = y_train[10000:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 15000 samples, validate on 10000 samples\n",
      "Epoch 1/20\n",
      "15000/15000 [==============================] - 5s - loss: 0.5299 - acc: 0.7711 - val_loss: 0.4351 - val_acc: 0.8282\n",
      "Epoch 2/20\n",
      "15000/15000 [==============================] - 3s - loss: 0.3149 - acc: 0.9005 - val_loss: 0.3117 - val_acc: 0.8846\n",
      "Epoch 3/20\n",
      "15000/15000 [==============================] - 2s - loss: 0.2303 - acc: 0.9252 - val_loss: 0.3138 - val_acc: 0.8725\n",
      "Epoch 4/20\n",
      "15000/15000 [==============================] - 1s - loss: 0.1818 - acc: 0.9415 - val_loss: 0.2735 - val_acc: 0.8905\n",
      "Epoch 5/20\n",
      "15000/15000 [==============================] - 1s - loss: 0.1503 - acc: 0.9525 - val_loss: 0.2860 - val_acc: 0.8868\n",
      "Epoch 6/20\n",
      "15000/15000 [==============================] - 2s - loss: 0.1220 - acc: 0.9616 - val_loss: 0.3253 - val_acc: 0.8723\n",
      "Epoch 7/20\n",
      "15000/15000 [==============================] - 2s - loss: 0.1032 - acc: 0.9691 - val_loss: 0.3069 - val_acc: 0.88500.9\n",
      "Epoch 8/20\n",
      "15000/15000 [==============================] - 1s - loss: 0.0855 - acc: 0.9757 - val_loss: 0.3945 - val_acc: 0.8672\n",
      "Epoch 9/20\n",
      "15000/15000 [==============================] - 1s - loss: 0.0728 - acc: 0.9811 - val_loss: 0.3463 - val_acc: 0.8804\n",
      "Epoch 10/20\n",
      "15000/15000 [==============================] - 1s - loss: 0.0616 - acc: 0.9841 - val_loss: 0.3782 - val_acc: 0.8779\n",
      "Epoch 11/20\n",
      "15000/15000 [==============================] - 2s - loss: 0.0510 - acc: 0.9873 - val_loss: 0.3926 - val_acc: 0.8772\n",
      "Epoch 12/20\n",
      "15000/15000 [==============================] - 2s - loss: 0.0414 - acc: 0.9909 - val_loss: 0.4233 - val_acc: 0.8768\n",
      "Epoch 13/20\n",
      "15000/15000 [==============================] - 1s - loss: 0.0361 - acc: 0.9925 - val_loss: 0.4517 - val_acc: 0.8729\n",
      "Epoch 14/20\n",
      "15000/15000 [==============================] - 2s - loss: 0.0283 - acc: 0.9948 - val_loss: 0.4759 - val_acc: 0.8724\n",
      "Epoch 15/20\n",
      "15000/15000 [==============================] - 1s - loss: 0.0237 - acc: 0.9959 - val_loss: 0.5055 - val_acc: 0.8710\n",
      "Epoch 16/20\n",
      "15000/15000 [==============================] - 2s - loss: 0.0203 - acc: 0.9960 - val_loss: 0.5423 - val_acc: 0.8704\n",
      "Epoch 17/20\n",
      "15000/15000 [==============================] - 3s - loss: 0.0148 - acc: 0.9985 - val_loss: 0.5683 - val_acc: 0.86780.99\n",
      "Epoch 18/20\n",
      "15000/15000 [==============================] - 2s - loss: 0.0126 - acc: 0.9986 - val_loss: 0.5982 - val_acc: 0.8661\n",
      "Epoch 19/20\n",
      "15000/15000 [==============================] - 2s - loss: 0.0122 - acc: 0.9981 - val_loss: 0.6303 - val_acc: 0.8654\n",
      "Epoch 20/20\n",
      "15000/15000 [==============================] - 2s - loss: 0.0062 - acc: 0.9997 - val_loss: 0.6587 - val_acc: 0.8604\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(partial_x_train, partial_y_train, epochs=20, batch_size=512, validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['val_loss', 'val_acc', 'loss', 'acc'])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history_dict = history.history\n",
    "history_dict.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制训练损失和验证损失"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5zNdf7A8dfbRJL7pe2CGcrGuMSYqGRRklSskoitdLH8kpWtTekiZZNUYtWmNluutbVKRWpjk9pkCLkkEhq3UG6NMOP9++PzHR3jzMwZc77nnDnn/Xw8zmPO+Z7v+X7fc+bM930+d1FVjDHGJK5S0Q7AGGNMdFkiMMaYBGeJwBhjEpwlAmOMSXCWCIwxJsFZIjDGmARnicCElYgkich+Eakdzn2jSUTOEZGw97MWkfYisiHg8RoRaR3KvidwrpdE5P4TfX0Bx31MRP4Z7uOayDop2gGY6BKR/QEPywEHgRzv8R9VdUpRjqeqOUD5cO+bCFT13HAcR0RuA3qratuAY98WjmOb+GSJIMGp6tELsfeN8zZV/U9++4vISaqaHYnYjDGRYVVDpkBe0f81EZkmIvuA3iJyoYh8LiK7RWSriIwVkdLe/ieJiIpIivd4svf8bBHZJyL/E5E6Rd3Xe/4KEflGRPaIyDgR+VREbs4n7lBi/KOIrBORn0RkbMBrk0TkGRHZJSLrgY4FvD9DRWR6nm3jReRp7/5tIrLa+32+9b6t53esTBFp690vJyKTvNhWAs3z7PuAiKz3jrtSRDp72xsDfwNae9VuOwPe22EBr+/n/e67ROQtETkjlPemMCLS1Ytnt4jMFZFzA567X0S2iMheEfk64He9QESWeNu3i8iToZ7PhImq2s1uqCrABqB9nm2PAYeAq3FfHE4Bzgda4kqUdYFvgAHe/icBCqR4jycDO4F0oDTwGjD5BPY9DdgHdPGeGwwcBm7O53cJJca3gUpACvBj7u8ODABWAjWBasB8968S9Dx1gf3AqQHH/gFI9x5f7e0jwCXAAaCJ91x7YEPAsTKBtt790cB/gSpAMrAqz77dgTO8v8kNXgy/8Z67DfhvnjgnA8O8+x28GJsCZYHngLmhvDdBfv/HgH969xt4cVzi/Y3uB9Z49xsCG4HTvX3rAHW9+4uAnt79CkDLaP8vJNrNSgQmFAtU9R1VPaKqB1R1kaouVNVsVV0PTADaFPD6N1Q1Q1UPA1NwF6Ci7nsVsFRV3/aeewaXNIIKMcbHVXWPqm7AXXRzz9UdeEZVM1V1FzCygPOsB1bgEhTAZcBPqprhPf+Oqq5XZy7wERC0QTiP7sBjqvqTqm7EfcsPPO/rqrrV+5tMxSXx9BCOC9ALeElVl6rqL8AQoI2I1AzYJ7/3piA9gJmqOtf7G43EJZOWQDYu6TT0qhe/8947cAm9nohUU9V9qrowxN/DhIklAhOK7wMfiEh9EXlPRLaJyF5gOFC9gNdvC7ifRcENxPnte2ZgHKqquG/QQYUYY0jnwn2TLchUoKd3/wbvcW4cV4nIQhH5UUR2476NF/Re5TqjoBhE5GYRWeZVwewG6od4XHC/39Hjqepe4CfgrIB9ivI3y++4R3B/o7NUdQ3wZ9zf4QevqvF0b9c+QCqwRkS+EJFOIf4eJkwsEZhQ5O06+QLuW/A5qloReAhX9eGnrbiqGgBERDj2wpVXcWLcCtQKeFxY99bXgfYichauZDDVi/EU4A3gcVy1TWXggxDj2JZfDCJSF3ge6A9U8477dcBxC+vqugVX3ZR7vAq4KqjNIcRVlOOWwv3NNgOo6mRVbYWrFkrCvS+o6hpV7YGr/nsKeFNEyhYzFlMElgjMiagA7AF+FpEGwB8jcM53gTQRuVpETgL+BNTwKcbXgUEicpaIVAPuLWhnVd0GLAD+CaxR1bXeUycDZYAdQI6IXAVcWoQY7heRyuLGWQwIeK487mK/A5cTb8eVCHJtB2rmNo4HMQ24VUSaiMjJuAvyJ6qabwmrCDF3FpG23rnvwbXrLBSRBiLSzjvfAe92BPcL/EFEqnsliD3e73akmLGYIrBEYE7En4GbcP/kL+AadX2lqtuB64GngV3A2cCXuHEP4Y7xeVxd/le4hsw3QnjNVFzj79FqIVXdDdwFzMA1uHbDJbRQPIwrmWwAZgOvBhx3OTAO+MLb51wgsF79Q2AtsF1EAqt4cl//Pq6KZob3+tq4doNiUdWVuPf8eVyS6gh09toLTgZG4dp1tuFKIEO9l3YCVovrlTYauF5VDxU3HhM6cVWtxpQsIpKEq4ropqqfRDseY0oyKxGYEkNEOnpVJScDD+J6m3wR5bCMKfEsEZiS5GJgPa7a4XKgq6rmVzVkjAmRVQ0ZY0yCsxKBMcYkuBI36Vz16tU1JSUl2mEYY0yJsnjx4p2qGrTLdYlLBCkpKWRkZEQ7DGOMKVFEJN8R8lY1ZIwxCc4SgTHGJDhLBMYYk+BKXBtBMIcPHyYzM5Nffvkl2qGYEJQtW5aaNWtSunR+U+EYYyIpLhJBZmYmFSpUICUlBTcppYlVqsquXbvIzMykTp06hb/AGOO7uKga+uWXX6hWrZolgRJARKhWrZqV3oyJIXGRCABLAiWI/a2MiS1xkwiMMSZe7d4N990H69cXvu+JsEQQBrt27aJp06Y0bdqU008/nbPOOuvo40OHQptWvU+fPqxZs6bAfcaPH8+UKVPCETIXX3wxS5cuDcuxjDH++OUXGD0a6taFkSNhzhx/zhMXjcVFNWUKDB0KmzZB7dowYgT0KsayHNWqVTt6UR02bBjly5fn7rvvPmYfVUVVKVUqeO6dOHFioee54447TjxIY0yJkZMDkybBQw/B99/D5ZfD449Ds2b+nC/hSgRTpkDfvrBxI6i6n337uu3htm7dOlJTU+nVqxcNGzZk69at9O3bl/T0dBo2bMjw4cOP7pv7DT07O5vKlSszZMgQzjvvPC688EJ++OEHAB544AHGjBlzdP8hQ4bQokULzj33XD777DMAfv75Z6699lpSU1Pp1q0b6enphX7znzx5Mo0bN6ZRo0bcf//9AGRnZ/OHP/zh6PaxY8cC8Mwzz5CamkqTJk3o3bt32N8zYxKZKrzzDpx3HvTpA7/5DXz0Ebz/vn9JABKwRDB0KGRlHbstK8ttL06pID9ff/01r776Kunp6QCMHDmSqlWrkp2dTbt27ejWrRupqanHvGbPnj20adOGkSNHMnjwYF5++WWGDBly3LFVlS+++IKZM2cyfPhw3n//fcaNG8fpp5/Om2++ybJly0hLSyswvszMTB544AEyMjKoVKkS7du3591336VGjRrs3LmTr776CoDdu3cDMGrUKDZu3EiZMmWObjPGFN9nn8G998KCBVCvHrz+OnTrBpHoW5FwJYJNm4q2vbjOPvvso0kAYNq0aaSlpZGWlsbq1atZtWrVca855ZRTuOKKKwBo3rw5GzZsCHrsa6655rh9FixYQI8ePQA477zzaNiwYYHxLVy4kEsuuYTq1atTunRpbrjhBubPn88555zDmjVrGDhwIHPmzKFSpUoANGzYkN69ezNlyhQbEGZMGKxaBb//PbRqBWvXwvPPw8qVcN11kUkCkICJoHbtom0vrlNPPfXo/bVr1/Lss88yd+5cli9fTseOHYP2py9TpszR+0lJSWRnZwc99sknn1zoPieqWrVqLF++nNatWzN+/Hj++Mc/AjBnzhz69evHokWLaNGiBTk5OWE9rzGJIjMTbr0VGjeGuXPhscfg22+hXz+I9HeshEsEI0ZAuXLHbitXzm332969e6lQoQIVK1Zk69atzPGhC0CrVq14/fXXAfjqq6+CljgCtWzZknnz5rFr1y6ys7OZPn06bdq0YceOHagq1113HcOHD2fJkiXk5OSQmZnJJZdcwqhRo9i5cydZeevZjDEF+uknVwVUrx5MngwDB7puoUOHQsD3xohKuDaC3HaAcPYaClVaWhqpqanUr1+f5ORkWrVqFfZz3Hnnndx4442kpqYeveVW6wRTs2ZNHn30Udq2bYuqcvXVV3PllVeyZMkSbr31VlQVEeGJJ54gOzubG264gX379nHkyBHuvvtuKlSoEPbfwZh4dOAAjBvnev/s2QO9e8Pw4RAL62yVuDWL09PTNe/CNKtXr6ZBgwZRiii2ZGdnk52dTdmyZVm7di0dOnRg7dq1nHRSbOV8+5uZRKEKb7wBgwe76qBOnVwyaNIksnGIyGJVTQ/2XGxdHUyx7d+/n0svvZTs7GxUlRdeeCHmkoAxieKbb2DAAPjwQ2ja1I0NaNs22lEdz64QcaZy5cosXrw42mEYk9AOHIC//hVGjYKyZeHZZ+H//g9i9TtZjIZljDEl07vvugbg775zbY9PPglnnBHtqAqWcL2GjDHGDxs3uvEAV1/tSgFz57peQbGeBMASgTHGFMuhQ67xt0ED1xYwciQsXQrt2kU7stBZ1ZAxxpygjz6CO+6ANWvgmmvgmWf8G5zqJ19LBCLSUUTWiMg6ETl+shy3T3cRWSUiK0Vkqp/x+KVdu3bHDQ4bM2YM/fv3L/B15cuXB2DLli1069Yt6D5t27Ylb3fZvMaMGXPMwK5OnTqFZR6gYcOGMXr06GIfx5h4s2UL9OwJ7dtDdjbMmgVvvlkykwD4mAhEJAkYD1wBpAI9RSQ1zz71gPuAVqraEBjkVzx+6tmzJ9OnTz9m2/Tp0+nZs2dIrz/zzDN54403Tvj8eRPBrFmzqFy58gkfzxgTXHY2jBkD9evDjBkwbBisWAHe1GAllp8lghbAOlVdr6qHgOlAlzz73A6MV9WfAFT1Bx/j8U23bt147733ji5Cs2HDBrZs2ULr1q2P9utPS0ujcePGvP3228e9fsOGDTRq1AiAAwcO0KNHDxo0aEDXrl05cODA0f369+9/dArrhx9+GICxY8eyZcsW2rVrRzuvUjIlJYWdO3cC8PTTT9OoUSMaNWp0dArrDRs20KBBA26//XYaNmxIhw4djjlPMEuXLuWCCy6gSZMmdO3alZ9++uno+XOnpc6d7O7jjz8+ujBPs2bN2Ldv3wm/t8bEis8+g+bN4a673ARxK1bAww+7huGSzs82grOA7wMeZwIt8+zzWwAR+RRIAoap6vt5DyQifYG+ALULKXsNGuQaasKpaVP3LSA/VatWpUWLFsyePZsuXbowffp0unfvjohQtmxZZsyYQcWKFdm5cycXXHABnTt3znfd3ueff55y5cqxevVqli9ffsw00iNGjKBq1ark5ORw6aWXsnz5cgYOHMjTTz/NvHnzqF69+jHHWrx4MRMnTmThwoWoKi1btqRNmzZUqVKFtWvXMm3aNF588UW6d+/Om2++WeD6AjfeeCPjxo2jTZs2PPTQQzzyyCOMGTOGkSNH8t1333HyyScfrY4aPXo048ePp1WrVuzfv5+y8fCfYhLWpk1umcipU6FWLVcF1LVr5GYGjYRo9xo6CagHtAV6Ai+KyHF1Gqo6QVXTVTW9Ro0aEQ4xNIHVQ4HVQqrK/fffT5MmTWjfvj2bN29m+/bt+R5n/vz5Ry/ITZo0oUnAOPTXX3+dtLQ0mjVrxsqVKwudUG7BggV07dqVU089lfLly3PNNdfwySefAFCnTh2aNm0KFDzVNbj1EXbv3k2bNm0AuOmmm5g/f/7RGHv16sXkyZOPjmBu1aoVgwcPZuzYsezevdtGNpsSad8+NyfZuefCv//t7q9a5RqF4ykJgL8lgs1ArYDHNb1tgTKBhap6GPhORL7BJYZFJ3rSgr65+6lLly7cddddLFmyhKysLJo3bw7AlClT2LFjB4sXL6Z06dKkpKQEnXq6MN999x2jR49m0aJFVKlShZtvvvmEjpMrdwprcNNYF1Y1lJ/33nuP+fPn88477zBixAi++uorhgwZwpVXXsmsWbNo1aoVc+bMoX79+iccqzGRlJMDEyfCAw/A9u1uUNjjj7vSQLzys0SwCKgnInVEpAzQA5iZZ5+3cKUBRKQ6rqpovY8x+aZ8+fK0a9eOW2655ZhG4j179nDaaadRunRp5s2bx8aNGws8zu9+9zumTnWdp1asWMHy5csBN4X1qaeeSqVKldi+fTuzZ88++poKFSoErYdv3bo1b731FllZWfz888/MmDGD1q1bF/l3q1SpElWqVDlampg0aRJt2rThyJEjfP/997Rr144nnniCPXv2sH//fr799lsaN27Mvffey/nnn8/XX39d5HMaEw0ffQRpaXD77XD22bBwoRsUFs9JAHwsEahqtogMAObg6v9fVtWVIjIcyFDVmd5zHURkFZAD3KOqu/yKyW89e/aka9eux/Qg6tWrF1dffTWNGzcmPT290G/G/fv3p0+fPjRo0IAGDRocLVmcd955NGvWjPr161OrVq1jprDu27cvHTt25Mwzz2TevHlHt6elpXHzzTfTokULAG677TaaNWtWYDVQfl555RX69etHVlYWdevWZeLEieTk5NC7d2/27NmDqjJw4EAqV67Mgw8+yLx58yhVqhQNGzY8utqaMbFqzRq45x63XnBKSmSXiYwFNg21iQr7m5lYsGsXPPKIWx7ylFNcddDAgfHREygvm4baGGMCHDoE48e7hWH27oW+fV1COO20aEcWHZYIjDEJQxXefttVA61bBx06wFNPgTeMJ2FFu/to2JS0Kq5EZn8rEw1ffgmXXOLGAJQpA7Nnw5w5lgQgThJB2bJl2bVrl11gSgBVZdeuXTbIzETM0qVw7bWuN9CKFa5KaNky6Ngx2pHFjrioGqpZsyaZmZns2LEj2qGYEJQtW5aaNWtGOwwT5zIy4NFHYeZMqFjRNQT/+c9g03AdLy4SQenSpalTp060wzDGxIDPP3cJYNYsqFLFNQjfeaclgILERSIwxpgFC9xF/8MPoVo1t2bwHXe40oApmCUCY0yJpQoff+wSwLx5rvvnqFHQvz94y32YEFgiMMaUOKpuOojhw+GTT+D0093qYH37Qrly0Y6u5ImLXkPGmMSg6rp9XnQRXHYZfPcdjBsH69e7KegtCZwYSwTGmJinCu++Cy1aQKdObqnIv//dDQobMMBND2FOnFUNGWNi2s6d0K+fWxCmbl146SX4wx/coDATHglRIpgyxc0oWKqU+zllSrQjMsaE4v33oXFjNxbg8cfh66/h1lstCYRb3JcIpkxxDUi5a7tv3Ogeg1twwhgTe7Ky3HxAzz0HDRu6dgFvQT3jg7gvEQwd+msSyJWV5bYbY2LPF19As2YuCQwe7EYIWxLwV9wngk2birbdGBMd2dmuO+hFF8GBA6576FNPxefaALEm7hNB7dpF226MibxvvoFWreDhh6FnT1i+3M0UaiIj7hPBiBHH9y0uV85tN8ZEl6rrBtqsGaxdC6+9BpMm2bxAkRb3iaBXL5gwAZKT3fqjycnusTUUGxNd27bBlVe66SAuvhi++gq6d492VIkp7nsNgbvo24XfmNjx73+73ns//wx/+xv83/8lzkLxsSjuSwTGmNixdy/06eMWiqlTx60adscdlgSizddEICIdRWSNiKwTkSFBnr9ZRHaIyFLvdpuf8Rhjomf+fGjSBF59FR58ED77DOrXj3ZUBnysGhKRJGA8cBmQCSwSkZmquirPrq+p6gC/4jDGRNfmzW51sFdegbPPhk8/hQsuiHZUJpCfJYIWwDpVXa+qh4DpQBcfz2eMiSE//wzDhsFvfwtTp8Ldd7uqIEsCscfPRHAW8H3A40xvW17XishyEXlDRGoFO5CI9BWRDBHJsHWJjYltOTkwcSLUqwePPAJXXeXmCBo1yhaLiVXRbix+B0hR1SbAh8ArwXZS1Qmqmq6q6TVq1IhogMaY0M2dC+npcMstbtDmp5+6sQG2pHhs8zMRbAYCv+HX9LYdpaq7VPWg9/AloLmP8RhjfLJmDXTuDJdeCj/9BNOmwf/+56aLMLHPz0SwCKgnInVEpAzQA5gZuIOInBHwsDOw2sd4jDFhtnMn3HknNGoE//0vjBzpqoF69LAuoSWJb72GVDVbRAYAc4Ak4GVVXSkiw4EMVZ0JDBSRzkA28CNws1/xGGPC5+BBNxDs0Udh3z43OOyRR9zi8abkEVWNdgxFkp6erhkZGdEOw5iEpOpWCrv3XrdO8BVXwJNPujUDTGwTkcWqmh7suWg3FhtjSogvvoDWreG669zEje+/D7NmWRKIB5YIjDH5UnXrAnTsCC1buhlCJ0xw4wEuvzza0ZlwsURgjDlOdrbr+dO8ObRvD0uXwmOPwbp1cPvtcFJCTFeZOOzPaYw5av9+eOkleOYZt4rfuefCiy9C7962Ulg8s0RgjGHbNhg7Fp5/HnbvdusDjBvnRgWXsnqDuGeJwJgEtnq1Wxd40iQ4fBiuucbNCWTzASUWSwTGJBhVWLDAdft85x1X5XPrrTB4MJxzTrSjM9FgicCYBJGTAzNmwOjRsHAhVKvmFou/4w6wKbwSmyUCY+LckSOuB9CwYa7Xz9lnw/jxcPPNbjyAMZYIjIlTqjBnDgwZAsuWQdOm8K9/QdeukJQU7ehMLLH+AMbEoUWL3EygV1zh1gmeOhUWL4Zu3SwJmONZIjAmjnzzjZsCokULWLHCdQn9+mvo2dO6gZr8JVTVkKpNjWvi09atMHy4G/xVtqxrBP7zn6FChWhHZkqChPmOMHmyWznp8OFoR2JM+OzdCw8+6Lp9vvQS9OsH337rGoYtCZhQJUwiqFQJlixx3eeMKekOHoQxY6BuXTcH0NVXu8Fhf/sb/OY30Y7OlDQJkwg6dXL/NGPHRjsSY05cTo4bBXzuuXDXXa4nUEYGTJ9ug8HMiUuYRJCUBAMGuMW0lyyJdjTGFI0qzJ4NaWlw441QtSp88AH85z9uhlBjiiNhEgFAnz5uAM24cdGOxJjQrV7tuoF26uSWhZw61ZUCLrss2pGZeJFQiaByZbjpJjfKcseOaEdjTMF274ZBg6BxY/j8czc5nHUFNX5IuI/TgAGuoe3FF6MdiTHB5eTACy9AvXquTevWW934gMGDoUyZaEdn4lHCJYLUVLfi0nPPWVdSE3vmz3d1/v36QYMGbjTwCy/AaadFOzITz3xNBCLSUUTWiMg6ERlSwH7XioiKSLqf8eQaOBA2b4a33orE2Ywp3MaNcP310KYN/PgjvPYafPwxNGsW7chMIvAtEYhIEjAeuAJIBXqKSGqQ/SoAfwIW+hVLXtaV1MSKrCw3Crh+fbc2wCOPuHaA7t1tFLyJHD9LBC2Adaq6XlUPAdOBLkH2exR4AvjFx1iOkZTk5mBfsAC+/DJSZzXmV6qu73/9+m5qiC5dXAJ46CGbGtpEnp+J4Czg+4DHmd62o0QkDailqu/5GEdQt9xiXUlNdHz5Jfzud673T/Xqrl1g+nSoXTvakZlEFbXGYhEpBTwN/DmEffuKSIaIZOwIU7/P3K6kU6daV9KSICvL1ZsfOhTtSE7cDz9A376uMXjNGtdzbdEiaN062pGZROdnItgM1Ap4XNPblqsC0Aj4r4hsAC4AZgZrMFbVCaqarqrpNcK4pp51JS057r0XevRwc+rs3x/taEL3/ffw8svu2/8558DEiW5qiG++gdtus7UBTGwQVfXnwCInAd8Al+ISwCLgBlVdmc/+/wXuVtWMgo6bnp6uGRkF7lIkl13m6mbXr4fSpcN2WBNGS5bA+edDy5Zurd30dHjvPVetEmv273e9fT74AD780I0KBjj9dOjY0SW0+vWjG6NJTCKyWFWD9sz0bT0CVc0WkQHAHCAJeFlVV4rIcCBDVWf6de6iGDgQOnd2XUmvuy7a0Zi8jhxxDfvVq8OsWa4+/frr4eKL3TKMycnRjS8nx9X5f/CBu332mRufcsoprh3gttugQwdo2NB6AZnY5VuJwC/hLhHk5LgRnGedBZ98ErbDmjD5xz/cxfSVV9xka+D+TldfDeXLu2TQsGFkY9q0yX3b/+AD+Ogj2LXLbW/WzJUwO3SAVq3cAjHGxIqCSgQJnwgAnn7area0ZIkN4Iklu3a56ZYbNHAlgcBv1MuXu6qWX36Bd9+Fiy7yN5ZNm9znZM4cV5UIcOaZ7qJ/2WVutLqN/jWxrKBEkHBTTARjXUlj09ChbuK15547vlqlSRM3pXj16u4i/J5PHZAPHYLHH3f1+n//O9SpA88849YDzsx0jb833GBJwJRslghwXUlvvNG6ksaSRYtgwgTXhtO4cfB96tRxgwJTU92ArFdfDW8M//mPSzj33++mgf7mG9dOMWiQ1fmb+GKJwHPnna4r6UsvRTsSk5MD/fu7njbDhhW872mnwbx50LatGxcyenTxz5+Z6RqkL7sMsrPdgjBvvmkDvkz8skTgsVlJY8eLL7pZN596CipWLHz/ChVc1VD37nDPPfCXv7gpHIrq8GGXSOrXh5kz3dQPK1a4tghj4pklggB33um+DdqspNGzY4erimnXzg0gC9XJJ7uqvTvugCefdKvRFSWh5870ec897tyrVsGDD1rPH5MYLBEEuPJKV+9sjcbRc++9bjnGv/2t6HXwSUnub/fII667adeubmqKgmzbBr17u6qln392JYF33nGfA2MSRUiJQETOFpGTvfttRWSgiFT2N7TIy13g/pNPbFbSaPj0U9cLZ/BgV1V3IkTcDJ7PP+8adi+7zM3vn1d2Njz7rOue+q9/wQMPwMqVbnyCMYkm1BLBm0COiJwDTMDNITTVt6iiyLqSRkd2tqvWqVnTVckUV79+7gKfkeFG+G4OmOXq00/dxG+DBsGFF7p2gEcftemfTeIKNREcUdVsoCswTlXvAc7wL6zoCexKunNntKNJHM89B8uWuT765cuH55jXXgvvv+8Gg110ketq2qePm57ixx/hjTdcj6B69cJzPmNKqlATwWER6QncBLzrbYvbKdpyu5LarKSRsXWrKwV06OAu3uHUrh38979uBHLr1jB5smuH+Pprdy4bC2BM6ImgD3AhMEJVvxOROsAk/8KKrsCupNnZ0Y4m/v3lL+5CPW6cPxfmtDRXHfSnP7mpKUaOhFNPDf95jCmpQkoEqrpKVQeq6jQRqQJUUNUnfI4tqqwraWR8/LH7ln7PPfDb3/p3nnPOgTFj3LxFxphjhTTpnLdWQM8vyUwAABNwSURBVGfctNWLgR+AT1V1sK/RBeHHpHPB5M5KWrOmm/DMhN/hw67v/v79rt++NdYa459wTDpXSVX3AtcAr6pqS6B9uAKMRYFdSZcujXY08WnsWNdl89lnLQkYE02hJoKTROQMoDu/NhbHPetK6p/Nm908Qlde6RYGMsZET6iJYDhupbFvVXWRiNQF1voXVmzI7Uo6ZYp1Jd2yxc0GumZNeI43eLCrGho71nruGBNtoTYW/0tVm6hqf+/xelUNc0e/2JTblbRePShVClJSXGJIJLNnw3nnwR//6CZkS0tz8/ls2nRix/vPf+D11+G++6Bu3fDGaowpulCnmKgpIjNE5Afv9qaI1PQ7uFjw5ZcuAeze7Wa03LgR+vZNjGRw+LDrc9+pk1uNa/58N+CrdGnX5TM52Q3OGj8efvghtGMePOjaXs4+2x3bGBN9oVYNTQRmAmd6t3e8bXFv6FC3gHqgrCy3vShUjz9OLNuwwU3NMGqUm67h88/dgKxBg2DhQli3Dh57zCXIAQNcorj8cjdX0O7d+R/36add9dK4cTazpzGxItTuo0tVtWlh2yIhUt1Hc5Uqlf/c9nfd5WaszMo6/hZse+XKrp69W7eIhX9CZsxwDeVHjrjR1d27F7z/ihUwbZq7ffcdlCnjShE9e8JVV/3aI2jTJle1dPnl7hzGmMgp9uL1IvIRrgQwzdvUE+ijqpeGLcoQRToRpKS46qBgKlRwF7ly5dxI1dz7+W374AP3bXrgQFfHXqZMxH6NkBw86AZ2jRvnJmV77TVXhRMqVfjiC5g+3b1261b3HnTp4pLCP/7hFn9fvdpVKxljIqegRICqFnoDknFVQztwg8neAmqF8LqOwBpgHTAkyPP9gK+ApcACILWwYzZv3lwjafJk1XLlVN1lzt1OOUV10qSiH+vgQdVBg9wxWrZU3bgx/PGeqLVrVdPSXGyDBqn+8kvxjpedrTp3rurtt6tWqfLrezdiRHjiNcYUDZCh+V2r83uisBswqJDnk4BvgbpAGWBZ3gs9UDHgfmfg/cLOG+lEoOqSQXKyqoj7OXly8Y73xhuqFSuqVq2qOmtWOCIsnmnTVCtUcBfst98O//EPHlR9913Vv/61+AnGGHNiCkoExVmhrLDpJVoA69R1NT0ETAe6BO6gbrRyrlOBE1hp1n+9ernG0yNH3M9evYp3vGuvdWvy1qrl6tKHDo3O5HZZWa4HVM+e0LixG0Htx+CuMmXcwLH77nNLShpjYktxEkFhw4DOAr4PeJzpbTv2ICJ3iMi3wChgYNATifQVkQwRydixY8eJxhtTzjkH/vc/uO02+Otf3Upa27ZF7vyrVkHLlq4x+L773FTNtWtH7vzGmNhRnEQQlm/vqjpeVc8G7gUeyGefCaqarqrpNWrUCMdpY8Ipp7gL8T//6RqRmzVzF2Q/qbounuefD9u3u4Vb/vpXNzbAGJOYCkwEIrJPRPYGue3DjScoyGbckpa5anrb8jMd+H1IUceZm25yiaBSJbj0Unj8cX/GHOzb56bMuOUWVxpYtsx15TTGJLYCE4GqVlDVikFuFVT1pEKOvQioJyJ1RKQM0APX8+goEQlcJPBKEmD+ovw0bgyLFsF118H997tF1HftKv5xc3Lc6OixYyE93S3B+cgj8OGHcEZcLjZqjCmqwi7mJ0xVs0VkAG6yuiTgZVVdKSLDca3XM4EBItIeOAz8hFsKM2FVqOAGZbVu7QarpaW5BdhbtAj9GAcOuL78n3zi1uj97DNXEgA3X9JHH0Hbtr6Eb4wpoUIaUBZLIj2gLFpySwdbtsBTT7lpHILN0vnjj24ZxgUL3MU/I8PNESQCjRq5uYBat3Y/a9U6/vXGmMRQ0IAy30oEpnjOPx+WLHHtBwMHuov8Sy+5eXxyv+1/8olb2AVcY+/557vpnVu3hosugipVovs7GGNKBksEMaxqVXj7bRg92rUbvPOOW+QdoGJFd7G/4Qb3bf/8810vJGOMKSpLBDGuVCk35fOFF8KkSa5RuXVr9zMpKdrRGWPigSWCEqJ1a3czxphwK86AMmOMMXHAEoExxiQ4SwTGGJPgLBEYY0yCs0RgjDEJzhKBMcYkOEsExhiT4CwRGGNMgrNEEAFTpkBKihslnJLiHhtjTKywkcU+mzLFrQucleUeb9zoHkPx1z42xphwsBKBz4YO/TUJ5MrKctuNMSYWWCLw2aZNRdtujDGRZonAZ7VrF227McZEmiUCn40YAeXKHbutXDm33RhjYoElAp/16gUTJkBysls+MjnZPbaGYmNMrLBeQxHQq5dd+I0xsctKBMYYk+B8TQQi0lFE1ojIOhEZEuT5wSKySkSWi8hHIpLsZzzGGGOO51siEJEkYDxwBZAK9BSR1Dy7fQmkq2oT4A1glF/xGGOMCc7PEkELYJ2qrlfVQ8B0oEvgDqo6T1Vzh1t9DtT0MR5jjDFB+JkIzgK+D3ic6W3Lz63A7GBPiEhfEckQkYwdO3aEMURjjDEx0VgsIr2BdODJYM+r6gRVTVfV9Bo1akQ2OGOMiXN+dh/dDNQKeFzT23YMEWkPDAXaqOpBH+MxxhgThJ8lgkVAPRGpIyJlgB7AzMAdRKQZ8ALQWVV/8DGWEs2msTbG+Mm3EoGqZovIAGAOkAS8rKorRWQ4kKGqM3FVQeWBf4kIwCZV7exXTCWRTWNtjPGbqGq0YyiS9PR0zcjIiHYYEZOS4i7+eSUnw4YNkY7GGFNSichiVU0P9lxMNBab/Nk01sYYv1kiiHE2jbUxxm+WCGKcTWNtjPGbJYIYZ9NYG2P8ZtNQlwA2jbUxxk9WIjDGmARniSAB2IA0Y0xBrGooztmANGNMYaxEEOeGDv01CeTKynLbjTEGLBHEPRuQZowpjCWCOGcD0owxhbFEEOdsQJoxpjCWCOKcDUgzxhTGeg0lABuQZowpiJUIjDEmwVkiMIWyAWnGxDerGjIFsgFpxsQ/KxGYAtmANGPinyUCUyAbkGZM/LNEYApkA9KMiX+WCEyBbECaMfHP10QgIh1FZI2IrBORIUGe/52ILBGRbBHp5mcs5sSEY0Ca9ToyJrb51mtIRJKA8cBlQCawSERmquqqgN02ATcDd/sVhym+4gxIs15HxsQ+P0sELYB1qrpeVQ8B04EugTuo6gZVXQ4c8TEOE0XW68iY2OdnIjgL+D7gcaa3rchEpK+IZIhIxo4dO8ISnIkM63VkTOwrEY3FqjpBVdNVNb1GjRrRDscUgfU6Mib2+ZkINgO1Ah7X9LaZBBKOXkfW2GyMv/xMBIuAeiJSR0TKAD2AmT6ez8Sg4vY6ym1s3rgRVH9tbLZkYEz4iKr6d3CRTsAYIAl4WVVHiMhwIENVZ4rI+cAMoArwC7BNVRsWdMz09HTNyMjwLWYTW1JS3MU/r+Rk2LAh0tEYU3KJyGJVTQ/6nJ+JwA+WCBJLqVKuJJCXCByxvmbGhKygRFAiGotN4gpHY7O1MRhTMEsEJqYVt7HZ2hiMKZwlAhPTitvYbAPajCmcJQIT83r1cg3DR464n0WZmiIcA9qsasnEO0sEJq4Vt43BqpZMIrBEYOJacdsYrGrJJAJLBCauFbeNwaqWTCKwxetN3CvONNq1awcf0FbUqiWbhtvEMisRGFMAq1oyicASgTEFiIWqJbDqJeMvqxoyphDRrFoCq14y/rMSgTE+Csc03OGoXrIShSmIJQJjfFTcqiUofvWSjYUwhbFEYIzPijMyGoo/KM5KFKYwlgiMiXHFrV6yEoUpjCUCY2JccauXrERhCmOJwJgSoDjVS/FQorBE4i9LBMbEuZJeorBEEgGqWqJuzZs3V2NM5EyerFqunKq7DLtbuXJueyhEjn1t7k0ktNcnJwd/fXJyZOLPPUZysos5Oblorw3H68MBt1Z80Otq1C/sRb1ZIjAm8opzISvuhbykJ5JwJKJwKCgRWNWQMaZQ0WyjKG7VVHHbOIpbtVUSGtstERhjfFXcNoqSnkhiobG9ML4mAhHpKCJrRGSdiAwJ8vzJIvKa9/xCEUnxMx5jTHQUp0RR0hNJtBvbQ+FbIhCRJGA8cAWQCvQUkdQ8u90K/KSq5wDPAE/4FY8xpuQqyYkk2t13Q+FniaAFsE5V16vqIWA60CXPPl2AV7z7bwCXioj4GJMxJgFFM5FEu/tuKPxMBGcB3wc8zvS2Bd1HVbOBPUC1vAcSkb4ikiEiGTt27PApXGOMCa6480VFs7E9FCWisVhVJ6hquqqm16hRI9rhGGNMxIRjBtvC+LkwzWagVsDjmt62YPtkishJQCVgl48xGWNMiVOcxZFC4WeJYBFQT0TqiEgZoAcwM88+M4GbvPvdgLnewAdjjDER4luJQFWzRWQAMAdIAl5W1ZUiMhw3wm0m8A9gkoisA37EJQtjjDER5Ouaxao6C5iVZ9tDAfd/Aa7zMwZjjDEFKxGNxcYYY/xjicAYYxKclLS2WRHZAWyMdhz5qA7sjHYQBbD4iifW44PYj9HiK57ixJesqkH735e4RBDLRCRDVdOjHUd+LL7iifX4IPZjtPiKx6/4rGrIGGMSnCUCY4xJcJYIwmtCtAMohMVXPLEeH8R+jBZf8fgSn7URGGNMgrMSgTHGJDhLBMYYk+AsERSRiNQSkXkiskpEVorIn4Ls01ZE9ojIUu/2ULBj+RjjBhH5yjt3RpDnRUTGekuELheRtAjGdm7A+7JURPaKyKA8+0T8/RORl0XkBxFZEbCtqoh8KCJrvZ9V8nntTd4+a0XkpmD7+BDbkyLytff3myEilfN5bYGfBZ9jHCYimwP+jp3yeW2BS9r6GN9rAbFtEJGl+bzW1/cwv2tKRD9/qmq3ItyAM4A0734F4BsgNc8+bYF3oxjjBqB6Ac93AmYDAlwALIxSnEnANtxAl6i+f8DvgDRgRcC2UcAQ7/4Q4Ikgr6sKrPd+VvHuV4lAbB2Ak7z7TwSLLZTPgs8xDgPuDuEz8C1QFygDLMv7/+RXfHmefwp4KBrvYX7XlEh+/qxEUESqulVVl3j39wGrOX7ltVjXBXhVnc+ByiJyRhTiuBT4VlWjPlJcVefjZsANFLiU6ivA74O89HLgQ1X9UVV/Aj4EOvodm6p+oG5VP4DPcet9RE0+718oQlnSttgKis9bHrc7MC3c5w1FAdeUiH3+LBEUg4ikAM2AhUGevlBElonIbBFpGNHAQIEPRGSxiPQN8nwoy4hGQg/y/+eL5vuX6zequtW7vw34TZB9YuG9vAVXwgumsM+C3wZ41Vcv51O1EQvvX2tgu6quzef5iL2Hea4pEfv8WSI4QSJSHngTGKSqe/M8vQRX3XEeMA54K8LhXayqacAVwB0i8rsIn79Q4hYr6gz8K8jT0X7/jqOuHB5zfa1FZCiQDUzJZ5dofhaeB84GmgJbcdUvsagnBZcGIvIeFnRN8fvzZ4ngBIhIadwfbIqq/jvv86q6V1X3e/dnAaVFpHqk4lPVzd7PH4AZuOJ3oFCWEfXbFcASVd2e94lov38BtudWmXk/fwiyT9TeSxG5GbgK6OVdKI4TwmfBN6q6XVVzVPUI8GI+547qZ1HcErnXAK/lt08k3sN8rikR+/xZIigirz7xH8BqVX06n31O9/ZDRFrg3ueIrMUsIqeKSIXc+7hGxRV5dpsJ3Oj1HroA2BNQBI2UfL+FRfP9yyNwKdWbgLeD7DMH6CAiVbyqjw7eNl+JSEfgL0BnVc3KZ59QPgt+xhjY7tQ1n3OHsqStn9oDX6tqZrAnI/EeFnBNidznz6+W8Hi9ARfjimjLgaXerRPQD+jn7TMAWInrAfE5cFEE46vrnXeZF8NQb3tgfAKMx/XW+ApIj/B7eCruwl4pYFtU3z9cUtoKHMbVs94KVAM+AtYC/wGqevumAy8FvPYWYJ136xOh2Nbh6oZzP4N/9/Y9E5hV0Gchgu/fJO/ztRx3UTsjb4ze4064njLf+hVjsPi87f/M/dwF7BvR97CAa0rEPn82xYQxxiQ4qxoyxpgEZ4nAGGMSnCUCY4xJcJYIjDEmwVkiMMaYBGeJwBiPiOTIsTOjhm0mTBFJCZz50phYclK0AzAmhhxQ1abRDsKYSLMSgTGF8OajH+XNSf+FiJzjbU8RkbnepGofiUhtb/tvxK0RsMy7XeQdKklEXvTmnP9ARE7x9h/ozUW/XESmR+nXNAnMEoExvzolT9XQ9QHP7VHVxsDfgDHetnHAK6raBDfp21hv+1jgY3WT5qXhRqQC1APGq2pDYDdwrbd9CNDMO04/v345Y/JjI4uN8YjIflUtH2T7BuASVV3vTQ62TVWrichO3LQJh73tW1W1uojsAGqq6sGAY6Tg5o2v5z2+Fyitqo+JyPvAftwsq2+pN+GeMZFiJQJjQqP53C+KgwH3c/i1je5K3NxPacAib0ZMYyLGEoExobk+4Of/vPuf4WbLBOgFfOLd/wjoDyAiSSJSKb+DikgpoJaqzgPuBSoBx5VKjPGTffMw5lenyLELmL+vqrldSKuIyHLct/qe3rY7gYkicg+wA+jjbf8TMEFEbsV98++Pm/kymCRgspcsBBirqrvD9hsZEwJrIzCmEF4bQbqq7ox2LMb4waqGjDEmwVmJwBhjEpyVCIwxJsFZIjDGmARnicAYYxKcJQJjjElwlgiMMSbB/T+bNvu/VoDYmgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "history_dict = history.history\n",
    "loss_values = history_dict['loss']\n",
    "val_loss_values = history_dict['val_loss']\n",
    "\n",
    "epochs = range(1, len(loss_values) + 1)\n",
    "\n",
    "plt.plot(epochs, loss_values, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss_values, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 绘制训练精度和验证精度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxV1bn/8c/DZEBmgqighKpVsQpCilrFWS/iQEWqIt46FrXOP/3dUrXVWrVetV6r5WfFEWuUUriOdSgiFa0ToRLGKoigAcQwiEJApuf3x9qBk3hOcpIzhnzfr9d+nX32+Jx9TvaTvdbaa5u7IyIiUlOzXAcgIiL5SQlCRETiUoIQEZG4lCBERCQuJQgREYlLCUJEROJSgpCkmVlzM1trZnumc9lcMrO9zSztbb3N7HgzWxTz/iMzG5jMsg3Y1yNmdkND1xdJpEWuA5DMMbO1MW/bAN8CW6L3l7h7SX225+5bgLbpXrYpcPd907EdM7sYONfdj47Z9sXp2LZITUoQOzB333aCjv5DvdjdX0+0vJm1cPfN2YhNpC76PeaeipiaMDO7zcz+YmbPmNk3wLlmdpiZvWdmX5nZMjO738xaRsu3MDM3s6Lo/VPR/FfM7Bsze9fMetV32Wj+SWb2sZmtMbMHzOyfZnZ+griTifESM1tgZqvN7P6YdZub2f+Y2UozWwgMquX43Ghm42pMG21m90bjF5vZvOjzfBL9d59oW+VmdnQ03sbM/hzFNgfoX2PZm8xsYbTdOWZ2WjT9QOCPwMCo+G5FzLG9JWb9S6PPvtLMnjOz3ZI5NvU5zlXxmNnrZrbKzL4ws/+K2c+vomPytZmVmtnu8YrzzOztqu85Op5To/2sAm4ys33MbEq0jxXRcesQs37P6DNWRPP/YGYFUcz7xyy3m5lVmlmXRJ9X4nB3DU1gABYBx9eYdhuwETiV8M9Ca+CHwCGEq8vvAR8DV0TLtwAcKIrePwWsAIqBlsBfgKcasOwuwDfAkGje/wE2Aecn+CzJxPg80AEoAlZVfXbgCmAO0APoAkwNfwZx9/M9YC2wc8y2vwSKo/enRssYcCywHjgomnc8sChmW+XA0dH4PcA/gE5AT2BujWXPBHaLvpNzohi6RfMuBv5RI86ngFui8ROjGPsCBcD/A95I5tjU8zh3AJYDVwM7Ae2BAdG8XwJlwD7RZ+gLdAb2rnmsgbervufos20GLgOaE36P3weOA1pFv5N/AvfEfJ7Z0fHcOVr+8GjeGOD2mP1cBzyb67/DxjbkPAANWfqiEyeIN+pY73rgr9F4vJP+n2KWPQ2Y3YBlLwTeiplnwDISJIgkYzw0Zv7/AtdH41MJRW1V8wbXPGnV2PZ7wDnR+EnAR7Us+xJweTReW4L4LPa7AH4eu2yc7c4GTo7G60oQY4E7Yua1J9Q79ajr2NTzOP8nMC3Bcp9UxVtjejIJYmEdMQyr2i8wEPgCaB5nucOBTwGL3s8Ahqb772pHH1TEJJ/HvjGz/czsb1GRwdfArUBhLet/ETNeSe0V04mW3T02Dg9/0eWJNpJkjEntC1hcS7wATwPDo/FzovdVcZxiZu9HxR9fEf57r+1YVdmtthjM7HwzK4uKSb4C9ktyuxA+37btufvXwGqge8wySX1ndRznPQiJIJ7a5tWl5u9xVzMbb2ZLohieqBHDIg8NIqpx938SrkaOMLMfAHsCf2tgTE2WEoTUbOL5EOE/1r3dvT3wa8J/9Jm0jPAfLgBmZlQ/odWUSozLCCeWKnU1wx0PHG9m3QlFYE9HMbYGJgC/IxT/dAT+nmQcXySKwcy+BzxIKGbpEm333zHbratJ7lJCsVXV9toRirKWJBFXTbUd58+BvRKsl2jeuiimNjHTdq2xTM3P99+E1ncHRjGcXyOGnmbWPEEcTwLnEq52xrv7twmWkwSUIKSmdsAaYF1UyXdJFvb5EtDPzE41sxaEcu2uGYpxPHCNmXWPKix/UdvC7v4FoRjkCULx0vxo1k6EcvEKYIuZnUIoK082hhvMrKOF+0SuiJnXlnCSrCDkyp8RriCqLAd6xFYW1/AMcJGZHWRmOxES2FvunvCKrBa1HecXgD3N7Aoz28nM2pvZgGjeI8BtZraXBX3NrDMhMX5BaAzR3MxGEpPMaolhHbDGzPYgFHNVeRdYCdxhoeK/tZkdHjP/z4QiqXMIyULqSQlCaroOOI9QafwQoTI5o9x9OXAWcC/hD34v4EPCf47pjvFBYDIwC5hGuAqoy9OEOoVtxUvu/hVwLfAsoaJ3GCHRJeNmwpXMIuAVYk5e7j4TeAD4IFpmX+D9mHUnAfOB5WYWW1RUtf6rhKKgZ6P19wRGJBlXTQmPs7uvAU4AziAkrY+Bo6LZdwPPEY7z14QK44Ko6PBnwA2EBgt71/hs8dwMDCAkqheAiTExbAZOAfYnXE18RvgequYvInzP37r7O/X87ML2ChyRvBEVGSwFhrn7W7mORxovM3uSUPF9S65jaYx0o5zkBTMbRGgxtJ7QTHIT4b9okQaJ6nOGAAfmOpbGSkVMki+OABYSyt7/AzhdlYrSUGb2O8K9GHe4+2e5jqexUhGTiIjEpSsIERGJa4epgygsLPSioqJchyEi0qhMnz59hbvHbVa+wySIoqIiSktLcx2GiEijYmYJexNQEZOIiMSlBCEiInEpQYiISFxKECIiEpcShIiIxJWxBGFmj5nZl2Y2O8F8ix4tuMDMZppZv5h555nZ/Gg4L1Mxiog0ZiUlUFQEzZqF15KS9G4/k1cQT1DL834JT+faJxpGEnrZJOoW+GbCow4HADebWacMxikiTVSqJ9hMn6Dr2vfIkbB4MbiH15Ej0xtDxhKEu08ldIOcyBDgSQ/eAzpaeLj6fwCT3H2Vu68mdG9cW6IREam3VE+w6ThBp5JgbrwRKiurT6usDNPTJZd1EN2p/njB8mhaounfYWYjzazUzEorKioyFqiI5KdcnmBTXT/VBPNZgi4IE01viEZdSe3uY9y92N2Lu3at7QFkIpIJuSyiyfUJNtX1U00weyZ4WG6i6Q2RywSxhOrP5e0RTUs0XUTySK6LaHJ9gk11/VQTzO23Q5s21ae1aROmp427Z2wAioDZCeadTHjcogGHAh9E0zsDnxIetN4pGu9c17769+/vIlI/Tz3l3rOnu1l4feqp5Nft2dM9nNqrDz17Zmd9s/jrmyW3/lNPubdpU33dNm2SPwaprp/q56+KoaHfXxWg1BOdwxPNSHUgPDx9GeHJYOXARcClwKXRfANGA58QnhtbHLPuhcCCaLggmf0pQYjUT6onuFRP0Kmunw8n2FTWT/X4p0tOEkS2ByUIaYoa8xVAquvnywk2Fem4AkiVEoTIDijXVwC5LqKp2kauT7CNXW0JYod55GhxcbHreRDSlBQVhYrdmnr2hEWLMr8+hArlG28MFat77hkqSEeMSG7ddKwvqTOz6e5eHG9eo27mKtLYpdLMMx9awYwYEZLJ1q3htb4n91TXl8xSghDJkVSbeabazHLECBgzJlwxmIXXMWN0kpbtVMQkkiOpFvFUJZjYewHatNFJXupHRUwiGZLLIiJdAUimtch1ACKNVc3/4KuKiCC5k/See8a/gqhPVwkjRighSOboCkKatFx29paVrhJEUqAEIU1Wrjt7UxGR5DtVUkuTlQ/3EYjkmiqpReLIh/sIRPKZEoQ0WbqPQKR2ShDSqKVSyZwPdxKL5DMlCGm0Uq1k1hWASO1USS2NliqJRVKnSmrZIWXjoe0iTZkShDRa2Xhou0hTpgQhjZaamYpklhKE5FQqrZBUySySWeqsT3Im1c7uqpZTQhDJDF1BSM6k2tmdiGSWEoTkjFohieQ3JQjJGbVCEslvShCSM2qFJJLflCAkJWqFJLLjUismaTC1QhLZsekKQhpMrZBEdmxKENJgaoUksmNTgpAGUyskkR2bEoQ0mFohiezYlCCkwdQKSWTHplZMkhK1QhLZcekKoolL5T4GEdmx6QqiCUvHfQwisuPK6BWEmQ0ys4/MbIGZjYozv6eZTTazmWb2DzPrETNvi5nNiIYXMhlnU6X7GESkNhm7gjCz5sBo4ASgHJhmZi+4+9yYxe4BnnT3sWZ2LPA74D+jeevdvW+m4hPdxyAitcvkFcQAYIG7L3T3jcA4YEiNZXoDb0TjU+LMlwzSfQwiUptMJojuwOcx78ujabHKgKHR+OlAOzPrEr0vMLNSM3vPzH4cbwdmNjJaprSioiKdsTcJuo9BRGqT61ZM1wNHmdmHwFHAEmBLNK+nuxcD5wD3mdleNVd29zHuXuzuxV27ds1a0DsK3ccgIrXJZCumJcAeMe97RNO2cfelRFcQZtYWOMPdv4rmLYleF5rZP4CDgU8yGG+TpPsYRCSRTF5BTAP2MbNeZtYKOBuo1hrJzArNrCqGXwKPRdM7mdlOVcsAhwOxldsiIpJhGUsQ7r4ZuAJ4DZgHjHf3OWZ2q5mdFi12NPCRmX0MdAOqSr/3B0rNrIxQeX1njdZPIiKSYebuuY4hLYqLi720tDTXYWRdSUm4b+Gzz0Lro9tvV5GRiCTPzKZH9b3foTupGzHdCS0imZTrVkySAt0JLSKZpATRiOlOaBHJJCWIRkx3QotIJilBNGK6E1pEMkkJohHTndAikklqxdTI6U5oEckUXUGIiEhcShAiIhKXEoSIiMSlBCEiInEpQYiISFxKECIiEpcSRI6VlEBRETRrFl5LSnIdkYhIoPsgcki9sYpIPtMVRA6pN1YRyWdKEDmk3lhFJJ8pQeSQemMVkXymBJFD6o1VRPKZEkQOqTdWEclnasWUY+qNVUTyla4gREQkLiUIERGJSwlCRETiUoIQEZG4lCBERCQuJQgREYlLCUJEROJSghARkbiUIEREJK46E4SZXWlmnbIRjIiI5I9kriC6AdPMbLyZDTIzy3RQIiKSe3UmCHe/CdgHeBQ4H5hvZneY2V4Zjk1ERHIoqToId3fgi2jYDHQCJpjZXRmMTUREciiZOoirzWw6cBfwT+BAd78M6A+cUce6g8zsIzNbYGaj4szvaWaTzWymmf3DzHrEzDvPzOZHw3n1/mRZUlICRUXQrFl4LSnJdUQiIumRTHffnYGh7r44dqK7bzWzUxKtZGbNgdHACUA5oR7jBXefG7PYPcCT7j7WzI4Ffgf8p5l1Bm4GigEHpkfrrq7Ph8u0khIYOXL7c6UXLw7vQV14i0jjl0wR0yvAqqo3ZtbezA4BcPd5taw3AFjg7gvdfSMwDhhSY5newBvR+JSY+f8BTHL3VVFSmAQMSiLWrLrxxu3JoUplZZguItLYJZMgHgTWxrxfG02rS3fg85j35dG0WGXA0Gj8dKCdmXVJcl3MbKSZlZpZaUVFRRIhpddnn9VvuohIY5JMgrCokhoIRUuk70l01wNHmdmHwFHAEmBLsiu7+xh3L3b34q5du6YppOTtuWf9pouINCbJJIiFZnaVmbWMhquBhUmstwTYI+Z9j2jaNu6+1N2HuvvBwI3RtK+SWTcf3H47tGlTfVqbNmG6iEhjl0yCuBT4EeEEXQ4cAoxMYr1pwD5m1svMWgFnAy/ELmBmhWZWFcMvgcei8deAE82sU3QX94nRtLwyYgSMGQM9e4JZeB0zRhXUIrJjqLOoyN2/JJzc68XdN5vZFYQTe3PgMXefY2a3AqXu/gJwNPA7M3NgKnB5tO4qM/stIckA3Oruq76zkzwwYoQSgojsmCymeiH+AmYFwEXAAUBB1XR3vzCzodVPcXGxl5aW5joMEZFGxcymu3txvHnJFDH9GdiV0PT0TUJ9wDfpC09ERPJRMglib3f/FbDO3ccCJxPqISRPbN4MW7fmOgoR2dEk01x1U/T6lZn9gNAf0y6ZC0nq4+23YciQkCT69YP+/aG4OLzutVfoAkREpCGSSRBjopZENxFaIbUFfpXRqCQpU6bAKafAHnvAscdCaSn88Y/w7bdhfvv225NGVeJQ0hCRZNWaIKImqF9H3V1MBb6XlaikTq+9Bj/+Mey9N7z+OnTrFqZv2gRz5sD06WGoK2n07x+2oaQhIjUl04qpNFENdz5pSq2YXnoJzjgDeveGSZOgsLD25WsmjenToaysetK4/vrQh5QShUjTUlsrpmQSxJ3ACuAvwLqq6fl2X0JTSRD/+79w9tnQp0+4iujcuWHbiU0aL70Ezz0HgwfDU09Bpyw/YHb69FD01bFjdvcrIqk3cz2LcAPbVGB6NOz4Z+IkrV0LBx4Il14KX3+d2X2NGwdnnhnqEl5/veHJAaBlS+jbFy66KCSd0aPD1UhxMcyYkb6Ya/PNN/Czn4V9HnYYLF2anf2KSHKSeeRorziD6iIif/sbzJ4NDz0EBxwAL7+cmf08+WS4Y/vww8OVQ4cO6du2Gfz85/Dmm7BhQzhZ//nP6dt+PG++CQcdBI89BhdfDOXlcOSR4ZkaIpIfknmi3E/jDdkIrjGYMAF22w3efTectE8+GX76U1i5Mn37eOQROP98OOaYkIDatUvftmMddhj8619wyCHhM1x+OWzcmN59bNgA110XPkvz5vDWW/Dww+GKaOVKGDgQFixI7z5FpIHcvdYBeCBmeJjQk+uEutbL9tC/f3/PtrVr3du0cb/88vB+wwb3X/3KvUUL9112cf/rX1Pfxx//6A7uJ53kXlmZ+vaSsWmT+3XXhf0eeqh7eXl6tlta6t67d9juZZe5f/NN9fkffuheWOi+227uc+akZ5/JmDbNfdIk94ULw2cXaUoIfePFP/8nmpFwBegIvFrf9TI95CJB/PWv4QhOmVJ9+owZ7v36hXlDh7ovW9aw7f/+92EbQ4aE5JNt48e777xzSHY1P2N9bNzo/pvfhMS5++7ur76aeNk5c9x33TUkig8/bPg+k1FZGZI7bB9atHDfay/3E090v/RS97vvdp84MXynX3+d2XhEciHdCaIl8FF918v0kIsEcdZZ7l27um/e/N15mza533mn+047uXfq5D52rPvWrclv+447wrczbFg4webKnDnu++7r3ry5+z331O8zuLvPm+deXBw+y4gR7qtW1b3Oxx+777GHe8eO7u+/37C46zJjxvarmWuvdX/jDfdHHnH/5S/D91pc7N65c/XkAeH7PvRQ93POCVeLTzzhPnWq+9y57osWuS9fHq6M4v0mRPJRbQkimWauLwJVCzUjPEd6vLuPSnNpV0qy3cx1/Xro2hXOPRf+9KfEy330UWgp9M9/wqBBoTK7tifOucNvfhOGc86BsWOhRbqe39dAX38NF1wQWjv95Cfw6KN114Ns3Qr33w+//CXsvDM8+GBYN1mLF4e7wysqQkOAgQNT+wyxcf3hDzBqVGgFNnYsnHhi4uW/+goWLoRPPvnu62ef1d4HVqtW4QFSbdpA69bVX2uO77RTaFnWqtX219jxRK9V40VF4Y56s/QcJ2k6Ur0P4qiYt5uBxe5ensb40iLbCeK55+D000PT0OOPr33ZrVtDM9JRo8KNaHfdBZdc8t2b0tzhhhvgzjtDpfQjj4SK3HzgDnffHU74++4Lzz4bXuNZvDjE/49/hK5AHn4Ydt21/vtcsiQc28WL4YUX6j7OdVm2LMT197/DaaeF45vKk2o3bgxJYuFCWL0aKivDsH59/cYrK8O2Nm4M96ds2lT3vuPZZRcYMGD78MMfptYUWpqG2hJEMkVKvYCCmPetgaK61sv2kO0iphEj3Lt0qV/xz8KF7scdF4oqjjrKff787fO2bg1FHeB+ySXuW7akPeS0mDw51A+0axfK5mNt3er+2GNhXtu27o8+Wv8iqZqWL3c/6KBQVPfiiw3fzvPPh7hbt3b/059SjyuTtm4Nv6u1a0OR3PLl7p9/Hn4///63+6xZ7tOnu7/3XijemjTJffRo9/POc99/f3ez7UVie+8disPuu8/9nXey19BBGg9SqYMg3BTXKuZ9K2BaXetle8hmgtiwIZwEL7qo/utu3RrKujt0CCere+4JJ4Of/zx8G1demd8nL3f3zz5zHzAgxPuLX4T6li++cD/11DDtyCPDySxdVq4MdQItWoSK8/pYty5UNoN7376hrmBHt2ZNqFO5887QSKJHj+0Jo0WL0IDi0ktDMp89W/UlTV1tCSKZIqYZ7t63xrQyd++TymVNumWziOmll+DUU+GVV0K9QkMsWQKXXQYvvhjuo1i2LPSHdNddjaMc+dtv4eqrQ53Kj34EH38c7oy+4w645pr09+m0Zk24x+Tdd0O9wbnn1r3OjBmhHmfevHBsb7stlPU3RUuXwrRp8MEHYZg2LRxTgLZtoXv3UJ+x0061D4mWad8eunQJQ2FheO3cOff1Z1K3VOsgJgEPeHiGNGY2BLjK3Y9Le6QpyGaCOP98eP55WL48/ME0lHvoPuMXv4ALL4Sbb24cySHWE0+ERHfAAeFu7969M7evtWvDsy+mTAkNA0aOjL/c1q3wP/8T6ku6dg0JJdX6ix3N1q0wf/72ZPHllyHp12fYsqXu/XTsWD1pJBpv3x4KCsLQuvX28YICdSCZaakmiL2AEmD3aFI58FN3z6v7XbOVIDZuDF1r//jH8PjjGd9do7BqVbiLPBsV6uvXw7Bh4Y7y++4LVzGxli6F884Ld2b/+MehIrpLl8zH1RRt2RL+HjZsCC3dVq6EFSvCa13j69bVvf0qrVrFTx6x47vsErqtjx2y3elkY1VbgqjzAtDdPwEONbO20fu1aY6vUXnjjdD0cdiwXEeSP7LZUqZ169CCavjwUJRVWRmuFCC0LLvoonDCGjMm9PHU2K7IGpPmzcP30bp1OBn37Jn8uhs2bE8YK1eG4skNG8Kwfn3d47HvV6wI/aE9+WT1fXTu/N2kUTUUFuq3kYw6E4SZ3QHc5e5fRe87Ade5+02ZDi4fTZgQLodVZJE7rVrBX/4SrhRuuCGcXFauDEmhf38oKUncBFfyQ0FBqPfo3j1921y/PjQ5XrCg+vDOO6EoN/aelXbtqieMnj1DyUC3bqFJdrdu4f6Upi6ZIqYP3f3gGtP+5e79MhpZPWWjiGnTpvDjGTw4872dSt22bAn3kzz6aPhv8L/+C269NbV6IdkxbdwIixZ9N3ksWACffhqe6V5Tu3bVE0ai127dQsJrrFIqYgKam9lO7v5ttLHWQJNsC/Lmm6G8XcVL+aF583DVcPDB4ZkcRx6Z64gkX7VqBd//fhhq2rw5VNJ/8UVoeBLvdc6cULy8enX87VcVtVXdGV/bXfPxxnv1Cl35t26d2eNQX8kkiBJgspk9DhhwPjA2k0HlqwkTQpPA2rpmkOxq1ix0Sy7SUC1awO67h6Eu33773WTyxRehyXCiu+VXrow/vaaCgtClzAknhOGgg3LfgiuZSur/NrMy4HhCn0yvAfWojtoxbNkS+iI65ZT8y/Iikh077RT6vNpjj9S24769on3dOpg1K3TbM2lSKCqF0ET7+OO3J4wePVKPv76SvY1lOSE5/AT4FJiYsYjy1NSpoeM4FS+JSKrMthdLde4cEs7gwWHe0qWhmXZVwnjmmTB9v/22J4ujj87cg8OqxZmoktrMvg8Mj4YVwF+A6909L68eMl1Jffnl4aawigq1bhCR7HCvfnUxdWq46mjRIjwBsiphFBc3/K71Bt0oZ2ZbgbeAi6puijOzhZ6nz6POZILYsiVc3h1xBPz1rxnZhYhInTZsCM12J00KvRJ/+GFIIgcfHB4X3BANbcU0FDgbmGJmrwLjCJXUTc4774SKKBUviUguFRSE56Qceyz87nfhJsHJkxveRXxdEiYId38OeM7MdgaGANcAu5jZg8Cz7v73zISUfyZMCF9MVRmhiEg+KCyEs87K3PbrbETl7uvc/Wl3PxXoAXwI/CJzIeWXrVth4sTQa2s2KoVERPJFvVrZuvtqdx+Tbz25ZtL774euuVW8JCJNjTrSrcOECeEuzFNOyXUkIiLZpQRRC/eQIE48MXRnLSLSlGQ0QZjZIDP7yMwWmNmoOPP3NLMpZvahmc00s8HR9CIzW29mM6LhT5mMM5HS0vBQehUviUhTlLEHAppZc2A0cALhIUPTzOwFd58bs9hNwHh3f9DMegMvA0XRvE9qPuo02yZMCDefnHZaLqMQEcmNTF5BDAAWuPtCd99IuI9iSI1lHGgfjXcAlmYwnnqpKl46/ng9mUpEmqZMJojuwOcx78ujabFuAc41s3LC1cOVMfN6RUVPb5rZwHg7MLORZlZqZqUVFRVpDD088H7hQhUviUjTletK6uHAE+7eAxgM/NnMmgHLgD2jBxX9H+BpM2tfc+WoyW2xuxd37do1rYFNmBCeNzCk5jWPiEgTkckEsQSI7RS3RzQt1kXAeAB3fxcoAArd/Vt3XxlNnw58AsR51EdmuIc+l445JtypKCLSFGUyQUwD9jGzXmbWitCv0ws1lvkMOA7AzPYnJIgKM+saVXJjZt8D9gEWZjDWambNgvnzVbwkIk1bxloxuftmM7uC8ICh5sBj7j7HzG4FSt39BeA64GEzu5ZQYX2+u7uZHQncamabgK3Ape6+KlOx1jRhQniS0+mnZ2uPIiL5J2F3341NOrv77t07PJD8jTfSsjkRkbxVW3ffua6kzjtz58K8eSpeEhFRgqhhwoTwOEAVL4lIU6cEUcOECeHJcbvtlutIRERySwkixkcfhRZMKl4SEVGCqGbixPA6dGhu4xARyQdKEDEmTIDDDoMePXIdiYhI7ilBRD75BD78UMVLIiJVlCAiVcVLZ5yR2zhERPKFEkRkwgT44Q+hZ89cRyIikh+UIIDFi2HaNBUviYjEUoJAxUsiIvEoQRCKlw4+GPbaK9eRiIjkjyafID7/HN59V8VLIiI1Zay778aiWzf429/gwANzHYmISH5p8gmiVSsYPDjXUYiI5J8mX8QkIiLxKUGIiEhcShAiIhKXEoSIiMSlBCEiInEpQYiISFxKECIiEpcShIiIxKUEISIicSlBiIhIXEoQIiISlxKEiIjEpQQhIiJxKUGIiEhcShAiIhKXEoSIiMSlBCEiInEpQYiISFxKECIiEldGn0ltZoOAPwDNgUfc/c4a8/cExgIdo2VGufvL0bxfAhcBW4Cr3P21TMYqIg23adMmyiWt6TkAABGWSURBVMvL2bBhQ65DkQQKCgro0aMHLVu2THqdjCUIM2sOjAZOAMqBaWb2grvPjVnsJmC8uz9oZr2Bl4GiaPxs4ABgd+B1M/u+u2/JVLwi0nDl5eW0a9eOoqIizCzX4UgN7s7KlSspLy+nV69eSa+XySKmAcACd1/o7huBccCQGss40D4a7wAsjcaHAOPc/Vt3/xRYEG1PRPLQhg0b6NKli5JDnjIzunTpUu8rvEwmiO7A5zHvy6NpsW4BzjWzcsLVw5X1WBczG2lmpWZWWlFRka64RaQBlBzyW0O+n1xXUg8HnnD3HsBg4M9mlnRM7j7G3Yvdvbhr164ZC1JEpCnKZIJYAuwR875HNC3WRcB4AHd/FygACpNcV0QaqZISKCqCZs3Ca0lJattbuXIlffv2pW/fvuy6665079592/uNGzcmtY0LLriAjz76qNZlRo8eTUmqwTYimWzFNA3Yx8x6EU7uZwPn1FjmM+A44Akz25+QICqAF4CnzexeQiX1PsAHGYxVRLKkpARGjoTKyvB+8eLwHmDEiIZts0uXLsyYMQOAW265hbZt23L99ddXW8bdcXeaNYv/f/Hjjz9e534uv/zyhgXYSGXsCsLdNwNXAK8B8witleaY2a1mdlq02HXAz8ysDHgGON+DOYQri7nAq8DlasEksmO48cbtyaFKZWWYnm4LFiygd+/ejBgxggMOOIBly5YxcuRIiouLOeCAA7j11lu3LXvEEUcwY8YMNm/eTMeOHRk1ahR9+vThsMMO48svvwTgpptu4r777tu2/KhRoxgwYAD77rsv77zzDgDr1q3jjDPOoHfv3gwbNozi4uJtySvWzTffzA9/+EN+8IMfcOmll+LuAHz88ccce+yx9OnTh379+rFo0SIA7rjjDg488ED69OnDjZk4WHFktA7C3V929++7+17ufns07dfu/kI0PtfdD3f3Pu7e193/HrPu7dF6+7r7K5mMU0Sy57PP6jc9Vf/+97+59tprmTt3Lt27d+fOO++ktLSUsrIyJk2axNy5c7+zzpo1azjqqKMoKyvjsMMO47HHHou7bXfngw8+4O67796WbB544AF23XVX5s6dy69+9Ss+/PDDuOteffXVTJs2jVmzZrFmzRpeffVVAIYPH861115LWVkZ77zzDrvssgsvvvgir7zyCh988AFlZWVcd911aTo6tct1JbWINDF77lm/6anaa6+9KC4u3vb+mWeeoV+/fvTr14958+bFTRCtW7fmpJNOAqB///7b/ouvaejQod9Z5u233+bss88GoE+fPhxwwAFx1508eTIDBgygT58+vPnmm8yZM4fVq1ezYsUKTj31VCDc3NamTRtef/11LrzwQlq3bg1A586d638gGkAJQkSy6vbboU2b6tPatAnTM2HnnXfeNj5//nz+8Ic/8MYbbzBz5kwGDRoU996AVq1abRtv3rw5mzdvjrvtnXbaqc5l4qmsrOSKK67g2WefZebMmVx44YV5eRe6EoSIZNWIETBmDPTsCWbhdcyYhldQ18fXX39Nu3btaN++PcuWLeO119Lfg8/hhx/O+PHjAZg1a1bcK5T169fTrFkzCgsL+eabb5g4cSIAnTp1omvXrrz44otAuAGxsrKSE044gccee4z169cDsGrVqrTHHU9G+2ISEYlnxIjsJISa+vXrR+/evdlvv/3o2bMnhx9+eNr3ceWVV/LTn/6U3r17bxs6dOhQbZkuXbpw3nnn0bt3b3bbbTcOOeSQbfNKSkq45JJLuPHGG2nVqhUTJ07klFNOoaysjOLiYlq2bMmpp57Kb3/727THXpNV1Zw3dsXFxV5aWprrMESapHnz5rH//vvnOoy8sHnzZjZv3kxBQQHz58/nxBNPZP78+bRokfv/x+N9T2Y23d2L4y2f+4hFRHYga9eu5bjjjmPz5s24Ow899FBeJIeGaJxRi4jkqY4dOzJ9+vRch5EWqqQWEZG4lCBERCQuJQgREYlLCUJEROJSghCRRu+YY475zk1v9913H5dddlmt67Vt2xaApUuXMmzYsLjLHH300dTVhP6+++6jMqYHwsGDB/PVV18lE3peU4IQkUZv+PDhjBs3rtq0cePGMXz48KTW33333ZkwYUKD918zQbz88st07NixwdvLF2rmKiJpdc01EKd365T07QtRL9txDRs2jJtuuomNGzfSqlUrFi1axNKlSxk4cCBr165lyJAhrF69mk2bNnHbbbcxZMiQausvWrSIU045hdmzZ7N+/XouuOACysrK2G+//bZ1bwFw2WWXMW3aNNavX8+wYcP4zW9+w/3338/SpUs55phjKCwsZMqUKRQVFVFaWkphYSH33nvvtt5gL774Yq655hoWLVrESSedxBFHHME777xD9+7def7557d1xlflxRdf5LbbbmPjxo106dKFkpISunXrxtq1a7nyyispLS3FzLj55ps544wzePXVV7nhhhvYsmULhYWFTJ48OaXjrgQhIo1e586dGTBgAK+88gpDhgxh3LhxnHnmmZgZBQUFPPvss7Rv354VK1Zw6KGHctpppyV8RvODDz5ImzZtmDdvHjNnzqRfv37b5t1+++107tyZLVu2cNxxxzFz5kyuuuoq7r33XqZMmUJhYWG1bU2fPp3HH3+c999/H3fnkEMO4aijjqJTp07Mnz+fZ555hocffpgzzzyTiRMncu6551Zb/4gjjuC9997DzHjkkUe46667+P3vf89vf/tbOnTowKxZswBYvXo1FRUV/OxnP2Pq1Kn06tUrLf01KUGISFrV9p9+JlUVM1UliEcffRQIz2y44YYbmDp1Ks2aNWPJkiUsX76cXXfdNe52pk6dylVXXQXAQQcdxEEHHbRt3vjx4xkzZgybN29m2bJlzJ07t9r8mt5++21OP/30bT3KDh06lLfeeovTTjuNXr160bdvXyBxl+Ll5eWcddZZLFu2jI0bN9KrVy8AXn/99WpFap06deLFF1/kyCOP3LZMOroEb/J1EOl+Nq6I5MaQIUOYPHky//rXv6isrKR///5A6PyuoqKC6dOnM2PGDLp169agrrU//fRT7rnnHiZPnszMmTM5+eSTU+qiu6qrcEjcXfiVV17JFVdcwaxZs3jooYey3iV4k04QVc/GXbwY3Lc/G1dJQqTxadu2LccccwwXXnhhtcrpNWvWsMsuu9CyZUumTJnC4sWLa93OkUceydNPPw3A7NmzmTlzJhC6Ct95553p0KEDy5cv55VXtj/osl27dnzzzTff2dbAgQN57rnnqKysZN26dTz77LMMHDgw6c+0Zs0aunfvDsDYsWO3TT/hhBMYPXr0tverV6/m0EMPZerUqXz66adAeroEb9IJIpvPxhWRzBs+fDhlZWXVEsSIESMoLS3lwAMP5Mknn2S//fardRuXXXYZa9euZf/99+fXv/71tiuRPn36cPDBB7PffvtxzjnnVOsqfOTIkQwaNIhjjjmm2rb69evH+eefz4ABAzjkkEO4+OKLOfjgg5P+PLfccgs/+clP6N+/f7X6jZtuuonVq1fzgx/8gD59+jBlyhS6du3KmDFjGDp0KH369OGss85Kej+JNOnuvps1C1cONZnB1q1pCkykCVB3341Dfbv7btJXENl+Nq6ISGPSpBNEtp+NKyLSmDTpBJHLZ+OK7Gh2lOLqHVVDvp8mfx9Erp6NK7IjKSgoYOXKlXTp0iXhDWiSO+7OypUrKSgoqNd6TT5BiEjqevToQXl5ORUVFbkORRIoKCigR48e9VpHCUJEUtayZcttd/DKjqNJ10GIiEhiShAiIhKXEoSIiMS1w9xJbWYVQO2drORWIbAi10HUQvGlRvGlRvGlJpX4erp713gzdpgEke/MrDTR7ez5QPGlRvGlRvGlJlPxqYhJRETiUoIQEZG4lCCyZ0yuA6iD4kuN4kuN4ktNRuJTHYSIiMSlKwgREYlLCUJEROJSgkgTM9vDzKaY2Vwzm2NmV8dZ5mgzW2NmM6Lh1zmIc5GZzYr2/51H8Flwv5ktMLOZZtYvi7HtG3NsZpjZ12Z2TY1lsnoMzewxM/vSzGbHTOtsZpPMbH702inBuudFy8w3s/OyGN/dZvbv6Pt71sw6Jli31t9CBuO7xcyWxHyHgxOsO8jMPop+i6OyGN9fYmJbZGYzEqybjeMX97yStd+gu2tIwwDsBvSLxtsBHwO9ayxzNPBSjuNcBBTWMn8w8ApgwKHA+zmKsznwBeEmnpwdQ+BIoB8wO2baXcCoaHwU8N9x1usMLIxeO0XjnbIU34lAi2j8v+PFl8xvIYPx3QJcn8T3/wnwPaAVUFbz7ylT8dWY/3vg1zk8fnHPK9n6DeoKIk3cfZm7/ysa/waYB3TPbVQNMgR40oP3gI5mtlsO4jgO+MTdc3p3vLtPBVbVmDwEGBuNjwV+HGfV/wAmufsqd18NTAIGZSM+d/+7u2+O3r4H1K+P5zRKcPySMQBY4O4L3X0jMI5w3NOqtvgsPNjiTOCZdO83WbWcV7LyG1SCyAAzKwIOBt6PM/swMyszs1fM7ICsBhY48Hczm25mI+PM7w58HvO+nNwkurNJ/IeZ62PYzd2XReNfAN3iLJMvx/FCwhVhPHX9FjLpiqgI7LEExSP5cPwGAsvdfX6C+Vk9fjXOK1n5DSpBpJmZtQUmAte4+9c1Zv+LUGTSB3gAeC7b8QFHuHs/4CTgcjM7Mgcx1MrMWgGnAX+NMzsfjuE2Hq7l87KtuJndCGwGShIskqvfwoPAXkBfYBmhGCcfDaf2q4esHb/aziuZ/A0qQaSRmbUkfIkl7v6/Nee7+9fuvjYafxloaWaF2YzR3ZdEr18CzxIu5WMtAfaIed8jmpZNJwH/cvflNWfkwzEEllcVu0WvX8ZZJqfH0czOB04BRkQnkO9I4reQEe6+3N23uPtW4OEE+8318WsBDAX+kmiZbB2/BOeVrPwGlSDSJCqvfBSY5+73Jlhm12g5zGwA4fivzGKMO5tZu6pxQmXm7BqLvQD8NGrNdCiwJuZSNlsS/ueW62MYeQGoahFyHvB8nGVeA040s05REcqJ0bSMM7NBwH8Bp7l7ZYJlkvktZCq+2Dqt0xPsdxqwj5n1iq4ozyYc92w5Hvi3u5fHm5mt41fLeSU7v8FM1sA3pQE4gnCZNxOYEQ2DgUuBS6NlrgDmEFpkvAf8KMsxfi/ad1kUx43R9NgYDRhNaEEyCyjOcow7E074HWKm5ewYEhLVMmAToQz3IqALMBmYD7wOdI6WLQYeiVn3QmBBNFyQxfgWEMqeq36Hf4qW3R14ubbfQpbi+3P025pJONHtVjO+6P1gQqudT7IZXzT9iarfXMyyuTh+ic4rWfkNqqsNERGJS0VMIiISlxKEiIjEpQQhIiJxKUGIiEhcShAiIhKXEoRIHcxsi1XvZTZtPYuaWVFsT6Ii+aRFrgMQaQTWu3vfXAchkm26ghBpoOh5AHdFzwT4wMz2jqYXmdkbUWd0k81sz2h6NwvPZyiLhh9Fm2puZg9H/f3/3cxaR8tfFT0HYKaZjcvRx5QmTAlCpG6taxQxnRUzb427Hwj8EbgvmvYAMNbdDyJ0lHd/NP1+4E0PHQ32I9yBC7APMNrdDwC+As6Ipo8CDo62c2mmPpxIIrqTWqQOZrbW3dvGmb4IONbdF0Ydqn3h7l3MbAWh+4hN0fRl7l5oZhVAD3f/NmYbRYQ++/eJ3v8CaOnut5nZq8BaQo+1z3nUSaFItugKQiQ1nmC8Pr6NGd/C9rrBkwn9YvUDpkU9jIpkjRKESGrOinl9Nxp/h9D7KMAI4K1ofDJwGYCZNTezDok2ambNgD3cfQrwC6AD8J2rGJFM0n8kInVrbdUfXP+qu1c1de1kZjMJVwHDo2lXAo+b2f8FKoALoulXA2PM7CLClcJlhJ5E42kOPBUlEQPud/ev0vaJRJKgOgiRBorqIIrdfUWuYxHJBBUxiYhIXLqCEBGRuHQFISIicSlBiIhIXEoQIiISlxKEiIjEpQQhIiJx/X8TyxAdyKn39wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.clf()  # 清空图像\n",
    "%matplotlib inline\n",
    "\n",
    "acc = history_dict['acc']\n",
    "val_acc = history_dict['val_acc']\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 15000 samples, validate on 10000 samples\n",
      "Epoch 1/4\n",
      "15000/15000 [==============================] - 3s - loss: 0.5157 - acc: 0.7895 - val_loss: 0.4009 - val_acc: 0.8653\n",
      "Epoch 2/4\n",
      "15000/15000 [==============================] - 2s - loss: 0.3146 - acc: 0.9029 - val_loss: 0.3247 - val_acc: 0.8787\n",
      "Epoch 3/4\n",
      "15000/15000 [==============================] - 1s - loss: 0.2319 - acc: 0.9246 - val_loss: 0.2808 - val_acc: 0.8923\n",
      "Epoch 4/4\n",
      "15000/15000 [==============================] - 1s - loss: 0.1816 - acc: 0.9430 - val_loss: 0.2729 - val_acc: 0.8905\n",
      "24832/25000 [============================>.] - ETA: 0s评估： [0.2911555328845978, 0.88292]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[0.92948604],\n",
       "       [0.82596517],\n",
       "       [0.99709994],\n",
       "       ...,\n",
       "       [0.77651155],\n",
       "       [0.02592379],\n",
       "       [0.8645573 ]], dtype=float32)"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "from keras.datasets import imdb\n",
    "from keras import models\n",
    "from keras import layers\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "\n",
    "# 字符转换向量函数\n",
    "def vectorize_sequences(sequences, dimension=10000):\n",
    "    results = np.zeros((len(sequences), dimension))\n",
    "    for i, sequence in enumerate(sequences):\n",
    "        results[i, sequence] = 1.\n",
    "    return results\n",
    "\n",
    "# 加载数据\n",
    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)\n",
    "\n",
    "# 对数据和标签进行向量化\n",
    "x_train = vectorize_sequences(train_data)\n",
    "x_test = vectorize_sequences(test_data)\n",
    "y_train = np.asarray(train_labels).astype('float32')\n",
    "y_test = np.asarray(test_labels).astype('float32')\n",
    "\n",
    "# 构建网络\n",
    "model = models.Sequential()\n",
    "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
    "model.add(layers.Dense(16, activation='relu'))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))\n",
    "\n",
    "# 编译模型\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])\n",
    "\n",
    "# 留出验证数据\n",
    "x_val = x_train[:10000]  # 验证用\n",
    "partial_x_train = x_train[10000:]  # 训练用\n",
    "\n",
    "y_val = y_train[:10000]  # 验证用\n",
    "partial_y_train = y_train[10000:]  # 训练用\n",
    "\n",
    "# 训练\n",
    "history = model.fit(partial_x_train, partial_y_train, epochs=4, batch_size=512, validation_data=(x_val, y_val))\n",
    "\n",
    "# 评估\n",
    "results = model.evaluate(x_test, y_test)\n",
    "print('评估：', results)\n",
    "\n",
    "# 预测\n",
    "model.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**常见的网络拓扑结构**\n",
    "* 线性堆叠(Sequential)的网络\n",
    "* 双分支(two-brach)网络\n",
    "* 多头(multihead)网络\n",
    "* Inception模块\n",
    "\n",
    "**选择损失函数的一些简单的原则**\n",
    "* 二分类问题，可以使用二元交叉墒(binary crossentropy)\n",
    "* 多元分类问题，可以使用分类交叉墒(categorical crossentropy)\n",
    "* 回归问题，可以使用均方误差(mean-squared error)\n",
    "* 序列学习问题，可以使用联结主义时序分类(CTC)\n",
    "\n",
    "**本例中，单词序列可以编码为二进制向量，但也有其他编码方式**  \n",
    "**对于二分类问题的sigmoid标量输出，应该使用binary_crossentropy损失函数**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}