单变量线性回归.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 前情函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## GPU相关语句"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 GeForce GTX 960M 0\n"
     ]
    }
   ],
   "source": [
    "'''以下通过先判断GPU是否存在，\n",
    "   在通过对应的语句打印出GPU的信息\n",
    "'''\n",
    "\n",
    "if torch.cuda.is_available():\n",
    "    # 返回gpu数量\n",
    "    GPU_num = torch.cuda.device_count()\n",
    "    # 返回gpu名字，设备索引默认从0开始\n",
    "    GPU_name = torch.cuda.get_device_name(0)\n",
    "    # 返回当前设备索引\n",
    "    GPU_index = torch.cuda.current_device()\n",
    "    print(GPU_num, GPU_name, GPU_index)\n",
    "else:\n",
    "    print('Use CPU for work')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面语句用于将模型放在GPU上训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cuda', index=0)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "device"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## torch.unsqueeze 与 torch.squeeze "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "torch.unsqueeze(input, dim) → Tensor  \n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;返回插入指定位置尺寸为1的新张量\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;返回的张量与此张量共享相同的基础数据\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;可以通过`dim`控制指定的轴向\n",
    "\n",
    "这个函数主要是对数据维度进行扩充。给指定位置加上维数为1的维度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x0 维度: torch.Size([4]) \n",
      " tensor([1, 2, 3, 4])\n",
      "x1 维度: torch.Size([1, 4]) \n",
      " tensor([[1, 2, 3, 4]])\n",
      "x2 维度: torch.Size([4, 1]) \n",
      " tensor([[1],\n",
      "        [2],\n",
      "        [3],\n",
      "        [4]])\n"
     ]
    }
   ],
   "source": [
    "x0 = torch.tensor([1, 2, 3, 4])\n",
    "print('x0 维度:', x0.size(),'\\n', x0)\n",
    "\n",
    "x1 = torch.unsqueeze(x0, 0)\n",
    "print('x1 维度:', x1.size(),'\\n', x1)\n",
    "\n",
    "x2 = torch.unsqueeze(x0, 1)\n",
    "print('x2 维度:', x2.size(),'\\n', x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "torch.squeeze(input, dim=None, out=None) → Tensor\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;移除输入张量维度为1的轴\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;例如输入张量的shape为: $(A \\times 1 \\times B \\times C \\times 1 \\times D)$输出张量的shape则为:$(A \\times B \\times C \\times D)$\n",
    "\n",
    "&ensp;&ensp;&ensp;&ensp;可以通过`dim`控制指定的轴向，例如输入张量的shape为: $(A \\times 1 \\times B)$，通过`squeeze(input, 1)`，输出张量的shape为: $(A \\times B)$\n",
    "\n",
    "\n",
    "这个函数主要对数据的维度进行压缩，去掉维数为1的的维度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x0 维度: torch.Size([2, 1, 2, 1, 2])\n",
      "x1 维度: torch.Size([2, 2, 2])\n",
      "x2 维度: torch.Size([2, 1, 2, 1, 2])\n",
      "x3 维度: torch.Size([2, 2, 1, 2])\n"
     ]
    }
   ],
   "source": [
    "x0 = torch.zeros(2, 1, 2, 1, 2)\n",
    "print('x0 维度:', x0.size())\n",
    "\n",
    "x1 = torch.squeeze(x0)\n",
    "print('x1 维度:', x1.size())\n",
    "\n",
    "x2 = torch.squeeze(x0, 0)\n",
    "print('x2 维度:', x2.size())\n",
    "\n",
    "x3 = torch.squeeze(x0, 1)\n",
    "print('x3 维度:', x3.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 正式开始"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 导入必要模块  \n",
    "安装`pytorch`时，不会安装`torchsummary`，需要手动通过`!pip install torchsummary`安装，如果不在`jupyter`中，则去掉感叹号进行安装"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.6.0'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch\n",
    "from torchsummary import summary\n",
    "import torch.nn.functional as F\n",
    "import matplotlib.pyplot as plt\n",
    "import pylab as pl\n",
    "from IPython import display\n",
    "\n",
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 构建数据集  \n",
    "本案例中将以一个正弦函数$y=sin(x)$为例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果使用多个GPU，应该使用`torch.cuda.manual_seed_all()`为所有的GPU设置种子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPU is available\n"
     ]
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    torch.cuda.manual_seed(1) #为当前GPU设置随机种子用于生成随机数，以使得结果是确定的\n",
    "    print('GPU is available')\n",
    "else:\n",
    "    torch.manual_seed(1)      #为CPU设置随机种子用于生成随机数，以使得结果是确定的\n",
    "    print('Use CPU for work')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`squeeze`是删除维度为1的轴，`unsqueeze`则是增加一个维度为1的轴，通过`dim`控制增加轴的方向，其中可以通过`x.size()`获取`x`的维度为`torch.Size([100, 1])`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "PI = math.pi\n",
    "\n",
    "x = torch.unsqueeze(torch.linspace(-PI, PI, 200), dim=1)\n",
    "\n",
    "y = torch.sin(x) + 0.07 * torch.randn(x.size())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAA500lEQVR4nO3dd3yTVRfA8d/tHjJkONkOZJehyKsgAhYoWhEF91aGLQoOpqK4WIIooAIKiiIu0EZFqYDIKyqKWECGyBZ8QWZBWlranvePp2nTkC6aNElzvp9PPjTJ0+e5acvJzbn3nmtEBKWUUhVfkLcboJRSqnxowFdKqQChAV8ppQKEBnyllAoQGvCVUipAhHi7AYWpUaOG1KtXz9vNUEopv/Lrr78eEJGarp7z2YBfr149Vq1a5e1mKKWUXzHG7CzsOU3pKKVUgNCAr5RSAUIDvlJKBQifzeG7cvLkSXbv3s2JEye83ZSAFRERQa1atQgNDfV2U5RSpeRXAX/37t1UqlSJevXqYYzxdnMCjohw8OBBdu/eTf369b3dHKVUKflVSufEiRNUr15dg72XGGOoXr26fsJSyk/5VcAHNNh7mf78lSs2GyQmWv8q3+VXKR2llG+w2SA5GWJjrfu33gppaTB7NsybB/HxpT9PSb9HnT6/6+H7gn379nHbbbfRoEEDWrduTbt27fj000/L7fo7duygadOmLFq0iJiYGGJiYjjjjDNo2LAhMTEx3HXXXSU6T0pKCgsXLsy7/8wzz/DSSy95qtmqgrDZrAA/bZr17/TpVrAH69/k5KK/1/5JwPE8vXtDjx76CcHTtIdfSiJCz549ufvuu3n//fcB2LlzJzanv9SsrCxCQjz74+3atStdu3YFoGPHjrz00ku0adOmwDHZ2dkEBwe7/P6UlBRWrVpFXFycR9up/JtzLzw5uWCA37cPoqKsr6Oi8nv9rs7j+EmgY8f882RmwsKFsGxZ6T4hqNLRHn4pLV26lLCwMPr375/3WN26dRk4cCBvv/028fHxdOrUic6dO3Po0CF69uxJ8+bNufzyy1m7di1wak+6adOm7Nixgx07dtCoUSMefPBBmjRpQmxsLOnp6QD8+uuvtGjRghYtWjBt2rQi21ivXj2GDh1Kq1at+Pjjj+nYsWNemYoDBw5Qr149MjMzGTVqFB9++CExMTF8+OGHAGzYsIGOHTvSoEEDXn31Vbf+7JT/ce7N22xWQA8Lyz/m999h0CBISCg6WDu/UYD1BuEoLc36xKDjAZ7hvz38QYMgJcW954yJgcmTizxk/fr1tGrVqtDnV69ezdq1a6lWrRoDBw6kZcuWfPbZZyxdupS77rqLlGLa/OeffzJv3jxmzpxJnz59mD9/PnfccQf33nsvU6dOpUOHDjzxxBPFvpTq1auzevVqAN54441Tng8LC+PZZ59l1apVTJ06FbDeiDZt2sS3337LsWPHaNiwIQMGDNA59wHMOUgnJ8PUqdCli9UjB8jIgNRU6/GixMZaPXv7J4F+/azb9OmwZIl1nrCw/K9LOx6giqc9/DJKSEigRYsWXHrppQBcc801VKtWDYDvv/+eO++8E4BOnTpx8OBBjh49WuT56tevT0xMDACtW7dmx44dHDlyhCNHjtChQweAvHMW5eabbz6t19OjRw/Cw8OpUaMGZ511Fvv27Tut86iKITY2vxfumK7p18/1467YbFZ+fvr0Uz8JxMfDl1/CRx9Zj3fpYgV7KH48wNV19JNB0dzSwzfGzAKuBf4RkaYunjfAK0AckAbcIyKry3TRYnrintKkSRPmz5+fd3/atGkcOHAgL3ceHR1d7DlCQkLIycnJu+84rz08PDzv6+Dg4LyUTmk5tsPxesXNoXe+flZW1mldX/k3x7z9vHmnzqSJj3f9uKvz9O5t5ejB6r1/9NGpx9uDv81m5fGLGw9wdZ3TnSkUSNzVw38b6FbE892Bi3JvfYHX3XTdctepUydOnDjB66/nv4Q0+2deJ+3bt2fu3LkALFu2jBo1alC5cmXq1auXl25ZvXo127dvL/KaVatWpWrVqnz//fcAeecsqXr16vHrr78C8Mknn+Q9XqlSJY4dO1aqc6mKb+RIuPHG/Lw9WOkaV0Ha1eOOkpPzgz1Yvfeieu32N5LixgNcXaekM4UCmVt6+CKy3BhTr4hDrgfmiIgAPxljqhpjzhWR/7nj+uXJGMNnn33G4MGDGT9+PDVr1iQ6Oppx48ad0ht/5plnuO+++2jevDlRUVG88847ANx4443MmTOHJk2a0LZtWy6++OJirzt79mzuu+8+jDHElrTbk+vxxx+nT58+zJgxgx49euQ9fvXVVzN27FhiYmIYPnx4qc6pKiabDcaNg+xs6749eJYo8B48CP/9rzWKu307bN/OCzsOMcBkESxZnCCCvUHn0Wjz+fBifWjbFi69FCpXLnAae2+/NJzHB0r5XyRgGCsGu+FEVsD/opCUzhfAWBH5Pvf+EmCoiKxyOq4v1icA6tSp03rnzoJ1/Ddu3EijRo3c0l51+vT3UHElJlo9e7uQEJg/v5AALGJNnJg713pXWLcu76lD4edwrGYDUkNqEBQeyqGjwYRnp9Gw0t9UPf437N1rHWiMNVnixhut3E8JOj+F0UVcFmPMryLSxtVzPjVLR0RmADMA2rRp4553IqVUiTn2lIODYcgQF8Hz2DGYMcM6cP16CA2Fq65iY/ObeejDq1iZ1Yr0jCjYbR0eFeUiPXP4MPz8M6xcCYsWwZNPWrdWreDhh+GWW8BhPKkkTueTQaApr1k6e4DaDvdr5T6mlPIhjjn0BQusrEvezJcDB/jj5lGk1awDjz8OVarA66/z1az/kdjwGx4/PJJlWVeSTsHJ9c45dZsNEp86E1tGVxg1ClasgL/+siZiZGbCPfdAvXrw4otw/HiBc+lMnDISEbfcgHrA74U81wP4CjDA5cDPxZ2vdevW4mzDhg2nPKbKn/4eAkNSkkhUlEgoGTIidLykh1USAfmEXnJF+C+SlJR/DIiEhYmEhFhfO96ioqzjHM9pPz4uLv85ERHJyRFJThbp2tU66NxzZfXAtyRxQJaMGJH/vY7nVAUBq6SQuOquaZnzgI5ADWPMbuBpIDT3DeUNYCHWlMwtWNMy73XHdZVSnpOcDFemLeJVHqbhyc0sq3QtD2WOYyONwWG2jWN5BMcqHTEx1oIsx5y642wal+UUjIFrrsGWfg1/Rv9A75WP0XLK/dzPq/QLepO0nDZ517RfX/P2pVDYO4G3b9rD9136ewgAR4/Kji73i4D8wUVyQ/iXctNN+T14ew/bscdekl634/GOt4QE18cEB+VIHz6Q3ZwnJwmWF8xICeOEREVJqXr8SUnWNQLhUwFF9PC9HtgLu2nA9136e6jgVqwQadBAxBjZfOMweaT/iQLBNTjYCrZ2pQ2mSUlWKic8/NQ3j4QE6znHN4OQEJEqHJY5wfeIgOyu3kyWTN0gCQmFv2k4Xy+QUkFFBXwtrVBKwcHBxMTE0KRJE1q0aMHEiRMLrJp1ZceOHXmVNd1p8uTJhS76UqrURODll6F9e8jJgeXLueiTMUx+PZzU1PxUTHa2laqxK8kCLEfO5RTmzbMetxdpW7w4f4JOVJQ1U+iOhKpsGjqb13t8QY2sfXQadhn3RH1UovIOuijLQWHvBN6++WoPPzo6Ou/rffv2SefOnWXUqFFFfs+3334rPXr0cHtb6tatK/v373f7eYvjC78H5WbHj4vcfrvVDe7VSyQ1tcDTnu4lO/fW4+IKfmpwvP4FEbvl4CX/EQHZct0gGdg/s9h0jvbwNaVzWhwDvojI1q1bpVq1apKTkyPbt2+XK6+8Ulq2bCktW7aUFStWiIhI27ZtpXLlytKiRQuZNGlSocf9/fff0r59e2nRooU0adJEli9fLiIiixYtkssvv1xatmwpN910kxw7dkxeeeUVCQ0NlaZNm0rHjh3L9WfgC78H5UZ79oi0bClijMjzz4tkZ7s8zJN58MKCcmFpnoH9M0UeeUQEZG+rbjL4wWOaw88V0AHf3b9o54AvIlKlShXZu3evHD9+XNLT00VEZPPmzWJ/Dc49/MKOe+mll+T5558XEZGsrCw5evSo7N+/X9q3by///vuviIiMHTtWRo8eLSLaw1fFK/bvf9Mmkbp1Rc44Q3568nOvBkXntjpP4XTO+YuI/JYwU04SLKtoJQ0i/y7wvXFxLqZ9BoCiAr5PrbR1t/KuoHfy5EkSExNJSUkhODiYzZs3l+q4Sy+9lPvuu4+TJ0/Ss2dPYmJi+O6779iwYQNXXHEFAJmZmbRr185zL0JVGMX+/f/8szWPMiiI70YvI+6p1l6tNum8UtZ5CmdcHNSvn5+rT0yE7dsfIIfz+ZjeLElvx5yPkoGLS1ShMxBV6EHb8his2bZtG8HBwZx11lm8/PLLnH322axZs4ZVq1aR6Vgm0EFhx3Xo0IHly5dz/vnnc8899zBnzhxEhGuuuYaUlBRSUlLYsGEDb731lvtfiKpwCvv7t9nglRuWkXVVJ6tw2YoVjF/S2ucGNp1r8ffrl7/JiuMA77fh3enAcqJJY8jCq1jzwcZTKnSOHKmrc6GCB/zCNm9wl/3799O/f38SExMxxpCamsq5555LUFAQ7777Ltm5JQedyxAXdtzOnTs5++yzefDBB3nggQdYvXo1l19+OStWrGDLli0AHD9+PO8TgZY3VkWpUsUqfgb5f/82G0zr8x0PfNaDzRl1WfT0D9g2XsTixfnfFx7uG9UmCyuV7Nzz79wZ/pPQinVTlhERJgxZ2JGYkN8LnOv33/O3aAxoheV6vH3z1Rx+UFCQtGjRQho3bizNmzeXCRMmSHbuINfmzZulWbNm0rx5cxkyZEhevj8zM1Ouvvpqad68uUyaNKnQ495++21p0qSJxMTEyJVXXinbtm0TEZElS5ZImzZtpFmzZtKsWTNJyn0xr776qlx88cU6aKtOUWDxksO8+Uk3fCf/EiXraSRnsVcSElzPkPFlRc662bRJ5Lzz5ESl6jLgyrVSt27B11bYXP2KhEAetFXup78H3+dyUdKPP8rJiGjZaC6Rs/lfXi0bf6xRU2RH7s8/Rc47T+Scc+Sb6Vv97rWVVVEBv0KndJQKVM7pzF6NNkKPHoScfw47Z39L67hzMMaqZTN58ql7zfq6Ihd7XXghfPMNZGbSZWwXFkz9u8ACr0CutqkBX6kKyDH//emrf9FpbCyEhUFyMl3vPof69QtuFp6aWrrVsr4qr3zylsbw1Vfwzz90ndSVqc8eAvIHe3v1sgZyA43fBXzrE4vyFv35+4/4eJj67CFiJ3aFo0dZNuxrEic1wGbz/IQGb7BPQ7XvxWvbexkkJcHmzXDDDSz9KqNAeYjx4wOvp+9XAT8iIoKDBw9q0PESEeHgwYNERER4uymqCPZe7ucLTsJNN8HWrawYaqPHiBYFNiafN8+a296xo1eb6zYup6F27mwtLFi+nMc2PUhwUH7syMryjemn5cmvFl7VqlWL3bt3s3//fm83JWBFRERQq1YtbzdDFSJ/sZXQ6o0EyP4W3n2XeT9ddUowjI21atGnpTnVpPdThW5kftttsGULtZ9+mq87XET3H54iK6vifLIpDb8K+KGhodSvX9/bzVDK6wrbsNveyx3EZO7Lnsmi1iPoescdxFY+NRi66hH7c8C3j1u43BDlqafgzz/p8t4ofnz8Yt5OvzkwN00pbPqOt2+upmUqpYqeh56UJNIzfKFkY+TT4F6S9Gl2gecKq1UTEFMWT5wQueIKkchIkTVrvN0aj6GIaZlGfDQf3qZNG1m1apW3m6GUz0lMtAYm7RIS8ksOsG0bmc1b809EHda89gM9+kQXea7CPilUWHv3QqtWEBkJv/wC1ap5u0VuZ4z5VUTauHrOrwZtlVJFzLBJT4cbbyQsFGqtXFAg2OdNV3SalVLazUv83jnnwPz58Ndf/NPlNgY+lB1QM3U04CvlZ1zWmBGBAQMgJQXeew8uuCDv+FOmKwZQgHOpXTvWPDiVs35bRM3XRwfUz0QDvlJ+6JSe+ZtvwjvvwNNPQ48eBY7VLf5ONdP0ZRb38iTP0zZtacD8TDTgK+WDCkvBuLRhAzzyCHTpAqNGnfJ0RVxkVVaxsTA0cgp/0JD3uZ1rL/vH200qFzpoq5SPcdy4JCqqmPnxJ07AZZdZg5Fr1mD75VyXg7ABNzhbAjYb/P7+WoYuuIzgzldbO6sH+X8fuKhBW69PvyzsptMyVaByWelSCtm2LzFRBOTHp76UuDjX2wCqYrz2mvVDGz/e2y1xC7RaplL+w1UKxmaD3r2t6pYLF0KfPrBypA2mTmVr/GA6T4xj4cKCBdECJS9dZv37w403wogRsHKlt1vjURrwlfIxrmbhJCdTYNu+6hl7aDrxXmjZkinnjckblLXTXH3J2GyQONDwVa+ZcP75cMstVunQCkoDvlI+yHkWTmxudWMAQw5zzZ2Emwz44AM6dQ/P+0QQFmYVRPP3ujjlwXG66k0PnskbV80jZ+cudt70qLeb5jEa8JXyA/Hx8PHHVjB/vdEUOsq3hEx9BS6+GLAqXsbFWcd8+aUG+5Jwnq468P12jJWh1F08i5VPfeHdxnmIztJRyp9s3gwxMdCpE7YHP2f6DMOSJVbuvtgZPaoAx9lQwcFWjfwwMviFS6kbtZ8qu36H6tW93cxS09IKSlUE2dlw770QHs6iG2dw621GB2rLwHGsZOhQ6w0zk3D6RcyhUsYBayFEBeNX5ZGVCmiTJ8MPP8C77/L5T+fpQK0bxMdbN5stfyOYmJgYvlzyNNd98BSrat/A22l9Ksz6BU3pKOUPNm2yUjndusGnn2L73OSlI8LCrEW2/fpVjKBU3hxTO2FhYAxkZWTxo/kP9WUbTfmdY1HnMGiQNYHH14N/USkdDfhK+bqsLLjiCtiyBdavtyo+oqtn3cW53LTdJWzkN1ryNd24gU8JCTF5O2X58liJ5vCV8meTJsHPP1tRKTfYQwCWNvYQx4VuYWEQHm59vS2sEaODn6MnSdwUtICsLOtxfx4r0Ry+Ur5s61arAmbPnnDzzd5uTYXkvDUiwPTp1r8hzQfz15vzmJWeyIqczvwvvapfj5W4pYdvjOlmjPnDGLPFGDPMxfP3GGP2G2NScm8PuOO6SlVoItay/9BQqytvjLdbVGE5f1patswqYTHp1RC2DZ9JpfR/+KH9kIJ7EPihMgd8Y0wwMA3oDjQGbjXGNHZx6IciEpN7e7Os11Wqwps7FxYvZu1tY0kcc37AbNLhbc4Lsj7e1hoGD6Ze8kym9v7Ob4M9uKeHfxmwRUS2iUgm8AFwvRvOq1RAstlg6P0HyEgYzKGGl/OfOf11t6py5HL/gNGjoV496NvXKkntp9wR8M8H/nK4vzv3MWc3GmPWGmM+McbUdsN1lapw7FMEL5n1BEFHjzAgeAbH063/pv48WOhPXG4hGR1tJfY3b4YXXvB2E09bec3S+RyoJyLNgW+Ad1wdZIzpa4xZZYxZtX///nJqmlK+IzkZ2qYt5V7eZjxD+GRTs7znwsP9d7DQ37icARUbC3feCWPHwu+/e61tZVHmefjGmHbAMyLSNff+cAARGVPI8cHAIRGpUtR5dR6+qsgKm0P/xScnuKRPM0SgZdBajudE5j0XF2cVRlNedOAANGzIgXOb8sxVy6hS1fjcYixPz8P/BbjIGFPfGBMG3AIUyDQaY851uBsPbHTDdZXyS45leZ3z8tduGM+FsoWv41/nkWGRBXLJ/fp5p73KQY0apNw6lhrrl5P62nu8+KLr36OvKnPAF5EsIBFYhBXIPxKR9caYZ40x9ve8h40x640xa4CHgXvKel2l/JXzLBB7Xn7xzO1kPjuGPVf0YWBSF154wUUuWXndW3I/K7mMl3icKhwB/Gd8RUsrKFXOXG1SDkCvG+icnUxMxB9M/LCWBngf4piCW7kSFo/7lR+zL2UqiTzCqz5VbqGolI6utFWqnDmv7IyPh2nXfU1C9mcMYwxbTtQiOdk3gocq+AY9c6a1/i0juzVvmAEkyjTS+txL49tb+sXvS3v4SnlbRgb/1m/K33uDaCrrCI0K85neoiq8uFpVDrMrsiGVWlwAK1ZAkG+UJtPiaUr5sA0PTuKM/21hxc1T6Jugwd7XFFZcLTPqTLb0nQA//cR7Xd72i0Fb7eEr5UXfvLWL/zzQiEV05c6oBRrsfZRjDh8cvhahWq+raJizkRaRf/LGB1W9/vvTHL5SPsQxeNQa8zhB5DCYl/Nmeng7YKhT2XfGcrwPkJho+D7nVVbTisfTn2XkyEkFnvc1mtJRqhw5zsGf0WcxrbZ+zITQEeyirl+X3Q1UsbHwZ1QMM3mQgUzh5O+bfHpOvgZ8pcqJzQYjR1qzPULJZELGQPZXvoBW7z+hc+39lH3G1dxLnuc40UziUZ+ek68pHaXKgePUPoAEptGITczp+jk/L4vwqaX5qnSs31tNxt40irEnH+PaoIVUqRLn7Wa5pD18pcqB4+ra6hxgdNBoNjfoyoAvr/WrpfnKtfh4CHs0kT+4mAk5jzLt5Uyf/H1qwFeqHDhO7Xs+5BnO4F8+umyiyxILyj8d+jeMwbzMJfzBvenTfPL3qQFfqXLSsSP0a7+BvjlvENS/H81vbXLqRhvKb8XGwndRcSykO08zmmvb+l6Jd52Hr5SHOebvFwV15+rIHwndsQVq1Ci0TLLyTzYbrPlwEyM/bEbQA/fDG2+Uext0Hr5SXmTP33fjK2JzvmZBzER61agBnDq/W/k36/d5CZyVCK+8Ym1CHxPj7Wbl0ZSOUh4WGwuVI08ykcfYYi4kdHCit5ukPG3UKKheHQYNAh/KomjAV8rD4uPhv3fOoDEbOTT8Ja67MczbTVKeduaZ8Nxz8N13PjX9SnP4Snna4cNw4YXQogUsWWLV11UVX1YWNG8O2dnWHrihoQWe9tT4jVbLVMqbnnvOCvovv6zBPpCEhMCECbB5M8yYUeCpora59CQN+Ep50ubNMGUK3H+/1cNXgSUuDjp1ImPEMzz+YCo2W8ESG1C+azA04CvlScOGQUSE1ctXgccYll33EqFHD1L9zbH07g19+lgZHrvyXIOhAV8pT1mxAj79FIYMgXPO8XZrVDmy2aydsmw2+GRLS97lTgbzMmdn7iIjI/+4pk3Lt2ieDtoq5QkicMUVsGMH/PknREd7u0WqnDgutAsOhrZt4e+Vf7Eh+2IWBN3E/aHvkpGBxzY+14VXSpW3Tz+FH3+0dr3WYB9QHAvlZWfDDz8A1GayeZThOS9S68VBfLyttVdWV2sPXyl3O3kSmjSB0FA+f34Ni5aEaOmEAGKzQa9eVrB3VImj7Im8kEptm8DSpR6bsaXTMpUqTzNmwJ9/srLXOG65I0TLHweY+HgYOtSalekoO6oy2+8eDcuWwRdfeKVt2sNXyp2OHiWjzoXsim7MoBbfsvCr/F5cQgJMnerFtqlyZV9YVaUKpKbmLrDqftJajCUC69adshjLHTSHr1Q5+ePBCTRM3c9tqRNYe8AQHk7eAJ2WPw4srgvjhcL48dYTb74JAwaUa5u0h6/UaXJcGg+w8tO/eWrOhXyacz23MQ+w1t3Ur6/lj5UDEejUCdavhy1boHJlt55ee/hKuZnj1LuZM63xtykZTxNEFs+GvQCZEJZbI02DvbKzOgmG3te9xFXL2sDYsfDii+V2fR20Veo0OE69y8yECzLWcx+zmEYCDbo0IC7OehNYuFAHbJXFsX5O3FOt2X3V7TB5MuzZU25t0ICv1Glw3KM2LAwmBA3lGJWYFPkk/fpZaRz7ikrdr1ZBwU5CWhrMqPO8NXfz6afLrQ0BEfAdlzkr5Q7x8dYqyYQEWPrUt8TlfMmydiOY9kF14uMLviHogK2CU/8m2txUz/oDmj3byueXgwo/aOuYa/XUUmYVwHJyrLXz+/bBH39AZGTeU7pfrXJ2yt/EwYNwwQXQvj18/rlbrhHQg7bOH6OSk/U/n3Kjjz6CVavgnXcKBHvQ/WrVqU75m6he3aqoOnw4LF8OHTp49PoVPqWjH62Vx2RkWP9RW7SAO+4ANH2oTsMjj0CtWvDEEx7f/7bCB3zHXKumc5RbvfaaVQ1zwgQICvLaLkbKz0VGwrPPws8/wyefePRSFTLgO/ey4uOtJe0a7JXbHD5sbWoSGwvXXAO4Th8qVZS8WFX1Lqs4/ogRVvE9D3FLwDfGdDPG/GGM2WKMGebi+XBjzIe5z680xtRzx3Vd0V6WKhdjx8KRIzBuXN5/2ipVNH2oSq5ArLojmJ9uGAtbtrA2cYbH0oJlHrQ1xgQD04BrgN3AL8YYm4hscDjsfuCwiFxojLkFGAfcXNZru+Lcy5o+XXv26vQUOstm1y545RW4805su2IKzAIbNMihUJb+3akiOMeq9w7GcWHTqzh3xmje4S5mz67k9jS0O3r4lwFbRGSbiGQCHwDXOx1zPfBO7tefAJ2N8Uwx6B7/OcyUoEdoyCYAlizRXr4qvcI+KdpssDL2KbJzgOeeO+U/bWqqpg9VyZwyoaSrYfYl46nJfp5ggkfSgu6Ylnk+8JfD/d1A28KOEZEsY0wqUB044HiQMaYv0BegTp06p9WY7l1OkhY0m/NzdtGLT8nI0KmYqvQKy8ePuTmFFSfeZVLQE3w7oA4xMdZ/VnsPX9M4qqTsE0rsJZSTk6HKxZfxSXAfOmQvJypSiI11b7/Yp+bhi8gMYAZYC69O6yRnncXOm4dyw9wnuYLv+S3qSv1PqEotNtZaAOkYyJOT4ZkTQznMmTyfM5zUhdZeFprGUafL/vfimBYcMuhNDqRHM6+rcfvfkzsC/h6gtsP9WrmPuTpmtzEmBKgCHHTDtV1qNGMw6V+9xruhT7Bu+g/Ex3tmKzFVcTn2vuyBvGbKN7QjmcFMIpWqQME0jlKnw/nT5P4TlZg6zTPXckcO/xfgImNMfWNMGHAL4Jw1twF35359E7BUPFnTISqKyPHPUn/fT8RnLfDYZVTFVmA6b04Ojd8ewt7IevzU8iHCw61jNI2jysq5EN/27Z4bd3RLLR1jTBwwGQgGZonIC8aYZ4FVImIzxkQA7wItgUPALSKyrahzlrmWTlYWxMRYqyE3bPDIVmIqcKx+9D1avXwntzGXpKjbNI2j3Mpms2YULlmSv0Pa6c7Q8fgm5iKyUEQuFpELROSF3MdGiYgt9+sTItJbRC4UkcuKC/ZuERIC48ZZO8rMmOHxy6kK7MQJ6s0cya+04gNu0dk4yu3i48unpHaFXGmbJy4OOnaE0aPh6FFvt0b5q2nTqPbvLp4Kn4AQpGkc5RHlUferYgd8Y6wNg/fvt+qdKFVahw7B889D9+70/6iT1mRSHlMedb98alqmR1x6KdxyC0ycaO0Qf9553m6R8idjxiCpqYypMo6m6Gwc5VmeLqldsXv4WIMhT4e8QM7JrHLdSkxVADt3kv3KFN4LupuRHzTT2kzK71XogG9fHv/sew14jQRk1qxy20pM+Yci69c/9RTZOYYR2c8CWgFT+b8KHfAdFzQ8nfUkx4Mqse7aYdpLU0AxlVV/+w3ee4+dPQdxKMpaV6iDtcrfVeiA7zjq/W9YdcbIcJrt+ILX+izToK+Krl8/dChUq8ZFbw3TDXRUhREQm5gnJ1ur15YuTGczF7OXc5jz0EqmTKvQ73eqGK42uAfY9WYyiZ93hZdftgrlKOVHAnoTc/uot80Gy5ZF8mTa87zDPZioj/FQSX7lJ5zr5QDcdksO36cPYbupz/raA7jWu01Uyq0Cpotr/89decAdpNZrTqN3hzNoQIamdgKcY72c5GS4IX0uMaxhuLzI19+Ge7t5SrlVwAR8sP5TT3ktmA13jyd633bkjTfo1QtGjvR2y5QvqB59gud5kl9owxeRfXSAVlU4ARXw7ebuj+UbuvAUz3FG9hHGj9f51YHOZoOTL0+lLrsYHjSe7j2CSE7WvwtVsQRcwLfZYPsOwzAznhocZCjjyMrS+dWBzGaD8UMP8sTJF/iSOJbkXM1nnxUyXVMpPxZQAd8+K2PhQlgb3JL3zB0MYjIXR+zSj+8Byv430WfTaCpxjCeYQHCwVV0bdLGVqlgCKuA7zrvOyoLFHV8gOBiWtB2h86sDVHIy1E7bxEO8xgz6Ety0MUOHer5qoVLeEFAB37n8aK9BdQgb+ii1vpsLP//s3cYpr4iNhYnBQzhONOMjn+GFF+CFFzxftVApb6jwC6+c2Rdi5e1UdOwYXHwxNGgA339vlVRWgWPJEujShc/ajSNo2BAN7srvFbXwKuACvktvvQUPPAAffIAt8uaCbwiq4srOhlatrM1xNm6EiAhvt0ipMtOAX5zsbGjdmrT/HaHWsU0cTo8o056Syk/Y3+g//BD69PF2a5RyC4/vaev3goNh0iSi/tlJ3/TJgDW4O316EaVzld+y2eCxvsc48cST0K4d9O7t7SYpVS404JNbE31BJzZeFM8IXuQs9hEWZqV3dS52xWKfhll15ngiDu/lvzdM0nEbFTACPuA71kS/5a8JRAelM7/xKLp08fwO8qr8JSdDtbS/eJyXeJ9b+XDn5d5uklLlJuADvuPc/LUnLua7ZolcuelNHu+6TudiVyD2na2qVIHxwSMwCM9GjNHfqwooAR/wHefmBwfDD52egipVuPrzR5n3vuhc7ArA8VPcDxN/5Nbs90i64FEu6FTX201TqlwFfMCPj7f2uAgJsSbrjJlejXU3PgOLFxNvPs8rnav8l/1TXBDZTMgYyIHw8xj49wgWLtTxGRVYAj7gA6SmFqydMjNkADRuDIMHw4kTQDGbXSufZv8Udy+zacOvzLpkAvvTzwB0fEYFFg34nFpyoVK1UF69aAps2wYvvVTkZtf6RuD74uPhk5mHeTliOAcbX8klo2/V8RkVkCr8Focl4bjVXZUqMHkypKV1olbwTcQ/9yK/3HwnaWlWvtfeI7Rvm2jfE3X2bM31e9MpJTOcdF/5DGQegrlTiI8xBbY21N+ZChgi4pO31q1bizckJIiAdavNTskIiZQ9/7lJoqKsx6KiRJKSrGPj4vKPBet7VflLShKXv58869aJBAeL9O/vlfYpVZ6AVVJIXNUevpPYWKu3npYGB6PqsPWGETSa+xSPXLaENTU6069ffu9+8eL87wsP19SAtzhOrbWvkLZ/Wks9Ijz934epWbkyPP+8dxuqlJdpwHfimN6JjYWtmY8T/v5s7vh5INMi19CvXyhgPZ+Zmf99nTtrasBbHN+kQ0Jg0SJrxhXAjcynJt+ytt80mlev7t2GKuVlOmjrQnw8edMxv14WwSMymcZs5P70KXkzOpwHevv18157A539TTouDnJy8oN9JGlM5DFSaMHMIP0FKaUBvxixsbA08lq+JI5neIbrLt0L5AcZXZjlG+LjoX59K+DbDWUcddnFkPBXuaZbsPcap5SP0IBfjPh4mPeBYdXtk4kOyaDr0qF5UzEBXZjlQxw/dV0UtJXhQeNYddGtJH7UQX9HSqH18EtnxAgYM4bO4d+zNOMKrZnvIxynZAIkLxJG/dCNs7b+aG1scv753m2gUuVI6+G7y4gR/BNRm8kZ/QnhpK7S9AHOi+IApnb4iLNSkq3NaR2CvS6SU4GuTAHfGFPNGPONMebP3H/PLOS4bGNMSu7Nb/+72ZaeQf+saTTjdx5jok7F9AHOUzL/+/kRqzhS69bw0EN5xxW1WlqpQFHWHv4wYImIXAQsyb3vSrqIxOTe/DYBkpwMn2Zdx3x68TSjub3dNk3neJnzbKmH9oyEf/6xJuMH5w/UOr8x6CczFYjKGvCvB97J/fodoGcZz+fT7MHlYV7lJKGMSX3IWmRLwXSBpg7Kj+Nsqa9G/0z9r1+3fvitWwMF6+Br/RwV6Mo0aGuMOSIiVXO/NsBh+32n47KAFCALGCsinxVyvr5AX4A6deq03rlz52m3zVPsA4T9MqfQbObDMG8etqhb8mrqhIVZO+ZlZKCDuuUpKwsuvdTq3W/cCJUrF6h1FBVlZXpSU7V+jqrYihq0LXalrTFmMXCOi6dGOt4RETHGFPbuUVdE9hhjGgBLjTHrRGSr80EiMgOYAdYsneLa5g3x8bnBIvshSHkXHnmE/17XlbQ0a/jCcfWtY6E15WFTpkBKCnzyCVSuDJyaxklNtabRKhWoik3piEgXEWnq4pYE7DPGnAuQ++8/hZxjT+6/24BlQEu3vQJvCQ6GGTPg4EES/hpKWNiph2jqoJxs3QojR8K110KvXnkPO+f39XehAl1Zc/g24O7cr+8GkpwPMMacaYwJz/26BnAFsKGM1/UNMTEwaBD1kmfyaMtvCzzVtKmmczzNZoOBCTkc6PkAhIbC669b+bRcuhpaqYLKmsOvDnwE1AF2An1E5JAxpg3QX0QeMMb8B5gO5GC9wUwWkbeKO7dPLrxyJS0NWrTg+LFs6h9dy/70MzR3Xw7s+fk7097gDQaQkjCTmKkPeLtZSnldmXL4RRGRg0BnF4+vAh7I/foHoFlZruPToqJg1iyir7qKld2HMbH+VB0ULAfJyVA9bRcTeIJv6EKS3I+m55Uqmq60dYf27eHhh6m/cBpTb1qmwb4cxF4jvBXUF4PwcMRMYrua4r9JqQCnAd9dXngBLrgA7rsPjh/3dmsqLJsNevSArU+9zTU5i1jYYRzjPqynb7JKlYAGfHeJjoZZs2D7dhg+3NutqZBsNujdG1IW7uHedYP53rQnYvAADfZKlZAGfHfq0AEGDrTmhH/3nbdbU+EkJ8PJzBze5h5COcm98hbJi/VPWKmS0v8t7jZmjJXauesuOHLE262pEBzLIzwW/ArXsJjBvMxf4Rfp3HqlSkH3tHW36GiYOxeuuAL697fmZxodUDxdjuURLotYywqG8dNZ8exp/SAf9dfZUEqVhvbwPaFtWxg9Gj78EObM8XZr/Jq9PEI4J3jzxO0cCz6Ty39/ky8XGg32SpWSBnwPsTUexp/ndSCrfyJs2eLt5vit2FirIN1YhtGM37k7Zza2H2t6u1lK+SUN+B5gs8GtdwTT6e/3OHYihMNxt8PJk3lTCnv00NLJJRUfD0+0SGYQrzCFRD7P6q617JU6TRrwPcCehthNbfoygzP//JnNt4+md29YuNC69emjQb9E9u7lyS13s8E0ZgjjtQiaUmWgg7YeEBsLs2dbQX9hVG92tbuXCz9+kfZ0ZAldAKte/vTp+Ztvaz7aYt9voEoVOHo4m1Hf30bNE6nsm5zM/Zsj9WelVBmUqXiaJ/lN8bRC2ANXbCzEdz7O0UaXceKv/bTkN/7mfEJCrArLRW2UUuAcARDkHGfkAIxmFKN4jt8enk3LV+7xatuU8hdFFU/TlI6HxMdbm23ExwPR0VROnk+1iHS+ObMP13U7SWysFezB9R6rgbjptuOGJbEs4kmeZxb38lb2PV5tl1IVhQb8cmCzQeLUS0gZ+BaND/+A7ZIh9OtX9OYcgbjptn3DkrrsYC63s54mDImcqjl7pdxEc/ge5pimmB3Vh3XXrqDB5MnEt2nDvHm3F5qycRwHCJSByvh4+Gj2cVo81JPo49ksuHEBs/pEBUQ6S6nyoAHfw5x76pNrvcSrV62B++8nfvlFxE+9zOX32XdrCqQcPiL0+OReOLwOvvySp7td5O0WKVWhaErHw5z3Ve3SPdTaaPu886BnT9izp8Dx9roxNpvTOEAA2HD3WPj4YzbcOQa6dSvws1BKuYGI+OStdevWUlEkJYkkJFj/5lm3TuSMM0TatBE5fjzvuKgoEbD+LXB8BffzkI9FQOZyq0RF5siIEYH7s1CqLIBVUkhc1R6+hxU6tbJpU6vI2q+/wm23QXY206cXPlBboXu7P/xAzMQ7WMF/uI9ZpKUbbLbAG7RWytM04HtQsVMr4+Ph1VchKYnt1w1k8Tf5ayLCw/MHaiv0FM0//4T4eDLOrsOtkUlkEEFUlPWjKWoWk1Kq9HTQ1oNcTa209/Lze/6JxA/dTf1x43iMWoxhBACdO+cf63yeirBC12aDHz/dy5NfdyfaGM5Y/hVT19co8LratvX/16mUTyks1+PtW0XI4ReWk3d+fOTwbFly3h0iIA8y/ZSctePxYWEi4eH+kdt2OXaR+3ityAOylqZyjGhZPv5H7zRQqQqIInL4Xg/shd0qQsAXcR30EhKsn7z9FhIiEkqGLDRxIiCrH55d6Hni4gp+b0JC+b2W0ihqAPqxB1PlZ9pIOuFyNUt89jUo5Y+KCviaw/cwV1MrHadqBgdDVhacJIwbZD4ba19Dyyn3wfvvuzxPcSt0fUWhK4X//ZfhK64lhhRuZD4rozpRpUoFHpBWyodoDt8LHBdVVakCkydbQTE4KoJtEz+j0bQ4a09csGbwFPK9vpzbdrlS+MgR6N6d6n/8wqon3qd+Wg9iHF7/7Nmui8gppdyksK6/t28VJaVTEqekfY4dE7nqKhFjRF57reznK+Xz7lLgOv/8IxITIxIaKrJgQd4x/pKiUspfoDl8P5SWJnLddSIgG25/XhIeyilRgC5u8VZJnnfnm0VSksiTd+2U1NqNRSIiRL76qsBzYWH5wT483LcHoZXyBxrw/VVmpuzqaM3emcYAqRyZWWxAdB4Qdu4xF/V8Wd8snCUliVwZ/rP8j7PlCJXl++e/zXvc1QB0XFzJfixKqcIVFfB10NaXhYYyvvE7jGMID/E6C9K78b3tUJHf4ly7x3lQt6jniyvJXNKSzfZVwetGL2BRxlWkE0k7fmTe/zoWWES2eLG1wMzeln79ivl5KKXKprB3Am/ftIdvsfeq7+QdOUGY/HvOBVYdnmK+53TSMu7o4ScliVSKPCnPMVKyMfKjuVzOYm/e8c6fMOLiymc8QalAQRE9fN3i0Ec51uAB6+ub6/xI+0k3QGoqTJwIAwZg+9y4dcZOcdsqFva8/fGjG/fwwNJb6cB/eZP7+bLrFM6/MDLvdUyfDkuWFL21o1Lq9BW1xaHXe/KF3QK5h19kT3rvXpFu3URA/m57vdSO3O/1VbdJSSJRkTlyEx/JP9SQY0TL7bxb6OrisDCrZ6+9eqXcD83h+5eicuW2lWczsMGX/H7fJGr+spDV6ZdwH2+RnpZT5oqSzhU5S1qhc+WCPcxNv4GP6cNO6vJYh1VUTbijQO/d8TVlZkL9+tqzV6rcFfZO4O2b9vBP7eE7Pz74mnXyX64UAVkRdIUsm/Sry3OVJEfufO4S1aM/flxkzBjJjKosx4mUR3lJKkWedHnsiBFWCQlvfxpRqqJDp2X6n5LW4DFky31mlvwbWd168PrrRX77Le8cJZ1G6Xzupk0L3i8wvfPECZHXXxc591zryWuvlW+mby30jcWxHcHBVvBXSnlGUQG/TCkdY0xvY8x6Y0yOMcb1IIF1XDdjzB/GmC3GmGFluWagKGkNHiGIWXIvz9y+hU23jibtq2XQsiV068Y/r8/nZFomkF9WuTDO0zVd1qPftQtGjIDatWHAALjgAr4f+z2JdT8n7ZwGhW7H6JjOyc62xpyVUl5Q2DtBSW5AI6AhsAxoU8gxwcBWoAEQBqwBGhd37kDv4RfG3vN3TrnY71fhsIwOfVbSqtcSAdlHTZnKQ3IdSVI97GiRvXznTxVJSSLP3L5Z1t030Sr1EBRk3a6/XuSbbyTps5wSfYII5K0blSpveHpapjFmGfC4iJwyj9IY0w54RkS65t4fnvtGM6aocwb6tMyScJwimZxsLWaySxyQzZTrkllx75vE7PuaaNLIJJS/z4qhXrdGcMklULcunHEGREdb33TwoHXbtQtSUuC332DfPgD2VG/Gv5170nD8/db3YQ3oOl4zIcH6VFJcW3WwVinPKWpaZnlUyzwf+Mvh/m6gbTlct8KLj88PnitXQkiIleaJioLKZwaT+GV3qtzfnWtfziAm/QfighbRJftXzv5yCZFz5hR+4pAQaNIEunVjbXgbbpnTg40H6xP1Bcy7HViTX+kzKiq/Iqa9zLGroO7YVqWUdxQb8I0xi4FzXDw1UkSS3NkYY0xfoC9AnTp13HnqCs1ms0oMZ2VZuf24uPySw1FRMGhwOCkpV/PUkqsZchDCwqBy2FGqZ/5N1ZDjdGhznOuvE66Irw7Vq0ONGti+CiU5GbZvh40nrOvYxwGWLXM49yArJ+9Y5lnLHCvlm4oN+CLSpYzX2APUdrhfK/cxV9eaAcwAK6VTxusGDOdB0U2bCs7jT0215r1nZFiPZWbCASpzgMqQBSt/gmlrYV5TiG+av2l6Wpr15hAenr8y1n5Ox3NPnWr17Avbv1cp5RvKY+HVL8BFxpj6xpgw4BZA9zZyo5LMsHE8xh7EHTku8HJeJNW5s5Wfnzev8B23iivappTyvjLl8I0xNwBTgJrAl8aYFBHpaow5D3hTROJEJMsYkwgswpqxM0tE1pe55SqPq12w2rY9dZDU8Rg4ta6NY/B23K2qX7+CvXVXO275y05cSgUyLZ4W4IorhmZ/3NVxOvNGKd9T1CwdDfiqWI45fXuFSzj1MQ36SnlfUQFfi6epYrkq5lbSzVCUUr5DA74qlqsBWR2kVcr/lMfCK+XnChuQ1UFapfyL5vCVDr4qVYFoDl8VynFT8VtvLX6zE6WU/9KAH+B08FWpwKEBP8Dp4KtSgUMHbQOcrpBVKnBowFdaulipAKEpHaWUChAa8JVSKkBowFdKqQChAV8ppQKEBnyllAoQGvCVUipA+GwtHWPMfmCnh05fAzjgoXOXB22/9/n7a/D39oP/vwZPtb+uiNR09YTPBnxPMsasKqy4kD/Q9nufv78Gf28/+P9r8Eb7NaWjlFIBQgO+UkoFiEAN+DO83YAy0vZ7n7+/Bn9vP/j/ayj39gdkDl8ppQJRoPbwlVIq4GjAV0qpABGQAd8Y85wxZq0xJsUYk2yMOc/bbSotY8wEY8ym3NfxqTGmqrfbVBrGmN7GmPXGmBxjjN9MrTPGdDPG/GGM2WKMGebt9pSWMWaWMeYfY8zv3m7L6TDG1DbGfGuM2ZD79/OIt9tUWsaYCGPMz8aYNbmvYXS5XTsQc/jGmMoicjT364eBxiLS38vNKhVjTCywVESyjDHjAERkqJebVWLGmEZADjAdeFxEfH7HemNMMLAZuAbYDfwC3CoiG7zasFIwxnQA/gXmiEhTb7entIwx5wLnishqY0wl4Fegp5/9DgwQLSL/GmNCge+BR0TkJ09fOyB7+PZgnysa8Lt3PRFJFpGs3Ls/AbW82Z7SEpGNIvKHt9tRSpcBW0Rkm4hkAh8A13u5TaUiIsuBQ95ux+kSkf+JyOrcr48BG4Hzvduq0hHLv7l3Q3Nv5RKDAjLgAxhjXjDG/AXcDozydnvK6D7gK283IgCcD/zlcH83fhZsKhJjTD2gJbDSy00pNWNMsDEmBfgH+EZEyuU1VNiAb4xZbIz53cXtegARGSkitYG5QKJ3W+taca8h95iRQBbW6/ApJWm/UqfDGHMGMB8Y5PSJ3S+ISLaIxGB9Mr/MGFMu6bUKu6etiHQp4aFzgYXA0x5szmkp7jUYY+4BrgU6iw8OxpTid+Av9gC1He7Xyn1MlaPcvPd8YK6ILPB2e8pCRI4YY74FugEeH0ivsD38ohhjLnK4ez2wyVttOV3GmG7AECBeRNK83Z4A8QtwkTGmvjEmDLgFsHm5TQEld8DzLWCjiEzydntOhzGmpn1WnTEmEmsSQLnEoECdpTMfaIg1S2Qn0F9E/KqnZozZAoQDB3Mf+smfZhoZY24ApgA1gSNAioh09WqjSsAYEwdMBoKBWSLygndbVDrGmHlAR6zSvPuAp0XkLa82qhSMMVcC/wXWYf3/BRghIgu916rSMcY0B97B+hsKAj4SkWfL5dqBGPCVUioQBWRKRymlApEGfKWUChAa8JVSKkBowFdKqQChAV8ppQKEBnyllAoQGvCVUipA/B98G+2HDyz75wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, torch.sin(x).numpy(), color='r', label='GroundTruth')\n",
    "plt.scatter(x.numpy(), y.numpy(), s=10, color='b', label='Dataset')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型定义  \n",
    "采用`relu`激活函数，单隐层神经网络结构"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Net(torch.nn.Module):\n",
    "    def __init__(self, n_feature, n_hidden, n_output):\n",
    "        super(Net, self).__init__()\n",
    "        self.hidden = torch.nn.Linear(n_feature, n_hidden)\n",
    "        self.predict = torch.nn.Linear(n_hidden, n_output)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.hidden(x))\n",
    "        x = self.predict(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "开启GPU模式并查看网络摘要`summary`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------\n",
      "        Layer (type)               Output Shape         Param #\n",
      "================================================================\n",
      "            Linear-1                   [-1, 10]              20\n",
      "            Linear-2                    [-1, 1]              11\n",
      "================================================================\n",
      "Total params: 31\n",
      "Trainable params: 31\n",
      "Non-trainable params: 0\n",
      "----------------------------------------------------------------\n",
      "Input size (MB): 0.00\n",
      "Forward/backward pass size (MB): 0.00\n",
      "Params size (MB): 0.00\n",
      "Estimated Total Size (MB): 0.00\n",
      "----------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "net = Net(n_feature=1, n_hidden=10, n_output=1).to(device)\n",
    "\n",
    "summary(net, input_size=(1,))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型参数设置  \n",
    "优化器以及损失函数设置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = torch.optim.Adam(net.parameters(), lr=0.02)\n",
    "\n",
    "loss_func = torch.nn.MSELoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将数据拷贝至GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_data, y_data = x.to(device), y.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pytorch参数重置，避免调参后重新运行for循环时，原始已训练参数进行保留"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def weight_reset(m):\n",
    "    if isinstance(m, torch.nn.Conv2d) or isinstance(m, torch.nn.Linear):\n",
    "        m.reset_parameters()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 31.3 s\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABNc0lEQVR4nO3deXhM1xvA8e+ZyUYSkc2+q32X1FY7RbXWUktr6Y8QRVtapbRqqVJU1VJ7La2i1SpFqVqr1oTYFUWINbLJvsyc3x83E1kRkkySOZ/nmUdm5s697yTmnXPf+95zhZQSRVEUJf/TmTsARVEUJWeohK8oimIhVMJXFEWxECrhK4qiWAiV8BVFUSyElbkDyIibm5ssV66cucNQFEXJU3x9fR9IKd3Tey7XJvxy5crh4+Nj7jAURVHyFCGEf0bPqZKOoiiKhVAJX1EUxUKohK8oimIhcm0NPz3x8fEEBAQQExNj7lAslp2dHaVKlcLa2trcoSiKkkl5KuEHBATg6OhIuXLlEEKYOxyLI6UkKCiIgIAAypcvb+5wFEXJpDxV0omJicHV1VUlezMRQuDq6qr2sBQlj8pTCR9Qyd7M1O9fUfKuPJfwFUXJJXbsgF694ORJc0eiPCWV8J/BvXv36Nu3LxUqVMDDw4PGjRuzadOmHNv+9evXqVmzJjt37qRu3brUrVsXBwcHqlSpQt26denfv/9TrcfPz4/t27cn3Z80aRKzZ8/OrrCV/GTbNujcGX76CZo0gdWrzR2R8hRUws8kKSVdu3alefPmXL16FV9fX9avX09AQECK5RISErI9lvbt2+Pn54efnx+enp6sXbsWPz8/1qxZk7SMwWDI8PWpE76iPJWzZ+H11yE+HurWhZgYGDgQ3nkH4uLMHZ3yGCrhZ9KePXuwsbHB29s76bGyZcsycuRIVq1aRefOnWndujVt2rQhODiYrl27Urt2bRo1asTp06eBtCPpmjVrcv36da5fv061atXw8vKiRo0atGvXjujoaAB8fX2pU6cOderUYeHChY+NsVy5cowdO5b69evz888/07Jly6RpKh48eEC5cuWIi4tj4sSJbNiwgbp167JhwwYAzp8/T8uWLalQoQLz5s3L0t+dkg/Ex2vJPTYW+veHEydg2TKwsYFFi6BFC0g1+FFyjzzVlpmcmJw9Bw/lZ4+/5OO5c+eoX79+hs+fOHGC06dP4+LiwsiRI6lXrx6//fYbe/bsoX///vj5+T12/ZcvX2bdunUsW7aMN954g19++YW33nqLt99+mwULFtC8eXPGjBnzxPfh6urKiRMnAFi8eHGa521sbJgyZQo+Pj4sWLAA0L6ILl68yN69ewkPD6dKlSoMGzZM9dwrj8yYAb6+ULYsLFgAQsDgwVCnjjbqP3IEPDxgwwZo2dLc0SqpqBH+cxo+fDh16tThxRdfBODll1/GxcUFgIMHD9KvXz8AWrduTVBQEA8fPnzs+sqXL0/dunUB8PDw4Pr164SGhhIaGkrz5s0Bktb5OL169Xqm9/Pqq69ia2uLm5sbRYoU4d69e8+0HiUf8vODKVMAMC5fQSA2mK6JbfTw5MH+Q8g2beD+fWjbFmbPBnXN7Fwlz47wnzQSzy41atTgl19+Sbq/cOFCHjx4gKenJwD29vZPXIeVlRVGozHpfvK+dltb26Sf9Xp9Ukkns5LHkXx7T+qhT739nDgWoeQBcXFaKSchAfnOO/T5ryC+e3bjUdaZtYMa8uaKo/j6h+D5+iTWvfgiYsYMGDMGjh2DFSvA0REAo1ESFBmHm4MNQog095XspUb4mdS6dWtiYmJYtGhR0mNRUVHpLtusWTPWrl0LwL59+3Bzc6NQoUKUK1cuqdxy4sQJrl279thtFi5cmMKFC3Pw4EGApHU+rXLlyuHr6wvAxo0bkx53dHQkPDw8U+tSLI/RKImcOBlOnYIKFXjwyRR8/UNIMEp8/UO4EhiRdN/n5kMejJ8Ev/6qJfmff4aGDeHiRYxGSZ9lR2g8fTe9lx4hIcGYdL/XksPcC4tJ2mNQsodK+JkkhOC3335j//79lC9fngYNGjBgwAC+/PLLNMtOmjQJX19fateuzbhx41id2Lr2+uuvExwcTI0aNViwYAGVK1d+4nZXrlzJ8OHDqVu3bqY/FB9++CGLFi2iXr16PHjwIOnxVq1acf78+RQHbRUlOaNRMv7T1djOnKHdX/EdbsVc8CjrjF5A7VJOVCpij0dZZ6x0Ao+yzrg52EC3bnD8OFSvDhcuQIMGhK/7KcMvimPXQ2jy5Z6kL4LA8FiV/LOByK2/VE9PT5n6AigXLlygWrVqZopIMVF/B8sR+OAhwdVrUyXQn5WeXXhtzwbcHW1JSDDSc+lhTgeE4ZlY1gmJjk9bmomIgEGDtH59YHO7N/moXi/qlndjnVdD+iw7is/1YAyJaUgvoHbpwpwJCMOjrDPrvBqh06lST2YIIXyllJ7pPadG+IqiZMhtzgyqBPpz3bkEe/q/p43egZDoeM4EhGFIHK2HRMfj7mibtg7v4IDxx3VEfDETqdfT5c+1nDkyh/XdKqLT6Vjn1YjDH7ehQXkXrHSCOqULczogLGkvICjy6fv6jUap9gyeIEsSvhDiOyHEfSHE2QyeF0KIeUKIK0KI00KIjPsaFUUxO6NRErL3IHz5JVIInDb8wJoRLZMSupuDTdoyTgbr6bP8KHUeVufTkXORRYpgs38fwsMDjh1DpxMUKWTH+sTEv9G7MZ5Psd50t5Ps+IDRqJJ+erKqS2cVsABYk8HzrwCVEm8NgUWJ/yqKkssYjZL+3+5n0qR+OBuNyNGjcX65VYplhBCs82r0xA6boMi4pJLND7blufPeEpZvnYk4fBiaNYP588HLC51O4O6odYg9zXrT207y4wNBkXFJ61MeyZKEL6U8IIQo95hFugBrpLavdUQIUVgIUVxKeScrtq8oStYJDI+l+Q8LeCEogCuupXAa+ynu6SyXPEkDGKWRiw8ucj7wPNdDr3Mt5BqhMaHEOt0lJCoGpDW/RLtQblJH+q5youK6HTB0qHay1sKFUKBAuut9GqY9Dl//kEztGVianOrDLwncTHY/IPGxFAlfCDEEGAJQpkyZHApNURQTo1Ey//PVTDn6Kwah47tBnzHN3SnD5e+E3+Hn8z+z59oe/r7xN8HRwekvqH/048TDm5hYBd7qBku2QsGVKwk6sgerXzfjVLXOM8X9tHscli5XnXglpVwKLAWtS8fM4SiKxQkKDGXQiinokCxq+DrvT3grTfKMTYhlw7kNrDm1hr3X92KUj04itNO5IeIrYCWLYq8rgcHgQEU3J95vU4XAqHAi4gO5E36Hf4P+ZZvdMRoXDeHXDVDxgj9B9evyyUhPGv5vIq9WfhWdyNwhxmfZM7A0OZXwbwGlk90vlfhYnqPX66lVqxYJCQlUq1aN1atXU7BgwWda18CBA3nttdfo0aMHgwcPZvTo0VSvXj3dZfft24eNjQ1NmjQBtPlxChYs+NRTISvK03CbMQX3kNtccivLwTdH4J0sgYbFhLHYZzFfH/mGe5Hazrm1zprXKnWibflXaViiKb0X/YdRpvyCCLwvaF2uTZpkLKXkSvAV9vXYTOAHM2h0KogpM32YeLwzH3SpyMhG7+Hl4YWdlV3Sa9SZuc8np9oytwD9E7t1GgFhebV+X6BAAfz8/Dh79iw2NjZpJiZ71qkIli9fnmGyBy3hHzp0KOm+t7e3SvZK1vr7b8Q33yD1etw3/sgPw5sjJdwKDWf2odmUnVuWcbvHcS/yDk5WFVn86hLujL6L3cOxfLOlJLO3h+BRxiVpdY52VujT6bYxtU8ajZLCNmX4X+sPaHTiPpHjxyCE4PO9MPvb//h007tUnl+Z1X6rMRgNqhMnC2TJCF8IsQ5oCbgJIQKAzwBrACnlYmA70BG4AkQBb2fFds2tWbNmnD59mn379vHpp5/i7OzMxYsXuXDhAuPGjWPfvn3ExsYyfPhwhg4dipSSkSNHsmvXLkqXLo2NzaMPQcuWLZk9ezaenp7s2LGD8ePHYzAYcHNzY8WKFSxevBi9Xs8PP/zA/Pnz2b17Nw4ODnz44Yf4+fnh7e1NVFQUFStW5LvvvsPZ2ZmWLVvSsGFD9u7dS2hoKCtWrKBZs2Zm/I0puVZkJLz9NkiJ+PhjnFs0wWiUNJ8/i2PB3xCvuw2AnbEWheJ74BDrQfcqbTEaSeqOOXEjlH/GtQZAAK72NgRHpTwZy5S0fa4HU9DWiqg4A56JJ1jZT5sJL7VEvvkmnS+Fcuo7G17rcZOBDwfy5T+zmdn6G3z9Y1J04rja26gRfyZkVZdOnyc8L4HhWbGtJNn1x33KkzYSEhL4448/6NChA6DNiXP27FnKly/P0qVLcXJy4vjx48TGxvLSSy/Rrl07Tp48yb///sv58+e5d+8e1atX53//+1+K9QYGBuLl5cWBAwcoX748wcHBuLi44O3tnZTgAXbv3p30mv79+zN//nxatGjBxIkTmTx5MnPnzk2K89ixY2zfvp3Jkyfz119/ZcEvScl3xo2D//6D2rXh0095EPWAQZu9+Sf0F9CBtbEUK7vNZ+Mh96Sza02j9uTdMUVSnXyVuoxjap80SAiP0faGU7RRduyIPO6Df6tXKBdwmWMrrBnWuSCra56l009teaHw6+hCe+NRtgQuBa3ps+xI0rYfd1auKgVpctVB27wgOjo6afriZs2aMWjQIA4dOkSDBg0oX748AH/++SenT59OmqgsLCyMy5cvc+DAAfr06YNer6dEiRK0bt06zfqPHDlC8+bNk9Zlmmo5I2FhYYSGhtKiRQsABgwYQM+ePZOe7969O/BoqmVFSWPvXm1ueysrWL2aTf9tw3ubN/cj72MlCuAU34+WJd5k81FrTt8Mpk7pwvw4uGFS4sxMd4ypfTL5CN9U8jElZVmkJK/1nsHkPxbw+rm9rNoYRkP/Srzb/j+uRG+ktPsxhrf/nuCo+KfqvTftVTzNF0N+l3cTvplOnzbV8FNLPh2xlJL58+fTvn37FMuY43KCpumO1VTHSrrCwyFxLzNhwnjev72chce1K6q1LNeS5a+toJBNSaSUNJmxB4OE0wFhBEfFJyXXzHTHJG+fdClonVTykZKkpFy/TGGqVyzO2E4fEFzLg7d/nsuw45dpGlSJ971s2fPwLK3XtGZqq6nUL9OcEzfCHtt7r07KekTNpZMN2rdvz6JFi4iPjwfg0qVLREZG0rx5czZs2IDBYODOnTvs3bs3zWsbNWrEgQMHkqZMDg7W+pozmsrYyckJZ2dn/v77bwC+//77pNG+ojzRRx/B9evE1a5Jy+I7WHh8ITZ6G+a2n8vu/rup6FoBd0db3B1tn2oqhadh+oLQ63VJ8+8kT8onboQyv289Do9vy+AfZ6Lbvx9D8eLUunqZdZ/fou/99hikgfF7xhPuOI1t79dl/ZBGGe5dPO00EJYg747wc7HBgwdz/fp16tevj5QSd3d3fvvtN7p168aePXuoXr06ZcqUoXHjxmle6+7uztKlS+nevTtGo5EiRYqwa9cuOnXqRI8ePdi8eTPz589P8ZrVq1cnHbStUKECK1euzKm3quRlu3bB4sUYra1o1/Y2/9wNpnSh0mx8YyMNSjZIsWh2n9iU+kxZ07EAo1ESVNsDDhzmaptONLxxhlWL/2L0h4Np5/ILO678QafglnzfZRMNSldPNy51UtYjanpkJdPU3yHvyPBg5cOHULMm3LzJxLZWTG2aQJvybVnQYRVV3EuYJSmmdzWs5GUencFA2zVfM/jYJgAie3al7Uv/cST0DDrpSHPXGewePgydTnttYEQsAtKfxTMfU9MjK4oFemzf+ujRcPMmx0rAF40TGFJ/CM5RE3lt7imz9bibSj2m5Jy6zPNNvxfp8tc65IYNYG+P/c+/sX9JArXu18YowtkX9B7LfL/HaJT0XnqYhl/spsEXu+m1RPXsm6iEryj5VHoHKwEMW7fBihXE6GFgV5j68hdMbjaXEzfCn2ke+uySuvZeJPFYgnjjDYxHjpJQqTI2Fy5weOVVep1vCCKBYdsHMu/IEnz9Q5LW4+MfzKV74WqefPJgwld/NPNSv/+8w6WgNbVKOaU429XwIIgHb2ptuxNbC8Z6rWbsS+N4d/1JDImj4PplcseBTVPt/fDHbVIclDUaJX3+eYhnp2kcq9sC++gI1v90lL+vtkQYJaN2DaOQ+66k9djbWvHq/IPq7FzyWMK3s7MjKChIJR0zkVISFBSEnZ3dkxdWzMpolPRdfpTTN0OpU8qJHwdrl5842qspRR9Gc7gU/OA5lo4VexEUGceJG6GAdonBBX3r5Zqad+oyDzzacwm1LkDfDmOImPw56HQ0XbOPqzur4hIFfuHf0L/dKf54tylRcYakK3Plhj0Xc8pTXTqlSpUiICCAwMBAc4disezs7ChVqpS5w1CeIPkZracDwgiKjGPLnF4M3nORaCsY89o7NC3TCSllmg6Z3N6jnjpe+yHj4aVG0Ls3ZY9e5PoNV1p2CmLq3xNwt3fEs6ynmic/UZ7q0lEU5elIKem99NHZpfWL/0b/N2dSPALOjRuM28fzGbnuJCcSn8/wIuS5VLrdRzduQI8ecPw4CTZWeL2SwKp6sKzTcrpUeivpBK/83r3zuC6dPDXCVxTl6STvPV97bhFuQ7Rk/6B+VWp8vpjAqAROJDuga7oIeV6R7tm9Zcpg3H+A2OEjKLByBSs3Q4NbMNQ4GMeeDvSs/ga9lx7m2HXtgG6Dci6sH2JZ0yyohK8o+ZROJ9h57Wf2zx3FpjMQb2eDWPELUqdLMadN7VJOuNpbmzvc52Y0SvqsOYlvse580K8o3j99xTCfWPydoL91f2x1zvj6xyct73vD8qZZyFMHbRVFeTLTfPN7r+3lox/fZsnv2uPfdxlBw5+u03vpEaSEtYMaUrt0YU4FhNFn2dE838GSvA31q1Iv8fD79QBM2yvwvBbHwC1v8ELJR+2alljTVyN8RclHTCdbHb5xiru2H/H91gSKREFcs+ZML98qTZ/9mYCwFB0seXm0m/pgbqEejWDMGPSzZrH5twJUGRzGBdvx/P7uPoo7lMi3NfzHUSN8RcmDTKP41E0XQZFxHPW/ym2rSXQ9E06vcyDt7bFetZL65VxTTCCW3yYVS7dvf9o0aNwYt+BoNu9w5mbYTd7+vQcOBYwWl+xBdekoSp7zuPndYxNiKTWzAbqQ05z/Vo9rlAEWLQJv73Q7WyziwiA3bkDduhASwhddXJhQL5heNXqx7vV1+fI9q7l0FCUfyWjKBID3drzHg7jTrNxhpyX7tm1h6NAME3t6JzblO2XKwKpVAHy87SGt7hZgw7kNTD843bxxmYFK+IqSx2RUilnis4Qlvkvof86KjmdjwNERli/HmHhxEYu++HfnzjBqFCIhga1bHHGOhk/2fML3fr9Y1Jn76qCtouQx6c3v/s+Nfxj5x0iKhcPSP22BBJgzB8qWJSg8Vl3xCWDGDDh4kILHj3P4n2pUbXOBgb8NYP0/Rn4f1sMi+vHVCF9R8qDkpZjAyEDe2PgG8YZ4dh2sgO3DSOjQAQYNAtQVn5LY2MCGDeDkRJWDF/jwSDmMIpK/7n/MrbCH5o4uR6iEryi5UEZdOGmWk0b6berH7fDbTA2oRM2jVzE6OSGXLoXEunxGs05apPLl4bvvAPjirwAa3nIhTvcfXxwaZ+bAcoZK+IqSy2R04RKjUXLvYQz3H8YkfRHM/GcmO//bSY24woz/+R4AY5v+j97bA1LU6i3i4OzT6t4dRozA2pDAnh12uMVbs9hnMevOrDN3ZNlO1fAVJZdJrwvH1d4mzTwwIzok8MmeT0DC7oMV0T30ZfcLDfi5RmusLLlW/zRmz4ZDhyh44gRHDtfjhWYn8frdi3rF61HVraq5o8s2aoSvKLlMejV305eAybEb/vT5pQ8GaWBDeHuKHvAl1M6BaZ3fQ6/XWXat/mnY2sKGDUhHRyruOcmKWx5ExkfS8+eeRMVHmTu6bKMSvqLkMunV3E1fAgASI3GF5nM74hZdC9Snx7JDAHzWdij+Nk5sG9lU1eqfgrFCRb7pNQaAfqtO0yW6LGfvn+XDPz80c2TZRyV8RcmFUtfchRCsH9KYo+PbMPSVy9yJPYSzbWF+/KMguvBwjtVtwbaarfAs50KVYo4q2T+FoMg4Frh7sLZuB6wT4vlxgxHneCsW+Sxi++Xt5g4vW6iEryh5hE4nCI69xtSDEwDYFf0GBfYdRLq6Uv7n1RxSXTiZYtpr+qLtEK6XfIGC12+y9LcyIOF/m/9HYGT+u7KeSviKkkfEG+Lpt6kfMQkxjC7anRpffg/AmFZDafTdOUauO4kFnTT63Eyls30TO+K4+Rcire3oce4qw3xLci/yHl6/e+W7s3BVwleUPOLzA5/je8eXso6l+XztPexio9lapSkbKzZRF+l+RqbSWeG6NVna9yMAvt4ZSKMgezb/u5nlJ1Y81fkQeYVK+IqSBxy7dYxpf09DIPgzohsFDv5DmGNhJrcfhqOdFXpLP4v2ORiNkr7Lj7KgeAN2NX4V2/g41vxog30sDNv2Lp7Tf8g3cxCpPnxFyeWi4qPot6kfBmlgWpn/UXnEcgAcVyxje4dOuBS0Jjgq71yAPLcxtbwaJLzf5G02XT5L5Qf+LN1ahDe73+ee1Wx8/GcSGBGLLrFjKq/+ntUIX1FyubG7xnIp6BI1Xasz7rt/ISoK+vRB17MH7o626PU6dRbtc0h+3kONF4qz6J0viLK2pe+Z+ww+7UCs/iKOrrsYue5knp9xVI3wFSUX+9v/bxYcX4CVzoodwR3RHZwNxYrB/PnmDi3fSD37qJSNCS8WCe8MYdGOeP4pDpeslhMSVBthLJ6nZxxVI3xFyaWi46MZtEWb8fKrskMpOX2B9sSSJeDqasbI8p/U5z0M0dXk15ptsIqOZefvThAVTbTDt+h1Mk8fK8mShC+E6CCE+FcIcUUIkWbaOSHEQCFEoBDCL/E2OCu2qyj52eT9k7kcfJlartUZuuA4xMRA//7axTyUbGOq6X/ysjdXXEtR+mYYy3fZERh/kqEdr+Xpcx2eO+ELIfTAQuAVoDrQRwhRPZ1FN0gp6ybelj/vdhUlvzIaJX9dOczsQ7MRCIb/VR3b48cIdnLDOOdrc4eX75lq+nF2BVn8znSknR19j8fw5imY+vd4boTdMHeIzywravgNgCtSyqsAQoj1QBfgfBasW1EsitEo6bX0IFvuDMCgMzDR7U0Gbv0JgDHtRjDDxh53M8eY36W5oliZGPDyYvl2PT4lIhiydQg73tyRJ0f5WVHSKQncTHY/IPGx1F4XQpwWQmwUQpROb0VCiCFCCB8hhE9gYP47rVlRniQoMo49t5YRp7uGbUJR+s7ywdYQz4ZaLxPeul2erR3nNSlq+oMGQZ8+2MUa2PirngMX/2SV3ypzh/hMcuqg7e9AOSllbWAXsDq9haSUS6WUnlJKT3d3NY5R8q+MrmgVGP0foVbrAfjm5ItU8f+XW47ufNFmMAv61suTo8o8TwjtQHmlStS8Y+DrHfDhrg+5cC8Ag8GYp87EzYqSzi0g+Yi9VOJjSaSUQcnuLgdmZsF2FSVPMl3Rytc/BI+yzqzzaoROJ5BS8s72dzASz1jHLgz5U5uxcfyr71G1auk82QaYbzg6wk8/IRs1wts3lr3lg2kQMZByujFExRnwTPZ3zM2yYoR/HKgkhCgvhLABegNbki8ghCie7G5n4EIWbFdR8qT0rmgFsNpvDfv991Pc1o3Pv7+OiI9HDhnC7KUf5unOkHyjbl3E19pB82VboGjobgLjTuSpeYyeO+FLKROAEcBOtET+k5TynBBiihDC1D/2rhDinBDiFPAuMPB5t6soeVV6V7R6EBmE99b3AfjsQA2s/E5B2bKI2bPVWbS5QFIJbuhQjD16UCgOfvoZonQL0OkS8kxvvsittSdPT0/p4+Nj7jAUJVsYjfJRF4gQDNjkxZrTy3nxVkX+WXEda6MBdu+G1q3NHarFS16Cq1+mMAWiI5kypR9lQ++x4EW4+fmnzHh5cq75UhZC+EopPdN7Tp1pqyhmkLwL5GjAUb4/vQKbBB0rN8dibTQg33lHJftcInUJ7mBgPCM6jyNOp2fEcbi+cjqXgy+bO8ynohK+ophRgjEB763eSCTbrjeixv0AZIUKiC+/NHdoSqLUJTiPss5cKFmZtT1GArBkUwKfLu+P0Wg0c6RPpiZPUxQzmn90Pn73/Ghwy5lWPx5BCoFYuRIcHMwdmpIo7eRq2qjfzb4DMdGXKPz7dj6Yd5QW+knsHz05V3fqqBG+ouQw0wHAgLAAPt07EZsEWLXJCr3RSPSwEdC8ublDVFJJXoJL+lmnI2LBCvydCtHgNvTYOJOTAXdydU++SviKkoNMBwAbT99N48UDiIyPYPbB0lR7EMjtIqUpMHO6uUNUMsG1dFGWDJ1FvA7eOxrLVx/2zNXz5auEryg5xGiUXLoXjo9/COH4EBCzh2Z3bRlx4BZSCIr/ug5hb2/uMJVMEELw+XQvzo0aBsDC3w9x48y+XNuTrxK+ouQA08j+1Xl/Y2djJMR6KbbxsOrXAgijkd/b9kE2bmLuMJVnoNMJ6sxcwO5qRXGOgflbpuFirUb4imKxkl839W7CJuJ1t5h/2JkK90O54lKKj+v0yLWjQuXJhE5HjW17ueEk8LwZyfpX2+fKso5K+IqSA0ytfehCeGiznsY3YPDeUIxCx9jXRlHzhWJ54kxNJWN6twp4d+1CgoA39xzgwU8/mTukNFTCV5QcICXM61OPxvX+xDo2ip+32yOkRIz9iMULhqu5cvIBNwcb7BuNZmJrZwDsh74NAQFmjiollfAVJZuZ6vceX37L+vPfM2OvjpJ3I6FGDcSkSWqunHxCCMGGIU1pPO8ntr8A9g+jie7ZDRISzB1aEpXwFSWbBUXG4eMfRKDVYhrfgBFHjKDXw+rVYKumPM5PdDpBp+pt2TqhB7ccocARH5g40dxhJVEJX1GymZuDDa7uR8B4mRW/69FJ4KOPwMPD3KEp2eTT1+cxqHcBDAKYPh127jR3SIBK+IqS7R7GPuSGYTnjDkK1QANUrpyrRn1K1ivuWJw2AybzWUvtvuzXD27fNmtMoBK+omS7qQem4nLtHp/8nVinX7oU7OzMG5SS7d5t+C4/da7ArgogAgPhzTfBYEh6PqPLXGYnlfAVJRtdfHCReYfnsux3sDZI8PKCFi3MHZaSA2ytbJnZ4Sve6g53HQXs2wdTpmA0Su49jKF34hQbOTkVg5otU1Gy0eidoxl8zMBLN4FixWCmupyzJelSpQvzaraib7e9/PU9iKlTmfbQjVV2FTAk5njT5RFz4prFaoSvKNlk55WdnPH9gxm7Ex9YuBAKFzZnSEoOE0Lw1ctz2FtBMKW5QEiJ97KJuISHAKBPdpnLnKASvqJkgwRjAh/sHM3CbVAoFujaFbp3N3dYSg4xlW3uP4yhhH0VHA3tmNpCsr+sI+6RoczdNpuGZQpxeFzrHD3pTpV0FCUbrDixgmr7z9P5EshChRALFpg7JCWHGI2S3ksPc+y6Nop3sNHjFPcWEXYH6PN6ONdWOfHS9VM0CdqHKNQsR2NTI3xFyWIPYx/y1bYJLNiu3Y+YMg1ZooR5g1JyjGmiPJOIOAN6nHFO6MWdQjCir5N2ZbNJk2D//hyNTSV8RcliX/z9BWM3B1E0Es6/UIf6d8vl6otiKFkraaK8RI52Vuh1glalBlLBuQLL3W5wYkA7MBqhb18IDMyx2FRJR1Gy0LWQa5z88StmnASjtTXvtx5GvBQ52omhmJcQgvVDGhMYEYsAXO1tCI6Kx83Bht8uzqb7T93pWPk4AS81xvqfw9CvH2zfDrrsH3+rEb6iZKFPt49h4W/aZFk/dRhI4Xq1scrhTgzF/HQ6QdFCdhQpZIder0uaIK9r1a60LNeS+3HBTBtaHVxdtWkXcqhdV+TWC+56enpKHx8fc4ehKBkyGiVBkXG4OdggJfxxeT9nBrVi3D9w0b0k3d5eyL4J7dEJgZuDjZoRUwHA764f9ZfUR6/Tc+2FhZTqO1SbTG//fnjppedevxDCV0rpmd5zqqSjKM/ANOWxr38I9csURkrJjZMD+OcQGAVMeGU0tSsUoYia+lhJxmiUlLSvxv/q/Y8VJ1cwjN/5fcwYmDULevcGPz9t1J9NVMJXlGdg6sRIMEp8/UOIFnv5fesNrCSEeXmx+KvhuBS05kFEnBrdK0DKQUKNUt1wtPmJrZe2smfQCFofPAiHD8PAgbBlC2TT/xdVw1eUZ2DqxLDSCeqWKUj/E0vwuAN3nAtRaPZsXO1t6Lv8aI7PlaLkXskHCecCBCNf/BCA0XvHYvhxLTg7w9atMGdOtsVgEQnfHLPSKfmbEIJ1Xo04/HEbWjtvZeLuCACKrFmLKFQozR6AukC5knyQ4FHWmQnNP6R0odKcuneKNSH7YNUqbcFx4+DIkWyJId8nfNNulBppKVlNpxMYCKbap99QMAHudm6D/rXXgLQfbtWhoyQfJKwf0oiCNgWZ3mY6ABP2TCCiQ2sYNUq7JGLv3hAS8oQ1Zl6+T/hqpKVkp+2f9qb1FQMPHWwotnxd0uOpP9yqhq+ANkhIfg3jPrX64FnCkzsRd5h9aDbMmAEvvghOThAamvXbz/I15jJqpKVkl3Nn99Jl6QEAImdMAXf3FOXD1B9uRUlNJ3TMaafV7GcdmsWtmEDYvFkr6ZQvn+Xbs4g+/OT90urDp2QFKSV7mxSn9ZF7XKxXmqq+/hglSV0YHmWdWefVCJ1O/X9Tnuz1n17n1wu/MrDuQFZ2Wflc63pcH36+HOGnPkirRlpKVju+bBKtj9wj0hqKrd0MQqjyoZJpplw1o80MrHXWrPZbjd9dv2zbXpYkfCFEByHEv0KIK0KIcek8byuE2JD4/FEhRLms2G561EFaJbvFhwZTatwXAPgM60KhKnUJDI/F1d5alQ+Vp5Y8V32y8QHvvPgOEskHOz/g/sOYbOkqfO4Tr4QQemAh8DIQABwXQmyRUp5PttggIERK+YIQojfwJdDrebednqDIOI7538Rg1ONzXfIgIpYihdQFo5XMy6gUeH5od+qEJHCmtC0NZ6xNUcZZO6ghIdHxqnyoPFHqPcLt3cay5tQa9lzfQ52ZM2heun2WlwWzYoTfALgipbwqpYwD1gNdUi3TBVid+PNGoI3Ipk/DvhtbuG03hIdWmzBIGLHupBrlK5mW0Z5i2N6d1Pp5PwkCrs2cysN4fYoPbUh0vCofKk8ldUNJJfdijG40HoAH+hX4+AdmeVkwKxJ+SeBmsvsBiY+lu4yUMgEIA9JMGCGEGCKE8BFC+AQ+4xzRReyL0PxKGDaxP5NAECdULVV5BunV440xsdzr0wudhLlN3Hn/dHVG/HiC+mVUGUfJvOStuz8ObsiDiDg+bDICB30ppIigUonwLP//lKvm0pFSLgWWgtal8yzraLHZjxbfw7qacQzt9gMvlZysPoRKpplGX6ZSjZuDDbfGvEvle2FcdoGFjcYgjXDiRij/jGutZsRUnolOJ3C1t0lRFvz7f9twsi1BORfXLP//lBUJ/xZQOtn9UomPpbdMgBDCCnACgrJg22l17oxx/Mf0ORvNTzV2MdZ7hvoQKplmGn0l1fAvXqToN98C8N6r9XBwqE1UnAGPss5qRkzluaTemyzpWD/bLpSTFSWd40AlIUR5IYQN0BvYkmqZLcCAxJ97AHtkdp0AUL48uhlfAvDtNpj023tqDh3lmSS180pJWP/eWCcYWVFPh3xxIicmtOWIOotWyQLJa/mmqbazK2c9d8JPrMmPAHYCF4CfpJTnhBBThBCdExdbAbgKIa4Ao4E0rZtZavhwEho3pHgEdF12kC3/pv7+UZSnZ1yyGCef09y1hymtunH5jg2hMQnq4KySJUx7k/+Maw1C0GTGnmxrKc+SPnwp5XYpZWUpZUUp5bTExyZKKbck/hwjpewppXxBStlASnk1K7abIZ0Oq5WrSbCx4n9+sHneO8QZ1IFb5RncuoVhzAcAjH7NEb1tX3VwVslyOp1AJwQnsvnEvXx5pi0AVaogJk8B4LMfb7N839dmDkjJc6QkYZg31pExbK4CbT+ay9GPX1VlHCVb5MS8X/l7Lp2EBELrVqPwuSssb2RL9z23cSngkjUBKvnfL79Ajx48tIHuE6ux8+Mz6HV6c0el5GNZMe+Xxc2lk8TKCscfNhKvFww+EsvaBUPNHZGSV4SEYBj+DgBjX4bzUf3ou+yYOolPyVbZPe9Xvk74RqOk79Eo5jduD0CHGRu5fPOUmaNScpMMr4b20Ufo793nYGn4vl4DbAy11YRoSp6XrxO+qb91UaOhnHN3oFIwnHunh7nDUnKJDCfa27cPli8nVg/DuuppVmyUOpNWyRfydcI3HQSR1jaseHsyBgGdtl3Bd9O35g5NyQXSnc44Oho5ZAgA05pB61eGs21YL3XlKiVfyFVTK2S15GdLuhTswOpjq/nfvtMU9B5FQvuBWBUsaO4QFTNKb/oEJkxAXL7MWXdY3NaJCy0mJtVVFSWvy9cJHx4dBAkMj2WW5ziann6LavfjODaiFw2++93c4SlmlGb6hNOnkTNnIgV4dYZxrSfiWjDNHH+Kkmfl65JOcm4ONtR+oRwfvNYTI1Bv9VZuHTigpl2wcEldEUYjcvBghMHAwhfhbLmSDPN4x9zhKUqWspiEbxrNLZ+/kp9aFcHaCA/e6MobCw5gMBjNHZ5ibvPmIXx8uFlIML4N2McMICJW1euV/MViEj5oo7mihQpQaOYKrhWGOvdCqL9xAT2WHFb91Zbs2jX45BMAhr0qSbCuQfNSr6qOHCXfsaiED1ornmfltnzcvTYAo/75lUi/M6q/2lJJCd7eEBXF+pqCbVWgkcso1g7SLkihSn5KfmJRCd/Ud91kxh7C6k5gRT2BncHI13/Nxq2AOmXeIv3wA/z5Jw/trXmvg8Q+oRU37hbjjfT68xUlj7OohJ+87/ryHQfOfzSUW45Q49p/MH++ucNTclpgIHLUKADebRvPA3s7XA1vU6d0YU4HhGXrrIWKYg4WlfBTz0b3aZfpjOvhBIBh/Dj47z8zR6jkqFGjEEFB7C5vx+q64JTwBjtHdmGjd2M8s3nWQkUxh/w9W2Y6Us9G993J77AdMIg3z4ChZXP0e/aBOpsy//vjD+jYkXhba6oOjeeGczG6FFvHz0NbIITIklkLFcUcLHe2zHSkno1uYN2BfDewDvcLgn7fAYxLlqY/mZaSf0REaAdqgc9bW3HVBZZ3mZeU7CH7Zy1UFHOwuISfmk7omNrzW0Z01O7HjBpF94/Xq4N1+dknn8CNG9ys6MY0z2jalG9D/7o9VHJX8j2LT/gATUo3waZ3X36tCgVjopn8xwJ8rwcTGBGrRvv5zdGjMG8eUq+nW9sgpJWOuR3mqmSvWASV8NHq+h81nsqYLgUIsYPWV3147+5RRq47qVrz8pO4OPDyAin58eUS+BaXDPMcRs0iNc0dmaLkCItP+Kbe/O4LLlLQvT/vd9AeH/7bAm6c+0+15uUns2bBmTPccnXBq/5NbEQhJrWYbO6oFCXHWHzCT96bHxnUgf3Ny/HHC6ALDWHuwRWqNS+fMF64iJyiXdT+7Y5Gom2gUNybSKODmSNTlJxj8Qnf1JuvF1C3lDtz2n/F0Ncg3AYandyHb7UwdeGLPM6YYOBilz6IuDh+fbESuyqGYm0sw0vFe+Fqb23u8BQlx1h8whdCsHZQQ2qXLsypgDDWHyhG5Xpt+ehl7XmnD99HBAebN0jluUR+u5jql/0ILOiIV6trADR0HsOZW5H0WXZUHZ9RLIbFJ3yAkOh4zgSEYTBKTtwI5bNms1jxop59ZYH79+H99zO+2LWSu92+jcOn4wH4oIM7wQUT6FG1D3fuV8Cgjs8oFkYlfFJOuVC/TGHm/xmNfUJ3BneGaGsBP/zAzHe/SrdjR30R5HIjRiAePuROS0++r3WVQraFmNfxqxRTbKjjM4qlyPeXOHwayS91J6WkyYw9OBp74e+yjwmtApnzJwxcM4O1g77F11870OvuaJvU4WO6Juo6r0bodKrWbw7pToXw66+waRPSwYHuze+AgKmtplLcsTjrvIqpqRMUi6NG+IlMp9K7O9riUdYZG10BGjp/wDeN4HhpPcXCgxi/b2WKEWFgeCw+iR0+qjRgPqYvXtMeWEKCkQc37yFHjADg97ebcER3i7rF6vLOi9plC9XUCYolUgk/FdNo//DHbTgwcgwdq77GgE4G4q109PHbwfoXopIm1xq57gSGxPJO/TKqNGAuyVtrfa4H03PpYXZ1Hoi4c4eH9evQ03UPAN92/BYrndqpVSyXSvjpMI3+dDodc9t9w6UiNkxqnnjdWy8viIwkKDKOEzdCAdALWNC3nhotmknyYzC1Szlhd+ggffx2EKezolWDGOJI4O26b9O4dGNzh6ooZqUS/hMUsimJU0IvZr4EfsWsEdeuwYQJKZKMZzkX3B1tzR2qxTLtlf0zrjUFjPF8/od2MZu5TRpyoti/6KQDYxpNMXOUimJ+av/2CdwcbGhVchBb7uzh7S638F2uQzdvHqJnT9Z5NVEH/nIJnU6gE4KX1i+hYvAtLruWZFrrfwGo5/QOVYuUNHOEimJ+aoT/BEIIfhrSnB97LsWvOMxuqgMpSXj7f4jYGHXgLxdxu3qRoUc2AvBpryo81D2gXlFPjrw7Tf2NFAWV8J+KTifoUbMjvWr0YmLTBC4VKYDV5Uts7jZUnaWZCxiNksDQKPDywspowL9vV34qshe90LOiy1Ks9FaPllPnTCgW7LkSvhDCRQixSwhxOfFf5wyWMwgh/BJvW55nm+b0dfuvsbV3on+naIzAa3+uJezgEXOHZdFMLZlLeryPOH4cWbIkvT0uIZF80PgD6hWvl2I5Nd21Ysmed4Q/DtgtpawE7E68n55oKWXdxFvn59ym2RS1L0YlW2+OloZvGtlgJY0UHumtzbOumEVQZBx3T11k1IHvAfhhSHOOhJ+ngnMFPmv5WYrlfNU5E4qFe96E3wVYnfjzaqDrc64vVwuKjCPkQTNsDbWZ0DqOu0UdEKdPw5dfqnKBmbjZW/PN/iXYx8fwd71GeOl+BWDpa0spaF3w0XLJuqrUdAqKpXreLp2iUso7iT/fBYpmsJydEMIHSABmSCl/S28hIcQQYAhAmTJlnjO0rOfmYINnWRcS/Edyy3Y4vTtEsG81yKlT+Si+PL8luOBR1pm1gxoSEh2vundygFi3jrrnj2J0KszXAwSxobEMrDuQNhXaACmnXDBNn6H+LoqlEk8akQoh/gKKpfPUBGC1lLJwsmVDpJRp6vhCiJJSyltCiArAHqCNlPK/x23X09NT+vj4PMVbyFmmBLLy9DeM/Wssa/50oN+hCE4Vr0z3t2YhdXrqlC7MmVthan6d7PbgAVSrBg8ecGDiAFroVuNe0J0Lwy/gWtBVzXWkWCQhhK+U0jO9555Y0pFStpVS1kznthm4J4QonriR4sD9DNZxK/Hfq8A+oN4zvhezM52FO7rxaOoXr8/w5hEEuztQ584l/nd8M0bg5M1QVSvOCaNHw4MHxDR/ic52mwCY98o8XAu6AqpuryipPW8NfwswIPHnAcDm1AsIIZyFELaJP7sBLwHnn3O7Zmels2JF5xVEFdDzVrsIAD44+APlg28BoFe14uy1cyd8/z3Szo5hnQVhcQ/pVLkTvWr0SlpE1e0VJRUp5TPfAFe07pzLwF+AS+LjnsDyxJ+bAGeAU4n/DnqadXt4eMi8YMLuCZJJyI0NHKQEebR0DfnGwr/lvbBoaTQazR1e/hQeLmXZslKC3O/dWTIJ6fKli7wTfifNogaDUd5/GKP+ForFAHxkBnn1uQ7aSimDgDbpPO4DDE78+RBQ63m2k5tNbDGRrZe24tXyFG0vFaDBzXOsN/ohCjU1d2j518SJ4O/Pf6XK0dZtJwALXllIMYe0h5pMJThFUdSZts/NRm/Dmm5riHCwZmD7aADEuHFw/bp5A8uvjh+Hb75B6nT062BFvFUs9oamtC3XzdyRKUqupxJ+FqhdtDZTWk3ht2rwex07iIyEIUNA9eRnrfh45ODBYDRytEczDpe6gl4Wpm3x8ao+ryhPQSX8LDKmyRgal2rM4JdjCHe0hV27YNUqc4eVrxhnzUKcPo2/kxttKx4GYHmnJWzy7qD66hXlKaiEn0X0Oj2ru64mwrkg3i/Hag+OGgW3b5s3sPzi0iXEFG1O+2Gv2RBpG4eDoRWvVu6ikr2iPCWV8LNQJddKzHp5Fj/Wgh3VrCEsDIYNU6Wd52U0wpAhiNhYfvUozx+VbqM3FqVN8XGqlKMomaASfhYb5jmMVyu/yqBX4okooIctW2DDBnOHlacZl6+A/fuJdXHCq8019ELP5j7r2eT9shrdK0omqISfxYQQrOq6CkqW4P22Bu3BkSMhMNCsceVVxlu3iX5/NABerRMILghTWk3h1aotVbJXlExSCT8buBV044dua1npIfirPNqcL+++a+6w8qT44SOwj47gjxcK8X2NSF4q3YKxL401d1iKkiephJ8NjEbJ4j9tcUrog1dniLIWsH49bN6splHOjN9+w3bzJqJsrBn62kN0FMI5ZjRC/bdVlGeiPjnZwDRpl2N8L+461WJsWy25y2HD6D9nB43UVZeeLCwMhg8HYEJbAzcLg2vce1wIsFKToCnKM1IJPxuYJu2y1lnRtujnrG/uwsHSIO7cofOaORiMEh//EAIjYtVoPxXTHpBx7Fi4fZsTZayZ52mkUsFeFKKRmgRNUZ7DE+fDN5fcOh/+00p+4Y0dV3Ywal5H/BaBnQH695zMwxatsbHSc+Ixc7UnX4clHKA0zV/P3wfYsHYc8TqoNxScPZvyV7/dPIyRFvO7UJRn9Vzz4SvPxjRplxCCVyq9Qt8ek/mslfbcrL/ms7RLZU48Zq52S7zodlBkHGf+u8cXfywAYHpTuFjEmUWvfI+ttU3S71NRlGejEn4OMBolQ+uN4d8Br+JTHIqGPKDw5588dq52S7x4h5uDDZ+f2UTF4AAuuMG0ZoLmrtOoUaysuUNTlHxBJfxsZhqpvzRjLyL8PSYPKEucDmyWLGNt5RgOf9yG9UMapRm5WuLFO8TBg3T78weMArw6waR2X7F7hLca1StKFnnei5grT5B8pH7mZjxrvLcw53gDxu2OJfSt13G/HADpJDQhhGVddDs0lIS+fbAyGpnRFGp2H8q4Zu/n//etKDlIjfCzWeqReuMytXhxwSbOFAG32yGc8e6eYvnkffrJjwPkd/HDhmIVcAuf4rBvYEvmvzIfKVFdTIqShVTCz2amkXry0k2bqq9wZdbHGARU/2EnxzctBCzzQC1AwversV7/E5HWMKB7EQrEjUNIvUX+LhQlO6mEn80yaq3s1v8LDvR4Eb0Ee+93OXHtMIHhsfhkcKA2v56hK69eJX6oFwCjOhQg1HEqZ28auBIYYXEHrRUlu6mEn40eN2I3GiXV5+/gTnFHqt83sterLQO//w1D4jL1yzw6UJtvR/4JCfh3ak6B6Hi2VNdz7eV5FBCl8CjrTOWiDhZ30FpRsps6aJuN0mutdHe0TUrgvv4h9O77BVPmjOSD3VHc1Xtxpu58bHXuLOhbL2mPIPV6AiNi0QmR6w/mPu7EMaNRsndwe9qcv8UtR7Bf9SM7PXqmWN6iDlorSg5QI/xslFFrZfIEvt6mAiFTpgIw688IPto/gloltS+G9NZTv0xhRq47metH+0/auxn4fk9arNkDwLlZY2nz4htpDlJb0kFrRckJaoSfjTIapZoSuK9/CPXLFGaYaEOxzveZtW0B7x6NpHJMf+684UeJIhXTrEdKSZMZe9LsNeQ2Ge3dAMzZMZ1J3/+ClYSFjerzRt/JZo5WUSyDGuFns/RGqck7dxb0rc8J/xC2VGvPgF6f8LCAjg6nIrjTqCZ3/vVNsx53R9s8UdvOaO9mzuE5uI+fQIVQOF3UnUNvzsGloHW+PCCtKLmNmjzNzKSU9F6q1fM9yjqzuL6BmPatKR4ST5weYrp1otBHn8KLLya95kmTquWWSdeSxwHwyZ5P+G/RF6z/BeJtrQk/eBynerXou/xo0vtPbxI5RVGe3uMmT1MlHTNLr+wTcuwEu3u3oOWJYApt/B02/g6NG8N770H37uisrTMs4yQ/IJwds3Bm5rU6ncDV3oZ7D6OZ9Pdo/ti9hFPbtOes587DxbMO98Ji8PEPwZDLS1SKkh+okk4ukLrs4/xCTRodusGgr1sxswmE2AGHD0Pv3lC+PEyfrl02MR1PmnTtSS2ej+v3z2x7qNEo6bFkNxW/asWy40tYu0lQOAZkp84E9h2IwWBk5LoT6baiKoqS9VTCz6XsbexZNmIn5z4cQKnR4P0q/FekMNy6BePHQ+nSMHgwnDmT4nVPmnTtcV8IT0romZ3B88Ttf9l2dzBR+mOMO2hDU3+JLFYMr+ZDaTxjDz2XHMb3RigAekGKVlQlh0yaBG5u5o4i8zZvhlq1wM4OqleHDRue7nXnz0ObNlCwIJQoARMngsGQchkp4YsvtM9YgQLQvDn4+aVcZtUqbQ6s1LfFix8ts29f+ssIAe3bP8ebf3Yq4edi1nprZrVZjJ2uH0s8BS8MC2Xi2CYkdGgHMTGwYgXUrg2tW2sfAIMh3akcknvcF8KTEvrTzuBpNEp+PruNV9Y1I07nT+ObRZi8T/tQhX27jH3BkGCUnLoZSu1STljpBJ7lXFQpR3k6Bw/C669Dq1bwxx/w6qvQpw/8+efjXxcSAm3bagl382Yt2X/1FXz2WcrlZsyAqVNh7Fj4/XdwcNBed/du2nXu2aPtfZtu3ZPNjVW/fsrnDh9+9MX0yivP9zt4VlLKXHnz8PCQlsxgMMr7D2OkwWCQbyw+JItNmCitJ9tLJiErz68sTx34WcoRI6S0t5dSG5NIWaGClHPmSBka+lTrNhqNKR43Go3yjcWHZMWPt8k3Fh9K8/zjXmsSExcrq8x6UzIJySRk12VtZVy5clKCNL73nrwXFi17JttGQoLhsetTstlnn0np6mruKDKnXTspW7VK+dgrr0j50kuPf90XX0hZuLCUYWGPHvvySykLFHj0WHS0lIUKSTl58qNlIiKkdHOTcsKER4+tXKl95sLDMxf7zJlS6nRS3rqVuddlAuAjM8irZk/sGd0sOeEbDCkTb3y8lhQv3L8gayysIZmEtJpiJafunypjHwTK8C9mSmP58o8Sv4OD9mXw77/PtO1nScAGg1Eeun5a1lvsqSX7z3TS5ZN+MvKt/lqyr11bvjl/r6z48TbZc9E/8m5otEryucGTEv7u3VI2aCClra2URYpIOWxYyiQXFyflBx9IWbq0lDY2UhYvLmXXrlLGxmrPh4RIOWiQ9ritrbbc4MHPHm9MjJTW1lIuWpTy8dWrtUT6uMFOs2ZS9uqV8jF/f+0zs2XLo/cLUl64kHK5t9+Wsn79R/efNeHXq5f2yyqLPS7hq5JOLpS6tBISHY+7oy1V3aty3Os47zV8jwRjAp/u/ZRii5tRNcKKPmPWYPx1k7abGxEBCxZAlSrQsSPs3Kl9FTzBs3bvRMVFU2vOYJqsrM/Juz4U1BWlRPyXTLhXhYI/rAE7O0KWreLI7SgSjJITN0LR6YSq1+d2585Bhw5ajf+XX2DyZPjxR+jR49Ey06fD2rVaCWTXLpg7F5ycHtXFR4/WSjBff639P/zii5TXf5ASEhKefDP57z+Ij4eqVVPGWq0aGI1w6VLG7+fixbSvK1NGq+dfvPhoGb0eKlVKu37TMslVrAhWVtpnbcmSjLcNWmwnT2rlJzNRbZm5UPIzcVPXym31dkx46Us6vtCRgZvf5k7ERUKsP2D7nXb823051brtgdOnYd487YP4xx/arWpVePdd6N8f7O3TbDO9dk7gsV8AUkq2XtrK+ztGczXyCghwNLTj+LBVuAVF4vJSA23BWbNw8qhDLZ8oTgeE5eoTxpRkpk6FsmVhyxYtCQK4uECvXlo9unFjOHYM+vaFAQMeve6NNx79fOwYDB+uvcbkrbce/bx/vzZIeZJr16BcOa0OD1C4cMrnnZ21f03PpyckJO3rTK81vS4kRKvZm95v8mWioiAuDmxsoHhx7ffToIH25bZ+PXh7a8uMGpX+9tevB2tr7fiDmaiEnwtlNCVD8qRcv0xhGtqtZk/IYh5abSZcv5MXV1TlnRff4YPGH1B0+XKYMQPjkiXIbxehv3gR3nlH6/AZNAhGjNA+QInSm6Dt3XUn0+3nl1Ky6+ouPt37KcduHQOgkFV5HKK8aVqmGZVd3BG9+mofno4dMQ57h77Lj3L6Zih1Shfmx8EN1eg+Lzh2TBvNJ09+r7+ujWgPHtQSft26sGgRFC2q7Q3UqpVyBF+3Lsyapa2jbVuoXDnlNjw84PjxJ8dSokRWvKOs0759yk6bV17RGik+/1w7X0aXTvFk/Xpo10770jQTVdLJpdKbkiF1Uj59Kw7nhLcpFTeftuVfITI+klmHZlH+m/IM2zqMozFX6e3aihpvfcvcwVOQjRtDaKjWmVCxotZRsH8/SJmmA0dAmo6diLgIlvouxWOpB+1/aM+xW8coal+Uue3ncu+jC/iNe0/rDJozR2tJK1IEvvuOoKh4fP1DMEg4HRBGcFS8uX6tSmbcuaMl8uT0enB1heBg7f4nn2gj+G+/hTp1tFbGb755tPyCBdC1K0yZopU9KlXSEp+Jg4P2pfCkm03iHqFpJB8WljIu0wjd9Hx6nJ3Tvs70WtPrnJ21kmjqVs2QEK30Y/OYPdMePbTfy/XraZ87dQouXDBrOQeeM+ELIXoKIc4JIYxCiHRP5U1croMQ4l8hxBUhxLjn2aYlS52UTT83KVuHP/tt4+igY3So2InohGgW+y6m0YqG/HanL/esf2GWuwN3t+/SRm1vvaV9cDdtgpYtoV49xKpVrOtXN6md0zRnj9BF4F7Eh1G7BlHiqxIM3TqUk3dP4lrAlRltZnB5xBX61vDG1spG+4I6cQImTNACXrkSiha1yAuy5wvFi8P9+ykfMxggKOjRKNXOTkvm169rNepeveD992HHDu35woW18uLdu1rSa9gQ3nxT64cHbcBhbf3kmymJVqyo3U9dT794URtVp96DSK5q1bSvu3lTK8OYavtVq2rv8cqVtOtPXf9PzTQ4S2/vdf16rae/S5fHryObPddcOkKIaoARWAJ8KKVMM/mNEEIPXAJeBgKA40AfKeX5x63bUubSyazkB1alJMXPpnLPCyVCqPaCD2vPrCUwKjDptdbCkbYVm1HdvRr1ZDEabfWj9Prt2DzQRkdxLk7816MN+zvW4LD059TdU5y9fxaDfDTaaVqmKd4e3rxe/XVsdLYp6/59a6Hz9NA++CNHah/0dOJW5ZxcZNIkbRSe3pnbffqAr682MjWVdX7+WavRHzqklXRSk1JLbF98oR2wTe3OHa0888sv2h5meDj8+++T46xd+9Houn17LSn/9dej5197Tdt7PXgw43VMn66Vl/z9wdFRe2z2bK0f/+5dKFRIK8sULQpjxmh7L6B9IZQrB0OGaCWbjPTuDbt3w717aUs65ctr9f6nPUHsOWTbXDpSyguJG3jcYg2AK1LKq4nLrge6AI9N+Er6TKUe0AYSpp/vP3w0J82V286s7j+dDxtN4a+rfzDyt5VEi5PE6+7xx5Xt/HFlu7YyN7Dxht5n4b0jUP9uGNWW/soLy3+lUHW42AhEGSuKWNfHEFWfBsXasnWAdkBOm6o5NkXZJ+bd9yl46RKyRg3El19mGLeSy8TFwcaNaR/39oaXX9ZKMsOGQUCAdjJS+/aPkn23blodvl49LdFv3Kh11TRvrj3ftKm2TM2a2n/YZcu0poEGiQf0HR3BM8PiQPo+/VTbM33/fS227du1m2mvArSkXrEifPed1qhgej/z5mlfNGPHwtWr2hfe6NFasgdtj2XcOO2ArLOzNqqfM0frABo58tH6X39dew+1a2tfPhs2aLd589Im+yNHtD2Ur7/O3PvMBjlx0LYkcDPZ/QCgYXoLCiGGAEMAypQpk/2R5RNGo0wxJ0290tpFUk74h1C/TCk6lJjIiRshVC4RSY/GCdyK+I8VRw5yP+ouBQsk4NdOz8CXjbS4ZUWf3YE0PH6bvmclfc9CTP3afFyqDVsrN+HSLZsUB3PrlylM/TLOnLgRwtDgUxRctYI4vRUTuo3lS1s7dYAorwgPh5490z6+d6/W4TV+vJYkCxXSRv0zZz5apkkTLdHNmqUlxerVtdG7KYk3bqxNQ3D9uraXUK+ets5SpZ493qZNtS+WTz7RDhiXL6+1i7Zr92gZKbVEbDQ+eszZWRuBjxgBnTpp5aZRo7Skn9y4cdrrpk/XyleenlrLafLjGVWqaF8mN29q26peHdasgX790sa7fr3Wqmqus2uTeWJJRwjxF1AsnacmSCk3Jy6zj4xLOj2ADlLKwYn3+wENpZQjHrddVdJ5eoHhsTSevpsEo0QvYOvIpnRa8A8JRomVTvDPuNYASV8CtUs5cSogDEPi8tvebUaVYo6PuoGu+7PF+xNa7v+NwjERANxzcGFf69dp/fWnNF5+JsW69SEhuNSvhS44iCmtvVjTsCuHP26jRvSKYgaPK+k8cRAmpWwrpayZzm3zU27/FlA62f1SiY8pWST5QVHPci5UKeaY4iBpEUdbdEJwIrH8YprDRi+goK0Vr84/mGKytCDXYnxYvw+N3lnFhA4jSKhWnaIRwfTasgy36pVYvm8htQKvJq3bbdY0dMFBnKtcn+8bdFYHZhUll8qSC6A8YYRvhXbQtg1aoj8O9JVSnnvcOtUIP3NSHxRNfV+mutDKj4Mbcvl+BK/OP4ghcbRuGpWnXna9V0PE3r1au93WrUln7crmzRFNmmi7+EJgPHGSoPKV1YFZRTGjx43wn7dLpxswH3AHQgE/KWV7IUQJYLmUsmPich2BuYAe+E5KOe1J61YJP+s96Usg+eyaGX6B3L2BWLhQq1+Ghz9a+YgRMH++md6Zoigm2Zbws5NK+Dnjadol0512ITyc5cOnEXXjFk4lijBg+VR0DmmnbFAUJWepSxwqGXqadsl058nX2TKzTAsSSmnloNekFe45FLOiKM9Gdc4pT5TembLq7FlFyXtUSUd5KumVftTZs4qS+zxXW6aS/z3uwuUm6U3mlt5jiqLkXqqGb+HSOyBrmgZZUZT8RY3wLdyTLlyuKEr+oRK+hVMHXxXFcqiSjoXL6OpaiqLkPyrhK2rqYkWxEKqkoyiKYiFUwlcURbEQKuEriqJYCJXwFUVRLIRK+IqiKBZCJXxFURQLkWsnTxNCBAL+2bR6N+BBNq07J6j4zS+vv4e8Hj/k/feQXfGXlVKmO1t5rk342UkI4ZPRbHJ5gYrf/PL6e8jr8UPefw/miF+VdBRFUSyESviKoigWwlIT/lJzB/CcVPzml9ffQ16PH/L+e8jx+C2yhq8oimKJLHWEryiKYnFUwlcURbEQFpnwhRBThRCnhRB+Qog/hRAlzB1TZgkhZgkhLia+j01CiMLmjikzhBA9hRDnhBBGIUSeaa0TQnQQQvwrhLgihBhn7ngySwjxnRDivhDirLljeRZCiNJCiL1CiPOJ/3/eM3dMmSWEsBNCHBNCnEp8D5NzbNuWWMMXQhSSUj5M/PldoLqU0tvMYWWKEKIdsEdKmSCE+BJASjnWzGE9NSFENcAILAE+lFL6mDmkJxJC6IFLwMtAAHAc6COlPG/WwDJBCNEciADWSClrmjuezBJCFAeKSylPCCEcAV+gax77GwjAXkoZIYSwBg4C70kpj2T3ti1yhG9K9onsgTz3rSel/FNKmZB49whQypzxZJaU8oKU8l9zx5FJDYArUsqrUso4YD3QxcwxZYqU8gAQbO44npWU8o6U8kTiz+HABaCkeaPKHKmJSLxrnXjLkRxkkQkfQAgxTQhxE3gTmGjueJ7T/4A/zB2EBSgJ3Ex2P4A8lmzyEyFEOaAecNTMoWSaEEIvhPAD7gO7pJQ58h7ybcIXQvwlhDibzq0LgJRygpSyNLAWGGHeaNP3pPeQuMwEIAHtfeQqTxO/ojwLIYQD8Avwfqo99jxBSmmQUtZF2zNvIITIkfJavr2mrZSy7VMuuhbYDnyWjeE8kye9ByHEQOA1oI3MhQdjMvE3yCtuAaWT3S+V+JiSgxLr3r8Aa6WUv5o7nuchpQwVQuwFOgDZfiA9347wH0cIUSnZ3S7ARXPF8qyEEB2Aj4DOUsooc8djIY4DlYQQ5YUQNkBvYIuZY7IoiQc8VwAXpJRzzB3PsxBCuJu66oQQBdCaAHIkB1lql84vQBW0LhF/wFtKmadGakKIK4AtEJT40JG81GkkhOgGzAfcgVDAT0rZ3qxBPQUhREdgLqAHvpNSTjNvRJkjhFgHtESbmvce8JmUcoVZg8oEIURT4G/gDNrnF2C8lHK7+aLKHCFEbWA12v8hHfCTlHJKjmzbEhO+oiiKJbLIko6iKIolUglfURTFQqiEryiKYiFUwlcURbEQKuEriqJYCJXwFUVRLIRK+IqiKBbi/7+2ksw71qi7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "net.apply(weight_reset)\n",
    "\n",
    "loss_value = []\n",
    "\n",
    "for i in range(1000):\n",
    "    prediction = net(x_data)\n",
    "    \n",
    "    loss = loss_func(prediction, y_data)\n",
    "\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    loss_value.append(loss.cpu().detach().numpy())\n",
    "\n",
    "    if i % 10 == 0:\n",
    "        pl.axis([-4, 4, -2, 2]) \n",
    "        pl.clf()\n",
    "        pl.scatter(x.numpy(), y.numpy(), s=5)\n",
    "        pl.plot(x.numpy(), torch.sin(x).numpy(), 'g', lw=2, label='GroundTruth')\n",
    "        pl.plot(x.numpy(), prediction.cpu().detach().numpy(), 'r-', lw=2, label='Prediction')\n",
    "        pl.text(1.0, -1.0, 'Loss=%.4f' % loss.cpu().detach().numpy(), fontdict={'size': 15, 'color':  'red'})\n",
    "        pl.legend()\n",
    "        display.display(pl.gcf())\n",
    "        display.clear_output(wait=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "损失值可视化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAYfklEQVR4nO3df5RUdf3H8ed7fwAJIgprIQvuaiiyyRd0+aFoWSRifdNKKzh0II20UyqW5/s9eiwwNfv27Zd9T2hq8rUMRUUP7UmLbyFmohlLEPIjdDOMRfyybnw1lR+77Pv7x70LszOz7OzsLHfvndfjnDkz994Pc9937vKaz3zmzr3m7oiISPyVRF2AiIgUhgJdRCQhFOgiIgmhQBcRSQgFuohIQpRFteJhw4Z5VVVVVKsXEYmltWvXvu7uFdmWRRboVVVV1NfXR7V6EZFYMrNXOlumIRcRkYRQoIuIJIQCXUQkISIbQxeR+GtpaaGxsZG9e/dGXUriDBgwgMrKSsrLy3P+Nwp0EclbY2MjRx99NFVVVZhZ1OUkhrvT3NxMY2Mj1dXVOf87DbmISN727t3L0KFDFeYFZmYMHTq02598FOgi0iMK896Rz+sav0B/5hlYsABaWqKuRESkT4lfoD/3HNxyC+zfH3UlIpIwVVVVvP7663m1SZ1/9tln90p9XYlfoJeEJbe1RVuHiEgnnn322UjWq0AXkdjatm0bY8aM4XOf+xynnHIKs2fP5re//S1Tp05l9OjR/PGPfwTgH//4Bx//+McZN24cU6ZMYcOGDQA0Nzczffp0ampqmDdvHqlXcPv5z3/OpEmTGD9+PFdeeSUHDhzIua5BgwYB8NRTT3Heeedx6aWXMmbMGGbPnn1wHWvXruUDH/gAZ555JhdccAE7d+7s8esRv8MWFegifdO118L69YV9zvHj4fbbD9ukoaGBRx55hMWLFzNx4kQeeOABnnnmGerq6rjttttYvnw5CxcuZMKECSxfvpwnn3ySOXPmsH79er7xjW9wzjnnsGDBAh5//HHuvfdeALZs2cJDDz3E6tWrKS8v50tf+hJLlixhzpw53d6EdevWsWnTJk444QSmTp3K6tWrmTx5MldffTW/+MUvqKio4KGHHuLGG29k8eLFebxIhyjQRSTWqqurOf300wGoqalh2rRpmBmnn34627ZtA+CZZ57h0UcfBeBDH/oQzc3NvPnmmzz99NM89thjAHz0ox/l2GOPBWDlypWsXbuWiRMnArBnzx6OP/74vOqbNGkSlZWVAIwfP55t27YxZMgQNm7cyPnnnw/AgQMHGD58eH4vQAoFuogURhc96d7Sv3//g49LSkoOTpeUlNDa2prXc7o7c+fO5Vvf+lZB6ystLaW1tRV3p6amhueee67Hz58qpzF0M5thZlvNrMHMrs+yfJSZrTKzdWa2wcw+UtAqU7UHejfGs0SkuJ177rksWbIECMa1hw0bxuDBg3n/+9/PAw88AMCvfvUrdu/eDcC0adNYtmwZu3btAoIx+Fde6fSstd126qmn0tTUdDDQW1pa2LRpU4+ft8seupmVAouA84FGYI2Z1bn75pRmXwMedvc7zWws8ARQ1ePqslEPXUS66aabbuLyyy9n3LhxHHXUUfz0pz8FYOHChcyaNYuamhrOPvtsRo0aBcDYsWO59dZbmT59Om1tbZSXl7No0SJOPPHEgtTTr18/li1bxjXXXMMbb7xBa2sr1157LTU1NT16Xkv9VjdrA7OzgJvc/YJw+gYAd/9WSpu7gJfd/dth+++5+2EPxKytrfW8LnDxk5/AF74A27dDOC4lItHYsmULp512WtRlJFa219fM1rp7bbb2uYyhjwC2p0w3ApPT2twE/I+ZXQ0MBD6ca8Hdph66iEhWhToOfRZwn7tXAh8B7jezjOc2syvMrN7M6puamvJbkwJdRCSrXAJ9BzAyZboynJfq88DDAO7+HDAAGJb+RO5+t7vXunttRUXWa5zmULECXaQv6WrYVvKTz+uaS6CvAUabWbWZ9QNmAnVpbf4OTAMws9MIAj3PLngXFOgifcaAAQNobm5WqBdY+/nQBwwY0K1/1+UYuru3mtlVwAqgFFjs7pvM7Gag3t3rgOuAe8zsK4ADn/Pe2sMKdJE+o7KyksbGRvIeQpVOtV+xqDty+mGRuz9BcChi6rwFKY83A1O7teZ86Th0kT6jvLy8W1fUkd4Vv5NzlZYG9+qhi4h0EL9A15CLiEhWCnQRkYRQoIuIJIQCXUQkIRToIiIJoUAXEUmI+Aa6jkMXEekgfoGu49BFRLKKX6BryEVEJCsFuohIQijQRUQSQoEuIpIQCnQRkYSIb6DrsEURkQ7iG+jqoYuIdBC/QNdx6CIiWcUv0NVDFxHJSoEuIpIQCnQRkYRQoIuIJIQCXUQkIeIb6DoOXUSkg/gFug5bFBHJKn6BriEXEZGsFOgiIgmhQBcRSQgFuohIQijQRUQSQoEuIpIQ8Q10HYcuItJB/AJdx6GLiGQVv0DXkIuISFYKdBGRhFCgi4gkhAJdRCQhFOgiIgkR30DXYYsiIh3EN9DVQxcR6SCnQDezGWa21cwazOz6Ttp82sw2m9kmM3ugsGWm0HHoIiJZlXXVwMxKgUXA+UAjsMbM6tx9c0qb0cANwFR3321mx/dWwZgF9wp0EZEOcumhTwIa3P1ld98PLAUuTmvzBWCRu+8GcPddhS0zhVlwU6CLiHSQS6CPALanTDeG81KdApxiZqvN7A9mNiPbE5nZFWZWb2b1TU1N+VUMwTi6Al1EpINCfSlaBowGzgNmAfeY2ZD0Ru5+t7vXunttRUVF/mtToIuIZMgl0HcAI1OmK8N5qRqBOndvcfe/AS8SBHzvUKCLiGTIJdDXAKPNrNrM+gEzgbq0NssJeueY2TCCIZiXC1dmmpISHYcuIpKmy0B391bgKmAFsAV42N03mdnNZnZR2GwF0Gxmm4FVwL+5e3NvFU1pqXroIiJpujxsEcDdnwCeSJu3IOWxA18Nb71PQy4iIhni90tRUKCLiGShQBcRSQgFuohIQijQRUQSQoEuIpIQ8Q10HYcuItJBPANdx6GLiGSIZ6BryEVEJIMCXUQkIRToIiIJEd9A15eiIiIdxDPQS0sV6CIiaRToIiIJEc9ALytToIuIpIlnoJeWQmtr1FWIiPQp8Q109dBFRDqIZ6BryEVEJEM8A11DLiIiGeIb6Oqhi4h0oEAXEUmIeAZ6WZmGXERE0sQz0NVDFxHJoEAXEUmIeAa6DlsUEckQz0DXYYsiIhniG+jqoYuIdBDPQNeQi4hIhngGuoZcREQyxDfQ1UMXEekgnoGuIRcRkQzxDHQNuYiIZIhvoKuHLiLSQTwDXUMuIiIZ4hnoGnIREckQ30BXD11EpAMFuohIQsQz0HU+dBGRDPEM9NJSaGsD96grERHpM3IKdDObYWZbzazBzK4/TLtLzMzNrLZwJWZRWhrct7X16mpEROKky0A3s1JgEXAhMBaYZWZjs7Q7GpgPPF/oIjOUlQX3GkcXETkolx76JKDB3V929/3AUuDiLO1uAb4N7C1gfdm199A1ji4iclAugT4C2J4y3RjOO8jMzgBGuvvjh3siM7vCzOrNrL6pqanbxR7UHujqoYuIHNTjL0XNrAT4PnBdV23d/W53r3X32oqKivxXqiEXEZEMuQT6DmBkynRlOK/d0cD7gKfMbBswBajr1S9GNeQiIpIhl0BfA4w2s2oz6wfMBOraF7r7G+4+zN2r3L0K+ANwkbvX90rFoCEXEZEsugx0d28FrgJWAFuAh919k5ndbGYX9XaBWWnIRUQkQ1kujdz9CeCJtHkLOml7Xs/L6oKGXEREMsT3l6KgHrqISIp4BrqGXEREMsQz0DXkIiKSId6Brh66iMhBCnQRkYSIZ6C3j6FryEVE5KB4Brp66CIiGRToIiIJEc9A15CLiEiGeAa6eugiIhniGejqoYuIZIhnoJeXB/cKdBGRg+IZ6P36Bff790dbh4hIHxLPQG/voSvQRUQOimegq4cuIpIh3oHe0hJtHSIifUi8A109dBGRgxToIiIJEc9A15eiIiIZ4hno6qGLiGSId6DrS1ERkYPiGejtP/3fvz+47d0bbT0iIn1APAPdLBhH370bBg6EefOirkhEJHLxDHQIhl3WrAnO57JkCbhHXZGISKTiG+j9+8O6dYem33wzulpERPqA+Ab6oEGwb9+h6Vdfja4WEZE+IL6BPnBgx2kFuogUufgHevuPjBToIlLk4h/oEyYE9wp0ESly8Q30QYOC+5NPhsGDYceOaOsREYlYfAO9vYf+nvfACSfAzp3R1iMiErH4BnppaXA/YkQQ6gp0ESly8Q/044+H4cMV6CJS9MqiLiBvt94anJzrE5+A9euDQHcPTgsgIlKE4ttDP/FEWLo0+HJ0+HDYs0e/FhWRohbfQE81fHhwr2EXESliCnQRkYRIVqC/9lq0dYiIRChZga4euogUsZwC3cxmmNlWM2sws+uzLP+qmW02sw1mttLMTix8qYdxzDHBl6OvvHJEVysi0pd0GehmVgosAi4ExgKzzGxsWrN1QK27jwOWAf9Z6EK7KBJqauCFF47oakVE+pJceuiTgAZ3f9nd9wNLgYtTG7j7Knd/J5z8A1BZ2DJzMG4cbNigKxeJSNHKJdBHANtTphvDeZ35PPCrbAvM7Aozqzez+qamptyrzMXpp0NzM2zf3nVbEZEEKuiXomb2WaAW+E625e5+t7vXunttRUVFIVcN06cH91OnwowZOvuiiBSdXAJ9BzAyZboynNeBmX0YuBG4yN33pS/vdaeeCt/9Lpx0EqxaBZdddsRLEBGJUi6BvgYYbWbVZtYPmAnUpTYwswnAXQRhvqvwZebouuvgd7+D226D3/wG1q6NrBQRkSOty0B391bgKmAFsAV42N03mdnNZnZR2Ow7wCDgETNbb2Z1nTzdkTFvXnC+9DvuiLQMEZEjyTyio0Jqa2u9vr6+91Ywb15w8q6dO+Hoo3tvPSIiR5CZrXX32mzLkvFL0WzmzYO334YHH4y6EhGRIyK5gT55cnAo4+23w4EDUVcjItLrkhvoZrBgAWzZAvffH3U1IiK9LrmBDnDJJTBxInz96/DOO123FxGJsWQHuhl873vQ2Bgcoy4ikmDJDnSAc8+FT34yCPa33oq6GhGRXpP8QIfgB0dvvglLlkRdiYhIrymOQD/rLJgwAX70I52NUUQSqzgC3Qy+/GXYuBFWr466GhGRXlEcgQ4wcyYMHgx33RV1JSIivaJ4An3gQPjsZ+GRR4LzpouIJEzxBDrAlVfCvn3ws59FXYmISMEVV6CPGwdTpgTDLvpyVEQSprgCHeCLX4StW+Hpp6OuRESkoIov0D/9aRgyRF+OikjiFF+gv+tdMGcOLFsGhb5QtYhIhIov0CH4crSlBe67L+pKREQKpjgDfezY4Bwvd9wBra1RVyMiUhDFGegQnN9l27Zg6EVEJAGKN9A/9jE49VT4znd0CKOIJELxBnpJSdBL/9OfYMWKqKsREemx4g10CI52qa6G66+HtraoqxER6ZHiDvT+/eGb34Q//1nnSheR2CvuQAf4zGfgzDODXvobb0RdjYhI3hToJSVw553w2mtBqIuIxJQCHWDiRJg/H378Y1i5MupqRETyokBvd8stcNppMGsWNDZGXY2ISLcp0NsNHAiPPQZ79sCll8LevVFXJCLSLQr0VGPGBOd3ef55mD0bDhyIuiIRkZwp0NNdcgncfnvQW7/sMti/P+qKRERyUhZ1AX3S/Pnw1lvwta/Bzp3BdUiHDIm6KhGRw1IPvTM33hgMvzz1VHB2xmXLdM4XEenTFOiHM3cuPPccHH88fOpTMHky/PrXCnYR6ZMU6F2prYX6erj3Xti1Cy68EM44I+i960gYEelDFOi5KCuDyy+HF1+Ee+4JrnZ02WUwciRcdRU8+6x67SISOQV6d/TrB/PmwQsvBL8o/eAHg5771KnBWRvnz4fHH4e33466UhEpQuYR9Sxra2u9vr4+knUX1D//CcuXw9KlsGpV8MOk8nKYMiU4pcDEicGwTXU1lJZGXa2IxJyZrXX32qzLFOgFtHcvrF4dXDDj97+Hdetg375gWXl5EOqjR8PJJ8OIEVBR0fF2zDEwaFDwScAs2m0RkT7pcIGe03HoZjYD+CFQCvzE3f8jbXl/4GfAmUAz8Bl339aTomNpwACYNi24QTDWvnFjcFWkl16Chobg9tRThx+WKSsLgr2z21FHBesaMCA4p/vhHpeXB58Myso63qc+NgtuJSWHHqff8lnWPj9deiciW6eipCSoryQcFWx/nvT7dNnmH65t6vPlU2su21JWFtz0Jt19b78d/Cbk3e+OupLucY9kf3cZ6GZWCiwCzgcagTVmVufum1OafR7Y7e7vNbOZwLeBz/RGwbFSXg4TJgS3VO7BH2pTU8fbP/8Z/PF2dnv11aDNnj3Bp4F9+4L7lpZotk8KI5c3oVzfqLK9oaS/cXV2351ludSZ6zqyzdu/P/g/smdPMG/w4GC+e3DepfaOSGfr7Mlj98xbW1vu03v3QmsrjBoV/N8sLQ0+daeuZ+FCmDmTQsulhz4JaHD3l4NabClwMZAa6BcDN4WPlwE/MjPzqMZz+jqzQ73t6uqeP19b26FwTw369rA/cCD4A+vs/nB/qD1d1taWX2C1tQW1tz8PZN6nyzb/cG1Tn6/9lk+th5t2D17j1tbs9eRSc65tsgXc4V6/9O3PdVkudea6js7m9esXBPeQIcGnm8bGYH5pKbzzzqHzLGVbZ08et7+Oh/sE2tV0//7B/Y4dwXYcOHDoFCJmwd/20KH0hlwCfQSwPWW6EZjcWRt3bzWzN4ChwOupjczsCuAKgFGjRuVZsmQoKYF3vSu4iUjROqKHLbr73e5e6+61FRUVR3LVIiKJl0ug7wBGpkxXhvOytjGzMuAYgi9HRUTkCMkl0NcAo82s2sz6ATOBurQ2dcDc8PGlwJMaPxcRObK6HEMPx8SvAlYQHLa42N03mdnNQL271wH3AvebWQPwD4LQFxGRIyin49Dd/QngibR5C1Ie7wU+VdjSRESkO3QuFxGRhFCgi4gkhAJdRCQhIjs5l5k1Aa/k+c+HkfajpSKgbS4O2ubi0JNtPtHds/6QJ7JA7wkzq+/sbGNJpW0uDtrm4tBb26whFxGRhFCgi4gkRFwD/e6oC4iAtrk4aJuLQ69scyzH0EVEJFNce+giIpJGgS4ikhCxC3Qzm2FmW82swcyuj7qeQjGzkWa2ysw2m9kmM5sfzj/OzH5jZi+F98eG883M/it8HTaY2RnRbkF+zKzUzNaZ2S/D6Wozez7crofCM3xiZv3D6YZweVWkhefJzIaY2TIz+4uZbTGzs4pgH38l/JveaGYPmtmAJO5nM1tsZrvMbGPKvG7vWzObG7Z/yczmZltXZ2IV6CnXN70QGAvMMrOx0VZVMK3Ade4+FpgCfDnctuuBle4+GlgZTkPwGowOb1cAdx75kgtiPrAlZfrbwA/c/b3AboLr1ULKdWuBH4Tt4uiHwK/dfQzwLwTbnth9bGYjgGuAWnd/H8EZW9uvO5y0/XwfMCNtXrf2rZkdBywkuCrcJGBh+5tATtw9NjfgLGBFyvQNwA1R19VL2/oLggtzbwWGh/OGA1vDx3cBs1LaH2wXlxvBxVJWAh8CfgkYwa/nytL3N8Hpm88KH5eF7Szqbejm9h4D/C297oTv4/bLUx4X7rdfAhckdT8DVcDGfPctMAu4K2V+h3Zd3WLVQyf79U1HRFRLrwk/Zk4Angfe7e47w0WvAe8OHyfhtbgd+HegLZweCvyfu4dXVO6wTR2uWwu0X7c2TqqBJuC/w2Gmn5jZQBK8j919B/Bd4O/AToL9tpZk7+dU3d23PdrncQv0xDOzQcCjwLXu/mbqMg/eshNxnKmZ/Suwy93XRl3LEVQGnAHc6e4TgLc59BEcSNY+BgiHCy4meDM7ARhI5rBEUTgS+zZugZ7L9U1jy8zKCcJ8ibs/Fs7+XzMbHi4fDuwK58f9tZgKXGRm24ClBMMuPwSGhNelhY7blITr1jYCje7+fDi9jCDgk7qPAT4M/M3dm9y9BXiMYN8neT+n6u6+7dE+j1ug53J901gyMyO4lN8Wd/9+yqLU67XOJRhbb58/J/y2fArwRspHuz7P3W9w90p3ryLYj0+6+2xgFcF1aSFze2N93Vp3fw3YbmanhrOmAZtJ6D4O/R2YYmZHhX/j7duc2P2cprv7dgUw3cyODT/dTA/n5SbqLxHy+NLhI8CLwF+BG6Oup4DbdQ7Bx7ENwPrw9hGC8cOVwEvAb4HjwvZGcMTPX4EXCI4iiHw78tz284Bfho9PAv4INACPAP3D+QPC6YZw+UlR153nto4H6sP9vBw4Nun7GPgG8BdgI3A/0D+J+xl4kOB7ghaCT2Ofz2ffApeH298AXNadGvTTfxGRhIjbkIuIiHRCgS4ikhAKdBGRhFCgi4gkhAJdRCQhFOgiIgmhQBcRSYj/B3xKd0xPwmuzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "pl.plot([i for i in range(len(loss_value))], loss_value, label='model line', color='r')\n",
    "pl.legend()\n",
    "pl.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pytorch1.6.0",
   "language": "python",
   "name": "pytorch1.6.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "303.2px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}