Dropout.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout\n",
    "Dropout [1] is a technique for regularizing neural networks by randomly setting some features to zero during the forward pass. In this exercise you will implement a dropout layer and modify your fully-connected network to optionally use dropout.\n",
    "\n",
    "[1] Geoffrey E. Hinton et al, \"Improving neural networks by preventing co-adaptation of feature detectors\", arXiv 2012"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# As usual, a bit of setup\n",
    "\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from deeplearning.classifiers.fc_net import *\n",
    "from deeplearning.data_utils import get_CIFAR10_data\n",
    "from deeplearning.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array\n",
    "from deeplearning.solver import Solver\n",
    "\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "\n",
    "# for auto-reloading external modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "def rel_error(x, y):\n",
    "  \"\"\" returns relative error \"\"\"\n",
    "  return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_val:  (1000, 3, 32, 32)\n",
      "X_train:  (49000, 3, 32, 32)\n",
      "X_test:  (1000, 3, 32, 32)\n",
      "y_val:  (1000,)\n",
      "y_train:  (49000,)\n",
      "y_test:  (1000,)\n"
     ]
    }
   ],
   "source": [
    "# Load the (preprocessed) CIFAR10 data.\n",
    "\n",
    "data = get_CIFAR10_data()\n",
    "for k, v in data.iteritems():\n",
    "  print '%s: ' % k, v.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout forward pass\n",
    "In the file `deeplearning/layers.py`, implement the forward pass for dropout. Since dropout behaves differently during training and testing, make sure to implement the operation for both modes.\n",
    "\n",
    "Once you have done so, run the cell below to test your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running tests with p =  0.3\n",
      "Mean of input:  9.999078300567252\n",
      "Mean of train-time output:  3.012698173127907\n",
      "Mean of test-time output:  9.999078300567252\n",
      "Fraction of train-time output set to zero:  0.698764\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.6\n",
      "Mean of input:  9.999078300567252\n",
      "Mean of train-time output:  5.991648360434462\n",
      "Mean of test-time output:  9.999078300567252\n",
      "Fraction of train-time output set to zero:  0.400696\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n",
      "Running tests with p =  0.75\n",
      "Mean of input:  9.999078300567252\n",
      "Mean of train-time output:  7.50996772975719\n",
      "Mean of test-time output:  9.999078300567252\n",
      "Fraction of train-time output set to zero:  0.248852\n",
      "Fraction of test-time output set to zero:  0.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "x = np.random.randn(500, 500) + 10\n",
    "\n",
    "for p in [0.3, 0.6, 0.75]:\n",
    "  out, _ = dropout_forward(x, {'mode': 'train', 'p': p})\n",
    "  out_test, _ = dropout_forward(x, {'mode': 'test', 'p': p})\n",
    "\n",
    "  print 'Running tests with p = ', p\n",
    "  print 'Mean of input: ', x.mean()\n",
    "  print 'Mean of train-time output: ', out.mean()\n",
    "  print 'Mean of test-time output: ', out_test.mean()\n",
    "  print 'Fraction of train-time output set to zero: ', (out == 0).mean()\n",
    "  print 'Fraction of test-time output set to zero: ', (out_test == 0).mean()\n",
    "  print"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dropout backward pass\n",
    "In the file `deeplearning/layers.py`, implement the backward pass for dropout. After doing so, run the following cell to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dx relative error:  1.8928976827484326e-11\n"
     ]
    }
   ],
   "source": [
    "x = np.random.randn(10, 10) + 10\n",
    "dout = np.random.randn(*x.shape)\n",
    "\n",
    "dropout_param = {'mode': 'train', 'p': 0.8, 'seed': 123}\n",
    "out, cache = dropout_forward(x, dropout_param)\n",
    "dx = dropout_backward(dout, cache)\n",
    "dx_num = eval_numerical_gradient_array(lambda xx: dropout_forward(xx, dropout_param)[0], x, dout)\n",
    "\n",
    "print 'dx relative error: ', rel_error(dx, dx_num)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fully-connected nets with Dropout\n",
    "In the file `deeplearning/classifiers/fc_net.py`, modify your implementation to use dropout. Specificially, if the constructor the the net receives a nonzero value for the `dropout` parameter, then the net should add dropout immediately after every ReLU nonlinearity. After doing so, run the following to numerically gradient-check your implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running check with dropout =  0\n",
      "Initial loss:  2.3031564446027835\n",
      "W1 relative error: 4.41e-07\n",
      "W2 relative error: 2.06e-06\n",
      "W3 relative error: 9.60e-08\n",
      "b1 relative error: 2.94e-08\n",
      "b2 relative error: 3.73e-08\n",
      "b3 relative error: 1.12e-10\n",
      "\n",
      "Running check with dropout =  0.25\n",
      "Initial loss:  2.30269483887292\n",
      "W1 relative error: 6.80e-07\n",
      "W2 relative error: 1.01e-07\n",
      "W3 relative error: 2.55e-06\n",
      "b1 relative error: 1.32e-08\n",
      "b2 relative error: 1.53e-09\n",
      "b3 relative error: 1.93e-10\n",
      "\n",
      "Running check with dropout =  0.5\n",
      "Initial loss:  2.3010415621760134\n",
      "W1 relative error: 7.25e-07\n",
      "W2 relative error: 3.30e-06\n",
      "W3 relative error: 8.18e-07\n",
      "b1 relative error: 4.72e-09\n",
      "b2 relative error: 2.48e-09\n",
      "b3 relative error: 8.63e-11\n",
      "\n"
     ]
    }
   ],
   "source": [
    "N, D, H1, H2, C = 2, 15, 20, 30, 10\n",
    "X = np.random.randn(N, D)\n",
    "y = np.random.randint(C, size=(N,))\n",
    "\n",
    "for dropout in [0, 0.25, 0.5]:\n",
    "  print 'Running check with dropout = ', dropout\n",
    "  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,\n",
    "                            weight_scale=5e-2, dtype=np.float64,\n",
    "                            dropout=dropout, seed=123)\n",
    "\n",
    "  loss, grads = model.loss(X, y)\n",
    "  print 'Initial loss: ', loss\n",
    "\n",
    "  for name in sorted(grads):\n",
    "    f = lambda _: model.loss(X, y)[0]\n",
    "    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)\n",
    "    print '%s relative error: %.2e' % (name, rel_error(grad_num, grads[name]))\n",
    "  print"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regularization experiment\n",
    "As an experiment, we will train a pair of two-layer networks on 500 training examples: one will use no dropout, and one will use a dropout probability of 0.75. We will then visualize the training and validation accuracies of the two networks over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "(Iteration 1 / 125) loss: 7.799736\n",
      "(Epoch 0 / 25) train acc: 0.248000; val_acc: 0.216000\n",
      "(Epoch 1 / 25) train acc: 0.390000; val_acc: 0.230000\n",
      "(Epoch 2 / 25) train acc: 0.498000; val_acc: 0.282000\n",
      "(Epoch 3 / 25) train acc: 0.622000; val_acc: 0.250000\n",
      "(Epoch 4 / 25) train acc: 0.702000; val_acc: 0.292000\n",
      "(Epoch 5 / 25) train acc: 0.754000; val_acc: 0.255000\n",
      "(Epoch 6 / 25) train acc: 0.804000; val_acc: 0.273000\n",
      "(Epoch 7 / 25) train acc: 0.838000; val_acc: 0.272000\n",
      "(Epoch 8 / 25) train acc: 0.856000; val_acc: 0.277000\n",
      "(Epoch 9 / 25) train acc: 0.890000; val_acc: 0.288000\n",
      "(Epoch 10 / 25) train acc: 0.898000; val_acc: 0.279000\n",
      "(Epoch 11 / 25) train acc: 0.938000; val_acc: 0.294000\n",
      "(Epoch 12 / 25) train acc: 0.934000; val_acc: 0.302000\n",
      "(Epoch 13 / 25) train acc: 0.938000; val_acc: 0.320000\n",
      "(Epoch 14 / 25) train acc: 0.944000; val_acc: 0.287000\n",
      "(Epoch 15 / 25) train acc: 0.958000; val_acc: 0.265000\n",
      "(Epoch 16 / 25) train acc: 0.970000; val_acc: 0.282000\n",
      "(Epoch 17 / 25) train acc: 0.966000; val_acc: 0.291000\n",
      "(Epoch 18 / 25) train acc: 0.970000; val_acc: 0.281000\n",
      "(Epoch 19 / 25) train acc: 0.988000; val_acc: 0.286000\n",
      "(Epoch 20 / 25) train acc: 0.974000; val_acc: 0.288000\n",
      "(Iteration 101 / 125) loss: 0.012044\n",
      "(Epoch 21 / 25) train acc: 0.984000; val_acc: 0.302000\n",
      "(Epoch 22 / 25) train acc: 0.988000; val_acc: 0.286000\n",
      "(Epoch 23 / 25) train acc: 0.994000; val_acc: 0.289000\n",
      "(Epoch 24 / 25) train acc: 0.994000; val_acc: 0.293000\n",
      "(Epoch 25 / 25) train acc: 1.000000; val_acc: 0.304000\n",
      "0.75\n",
      "(Iteration 1 / 125) loss: 7.849005\n",
      "(Epoch 0 / 25) train acc: 0.260000; val_acc: 0.172000\n",
      "(Epoch 1 / 25) train acc: 0.336000; val_acc: 0.209000\n",
      "(Epoch 2 / 25) train acc: 0.482000; val_acc: 0.254000\n",
      "(Epoch 3 / 25) train acc: 0.592000; val_acc: 0.264000\n",
      "(Epoch 4 / 25) train acc: 0.698000; val_acc: 0.254000\n",
      "(Epoch 5 / 25) train acc: 0.718000; val_acc: 0.283000\n",
      "(Epoch 6 / 25) train acc: 0.744000; val_acc: 0.256000\n",
      "(Epoch 7 / 25) train acc: 0.832000; val_acc: 0.284000\n",
      "(Epoch 8 / 25) train acc: 0.860000; val_acc: 0.290000\n",
      "(Epoch 9 / 25) train acc: 0.870000; val_acc: 0.251000\n",
      "(Epoch 10 / 25) train acc: 0.902000; val_acc: 0.278000\n",
      "(Epoch 11 / 25) train acc: 0.910000; val_acc: 0.281000\n",
      "(Epoch 12 / 25) train acc: 0.922000; val_acc: 0.314000\n",
      "(Epoch 13 / 25) train acc: 0.950000; val_acc: 0.280000\n",
      "(Epoch 14 / 25) train acc: 0.952000; val_acc: 0.290000\n",
      "(Epoch 15 / 25) train acc: 0.958000; val_acc: 0.292000\n",
      "(Epoch 16 / 25) train acc: 0.964000; val_acc: 0.298000\n",
      "(Epoch 17 / 25) train acc: 0.950000; val_acc: 0.307000\n",
      "(Epoch 18 / 25) train acc: 0.972000; val_acc: 0.298000\n",
      "(Epoch 19 / 25) train acc: 0.962000; val_acc: 0.295000\n",
      "(Epoch 20 / 25) train acc: 0.972000; val_acc: 0.323000\n",
      "(Iteration 101 / 125) loss: 0.378467\n",
      "(Epoch 21 / 25) train acc: 0.952000; val_acc: 0.297000\n",
      "(Epoch 22 / 25) train acc: 0.984000; val_acc: 0.300000\n",
      "(Epoch 23 / 25) train acc: 0.980000; val_acc: 0.305000\n",
      "(Epoch 24 / 25) train acc: 0.992000; val_acc: 0.294000\n",
      "(Epoch 25 / 25) train acc: 0.992000; val_acc: 0.301000\n"
     ]
    }
   ],
   "source": [
    "# Train two identical nets, one with dropout and one without\n",
    "\n",
    "num_train = 500\n",
    "small_data = {\n",
    "  'X_train': data['X_train'][:num_train],\n",
    "  'y_train': data['y_train'][:num_train],\n",
    "  'X_val': data['X_val'],\n",
    "  'y_val': data['y_val'],\n",
    "}\n",
    "\n",
    "solvers = {}\n",
    "dropout_choices = [0, 0.75]\n",
    "for dropout in dropout_choices:\n",
    "  model = FullyConnectedNet([500], dropout=dropout)\n",
    "  print dropout\n",
    "\n",
    "  solver = Solver(model, small_data,\n",
    "                  num_epochs=25, batch_size=100,\n",
    "                  update_rule='adam',\n",
    "                  optim_config={\n",
    "                    'learning_rate': 5e-4,\n",
    "                  },\n",
    "                  verbose=True, print_every=100)\n",
    "  solver.train()\n",
    "  solvers[dropout] = solver"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAJNCAYAAABkwMu0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3X2c1mWZ8P/PwTDoaAYqpMLgUxq7hCSGmtUvnzI0FekJsc2yJ2rL2G13SblfRfy4d1eT7vS23N3MSntEdAmxJ9YsazFLIFxIjXwsGDQBAy1HGYbj/uO6hmaGGZxh5prvzFyf9+vF65rv8T2v73lcc10Mc3Ce3/OMzESSJEmSVD2GFJ2AJEmSJKlvWQhKkiRJUpWxEJQkSZKkKmMhKEmSJElVxkJQkiRJkqqMhaAkSZIkVRkLQUlSVYiImoj4U0QcXnQukiQVLdxHUJLUH0XEn1od7ge8ADSXjz+Umd/s+6wkSRocLAQlSf1eRDwOfCAzf7SHNkMzc0ffZdV3BvNrkyQVw6mhkqQBKSL+OSJujohvR8SzwLsi4pSI+EVEbI2IJyLi2oioLbcfGhEZEUeWj79RPv+DiHg2Iu6JiKM66WtIRNwaEU+Wr31XRPx1q/P7RcTVEfH7iNgWET+LiH3K595QzmlbRKyPiIvL8eURcUmra3wgIu5ql+tHIuJh4Dfl+BciYkNEPBMRKyLita2ePzQiPhURj5TPr4yI0RHxxYj4TLvX84OIuLTn74IkaaCyEJQkDWRvAb4FDAduBnYAfweMBF4HnA18aA/PfyfwKeAg4PfA/95D2+8CxwKHAr8Gvt7q3NXARODk8rX+F7CzXFh+D/gccDAwCVjbjdc3FTgROK58/MtyPwcBtwK3tBScwGzg7ZRe8wjgA8DzwE3AOyNiCEBEHAKcCizsRh6SpEHGQlCSNJAtz8zbM3NnZjZm5orM/GVm7sjMR4HrKRU9nbk1M1dmZhPwTeD4jhqVr39jZj6bmc8D84BXR8T+EVEDXALMyswnMrM5M5eXr/ku4IeZuaic0+bMvK8br+9fM/OPmdlYzuPrmfl0eZroVcBLgWPKbT8A/K/MfKic733ltj8HGlt9Hy4CfpSZm7uRhyRpkLEQlCQNZOtbH0TEX0XE98pTOJ8B5lMaHezMk62+fg54SUeNyiuOXhURj5av+3D51EjgEGAY8EgHTx3bSbyr2r++T0TEbyJiG/BHYH/+8vr21NfXKBWllB+/3kk7SVKVsBCUJA1k7Vc8+yKlaZvHZOZLgblA9EI/7wbeDJxBaRpqyyhcAH8AtgMv7+B56zuJA/yZ0mqoLQ7toM2u1xcRpwP/ALyN0tTPA4E/8ZfXt6e+vg68NSImldvc3kk7SVKVsBCUJA0mBwDbgD+XF3PZ0/2B3b3uC8AWSsXbv7ScyMxm4Ebgmog4tDx6+LryIjXfAM6OiLeVF3MZGRGvKj/1PuBtEVEXEa8A3teFHHYAm4FaStNT9291/gbgnyPi5VFyfEQcVM7xd+X+bgJuKU9vlSRVMQtBSdJg8o/Ae4BnKY0O3txL1/0qsLH8537g5+3Ofxx4EFgFPA38K6Utmh4DzgcuK8d/xV8WfvkspRG/p4CvUCoa9+T7wI+Ah4DHgWeAJ1qdXwAsAe4sn7se2LfV+ZvKfTstVJLkPoKSJFWDiDgD+DJwdPqPvyRVPUcEJUka5CJiGKVtNb5kEShJAgtBSZIGtYg4jtIKowcB1xacjiSpn3BqqCRJkiRVGUcEJUmSJKnKWAhKkiRJUpUZWnQCvWXkyJF55JFHFp2GJEmSJBVi1apVmzNzVFfaDppC8Mgjj2TlypVFpyFJkiRJhYiI33W1rVNDJUmSJKnKWAhKkiRJUpWxEJQkSZKkKmMhKEmSJElVxkJQkiRJkqpMIYVgRHwlIp6KiF93cj4i4tqIeDgi1kTECX2doyRJkiQNVkVtH3Ej8AXga52cPwc4tvznZODfy4+SJEmSVIglqxtYsGwdG7c2MnpEHbOnjGPapDFFp7VXChkRzMyfAU/vockFwNey5BfAiIg4rG+ykyRJkqS2lqxuYM7itTRsbSSBhq2NzFm8liWrG4pOba/01w3lxwDrWx1vKMeeKCYdSZIkSe0VNUK2YukXGfurBbwsN/FUjGL9CbM5ceqHKtrngmXrOKv5p3xi2CJGx2Y25kiu2jGdBcuGDchRwf66WEx0EMvdGkXMjIiVEbFy06ZNfZCWJEmSJChuhGzF0i8yYdUnOZRNDAk4lE1MWPVJViz9YkX7nfzMHVxZewP1QzYzJKB+yGaurL2Byc/cUdF+K6W/FoIbgLGtjuuBje0bZeb1mTk5MyePGjWqz5KTJEnSni1Z3cDrrvwxR13+PV535Y8H7PS5gaCo7/WCZetobGpuE2tsambBsnUV7XfsrxZQF9vbxOpiO2N/taCi/c4Zdgv7tet3v9jOnGG3VLTfSumvU0OXApdGxEJKi8Rsy0ynhUqSJA0ALSNFLUVCy0gRMCCn0HVVEdMkl6xuYPl3/o2bWcjofTaz8bmRXPOdGcBHKt73xq2N3Yr3lpflpg7nD74sN1e030Po+Pqdxfu7oraP+DZwDzAuIjZExPsj4sMR8eFyk+8DjwIPA18CPlJEnpIkSeq+okaKilTUNMn7vnc98+P6NtMV58f13Pe96yvaL8DoEXVMHbKc5cNm8eg+72T5sFlMHbKc0SPqKtrvU9HxTMCnYmRF+43h9d2K93dFrRp6UWYelpm1mVmfmV/OzP/IzP8on8/M/Ghmvjwzj8vMlUXkKUmSpO4raqSoSEUVvx/Y/o0Opyt+YPs3KtovwDXjH+Iz7e6Z+0ztDVwz/qGK9rv+hNk05rA2scYcxvoTZle0X86cC7XtitzaulJ8AOqv9whKkiSpp9YsgqsnwLwRpcc1i/qk285GhCo9UgSlhUSenHcMOz89nCfnHVPxBURaFFX8jh6ypVvx3nTiI5/v8F69Ex/5fGX7nfohfv3qf+ZJRrEzgycZxa9f/c8VXzWUidPh/Gth+FggSo/nX1uKD0D99R5BSZKkXjWYNoLukjWL2HHbxxja/HzpeNv60jFU/BfX2VPGtblHEKCutobZU8ZVtN+W1STrYjuUV5McvuqTrICKFwmjR9TR0EHRV+ni9/m6Q9mvcfelNJ6vO5T9KtozsG1D9+K96MSpH4Lye3po+U+fmDh9wBZ+7TkiKEmSBr2iN4IuYlXH534w9y9FYNnQ5ud57geVn8Y2bdIYrnjrcYwZUUcAY0bUccVbj6t44V3UapJQKn7ramva9t0Hxe9+58xnR82+bWI7avZlv3PmV7RfADq7N26A3jNXbSwEJUnSoNeyEXTrRS3Oav5pnyxeUlQRum/jk92K97ZpNXdz9z6zeGzfv+HufWYxrebuivf5sux4X+lKryYJxRW/TJzO0As+32a64tALPt83o1aD7J65auPUUEmS1KdWLP0iY3+1gJflJp6KUaw/YXbFp+1NfuYOrqi9YdeiGvVR2gh6zjMAZ1S07z0tIlLJImHjzoOpH7J7AbRx58FUfLxmzSK4fRY0ladKbltfOoaKFihPxSgOZfdi8KkY2SdTB6dNGlPMdOOipiu29Hnn/NJ00OH1pSJwkEydHOwcEZQkSX2m5R6uQ9nEkPI9XBNWfbLiC3oUuRF0UYuI3DDsXTzXbmXF53IYNwx7V0X7BUqFQVO719fUWIpXUGGrSVazidPh47+GeVtLjxaBA4aFoCRJ1aqAFSWLuoeryI2gi1pB8/hzZzI3Z7Jh50h2ZrBh50jm5kyOP3dmRfsFCltEpLDVJFsUtEqrtDecGipJUpHWLCpmWlVBU/delpsgOopXtiCL4fWl19hRvMJmTxnH8u/8G3/PQkbHZjbmSK5hBq+f8pGK9luaovgRLlx2Zt+vlNrJ97svFhEpbDXJgv5OSXvLQlCSpKIU+YvjnqbuDcZ7uM6c2/Z7DX22qMW0mrs5r/aGXSt41sdmrqy5gaE1rwIq+z4Xds9agd/vwhT0d0raW04NlSSpKAXdRwWQnUzR6yzeWwq7h6vIjaDvnN/hNg598T4XZpBtvN0lBe6pJ+0NRwQlSf1G1W34XeAvjn9gZIcjc6V45Zw49UOsgPKqoZt5Kkay/tWVXzUUKG5lxWotEAbRxttdUuB0WGlvWAhKkvqFlr3WWpbZb9lrDeiTYrCIIvS5ukPZr/GJjuMV7Rmu2P6ONtspQGlFySua3sH/rXDfhd3DVRQLhOpQjdNhNaA5NVSS1C/saa+1Sitqw++rmi7scHn/q5ourGi/ACtfehaXN32gzYqSlzd9gJUvPavifVcdN92uDtU4HVYDmiOCkqR+oai91qC4Db9v+tNJPD1kO58YuojRsYWNeTBX7ZjO7S+cxLyK9Voye8o45izeztLtr98Vq6ut4Yop4yrccxVy0+3qUW3TYTWgWQhKkvqF0SPqaOig6Kv0XmtQXBE6ekQdS7e+vk0xBjCmD15zS4FbVfdkFskCQVI/YyEoSeoXSiNUa9uMzNXV1jC7D0aoRo+o49XP3FEemSvt83bVjumsqvA0ySJfMxS4tYAkqXAWgpKkfqHIEaprxj/EhFU3UFdeOKU+NvOZ2hv49fgjgTMq1q+jcpKkokRmFp1Dr5g8eXKuXLmy6DQkSQPR1RM6WdVxLHz8132fjyRJeyEiVmXm5K60ddVQSVL/sWZRqSibN6L0uGZR3/Rbrfu8SZKqloWgJKl/WLOotAfXtvVAlh5vn9U3xWBn+7m5z5skaZAqpBCMiLMjYl1EPBwRl3dw/oiIuDMi1kTEXRHhv8SSNNjdOb/tRsxQOr5zfuX7dp83SVKV6fNCMCJqgOuAc4DxwEURMb5ds88CX8vMicB84Iq+zVKSqlwRUzSLnJ7pRtCSpCpTxKqhJwEPZ+ajABGxELgAeKBVm/HAx8tf/wRY0qcZSlI1W7OIHbd9jKHNz5eOt60vHUNlC6Ph9Z0s2NJHk0Lc502SVEWKmBo6Bmj9L/2Gcqy1/wHeVv76LcABEXFwH+QmSVXvuR/M/UsRWDa0+Xme+0GFp0k6PVOSpD5TxIhgdBBrv4fFPwFfiIhLgJ8BDcCO3S4UMROYCXD44Yf3bpaSVLAlqxsK2V9u38YnuxXvNS2jcXfOL00HHV5fKgIdpZMkqdcVUQhuAMa2Oq4HNrZukJkbgbcCRMRLgLdl5rb2F8rM64HrobSPYKUSlqS+tmR1A3MWr6WxqRmAhq2NzFm8FqDixeDGnQdTP2Rzx/GK9ozTMyVJ6iNFTA1dARwbEUdFxDBgBrC0dYOIGBkRLbnNAb7SxzlKUqEWLFu3qwhs0djUzIJl6yre9w3D3sVzOaxN7Lkcxg3D3lXxviVJUt/o80IwM3cAlwLLgAeBRZl5f0TMj4ip5WanAesi4rfAIcC/9HWeklSkjVsbuxXvTcefO5O5OZMNO0eyM4MNO0cyN2dy/LkzK963JEnqG0VMDSUzvw98v11sbquvbwVu7eu8JKm/GD2ijoYOir7RI+o6aN27SlNPP8KFy87s8/sTJUlS3yikEJQk7dnsKePa3CMIUFdbw+wp4/qk/2mTxlj4SZI0iFkISlI/NG3SGMas/y5jf7WAl+UmnopRrD9hNidOOrvo1CRJ0iBgIShJ/dGaRZy49tNAIwQcyiYOXftpOPJAV9WUJEk9VsSqoZKkF3PnfGhqd49gU2MpLkmS1EMWgpL0YtYsgqsnwLwRpcc1iyrf57YN3YtLkiR1g4WgJO3JmkXsuO1jsG09kLBtfem40sXg8E62bu8sLkmS1A0WgpK0B8/9YC5Dm59vExva/DzP/WBuJ8/oJWfOhdp2W0XU1pXikiRJPWQhKEl7sG/jk92K95qJ0+H8a2H4WCBKj+df60IxkiSpV7hqqCTtwcadB1M/ZHPH8Up3PnG6hZ8kSaoIRwQlaQ9uGPYunsthbWLP5TBuGPaugjKSJEnqOUcEJQ0YS1Y3sGDZOjZubWT0iDpmTxnHtEljKtrn8efOZO53dvD3uZDRsYWNeTDXMIPXnzuzov1KkiRVkoWgpAFhyeoG5ixeS2NTMwANWxuZs3gtQEWLwdK1P8KFy87s0wJUkiSpkiwEJQ0IC5at46zmn/KJYYsYHZvZmCO5asd0FiwbVvGibNqkMRZ+kiRpUPEeQUkDwuRn7uDK2huoH7KZIQH1QzZzZe0NTH7mjqJTkyRJGnAsBCUNCHOG3cJ+sb1NbL/YzpxhtxSUkSRJ0sBlIShpQDiE3bdw2FNckiRJnbMQlDQgxPCOd+3rLC5JkqTOWQhKGhjOnAu1dW1jtXWluCRJkrrFQlDSwDBxOpx/LQwfC0Tp8fxrS3FJkiR1i9tHSBo4Jk638JMkSeoFhYwIRsTZEbEuIh6OiMs7OH94RPwkIlZHxJqIeHMReUra3ZLVDbzuyh9z1OXf43VX/pglqxuKTkmSJEnd1OeFYETUANcB5wDjgYsiYny7Zp8EFmXmJGAG8G99m6WkjixZ3cCcxWtp2NpIAg1bG5mzeK3FoCRJ0gBTxIjgScDDmfloZm4HFgIXtGuTwEvLXw8HNvZhfpI6sWDZOhqbmtvEGpuaWbBsXUEZSZIkaW8UcY/gGGB9q+MNwMnt2swD/isiPgbsD7yxb1KTtCcbtzZ2Ky5JkqT+qYgRwegglu2OLwJuzMx64M3A1yNit1wjYmZErIyIlZs2bapAqpJaGz2irltxSZIk9U9FFIIbgLGtjuvZfern+4FFAJl5D7AvMLL9hTLz+sycnJmTR40aVaF0JbWYPWUcdbU1bWJ1tTXMnjKuoIwkSZK0N4ooBFcAx0bEURExjNJiMEvbtfk9cCZARPw1pULQIT+pYNMmjeGKtx7HmBF1BDBmRB1XvPU4pk0aU3RqkiRJ6oY+v0cwM3dExKXAMqAG+Epm3h8R84GVmbkU+EfgSxHxcUrTRi/JzPbTRyUVYNqkMRZ+kiRJA1whG8pn5veB77eLzW319QPA6/o6L0mSJEmqBoVsKC9JkiRJKo6FoCRJkiRVGQtBSZIkSaoyFoKSJEmSVGUsBCVJkiSpyhSyaqiknluyuoEFy9axcWsjo0fUMXvKOLd1kCRJUpdYCEoD0JLVDcxZvJbGpmYAGrY2MmfxWgCLQUmSJL0op4ZKA9CCZet2FYEtGpuaWbBsXUEZSZIkaSCxEJQGoI1bG7sV71VrFsHVE2DeiNLjmkWV71OSJEm9ykJQGoBGj6jrVrzXrFkEt8+CbeuBLD3ePstiUJIkaYCxEJQGoNlTxlFXW9MmVldbw+wp4yrb8Z3zoandqGNTYykuSZKkAcPFYqQBqGVBmD5fNXTbhu7FJUmS1C9ZCEoD1LRJY/p+hdDh9eVpoR3EJUmSNGA4NVRS1505F2rb3YdYW1eKS5IkacCwEJTUdROnw/nXwvCxQJQez7+2FJckSdKA4dRQSd0zcbqFnyRJ0gDniKA0ULmfnyRJkvaSI4LSQNSyn1/LVg4t+/mBo3WSJEl6UY4ISgOR+/lJkiSpBywEpYHI/fwkSZLUA4VMDY2Is4H/C9QAN2Tmle3OXw2cXj7cD3hZZo7o2yylF7dkdUPfb+oO7ucnSZKkHunzEcGIqAGuA84BxgMXRcT41m0y8+OZeXxmHg98Hljc13lKL2bJ6gbmLF5Lw9ZGEmjY2sicxWtZsrqh8p27n58kSZJ6oIipoScBD2fmo5m5HVgIXLCH9hcB3+6TzKRuWLBsHWc1/5Tlw2bx6D7vZPmwWZzV/FMWLFtX+c7dz0+SJEk9UMTU0DFA6zltG4CTO2oYEUcARwE/7oO8pG6Z/MwdXFF7A/vFdgDqYzNX1t7AnGcAzqh8Au7nJ0mSpL1UxIhgdBDLTtrOAG7NzOYOLxQxMyJWRsTKTZs29VqCUlfMGXbLriKwxX6xnTnDbikoI0mSJKlrelQIRsSlEXFgN5+2ARjb6rge2NhJ2xnsYVpoZl6fmZMzc/KoUaO6mYbUM4ewuVtxSZIkqb/o6YjgocCKiFgUEWdHREejfe2tAI6NiKMiYhilYm9p+0YRMQ44ELinhzlKFRGdrNDZWVySJEnqL3pUCGbmJ4FjgS8DlwAPRcS/RsTL9/CcHcClwDLgQWBRZt4fEfMjYmqrphcBCzOzs2mjUrFcuVOSJEkDVI8Xi8nMjIgngSeBHZRG8W6NiDsy8xOdPOf7wPfbxea2O57X09ykimpZqOXO+aWN3IfXl4pAF3CRJElSP9ejQjAiZgHvATYDNwCzM7MpIoYADwEdFoLSoOHKnZIkSRqAejoiOBJ4a2b+rnUwM3dGxHk9vLYkSZIkqQJ6uljM94GnWw4i4oCIOBkgMx/s4bUlSZIkSRXQ00Lw34E/tTr+czkmSZIkSeqneloIRutVPTNzJ72wAI0kSZIkqXJ6Wgg+GhGzIqK2/OfvgEd7IzFJkiRJUmX0tBD8MPBaoAHYAJwMzOxpUpIkSZKkyunRNM7MfAqY0Uu5SJIkSZL6QE/3EdwXeD/wSmDflnhmvq+HeUmSJEmSKqSnU0O/DhwKTAF+CtQDz/Y0KUmSJElS5fS0EDwmMz8F/DkzbwLOBY7reVqSJEmSpErpaSHYVH7cGhETgOHAkT28piRJkiSpgnq659/1EXEg8ElgKfAS4FM9zkqSJEmSVDF7XQhGxBDgmcz8I/Az4Ohey0qSJEmSVDF7PTU0M3cCl/ZiLpIkSZKkPtDTqaF3RMQ/ATcDf24JZubTPbyu1C1LVjewYNk6Nm5tZPSIOmZPGce0SWOKTkuSJEnql3paCLbsF/jRVrHEaaLqQ0tWNzBn8Voam5oBaNjayJzFawEsBiVJkqQO9KgQzMyjeisRaW8tWLZuVxHYorGpmQXL1lkISpIkSR3oUSEYEe/uKJ6ZX+vJdaXu2Li1sVtxSZIkqdr1dGroia2+3hc4E/gVYCGoPjN6RB0NHRR9o0fUFZCNJEmS1P/1aEP5zPxYqz8fBCYBw17seRFxdkSsi4iHI+LyTtpMj4gHIuL+iPhWT/LU4DZ7yjjqamvaxOpqa5g9ZVxBGUmSJEn9W09HBNt7Djh2Tw0ioga4DjgL2ACsiIilmflAqzbHAnOA12XmHyPiZb2cpwaRlvsAXTVUkiRJ6pqe3iN4O6VVQqE0ujgeWPQiTzsJeDgzHy1fYyFwAfBAqzYfBK4rb1ZPZj7Vkzw1+E2bNMbCT5IkSeqino4IfrbV1zuA32Xmhhd5zhhgfavjDcDJ7dq8AiAi7gZqgHmZ+cMe5qrBbM0iuHM+bNsAw+vhzLkwcXrRWUmSJEn9Uk8Lwd8DT2Tm8wARURcRR2bm43t4TnQQy3bHQylNMT0NqAf+OyImZObWNheKmAnMBDj88MP36gVoEFizCG6fBU3lBWO2rS8dg8WgJEmS1IEeLRYD3ALsbHXcXI7tyQZgbKvjemBjB21uy8ymzHwMWEcH9x5m5vWZOTkzJ48aNarbyWuQuHP+X4rAFk2NpbgkSZKk3fS0EByamdtbDspfv9iqoSuAYyPiqIgYBswAlrZrswQ4HSAiRlKaKvpoD3PVYLWtk9nIncUlSZKkKtfTQnBTRExtOYiIC4DNe3pCZu4ALgWWAQ8CizLz/oiY3+pay4AtEfEA8BNgdmZu6WGuGqyG13cvLkmSJFW5yGx/e143nhzxcuCbwOhyaAPw7sx8uBdy65bJkyfnypUr+7pb9Qft7xEEqK2D86/1HkFJkiRVjYhYlZmTu9K2R4vFZOYjwGsi4iWUispne3I9aa+0FHuuGipJkiR1SU/3EfxX4KqW1Twj4kDgHzPzk72RnNRlE6db+EmSJEld1NN7BM9pvaVDeQP4N/fwmpIkSZKkCuppIVgTEfu0HEREHbDPHtpLkiRJkgrW0w3lvwHcGRFfLR+/F7iph9eUJEmSJFVQTxeLuSoi1gBvBAL4IXBEbyQmSZIkSaqMnk4NBXgS2Am8DTiT0t6AkiRJkqR+aq9GBCPiFcAM4CJgC3Azpe0jTu/F3CRJkiRJFbC3U0N/A/w3cH7L5vER8fFey0qSJEmSVDF7OzX0bZSmhP4kIr4UEWdSukdQkiRJktTP7VUhmJnfycwLgb8C7gI+DhwSEf8eEW/qxfwkSZIkSb2sR4vFZOafM/ObmXkeUA/cB1zeK5lJkiRJkiqiN1YNBSAzn87ML2bmGb11TUmSJElS7+u1QlCSJEmSNDBYCEqSJElSlbEQlCRJkqQqYyEoSZIkSVXGQlCSJEmSqoyFoCRJkiRVGQtBSZIkSaoyhRSCEXF2RKyLiIcjYrcN6CPikojYFBH3lf98oIg8JUmSJGkwGtrXHUZEDXAdcBawAVgREUsz84F2TW/OzEv7Oj9JkiRJGuz6vBAETgIezsxHASJiIXAB0L4Q1ACzZHUDC5atY+PWRkaPqGP2lHFMmzSm6LQkSZIktVPE1NAxwPpWxxvKsfbeFhFrIuLWiBjbN6lpby1Z3cCcxWtp2NpIAg1bG5mzeC1LVjcUnZokSZKkdoooBKODWLY7vh04MjMnAj8CburwQhEzI2JlRKzctGlTL6ep7liwbB2NTc1tYo1NzSxYtq6gjCRJkiR1pohCcAPQeoSvHtjYukFmbsnMF8qHXwJe3dGFMvP6zJycmZNHjRpVkWTVNRu3NnYrLkmSJKk4RRSCK4BjI+KoiBgGzACWtm4QEYe1OpwKPNiH+WkvjB5Rx9Qhy1k+bBaP7vNOlg+bxdQhyxk9oq7o1CRJkiS10+eLxWTmjoi4FFgG1ABfycz7I2I+sDIzlwKzImIqsAN4Grikr/NU91wz/iEmrLqButgOQH1s5jO1N/Dr8UcCZxSamyRJkqS2IrP97XkD0+TJk3PlypVFp1G9rp4A29bvHh8+Fj7+677PR5IkSaoyEbEqMyd3pW0hG8prENq2oXtxSZIkSYWxEFTvGF7fvbgkSZKkwlgIqnecORdq2y0MU1tXikuSJEnqVywE1TsmTofzry3dE0iUHs+/thSXJEmS1K/0+aqhGsQmTrfwkyRJkgYARwQlSZIkqcpYCEqSJElSlbEQlCRJkqQqYyEoSZIkSVXGQlCSJEmSqoyFoCRJkiRVGQtBSZIkSaoyFoKSJEmSVGUsBCVJkiSpylgISpIkSVKVsRCUJEmalpL9AAAgAElEQVSSpCpjIShJkiRJVcZCUJIkSZKqjIWgJEmSJFUZC0FJkiRJqjKFFIIRcXZErIuIhyPi8j20e3tEZERM7sv8JEmSJGkw6/NCMCJqgOuAc4DxwEURMb6DdgcAs4Bf9m2GkiRJkjS4FTEieBLwcGY+mpnbgYXABR20+9/AVcDzfZmcJEmSJA12RRSCY4D1rY43lGO7RMQkYGxmfrcvE5MkSZKkajC0gD6jg1juOhkxBLgauORFLxQxE5gJcPjhh/dSegPfktUNLFi2jo1bGxk9oo7ZU8YxbdKYF3+iJEmSpKpQxIjgBmBsq+N6YGOr4wOACcBdEfE48BpgaUcLxmTm9Zk5OTMnjxo1qoIpDxxLVjcwZ/FaGrY2kkDD1kbmLF7LktUNRacmSZIkqZ8oohBcARwbEUdFxDBgBrC05WRmbsvMkZl5ZGYeCfwCmJqZKwvIdcBZsGwdjU3NbWKNTc0sWLauoIwkSZIk9Td9Xghm5g7gUmAZ8CCwKDPvj4j5ETG1r/MZbDZubexWXJIkSVL1KeIeQTLz+8D328XmdtL2tL7IabAYPaKOhg6KvtEj6grIRpIkSVJ/VMiG8qqc2VPGUVdb0yZWV1vD7CnjCspIkiRJUn9TyIigKqdldVBXDZUkSZLUGQvBQWjapDEWfpIkSZI65dRQSZIkSaoyFoKSJEmSVGUsBCVJkiSpylgISpIkSVKVsRCUJEmSpCpjIShJkiRJVcZCUJIkSZKqjIWgJEmSJFUZC8HBaM0iuHoCzBtRelyzqOiMJEmSJPUjQ4tOQL1szSK4fRY0NZaOt60vHQNMnF5cXpIkSZL6DUcEB5s75/+lCGzR1FiKS5IkSRIWgoPPtg3di0uSJEmqOhaCg83w+u7FJUmSJFUdC8HB5sy5UFvXNlZbV4pLkiRJEhaCg8/E6XD+tTB8LBClx/OvdaEYSZIkSbu4auhgNHG6hZ8kSZKkTjkiKEmSJElVppBCMCLOjoh1EfFwRFzewfkPR8TaiLgvIpZHxPgi8pQkSZKkwajPC8GIqAGuA84BxgMXdVDofSszj8vM44GrgM/1cZqSJEmSNGgVMSJ4EvBwZj6amduBhcAFrRtk5jOtDvcHsg/zkyRJkqRBrYjFYsYA61sdbwBObt8oIj4K/AMwDDijb1KTJEmSpMGviBHB6CC224hfZl6XmS8HLgM+2eGFImZGxMqIWLlp06ZeTlOSJEmSBqciCsENwNhWx/XAxj20XwhM6+hEZl6fmZMzc/KoUaN6MUVJkiRJGryKKARXAMdGxFERMQyYASxt3SAijm11eC7wUB/mJ0mSJEmDWp/fI5iZOyLiUmAZUAN8JTPvj4j5wMrMXApcGhFvBJqAPwLv6es8JUmS1LuamprYsGEDzz//fNGpSAPavvvuS319PbW1tXt9jcgcHAtyTp48OVeuXFl0GpIkSerEY489xgEHHMDBBx9MREfLRkh6MZnJli1bePbZZznqqKPanIuIVZk5uSvXKWRDeUmSJFWf559/3iJQ6qGI4OCDD+7xyLqFoCRJkvqMRaDUc73x98hCUJIkSVXjhz/8IePGjeOYY47hyiuv7LDNCy+8wIUXXsgxxxzDySefzOOPP77r3BVXXMExxxzDuHHjWLZs2Yv2d+ONN3LppZf2Vvrdds011/Dcc88V1n9v68r79/GPf5zjjz+e448/nle84hWMGDFi17mamppd56ZOnfqi/T3++ONMmDCh1/LvriVLlvDAAw9U5NpFbCgvSZIk9bnm5mY++tGPcscdd1BfX8+JJ57I1KlTGT9+fJt2X/7ylznwwAN5+OGHWbhwIZdddhk333wzDzzwAAsXLuT+++9n48aNvPGNb+S3v/0tNTU13c5lx44dDB1a+V/Fr7nmGt71rnex3377VbyvSuvq+3f11Vfv+vrzn/88q1ev3nVcV1fHfffd1+Nc+ur9W7JkCeedd95ur7E3OCJYKWsWwdUTYN6I0uOaRUVnJEmSNKAsWd3A6678MUdd/j1ed+WPWbK6oUfXu/feeznmmGM4+uijGTZsGDNmzOC2227brd1tt93Ge95TWrT+7W9/O3feeSeZyW233caMGTPYZ599OOqoozjmmGO49957d3v+V7/6VV7xildw6qmncvfdd++KX3LJJfzDP/wDp59+OpdddhlPP/0006ZNY+LEibzmNa9hzZo1AMybN4+LL76YM844g2OPPZYvfelLQGmRkNmzZzNhwgSOO+44br75ZgDuuusuzjvvvF39XHrppdx4441ce+21bNy4kdNPP53TTz+9R9+7vdLLvw939f1r7dvf/jYXXXRRt/pZtWoVr3rVqzjllFO47rrrdsVvvPFG3vGOd3D++efzpje9aY/vxxve8Abe8pa3MH78eD784Q+zc+fOXfkcd9xxTJgwgcsuu2zXtV/ykpfs+vrWW2/lkksu4ec//zlLly5l9uzZHH/88TzyyCPdeh0vxhHBSlizCG6fBU2NpeNt60vHABOnF5eXJEnSALFkdQNzFq+lsakZgIatjcxZvBaAaZPG7NU1GxoaGDt27K7j+vp6fvnLX+6x3dChQxk+fDhbtmyhoaGB17zmNW2e39DQtjh94okn+PSnP82qVasYPnw4p59+OpMmTdp1/re//S0/+tGPqKmp4WMf+xiTJk1iyZIl/PjHP+bd7373rtGqNWvW8Itf/II///nPTJo0iXPPPZd77rmH++67j//5n/9h8+bNnHjiibzhDW/o9PXOmjWLz33uc/zkJz9h5MiRe/U922sV+H24q+9fi9/97nc89thjnHHGGbtizz//PJMnT2bo0KFcfvnlTJs2bbfnvfe97+Xzn/88p556KrNnz25z7p577mHNmjUcdNBB/Od//men78e9997LAw88wBFHHMHZZ5/N4sWLee1rX8tll13GqlWrOPDAA3nTm97EkiVLOswB4LWvfS1Tp07lvPPO4+1vf3u3vldd4YhgJdw5/y8f+hZNjaW4JEmSXtSCZet2FYEtGpuaWbBs3V5fs6Nt0zpadKOzdl15/i9/+UtOO+00Ro0axbBhw7jwwgvbnH/HO96xayrp8uXLufjiiwE444wz2LJlC9u2bQPgggsuoK6ujpEjR3L66adz7733snz5ci666CJqamo45JBDOPXUU1mxYkUXX30fq8Dvw119/1osXLiQt7/97W2m7v7+979n5cqVfOtb3+Lv//7vdxtl27ZtG1u3buXUU08F2PX+tDjrrLM46KCDAPb4fpx00kkcffTR1NTUcNFFF7F8+XJWrFix67MxdOhQ/uZv/oaf/exne/fN6AUWgpWwbUP34pIkSWpj49bGbsW7or6+nvXr1+863rBhA6NHj95jux07drBt2zYOOuigLj9/T8XJ/vvvv+vrPRU27a/RWSEKpVHLlqmHQI+3FegVFfh9uKvf/xYLFy7cbVpoS/ujjz6a0047rc39g1B6T3ry/rXozvvXvn1fvX8WghXwXN2h3YpLkiSprdEj6roV74oTTzyRhx56iMcee4zt27ezcOHCDleOnDp1KjfddBNQul/rjDPOICKYOnUqCxcu5IUXXuCxxx7joYce4qSTTmrz3JNPPpm77rqLLVu20NTUxC233NJpPm94wxv45je/CZTuKxs5ciQvfelLgdJ9is8//zxbtmzhrrvu2jXt8Oabb6a5uZlNmzbxs5/9jJNOOokjjjiCBx54gBdeeIFt27Zx55137urjgAMO4Nlnn93r79leG17fvXgXdPX9A1i3bh1//OMfOeWUU3bF/vjHP/LCCy8AsHnzZu6+++7dFmEZMWIEw4cPZ/ny5QC73p+OdPZ+QGlq6GOPPcbOnTu5+eabef3rX8/JJ5/MT3/6UzZv3kxzczPf/va3d408HnLIITz44IPs3LmT73znO7v6qOT75z2CFXBV04V8Iv+N/WL7rthzOYyrmi5kXnFpSZIkDRizp4xrc48gQF1tDbOnjNvraw4dOpQvfOELTJkyhebmZt73vvfxyle+EoC5c+cyefJkpk6dyvvf/34uvvhijjnmGA466CAWLlwIwCtf+UqmT5/O+PHjGTp0KNddd91uK4YedthhzJs3j1NOOYXDDjuME044gebm5t1ygdKiMO9973uZOHEi++23367iE0pTC88991x+//vf86lPfYrRo0fzlre8hXvuuYdXvepVRARXXXUVhx5aGmiYPn06EydO5Nhjj21zT+LMmTM555xzOOyww/jJT36y19+7bjtzbtt7BAFq60rxvdTV9w9Ki7LMmDGjzUjbgw8+yIc+9CGGDBnCzp07ufzyyztcjfOrX/0q73vf+9hvv/2YMmVKp/l09n785je/4ZRTTuHyyy9n7dq1uxaOGTJkCFdccQWnn346mcmb3/xmLrjgAgCuvPJKzjvvPMaOHcuECRP405/+BMCMGTP44Ac/yLXXXsutt97Ky1/+8r3+/rUXexqiHEgmT56cK1euLDoNAI66/HucP2Q5nxi6iNGxhY15MFftmM7tO1/PY1eeW3R6kiRJhXjwwQf567/+6y63X7K6gQXL1rFxayOjR9Qxe8q4vV4oZiCZN28eL3nJS/inf/qnolPpmTWLSvcEbttQGgk8c25VLJx411138dnPfpbvfve7Fe2no79PEbEqMyd35fmOCFbA6BF1LN36epZuf32b+JgeTGWQJEmqNtMmjamKwm/Qmji9Kgq/gcpCsAIqMZVBkiRJ1WHevHlFp6AeOO200zjttNOKTuNFWQhWQMv/XFXjVAZJkiRJ/Z+FYIU4lUGSJGl3L7Y8v6QX1xvrvLh9hCRJkvrEvvvuy5YtW3rll1ipWmUmW7ZsYd999+3RdRwRlCRJUp+or69nw4YNbNq0qehUpAFt3333pb5+7/dkBAtBSZIk9ZHa2lqOOuqootOQhFNDJUmSJKnqWAhKkiRJUpWxEJQkSZKkKhODZdWmiNgE/K7oPDowEthcdBIa1PyMqZL8fKmS/Hypkvx8qZL66+friMwc1ZWGg6YQ7K8iYmVmTi46Dw1efsZUSX6+VEl+vlRJfr5USYPh8+XUUEmSJEmqMhaCkiRJklRlLAQr7/qiE9Cg52dMleTnS5Xk50uV5OdLlTTgP1/eIyhJkiRJVcYRQUmSJEmqMhaCFRQRZ0fEuoh4OCIuLzofDS4R8XhErI2I+yJiZdH5aOCLiK9ExFMR8etWsYMi4o6IeKj8eGCROWrg6uTzNS8iGso/x+6LiDcXmaMGrogYGxE/iYgHI+L+iPi7ctyfYeqxPXy+BvTPMKeGVkhE1AC/Bc4CNgArgIsy84FCE9OgERGPA5Mzsz/uYaMBKCLeAPwJ+FpmTijHrgKezswry/+hdWBmXlZknhqYOvl8zQP+lJmfLTI3DXwRcRhwWGb+KiIOAFYB04BL8GeYemgPn6/pDOCfYY4IVs5JwMOZ+WhmbgcWAhcUnJMkdSozfwY83S58AXBT+eubKP3DJ3VbJ58vqVdk5hOZ+avy188CDwJj8GeYesEePl8DmoVg5YwB1rc63sAg+MCoX0ngvyJiVUTMLDoZDVqHZOYTUPqHEHhZwflo8Lk0ItaUp446bU89FhFHApOAX+LPMPWydp8vGMA/wywEKyc6iDkPV73pdZl5AnAO8NHytCtJGkj+HXg5cDzwBPB/ik1HA11EvAT4T+DvM/OZovPR4NLB52tA/wyzEKycDcDYVsf1wMaCctEglJkby49PAd+hNB1Z6m1/KN8b0XKPxFMF56NBJDP/kJnNmbkT+BL+HFMPREQtpV/Sv5mZi8thf4apV3T0+RroP8MsBCtnBXBsRBwVEcOAGcDSgnPSIBER+5dvViYi9gfeBPx6z8+S9spS4D3lr98D3FZgLhpkWn5BL3sL/hzTXoqIAL4MPJiZn2t1yp9h6rHOPl8D/WeYq4ZWUHkJ2WuAGuArmfkvBaekQSIijqY0CggwFPiWny/1VER8GzgNGAn8Afg0sARYBBwO/B54R2a64Ie6rZPP12mUplQl8DjwoZb7uaTuiIjXA/8NrAV2lsP/i9J9XP4MU4/s4fN1EQP4Z5iFoCRJkiRVGaeGSpIkSVKVsRCUJEmSpCpjIShJkiRJVcZCUJIkSZKqjIWgJEmSJFUZC0FJktqJiOaIuK/Vn8t78dpHRsSA2mtKkjT4DC06AUmS+qHGzDy+6CQkSaoURwQlSeqiiHg8Ij4TEfeW/xxTjh8REXdGxJry4+Hl+CER8Z2I+J/yn9eWL1UTEV+KiPsj4r8ioq6wFyVJqkoWgpIk7a6u3dTQC1udeyYzTwK+AFxTjn0B+FpmTgS+CVxbjl8L/DQzXwWcANxfjh8LXJeZrwS2Am+r8OuRJKmNyMyic5AkqV+JiD9l5ks6iD8OnJGZj0ZELfBkZh4cEZuBwzKzqRx/IjNHRsQmoD4zX2h1jSOBOzLz2PLxZUBtZv5z5V+ZJEkljghKktQ92cnXnbXpyAutvm7Ge/YlSX3MQlCSpO65sNXjPeWvfw7MKH/9N8Dy8td3An8LEBE1EfHSvkpSkqQ98X8gJUnaXV1E3Nfq+IeZ2bKFxD4R8UtK/5l6UTk2C/hKRMwGNgHvLcf/Drg+It5PaeTvb4EnKp69JEkvwnsEJUnqovI9gpMzc3PRuUiS1BNODZUkSZKkKuOIoCRJkiRVGUcEJUlVJSKOjIiMCO+TlyRVLQtBSdKAEhHLImJ+B/ELIuJJCzxJkl6chaAkaaC5Ebg4IqJd/GLgm5m5o+9T6h1R4r/NkqSK8x8bSdJAswQ4CPj/WgIRcSBwHvC18vG5EbE6Ip6JiPURMa+rF4+IyyPikYh4NiIeiIi3tDv/wYh4sNX5E8rxsRGxOCI2RcSWiPhCOT4vIr7R6vltpqZGxF0R8S8RcTfwHHB0RLy3VR+PRsSH2uVwQUTcV359j0TE2RHxjohY1a7dP0bEkq6+dklS9bAQlCQNKJnZCCwC3t0qPB34TWb+T/n4z+XzI4Bzgb+NiGld7OIRSkXmcOD/B74REYcBRMQ7gHnla78UmApsiYga4LvA74AjgTHAwm68rIuBmcAB5Ws8RamwfSmlPQmvblVwnkSp4J1dfn1vAB4HlgJHRcRft7ruu4CvdyMPSVKVsBCUJA1ENwHviIi68vG7yzEAMvOuzFybmTszcw3wbeDUrlw4M2/JzI3l594MPAScVD79AeCqzFyRJQ9n5u/K50cDszPzz5n5fGYu78bruTEz78/MHZnZlJnfy8xHyn38FPgv/jIC+n7gK5l5RznHhsz8TWa+ANxMqfgjIl5JqSj9bjfykCRVCQtBSdKAUy6yNgEXRMTRwInAt1rOR8TJEfGT8jTNbcCHgZFduXZEvLs87XJrRGwFJrR67lhKI4btjQV+14P7E9e3y+GciPhFRDxdzuHNXcgBSsXwO8v3T14MLCoXiJIktWEhKEkaqL5GaSTwYuC/MvMPrc59i9JUybGZORz4D6D94jK7iYgjgC8BlwIHZ+YI4NetnrseeHkHT10PHN7JiqV/BvZrdXxoB212beobEfsA/wl8FjiknMP3u5ADmfkLYDul0cN34rRQSVInLAQlSQPV14A3Ah+k1bTQsgOApzPz+fI9de/s4jX3p1SUbQKIiPdSGhFscQPwTxHx6vIKn8eUi8d7gSeAKyNi/4jYNyJeV37OfcAbIuLwiBgOzHmRHIYB+5Rz2BER5wBvanX+y8B7I+LMiBgSEWMi4q9anf8a8AVgRzenp0qSqoiFoCRpQMrMx4GfUyrelrY7/RFgfkQ8C8yltLhMV675APB/gHuAPwDHAXe3On8L8C+URhyfpbyCaWY2A+cDxwC/BzYAF5afcwele/fWAKt4kXv2MvNZYFY55z9SKmKXtjp/L+UFZIBtwE+BI1pd4uuUildHAyVJnYrMfPFWkiRpQCgvoPMUcEJmPlR0PpKk/skRQUmSBpe/BVZYBEqS9qSjm9olSdIAFBGPU1pUpqt7JkqSqpRTQyVJkiSpyjg1VJIkSZKqjIWgJEmSJFWZQXOP4MiRI/PII48sOg1JkiRJKsSqVas2Z+aorrQdNIXgkUceycqVK4tOQ5IkSZIKERG/62pbp4ZKkiRJUpWxEJQkSZKkKmMhKEmSJElVxkJQkiRJkqqMhaAkSZIkVRkLQUmSJEmqMhaCkiRJGjzWLIKrJ8C8EaXHNYuKzkjqlwbNPoKSJEmqcmsWwe2zoKmxdLxtfekYYOL04vKS+iFHBCVJkjQ43Dn/L0Vgi6bGUlxSGxUtBCPi7IhYFxEPR8TlHZz/cESsjYj7ImJ5RIwvx8+KiFXlc6si4oxK5ilJkqRBYNuG7sWlKlaxqaERUQNcB5wFbABWRMTSzHygVbNvZeZ/lNtPBT4HnA1sBs7PzI0RMQFYBoypVK6SJBVlyeoGFixbx8atjYweUcfsKeOYNsl/8qS9Mry+NB20o7ikNio5IngS8HBmPpqZ24GFwAWtG2TmM60O9weyHF+dmRvL8fuBfSNinwrmKklSn1uyuoE5i9fSsLWRBBq2NjJn8VqWrG4oOjVpYDpzLtTWtY3V1pXiktqoZCE4Bmj9XzIb6GBULyI+GhGPAFcBszq4ztuA1Zn5QkWylCSpIAuWraOxqblNrLGpmQXL1hWUkTTATZwO518Lw8cCUXo8/1oXipE6UMlVQ6ODWO4WyLwOuC4i3gl8EnjPrgtEvBL4DPCmDjuImAnMBDj88MN7IWVJkvrOxq2N3YpL6oKJ0y38pC6o5IjgBmBsq+N6YGMnbaE0dXRay0FE1APfAd6dmY909ITMvD4zJ2fm5FGjRvVCypIk9Z3RI+q6FZckqbdUshBcARwbEUdFxDBgBrC0dYOIOLbV4bn8v/buP0quskz0/feh04FGNA3CYUg6YDAxyxAi8TQg6iH80uAIIToYwoiKeiY6IzLqMpdwZ4xZOccDl8wZXChrrugoOlcNgcEQZtTIRNADghAMdoYwkQgI3XEGiCTK0EBInvtHVYfuppNUd3p3pWt/P2v1qtpv7V31VNfO7jz1Pu/7wsPV9lbgn4HLM/OuAmOUJKluFs6eSktzU5+2luYmFs6eWqeIJEllUVhpaGa+FBGXUJnxswn4emY+GBFLgbWZuQq4JCLOArYDz/ByWeglwGTgcxHxuWrbOzPzyaLilSRppPXMDuqsoZKkkRaZrxi2Nyq1t7fn2rVr6x2GJEmSJNVFRNyfme217FvogvKSJEmSpP2PiaAkSQAdK+Dq6bCktXLbsaLeEUmSVJgil4+QJGl06FgBt14K26vLNmx7orINTkMvSWpI9ghKkrRm6ctJYI/t3ZV2SZIakImgJEnbOgfXLknSKGciKEnSuLbBtUuSNMqZCEqSdOZiaG7p29bcUmmXJKkBmQhKkjRjHpx7DYybCETl9txrnChGktSwnDVUkiSoJH0mfpKkkrBHUJIkSZJKxkRQkiRJkkrGRFCSJKloHSvg6umwpLVy27Gi3hFJKjnHCEqSJBWpYwXceils765sb3uisg2OS5VUN/YISpIkFWnN0peTwB7buyvtklQnJoKSJElF2tY5uHZpNLHsedSyNFSSJKlI49oq5aADtUujWQnLnleu62LZ6o1s3trN+NYWFs6eytyZE+od1pDYIyhJklSkMxdDc0vftuaWSrs0mpWs7Hnlui4uv3k9XVu7SaBrazeX37yeleu66h3akJgISpIkFWnGPDj3Ghg3EYjK7bnXNGyPiUqkZGXPy1ZvpHv7jj5t3dt3sGz1xjpFtG8sDZUkicYq99F+aMY8E78SKN11pGRlz5u3dg+qfX9nj6AkqfQardxH0sgr5XWkZGXP41tbBtW+vys0EYyIsyNiY0RsiohFAzz+8YhYHxEPRMSdETGt12OXV4/bGBGzi4xTklRujVbuI2nklfI6UrKy54Wzp3L+2J9x59hLeeTAP+XOsZdy/tifsXD21HqHNiSFlYZGRBNwLfAOoBO4LyJWZeaGXrt9JzP/3+r+c4C/Bc6uJoTzgeOA8cC/RMQbMrPvvy5JkoZBo5X7SBp5db2OdKyoTNCyrbNSlnnm4pFLxkpU9jy36S7Oaf4aY3Y8D0BbPM2VTV9jTNObgNH3OyiyR/AkYFNmPpKZLwLLgfN675CZv++1+Sogq/fPA5Zn5guZ+Siwqfp8kiQNu0Yr95E08up2HelZwmHbE0C+vISD6/kNvzVLdyWBPcbseH7UzpJaZCI4Aeg9erSz2tZHRHwiIn4NXAVcOphjJUkaDgtnT6WlualPW0tz06gt95E08up2HSnZEg511WCzpBaZCMYAbfmKhsxrM/P1wGXAXw/m2IhYEBFrI2LtU089tU/BSpLKa+7MCVzx3uOZ0NpCABNaW7jivcc39mx/ZdSxAq6eDktaK7dl6DEp43uuk7pdRxosOdmv7W421FE6S2qRy0d0AhN7bbcBm/ew/3Lg7wZzbGZeB1wH0N7e/opEUZKkWs2dOcHEr5H1lM/19Jz0lM9B445vKuN7rrO5TXcx98ClcFAnHNgGTYspfOxYyZZwqKszF/f9NwWjepbUInsE7wOmRMSkiBhLZfKXVb13iIgpvTbfDTxcvb8KmB8RB0bEJGAKcG+BsUqSpEZWxvK5Mr7neqrXWL2SLeFQVw02S2phPYKZ+VJEXAKsBpqAr2fmgxGxFFibmauASyLiLGA78AzwoeqxD0bECmAD8BLwCWcMlSRJQ1bG8rkyvud62lPiXWSi0PPc9Zo1tGwaaJbUIktDyczvA9/v17a41/2/3MOxXwC+UFx0kiSpNMpYPlfv91zPJQ3qoZ6JdwMlJxo5hS4oL0mStF8oY/lcPd9zGZc0aLCJRNT4TAQlSfuNleu6eNuVP2bSon/mbVf+mJXruuodkhpFg43tqUk933MZxyeW8cuGenJG3H1WaGmoJEm1Wrmui8tvXk/39sqQ8K6t3Vx+83oAZ/PU8Chj+Vy93nMZxygF+u0AACAASURBVCc6Vm/kdKzgpVs++fLi7tueqGyDv+9BMBGUJO0Xlq3euCsJ7NG9fQfLVm80EZRGm3qPT6yXMn7ZUAfP/WAxB/ckgVVjdjxfaff3XzNLQ6V9YBmbNHw2b+0eVLtGMUu6Gp9lkirQQd3/Pqh2DcweQWmILGOThtf41ha6Bkj6xre2DLC3Rq2SLnK+cl0Xy1ZvZPPWbsa3trBw9tTG/lthmaQKtHnna2k74OmB2+sQz2hlj6A0RHsqY5M0eAtnT6WlualPW0tzEwtnT61TRCpECScR6fnisGtrN8nLXxw2fBXJjHnw6X+FJVsrtyaBGiZfG3sRz+XYPm3P5Vi+NvaiOkU0OpkISkNkGVuJWMY2IubOnMAV7z2eCa0tBDChtYUr3nt8Y/ea1Fs9zu0STiLiF4fS8Drh3QtYnAvo3Hk4OzPo3Hk4i3MBJ7x7Qb1DG1UsDZWGyDK2kihpGVu9zJ05wcRvpNTr3C7hJCJ+cahGVo+y58rz/wUXrD6zPOXWBTARlIZo4eypfcYIgmVsDWlPZWwFJ4KlG1OkkVWvc/vMxX0TUGj4SUT84nBkee0cOfWcL8EvDvedpaHSEFnGVhJ1KmMr7ZgijZx6lWiWcGF3x7+OHK+dI8uy59HNHkFpH/htVAnUqYzNNfVUuHqWaJZsrbWef7P2UhXPa+fIsux5dDMRlKQ9qVMZm39cVbT7Xv9Jpt//17TEi7vaunMs//r6T3JiHeNqVGX84rAeJZplvXbWqxzWsufRzdJQSdqTOpWx7e6PqH9cNVw+tWEKl23/731m3bts+3/nUxum1Ds0NYB6lWiW8dpZz3JYy55HN3sEJWlv6lDG5mREKtrmrd108XZWvfj2Pu3R4D0nGhn1KtEs47WznuWwlj2PbiaCkkaPjhWVGQ23dVbGMZ25uGHHGfnHVUWzpEtFqleJZhmvnfUuhy1j2XOjMBGUNDqUcD0//7iqSGXsOdHIqecXDWW7dvqljobKMYLSaNWxAq6eDktaK7cdK+odUbH2tOaZpEFzCRwVybFjI8fftYbKHkENGxdwHUEl7B2r25pnUgMrW8+Jf6dGThlLNOvF37WGKjKz3jEMi/b29ly7dm29wyitnhmr+pcY+e1yQa6evpv1vybCp/915OMZCWV8z5KGjX+nJJVBRNyfme217FtoaWhEnB0RGyNiU0QsGuDxz0TEhojoiIg1EXFMr8euiogHI+KhiLgmIqLIWLVv9jRjlQpQxt6xMxdX1u/rbQTW89MIK1vJs0ZMWf9OrVzXxduu/DGTFv0zb7vyxyOypICk0aGwRDAimoBrgXcB04ALI2Jav93WAe2ZOQO4CbiqeuxbgbcBM4DpwInArKJi1b6r94xVpTOubXDtjaBO6/lpBPWUPG97AsiXS54bPRk0+R0RZfw7Vc/15STt/4rsETwJ2JSZj2Tmi8By4LzeO2Tm7Zn5XHXzHqDnf7EJHASMBQ4EmoH/KDBW7aMyLuBaV2XtHZsxr1IGumRr5dYksDB16UUo44RAZU1+66CMf6fK2gsqqTZFJoITgN4DejqrbbvzUeAHAJl5N3A78Nvqz+rMfKigODUMnLFqhNk7pgLVrRehjCXPZUx+66SMf6fK2AsqqXZFzho60Ji+AWemiYiLgHaq5Z8RMRl4Iy/3EN4WEadm5k/7HbcAWABw9NFHD1PYGgpnrKqDGfNM/FSIZas38o4dP+H/GruC8fE0m/NwrnppHstWjy323/S4tt1MCNTAJc9lTH7rpIx/p1xfTtKeFJkIdgITe223AZv77xQRZwF/BczKzBeqze8B7snMZ6v7/AB4C9AnEczM64DroDJr6HC/AQ1O2aYhV3mUbcr59t/fxhXNX+PgeBGAtniaK5u/xuW/BzijuBc+c3HfZVGg8Uuey5j81lHZ/k4tnD11wJlSG7kXVFLtiiwNvQ+YEhGTImIsMB9Y1XuHiJgJfAWYk5lP9nrocWBWRIyJiGYqPYWWhkoacWWcbOHysTfuSgJ7HBwvcvnYG4t94TKWPJd1vK9GxNyZE7jivcczobWFACa0trhchqRdCusRzMyXIuISYDXQBHw9Mx+MiKXA2sxcBSwDDgFurK4O8XhmzqEyg+gZwHoq5aQ/zMxbi4pVknZnT5MtNOp/po7k6UG1D6uylTz3vNc1SyvloOPaKklgmX4HKlTZekEl1a7I0lAy8/vA9/u1Le51/6zdHLcD+FiRsUlSLco42ULsplwxLFcsRtmSX0nSfqHQBeUlabQr45TzlitKktT4TAQlaQ/KOOV8KcfqSZJUMoWWhkpqPGWbQbOMU84DlitKktTgTAQl1axnBs2eyVN6ZtAEGjoxcrIFSZLUaEwEJdWs3jNolq03UpIkqSgmgpJqVs8ZNMvaGylJklQEJ4uRVLN6zqC5p95ISZIkDY6JoKSa1XMGzTKu5ydJklQUE0FJNZs7cwJXvPd4JrS2EMCE1haueO/xI1KaWcr1/CRJkgriGEFJg1KvGTQXzp7aZ4wglGA9P0mSpIKYCEoaFUq7np8kSVIBTAQljRqu5ydJkjQ8HCMoSZIkSSVjIihJkiRJJWNpqCRJGlEr13U53leS6sxEUI2hYwWsWQrbOmFcG5y5GGbMq3dUkqR+Vq7r6jMDcNfWbi6/eT2AyaAkjSBLQzX6dayAWy+FbU8AWbm99dJKuyRpv7Js9cY+y8AAdG/fwbLVG+sUkSSVk4mgRr81S2F7d9+27d2VdknSfmXz1u5BtUuSimEiqNFvW+fg2iVJdTO+tWVQ7ZKkYpgIavQb1za4dklS3SycPZWW5qY+bS3NTSycPbVOEUlSORWaCEbE2RGxMSI2RcSiAR7/TERsiIiOiFgTEcf0euzoiPhRRDxU3ed1RcaqUezMxdDc75vk5pZKuyRpvzJ35gSueO/xTGhtIYAJrS1c8d7jnShGkkbYXmcNjYhLgG9n5jODeeKIaAKuBd4BdAL3RcSqzNzQa7d1QHtmPhcRfw5cBVxQfexbwBcy87aIOATYOZjXV4n0zA7qrKGSNCrMnTnBxE+S6qyW5SP+iEoS9wvg68DqzMwajjsJ2JSZjwBExHLgPGBXIpiZt/fa/x7gouq+04AxmXlbdb9na3g9ldmMeSZ+kiRJUo32WhqamX8NTAH+HrgYeDgi/ldEvH4vh04Anui13Vlt252PAj+o3n8DsDUibo6IdRGxrNrDKEmSJEnaRzWNEaz2AP579ecl4FDgpoi4ag+HxUBPNeCOERcB7cCyatMY4L8BnwVOBI6lkoT2P25BRKyNiLVPPfVULW9FkiRJkkpvr4lgRFwaEfdTGb93F3B8Zv458F+BP9nDoZ3AxF7bbcDmAZ7/LOCvgDmZ+UKvY9dl5iOZ+RKwEnhz/2Mz87rMbM/M9iOOOGJvb0WSJEmSRG1jBA8H3puZv+ndmJk7I+KcPRx3HzAlIiYBXcB84E977xARM4GvAGdn5pP9jj00Io7IzKeAM4C1NcQqSZIkSdqLWkpDvw/8rmcjIl4dEScDZOZDuzuo2pN3CbAaeAhYkZkPRsTSiJhT3W0ZcAhwY0Q8EBGrqsfuoFIWuiYi1lMpM/3qoN+dpOHXsQKung5LWiu3HSvqHZEkSZIGKfY2AWhErAPe3DNTaEQcAKzNzFeUatZTe3t7rl1rp6HKY+W6Lpat3sjmrd2Mb21h4eypxU/H3rECbr0Utne/3NbcAude46ytkiRJdRYR92dmey371tIjGL2Xi8jMndRWUiqpICvXdXH5zevp2tpNAl1bu7n85vWsXNdV7AuvWdo3CYTK9pqlxb6uJEmShlUtieAj1Qljmqs/fwk8UnRgGoXKWDJYp/e8bPVGurfv6NPWvX0Hy1ZvLPaFt3UOrl2SJEn7pVoSwY8Db6Uy4UsncDKwoMigNAr1lAxuewLIyu2tlzZ2MljH97x5a/eg2ofNuLbBtUuSJGm/VMuC8k9m5vzM/C+ZeWRm/mm/GT6lcpYM1vE9j29tGVT7sDlzcWVMYG/NLZV2SZIkjRp7HesXEQcBHwWOAw7qac/MjxQYl0abMpYM1vE9L5w9lctvXt+nPLSluYmFs6cW+8I9E8KsWVp5n+PaKkmgE8VIkiSNKrVM+vIPwL8Bs4GlwPupLAchvWxcW7VEcoD2RlXH99wzO+iIzxoKlaTPxG9kdKww6ZYkSYWoJRGcnJnvi4jzMvObEfEdKmsDSi87c/HAywqMUMlgXZZSqPN7njtzwsgkfqqP/kt19IxBBZNBSZK0z2qZLGZ79XZrREwHxgGvKywijU4z5lXWkhs3EYjK7QitLVe3pRTq+J5VAmUcdytJkkZMLT2C10XEocBfA6uAQ4DPFRqVRqc6lQzuaSmFwnvMLJNUUco47laSJI2YPSaCEXEA8PvMfAb4KXDsiEQlDULdllKQilTGcbeSJGnE7LE0NDN3ApeMUCzSkNRtKQWpSC7VIUmSClTLGMHbIuKzETExIg7r+Sk8MqlGC2dPpaW5qU/biCylIBXJMaiSJKlAkZl73iHi0QGaMzP3qzLR9vb2XLt2bb3DUJ3UZdZQSZIkaT8SEfdnZnst++51spjMnLTvIUnFcikFSZIkqXZ7TQQj4oMDtWfmt4Y/HEmSJElS0WpZPuLEXvcPAs4EfgGYCEqSJEnSKFRLaegne29HxDjgHwqLSJIkSZJUqFpmDe3vOWDKcAciSZIkSRoZtYwRvBXomVr0AGAasKLIoCRJkiRJxalljODf9Lr/EvCbzOwsKB5JkiRJUsFqKQ19HPh5Zv4kM+8CtkTE62p58og4OyI2RsSmiFg0wOOfiYgNEdEREWsi4ph+j78mIroi4su1vJ4kSZIkae9qSQRvBHb22t5RbdujiGgCrgXeRaWc9MKImNZvt3VAe2bOAG4Crur3+P8AflJDjJIkSZKkGtWSCI7JzBd7Nqr3x9Zw3EnApsx8pHrMcuC83jtk5u2Z+Vx18x6greexiPivwJHAj2p4LUmSJElSjWpJBJ+KiDk9GxFxHvB0DcdNAJ7otd1ZbdudjwI/qL7GAcD/BhbW8DqSJEmSpEGoZbKYjwPf7jVOrxP4YA3HxQBtOUAbEXER0A7Mqjb9BfD9zHwiYqCn2XXcAmABwNFHH11DSJIkSZKkWhaU/zXwlog4BIjM/EONz90JTOy13QZs7r9TRJwF/BUwKzNfqDafAvy3iPgL4BBgbEQ8m5l9JpzJzOuA6wDa29sHTDIlSZIkSX3ttTQ0Iv5XRLRm5rOZ+YeIODQi/mcNz30fMCUiJkXEWGA+sKrfc88EvgLMycwne9oz8/2ZeXRmvg74LPCt/kmgJEmSJGloahkj+K7M3NqzkZnPAH+8t4My8yXgEmA18BCwIjMfjIilvcYcLqPS43djRDwQEat283SSJEmSpGFSyxjBpog4sKdsMyJagANrefLM/D7w/X5ti3vdP6uG57geuL6W15MkSZIk7V0tieD/B6yJiG9Utz8MfLO4kCRJkiRJRaplspirIqIDOIvKTKA/BI4pOjBJkiRJUjFqGSMI8O/ATuBPgDOpjPmTJEmSJI1Cu+0RjIg3UJnp80JgC3ADleUjTh+h2CRJkiRJBdhTaei/Af8HODczNwFExKdHJCpJkiRJUmH2VBr6J1RKQm+PiK9GxJlUxghKkiRJkkax3fYIZub3gO9FxKuAucCngSMj4u+A72Xmj0YoRg3SynVdLFu9kc1buxnf2sLC2VOZO3NCvcOSJEmStJ/Y62QxmfmfmfntzDwHaAMeABYVHpmGZOW6Li6/eT1dW7tJoGtrN5ffvJ6V67rqHZokSZKk/USts4YCkJm/y8yvZOYZRQWkfbNs9Ua6t+/o09a9fQfLVm+sU0SSJEmS9jeDSgS1/9u8tXtQ7ZIkSZLKx0SwwYxvbRlUuyRJkqTyMRFsMAtnT6WlualPW0tzEwtnT61TRJIkSZL2N3taR1CjUM/soM4aKkmSJGl3TAQb0NyZE0z8JEmSJO2WpaGSJEmSVDImgpIkSZJUMiaCkiRJklQyJoKSJEmSVDImgpIkSZJUMiaCkiRJklQyhSaCEXF2RGyMiE0RsWiAxz8TERsioiMi1kTEMdX2EyLi7oh4sPrYBUXGKUmSJEllUlgiGBFNwLXAu4BpwIURMa3fbuuA9sycAdwEXFVtfw74YGYeB5wNfDEiWouKVZIkSZLKpMgewZOATZn5SGa+CCwHzuu9Q2benpnPVTfvAdqq7b/KzIer9zcDTwJHFBirJEmSJJVGkYngBOCJXtud1bbd+Sjwg/6NEXESMBb49bBGJ0mSJEklNabA544B2nLAHSMuAtqBWf3ajwL+AfhQZu4c4LgFwAKAo48+el/jlSRJkqRSKLJHsBOY2Gu7Ddjcf6eIOAv4K2BOZr7Qq/01wD8Df52Z9wz0Apl5XWa2Z2b7EUdYOSpJkiRJtSiyR/A+YEpETAK6gPnAn/beISJmAl8Bzs7MJ3u1jwW+B3wrM28sMMbCrFzXxbLVG9m8tZvxrS0snD2VuTP3VBkrSZIkSSOjsEQwM1+KiEuA1UAT8PXMfDAilgJrM3MVsAw4BLgxIgAez8w5wDzgVOC1EXFx9SkvzswHiop3OK1c18XlN6+ne/sOALq2dnP5zesBTAYlSZIk1V1kDjhsb9Rpb2/PtWvX1jsMAN525Y/p2tr9ivYJrS3cteiMOkQkSZIkqdFFxP2Z2V7LvoUuKF9WmwdIAvfULkmSJEkjyUSwAONbWwbVLkmSJEkjyUSwAAtnT6WlualPW0tzEwtnT61TRJIkSZL0siJnDS2tnglhnDVUkiRJ0v7IRLAgc2dOMPGTJEmStF+yNFSSJEmSSsZEUJIkSZJKxkSwEXWsgKunw5LWym3HinpHJEmSJGk/4hjBRtOxAm69FLZX1yzc9kRlG2DGvPrFJUmSJGm/YY9go1mz9OUksMf27kq7JEmSJGEi2Hi2dQ6uXZIkSVLpmAg2mnFtg2uXJEmSVDomgo3mzMXQ3NK3rbml0i5JkiRJmAg2nhnz4NxrYNxEICq3517jRDGSJEmSdnHW0EY0Y56JnyRJkqTdskdQkiRJkkrGRFCSJEmSSsZEUJIkSZJKxkRQkiRJkkqm0EQwIs6OiI0RsSkiFg3w+GciYkNEdETEmog4ptdjH4qIh6s/HyoyTkmSJEkqk8ISwYhoAq4F3gVMAy6MiGn9dlsHtGfmDOAm4KrqsYcBnwdOBk4CPh8RhxYVqyRJkiSVSZE9gicBmzLzkcx8EVgOnNd7h8y8PTOfq27eA7RV788GbsvM32XmM8BtwNkFxipJkiRJpVFkIjgBeKLXdme1bXc+CvxgiMdKkiRJkmpU5ILyMUBbDrhjxEVAOzBrMMdGxAJgAcDRRx89tCglSZIkqWSK7BHsBCb22m4DNvffKSLOAv4KmJOZLwzm2My8LjPbM7P9iCOOGLbAJUmSJKmRFZkI3gdMiYhJETEWmA+s6r1DRMwEvkIlCXyy10OrgXdGxKHVSWLeWW2TJEmSJO2jwkpDM/OliLiESgLXBHw9Mx+MiKXA2sxcBSwDDgFujAiAxzNzTmb+LiL+B5VkEmBpZv6uqFglSZIkqUwic8Bhe6NOe3t7rl27tt5hSJIkSVJdRMT9mdley76FLigvSZIkSdr/mAhKkiRJUsmYCEqSJElSyZgISpIkSVLJmAhKkiRJUsmYCEqSJElSyZgISpIkSVLJmAhKkiRJUsmYCEqSJElSyZgISpIkSVLJmAhKkiRJUsmYCEqSJElSyZgISpIkSVLJmAhKkiRJUsmYCEqSJElSyZgISpIkSVLJmAhKkiRJUsmYCEqSJElSyZgISpIkSVLJmAhKkiRJUskUmghGxNkRsTEiNkXEogEePzUifhERL0XE+f0euyoiHoyIhyLimoiIImOVJEmSpLIoLBGMiCbgWuBdwDTgwoiY1m+3x4GLge/0O/atwNuAGcB04ERgVlGxSpIkSVKZjCnwuU8CNmXmIwARsRw4D9jQs0NmPlZ9bGe/YxM4CBgLBNAM/EeBsUqSJElSaRRZGjoBeKLXdme1ba8y827gduC31Z/VmfnQsEcoSZIkSSVUZI/gQGP6sqYDIyYDbwTaqk23RcSpmfnTfvstABYAHH300fsQqiRJkoq2fft2Ojs7ef755+sdijSqHXTQQbS1tdHc3Dzk5ygyEewEJvbabgM213jse4B7MvNZgIj4AfAWoE8imJnXAdcBtLe315RkSpIkqT46Ozt59atfzete9zqcB1Aamsxky5YtdHZ2MmnSpCE/T5GlofcBUyJiUkSMBeYDq2o89nFgVkSMiYhmKhPFWBoqSZI0ij3//PO89rWvNQmU9kFE8NrXvnafe9YLSwQz8yXgEmA1lSRuRWY+GBFLI2IOQEScGBGdwPuAr0TEg9XDbwJ+DawHfgn8MjNvLSpWSZIkjQyTQGnfDce/o0LXEczM72fmGzLz9Zn5hWrb4sxcVb1/X2a2ZearMvO1mXlctX1HZn4sM9+YmdMy8zNFxilJkqRy+OEPf8jUqVOZPHkyV1555YD7vPDCC1xwwQVMnjyZk08+mccee2zXY1dccQWTJ09m6tSprF69eq+vd/3113PJJZcMV/iD9sUvfpHnnnuubq8/3Gr5/D796U9zwgkncMIJJ/CGN7yB1tbWXY81NTXtemzOnDl7fb3HHnuM6dOnD1v8g7Vy5Uo2bNiw9x2HoMgxgpIkSdKQrVzXxbLVG9m8tZvxrS0snD2VuTNrmoR+QDt27OATn/gEt912G21tbZx44onMmTOHadP6LnX993//9xx66KFs2rSJ5cuXc9lll3HDDTewYcMGli9fzoMPPsjmzZs566yz+NWvfkVTU9OgY3nppZcYM6b4/4p/8Ytf5KKLLuLggw8u/LVeoWMFrFkK2zphXBucuRhmzBvy09X6+V199dW77n/pS19i3bp1u7ZbWlp44IEHhhxDj5H6/FauXMk555zzivc4HArtEZQkSZKGYuW6Li6/eT1dW7tJoGtrN5ffvJ6V67qG/Jz33nsvkydP5thjj2Xs2LHMnz+fW2655RX73XLLLXzoQx8C4Pzzz2fNmjVkJrfccgvz58/nwAMPZNKkSUyePJl77733Fcd/4xvf4A1veAOzZs3irrvu2tV+8cUX85nPfIbTTz+dyy67jN/97nfMnTuXGTNm8Ja3vIWOjg4AlixZwgc+8AHOOOMMpkyZwle/+lWgMknIwoULmT59Oscffzw33HADAHfccQfnnHPOrte55JJLuP7667nmmmvYvHkzp59+OqeffvqQf29D0rECbr0Utj0BZOX21ksr7UNU6+fX23e/+10uvPDCQb3O/fffz5ve9CZOOeUUrr322l3t119/Pe973/s499xzeec737nHz+PUU0/lPe95D9OmTePjH/84O3fu3BXP8ccfz/Tp07nssst2Pfchhxyy6/5NN93ExRdfzM9+9jNWrVrFwoULOeGEE/j1r389qPexN/YISpIkab+zbPVGurfv6NPWvX0Hy1ZvHHKvYFdXFxMnvjypfVtbGz//+c/3uN+YMWMYN24cW7Zsoauri7e85S19ju/q6puY/va3v+Xzn/88999/P+PGjeP0009n5syZux7/1a9+xb/8y7/Q1NTEJz/5SWbOnMnKlSv58Y9/zAc/+MFdvVUdHR3cc889/Od//iczZ87k3e9+N3fffTcPPPAAv/zlL3n66ac58cQTOfXUU3f7fi+99FL+9m//lttvv53DDz98SL+zIVuzFLZ3923b3l1pH2KvYK2fX4/f/OY3PProo5xxxhm72p5//nna29sZM2YMixYtYu7cua847sMf/jBf+tKXmDVrFgsXLuzz2N13301HRweHHXYY//iP/7jbz+Pee+9lw4YNHHPMMZx99tncfPPNvPWtb+Wyyy7j/vvv59BDD+Wd73wnK1euHDAGgLe+9a3MmTOHc845h/PPP39Qv6ta2CMoSZKk/c7mrd2Daq9F5itXGxto0o3d7VfL8T//+c857bTTOOKIIxg7diwXXHBBn8ff97737SolvfPOO/nABz4AwBlnnMGWLVvYtm0bAOeddx4tLS0cfvjhnH766dx7773ceeedXHjhhTQ1NXHkkUcya9Ys7rvvvhrf/Qjb1jm49hrU+vn1WL58Oeeff36f0t3HH3+ctWvX8p3vfIdPfepTr+hl27ZtG1u3bmXWrFkAuz6fHu94xzs47LDDAPb4eZx00kkce+yxNDU1ceGFF3LnnXdy33337To3xowZw/vf/35++tM+q+ONKBPBonSsgKunw5LWyu0+dINLkiSVzfjWlkG116KtrY0nnnhi13ZnZyfjx4/f434vvfQS27Zt47DDDqv5+D0lJ6961at23d9TYtP/OXaXiEKl17Kn9BDY52UFhsW4tsG116DW33+P5cuXv6IstGf/Y489ltNOO63P+EGofCb78vn1GMzn13//kfr8TASLUEBNtCRJUpksnD2Vlua+k7C0NDexcPbUIT/niSeeyMMPP8yjjz7Kiy++yPLlywecOXLOnDl885vfBCrjtc444wwigjlz5rB8+XJeeOEFHn30UR5++GFOOumkPseefPLJ3HHHHWzZsoXt27dz44037jaeU089lW9/+9tAZVzZ4Ycfzmte8xqgMk7x+eefZ8uWLdxxxx27yg5vuOEGduzYwVNPPcVPf/pTTjrpJI455hg2bNjACy+8wLZt21izZs2u13j1q1/NH/7whyH/zobszMXQ3C9pb26ptA9RrZ8fwMaNG3nmmWc45ZRTdrU988wzvPDCCwA8/fTT3HXXXa+YhKW1tZVx48Zx5513Auz6fAayu88DKqWhjz76KDt37uSGG27g7W9/OyeffDI/+clPePrpp9mxYwff/e53d/U8HnnkkTz00EPs3LmT733ve7teo8jPzzGCRSigJlqSJKlMesYBDuesoWPGjOHLX/4ys2fPvu4oMwAAB7NJREFUZseOHXzkIx/huOOOA2Dx4sW0t7czZ84cPvrRj/KBD3yAyZMnc9hhh7F8+XIAjjvuOObNm8e0adMYM2YM11577StmDD3qqKNYsmQJp5xyCkcddRRvfvOb2bFjxytigcqkMB/+8IeZMWMGBx988K7kEyqlhe9+97t5/PHH+dznPsf48eN5z3vew913382b3vQmIoKrrrqKP/qjPwJg3rx5zJgxgylTpvQZk7hgwQLe9a53cdRRR3H77bcP+Xc3aD3/5x3GWUNr/fygMinL/Pnz+/S0PfTQQ3zsYx/jgAMOYOfOnSxatGjA2Ti/8Y1v8JGPfISDDz6Y2bNn7zae3X0e//Zv/8Ypp5zCokWLWL9+/a6JYw444ACuuOIKTj/9dDKTP/7jP+a8884D4Morr+Scc85h4sSJTJ8+nWeffRaA+fPn82d/9mdcc8013HTTTbz+9a8f8u+vv9hTF+Vo0t7enmvXrq13GBVLWoGBfq8BS7aOdDSSJEn7hYceeog3vvGN9Q5jv7dkyRIOOeQQPvvZz9Y7FA3BHXfcwd/8zd/wT//0T4W+zkD/niLi/sxsr+V4S0OLUEBNtCRJkiQNF0tDi3Dm4sqYwN7loftYEy1JkqRyWLJkSb1D0D447bTTOO200+odxl7ZI1iEGfPg3Gtg3EQgKrfnXuP4QEmSJEn7BXsEizJjnomfJElSP3ubnl/S3g3HPC/2CEqSJGlEHHTQQWzZsmVY/hMrlVVmsmXLFg466KB9eh57BCVJkjQi2tra6Ozs5Kmnnqp3KNKodtBBB9HWtm8TUZoISpIkaUQ0NzczadKkeochCUtDJUmSJKl0TAQlSZIkqWRMBCVJkiSpZKJRZm2KiKeA39Q7jgEcDjxd7yDU0DzHVCTPLxXJ80tF8vxSkfbX8+uYzDyilh0bJhHcX0XE2sxsr3ccalyeYyqS55eK5PmlInl+qUiNcH5ZGipJkiRJJWMiKEmSJEklYyJYvOvqHYAanueYiuT5pSJ5fqlInl8q0qg/vxwjKEmSJEklY4+gJEmSJJWMiWCBIuLsiNgYEZsiYlG941FjiYjHImJ9RDwQEWvrHY9Gv4j4ekQ8GRH/2qvtsIi4LSIert4eWs8YNXrt5vxaEhFd1evYAxHxx/WMUaNXREyMiNsj4qGIeDAi/rLa7jVM+2wP59eovoZZGlqQiGgCfgW8A+gE7gMuzMwNdQ1MDSMiHgPaM3N/XMNGo1BEnAo8C3wrM6dX264CfpeZV1a/0Do0My+rZ5wanXZzfi0Bns3Mv6lnbBr9IuIo4KjM/EVEvBq4H5gLXIzXMO2jPZxf8xjF1zB7BItzErApMx/JzBeB5cB5dY5JknYrM38K/K5f83nAN6v3v0nlD580aLs5v6RhkZm/zcxfVO//AXgImIDXMA2DPZxfo5qJYHEmAE/02u6kAU4Y7VcS+FFE3B8RC+odjBrWkZn5W6j8IQT+S53jUeO5JCI6qqWjlu1pn0XE64CZwM/xGqZh1u/8glF8DTMRLE4M0GYdrobT2zLzzcC7gE9Uy64kaTT5O+D1wAnAb4H/Xd9wNNpFxCHAPwKfyszf1zseNZYBzq9RfQ0zESxOJzCx13YbsLlOsagBZebm6u2TwPeolCNLw+0/qmMjesZIPFnneNRAMvM/MnNHZu4EvorXMe2DiGim8p/0b2fmzdVmr2EaFgOdX6P9GmYiWJz7gCkRMSkixgLzgVV1jkkNIiJeVR2sTES8Cngn8K97PkoaklXAh6r3PwTcUsdY1GB6/oNe9R68jmmIIiKAvwceysy/7fWQ1zDts92dX6P9GuasoQWqTiH7RaAJ+HpmfqHOIalBRMSxVHoBAcYA3/H80r6KiO8CpwGHA/8BfB5YCawAjgYeB96XmU74oUHbzfl1GpWSqgQeAz7WM55LGoyIeDvwf4D1wM5q8/9NZRyX1zDtkz2cXxcyiq9hJoKSJEmSVDKWhkqSJElSyZgISpIkSVLJmAhKkiRJUsmYCEqSJElSyZgISpIkSVLJmAhKktRPROyIiAd6/Swaxud+XUSMqrWmJEmNZ0y9A5AkaT/UnZkn1DsISZKKYo+gJEk1iojHIuL/iYh7qz+Tq+3HRMSaiOio3h5dbT8yIr4XEb+s/ry1+lRNEfHViHgwIn4UES11e1OSpFIyEZQk6ZVa+pWGXtDrsd9n5knAl4EvVtu+DHwrM2cA3wauqbZfA/wkM98EvBl4sNo+Bbg2M48DtgJ/UvD7kSSpj8jMescgSdJ+JSKezcxDBmh/DDgjMx+JiGbg3zPztRHxNHBUZm6vtv82Mw+PiKeAtsx8oddzvA64LTOnVLcvA5oz838W/84kSaqwR1CSpMHJ3dzf3T4DeaHX/R04Zl+SNMJMBCVJGpwLet3eXb3/M2B+9f77gTur99cAfw4QEU0R8ZqRClKSpD3xG0hJkl6pJSIe6LX9w8zsWULiwIj4OZUvUy+stl0KfD0iFgJPAR+utv8lcF1EfJRKz9+fA78tPHpJkvbCMYKSJNWoOkawPTOfrncskiTtC0tDJUmSJKlk7BGUJEmSpJKxR1CSJEmSSsZEUJIkSZJKxkRQkiRJkkrGRFCSJEmSSsZEUJIkSZJKxkRQkiRJkkrm/wfGdbFm7PxgzwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x113588350>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot train and validation accuracies of the two models\n",
    "\n",
    "train_accs = []\n",
    "val_accs = []\n",
    "for dropout in dropout_choices:\n",
    "  solver = solvers[dropout]\n",
    "  train_accs.append(solver.train_acc_history[-1])\n",
    "  val_accs.append(solver.val_acc_history[-1])\n",
    "\n",
    "plt.subplot(3, 1, 1)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].train_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Train accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "  \n",
    "plt.subplot(3, 1, 2)\n",
    "for dropout in dropout_choices:\n",
    "  plt.plot(solvers[dropout].val_acc_history, 'o', label='%.2f dropout' % dropout)\n",
    "plt.title('Val accuracy')\n",
    "plt.xlabel('Epoch')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.legend(ncol=2, loc='lower right')\n",
    "\n",
    "plt.gcf().set_size_inches(15, 15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question\n",
    "Explain what you see in this experiment. What does it suggest about dropout?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Answer\n",
    "From the graphs it can be seen that dropout regularizes the learning process, hence the neural network is less likely to overfit the data. This leads to a slighly lower training accuracy but the validation accuracy follows the training curve more closely. Hence it takes more"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}