2__Optimizing_Portfolios.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Optimizing Portfolios\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Python optimizers\n",
    "\n",
    "Optimizers is one of the most important library of a Machine Learning (ML) framowork. Most of ML models are \n",
    "formuled as an optimization problems, i.e. minimizing a given cost function. Historically, __Gradient Descent__ was one \n",
    "of the first optmization methods used for accomplishing such task. \n",
    "\n",
    "__Gradient Descent__ is based on the observation that if a \n",
    "multivariate function $f()$ is defined and differentiable in a neighborhood of a given point $a$ then for $\\alpha$ small \n",
    "enough  \n",
    "\n",
    "$$f(a) \\geq f \\big( a-\\alpha \\bigtriangledown f(a) \\big)$$\n",
    "\n",
    "and if $$f(a) = f\\big( a-\\alpha \\bigtriangledown f(a) \\big)$$ if and only if $\\alpha=0$ then $a$ is a __local minimum__. Also, \n",
    "__if $f()$ is a convex function__ then $a$ is also a __global minimum__. \n",
    "\n",
    "One of the main problems with __Gradient Descent__ is that it needs for each iteration you to provide $\\alpha$ also \n",
    "known as __learning rate__. If $\\alpha$ is too small you are going to converge too slowly. On the other hand, if $\\alpha$\n",
    "is too large you might diverge from the local optimum. \n",
    "\n",
    "There are more sophisticated algorithms than Gradient Descent that are able to select the best learning \n",
    "rate automatically and, hence, for which you only need to provide \n",
    "* $f()$, i.e. the cost-function  \n",
    "* $\\bigtriangledown f()$, i.e. the gradient \n",
    "\n",
    "The most popular of these algorithms are \n",
    "* [Conjugate Gradient](https://en.wikipedia.org/wiki/Conjugate_gradient_method)\n",
    "* [BFGS](https://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm)\n",
    "* [L-BFGS](https://en.wikipedia.org/wiki/Limited-memory_BFGS)\n",
    "\n",
    "Python offers the package [scipy.optimize](http://docs.scipy.org/doc/scipy/reference/optimize.html) \n",
    "where to minimize a function the following methods are available:\n",
    "* minimize(method=’Nelder-Mead’), usually applied to nonlinear optimization problems \n",
    "for which derivatives may not be known \n",
    "* minimize(method=’Powell’), usually applied to functions that need not be differentiable, \n",
    "and no derivatives are taken\n",
    "* minimize(method=’CG’), a.k.a. _Conjugate Gradient_ (see above)\n",
    "* minimize(method=’BFGS’), see above \n",
    "* minimize(method=’Newton-CG’), variant of Conjugate Gradient\n",
    "* minimize(method=’L-BFGS-B’), see above L-BFGS\n",
    "* minimize(method=’TNC’)\n",
    "* minimize(method=’COBYLA’), i.e. _Constrained optimization by linear approximation_  is \n",
    "a numerical optimization method for constrained problems where the derivative of the \n",
    "objective function is not known, invented by Michael J. D. Powell.\n",
    "* minimize(method=’SLSQP’), i.e. _Sequential quadratic programming_ is an \n",
    "iterative method for nonlinear optimization.\n",
    "* minimize(method=’dogleg’)\n",
    "* minimize(method=’trust-ncg’)\n",
    "\n",
    "Let's see these algorithms in action.  \n",
    "\n",
    "### Parabola\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Nelder-Mead</th>\n",
       "      <td>[2.00002441406]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Powell</th>\n",
       "      <td>1.9999999999683606</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CG</th>\n",
       "      <td>[1.99]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BFGS</th>\n",
       "      <td>[1.99999999056]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>L-BFGS-B</th>\n",
       "      <td>[2.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TNC</th>\n",
       "      <td>[2.00000000168]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>COBYLA</th>\n",
       "      <td>2.0001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[2.0]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            min\n",
       "Method                         \n",
       "Nelder-Mead     [2.00002441406]\n",
       "Powell       1.9999999999683606\n",
       "CG                       [1.99]\n",
       "BFGS            [1.99999999056]\n",
       "L-BFGS-B                  [2.0]\n",
       "TNC             [2.00000000168]\n",
       "COBYLA                   2.0001\n",
       "SLSQP                     [2.0]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd \n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import scipy.optimize as spo \n",
    "\n",
    "\n",
    "def parab(X):\n",
    "    ## X = 2 is the min\n",
    "    Y = (X - 2)**2 + 1.5 \n",
    "    return Y\n",
    "\n",
    "initial_guess = 3\n",
    "opt_methods_no_Jacobian = ['Nelder-Mead','Powell','CG','BFGS','L-BFGS-B','TNC','COBYLA','SLSQP']\n",
    "mins = [spo.minimize(parab,initial_guess,method=om,) for om in opt_methods_no_Jacobian]\n",
    "\n",
    "min_df = pd.DataFrame({'Method' : opt_methods_no_Jacobian,\n",
    " 'min' : [m.x for m in mins]\n",
    "  }).set_index('Method')\n",
    "min_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfwAAAFyCAYAAAAQ6Gi7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xd4FNXbxvHvCVISioCUiBoFfUFQgR9FRKUoCopkQQEB\nGwQ7YAENKiBNqRakiIIiiCWKgqigFEXQIIoQCwLBQolSpIiARGrO+8dJQtpM2u7OzO7zua69ILOz\nM8/cmeRkZs6cUVprhBBCCBHaIpwuQAghhBCBJw2+EEIIEQakwRdCCCHCgDT4QgghRBiQBl8IIYQI\nA9LgCyGEEGFAGnwhhBAiDEiDL4QQQoQBafCFEEKIMCANvhCiQJRSrZRSaUqpm/y4zOFKqTR/LU8I\nYU0afCFcSinVM72BzXj9p5TapJSarJSq5lBZ/h6LWwdgmUKIPJzmdAFCCFsaeBLYCpQBrgTuB65X\nSl2stT4S5HpUkNcnhPATafCFcL9FWuuk9P+/ppT6G+gPdATeLepClVKlgWNanqAlRFiQU/pCeM8y\nzJF2TaVUJaXUs0qpn5RSh5RSB5RSnyil6mf9QJbr792UUk8rpf4ADgPlC7qMdBoooZQarZTaqZT6\nVyn1oVLq7JwzKqW6KqXWKKVSlVJ7lFJvKKVq5LdxSqk4pdTnSqm/lFJHlFLrlVL3FS0qIUQGOcIX\nwnsuSP93H1AL8AHvAVuA6sC9wHKlVD2t9a4cn30SOAo8C5QGjgEXFWIZChgCpAFjgWqYsw1LlVIN\ntdZHAZRSvYDXgG+Bx9OX+TBwuVLqf1rrgzbbdx/wM/AhcAKIBaYqpZTW+qWChiSEyE4afCHc73Sl\n1Bmcuob/JJAKLAD2aq1rZ51ZKfUGsAm4ExiVY1mlgUZa62NZ5v+pkMuoBFyotU5Nn/d7YA5wNzBF\nKXUa5o+Bn4BWGetSSq1Mr7k/MMJme1tm/OGQbqpS6lNgACANvhBFJKf0hXA3BXwO7AH+AN4GDgKd\ntNY7tdbHM2dUKkIpVRnzx8AmoFEey5uVtbEHKMIyXs9o7NM//z6wE2ifPqkp5sh/atZ1aa0/AZKB\nG+w2OGtjr5SqkP7HzpdALaVUebvPCiGsyRG+EO6mgT7Ar5jT239prTdlvKmUUphT5fcDNYESWT63\nN4/lbc05oQjL+M1i2rnp/49J/+wvecyXDFyRx/Ss9VyBOQNwGRCV5S0NnA4csvu8ECJv0uAL4X7f\nZemln9NgYCQwA3Nt/W/M9fWJ5H0G7z8/LCMvyuL/haKUqgV8BmzEnPr/A9PP4AbMHyVyVlKIIpIG\nXwhv6wws01rfnXWiUqoi5jJAIJbxf3lMOx/4Mf3/WzGNfh1geY756gDbbGqJBUoBsVrr7VlqaWPz\nGSFEAchfy0J420lyHFErpboCZwVwGXcopcrlmPdM4JP0SWuA3cB9SqmSWea7HqiL6bhnVwtk+d2k\nlDod6FWQDRFCWJMjfCHcLb/T4wuAJ5VSrwFfA5cAtwK/F2IdhV3G30CiUmomEA08hLle/yqA1vqE\nUuoxzG15XyqlEtLnexDYDLxgU8sS4DiwQCk1DSgP3AX8lb4MIUQRSYMvhLvlNwreaEzHtluAm4G1\nmN7yY/P4rNWyCruM0UB9zP315YGlQN+sw/xqrV9XSh1On2csZpCfucDjedyDr7N87helVGfgaeAZ\nYBcwFTPmwAybHIQQ+VAyqqYQQggR+uQavhBCCBEGpMEXQgghwoA0+EIIIUQYkAZfCCGECAOO99JP\nHye7HWawjiP2cwshhBAiizLAecBirfU+uxkdb/Axjf1bThchhBBCeNitmIdrWXJDg78V4M0336Ru\n3boOl+JO/fv3Z8KECU6X4UqSjT3Jx5pkY0/yseambDZu3Mhtt90GeTwYKyc3NPhHAOrWrUujRnk9\niVOcfvrpko0Fycae5GNNsrEn+VhzaTb5XhKXTnsesGvXLqdLcC3Jxp7kY02ysSf5WPNqNtLge8D2\n7dvznylMSTb2JB9rko09yceaV7ORBt8DGjdu7HQJriXZ2JN8rEk29iQfa17NRhp8D+jRo4fTJbiW\nZGNP8rEm2diTfKx5NRvHH56jlGoErF27dq0bO0EIIYQQmY4fh507ISbG6UqMpKSkjDMOjbXWSXbz\nuqGXfoGkpKSwd+9ep8sQYahKlSrEuOWnWwjhqCVLIDYWfvkFLrjA6WoKxxMNfkpKCnXr1iU1NdXp\nUkQYioqKYuPGjZ5s9OPi4pg5c6bTZbiSZGNP8snb7NlQsWIcF1zgvWw80eDv3buX1NRUGZxHBF3G\noBZ79+71ZIPftm1bp0twLcnGnuST2/798OGH0KWLN7PxRIOfQQbnEaJwvNq5KBgkG3uST25z5sCJ\nE/Dss97MRnrpCyGEEAUwaxa0awfR0U5XUjSeOsIXQgghnLBpE3zzDbz7rtOVFJ2nGnytNUdPHHW6\njAIpVaIUSimnyxBhLjExkSuvvNLpMlxJsrEn+WRnOuuBz+fdbDzV4B9PO06ZUWWcLqNAjgw+QunT\nSjtdhghz48eP9+QvpmCQbOxJPqecPGka/O7doUwZ72Yj1/BDyPDhw4mI8P+39LzzzqN3795+X64I\nvHfeecfpElxLsrEn+ZzyxRfw55/Qs6f52qvZSIMfQpRSRb6MsGrVKkaMGMHBgwdzvRcRESGXJzwq\nKirK6RJcS7KxJ/mc8vrrULs2NGtmvvZqNtLgCwC+/vprRo4cyT///JPrvU2bNjF9+nQHqhJCCGcd\nPAhz50KvXuD14x5PXcMXgWP3TIWSJUsGsRIhhHCP99+HI0fg9tudrqT45AjfBf79918efvhhatas\nSZkyZahevTpt27blhx9+yJznvffeo0mTJkRFRVG1alVuv/12duzYYbvcbdu2ERERwezZs3O9FxER\nwciRIwEYMWIEAwcOBMz1+oiICEqUKEFKSkrmtJzX8Lds2ULXrl0544wzKFu2LM2bN+eTTz7JNs+K\nFSuIiIjgvffeY9SoUZxzzjlERkZyzTXX8Pvvv2eb97fffqNz586ceeaZREZGcs4559CjRw8OHTpU\nwBRFXuLj450uwbUkG3uSj/H669CmDZx99qlpXs1GjvBd4N5772XevHk88MAD1K1bl3379rFy5Uo2\nbtxIw4YNmTVrFr1796ZZs2aMHTuWv/76ixdeeIGvv/6a77//ngoVKhRr/TfddBO//PIL77zzDhMn\nTuSMM84AoGrVqgC5rt/v3r2b5s2bc+TIER566CEqV67M66+/TmxsLPPmzaNjx47Z5h87diwlSpQg\nPj6eAwcOMG7cOG677TZWrVoFwPHjx2nbti3Hjx/nwQcfJDo6mu3bt7NgwQL++ecfypcvX6ztC2de\nHA44WCQbe5IPbN4MX34Jb7yRfbpXs5EG3wU++eQT7r77bsaPH5857dFHHwXgxIkTPP7449SvX58V\nK1ZQqlQpAK644go6dOjAhAkTGDZsWLHWf8kll9CoUSPeeecdOnbsmO/OPGbMGPbs2UNiYiLNmzcH\n4K677qJ+/foMGDAgV4N/9OhRfvzxR0qUKAFAxYoVefjhh9mwYQP16tVjw4YNbN26lblz53LjjTdm\nfm7IkCHF2i4BDzzwgNMluJZkY0/yMbfilS8PWX4tAd7NplCn9JVS9ymlflRKHUh/fa2Uui7L+8uV\nUmlZXieVUlP9X3ZoqVixIqtXr2bnzp253luzZg27d++mT58+mY09QPv27bnwwgtZuHBhMEsF4NNP\nP+XSSy/NbOwBypYtyz333MPWrVvZsGFDtvl79+6d2dgDtGjRAq01mzdvBuD0008HYNGiRfz3339B\n2AIhhLCXlmYa/K5doWxZp6vxj8Jew/8DeAxonP5aBnyolMp4hJ0GpgPVgWjgTGCgf0oNXePHj2fd\nunWcc845NGvWjBEjRrBlyxbAXIdXSlG7du1cn7vwwgvZtm1bsMtl27Zt1KlTJ9f0jCcZ5qzpnHPO\nyfZ1pUqVANi/fz9g+gg88sgjvPrqq1SpUoXrrruOqVOn5nmLoBBCBENiImzZcure+1BQqAZfa71Q\na71Ia/1b+msI8C9wWZbZUrXWe7TWu9Nf//q14hDUtWtXNm/ezJQpUzjrrLN49tlnueiii1i8eLFt\n7/n8WN07n5aWVuRlFkXWo/ussm7bM888w08//cTgwYM5cuQIDz74IBdffHG+HROFveTkZKdLcC3J\nxl645zNrFtSsCXkNqOfVbIrcS18pFaGU6g5EAV9neetWpdQepdQ6pdRopVRksasMA9WrV+e+++5j\n3rx5bNmyhTPOOINRo0Zx3nnnobVm06ZNuT6zadMmzj33XMtlZhxJ57y3Pq+zAoUZWOfcc8/Ns56N\nGzdmvl8UF110EYMGDWL58uUkJiby559/8vLLLxdpWcLIuPtC5CbZ2AvnfA4fhvfeM0f3eQ1e6tVs\nCt3gK6UuVkodAo4CU4EbtdYZv/3fAm4DWgOjgduBN/JajjDS0tJynbquUqUKNWrU4OjRozRt2pRq\n1arx8ssvc/z48cx5Pv30UzZu3EiHDh0sl12+fHmqVKnCl19+mW36lClTcjXwZdMvUuU18E5O7du3\nZ/Xq1Xz77beZ0w4fPsz06dOpWbMm9erVy3cZWR06dIiTJ09mm3bRRRcRERHB0aPeeFiSW02ZMsXp\nElxLsrEXzvl88AH8+y/ccUfe73s1m6L00k8GGgAVgc7AbKVUS611stb61SzzrVdK7QI+U0rV1Fpv\n8UO9IefQoUOcffbZdOnShQYNGlCuXDmWLl3KmjVreP755ylRogTjxo2jd+/etGzZkh49erBr1y4m\nTZpErVq1ePjhh22Xf9dddzF27FjuvvtumjRpwpdffsmvv/6a61JB48aN0VozaNAgunfvTsmSJfH5\nfERG5j5B8/jjj5OQkMB1113Hgw8+SOXKlZk1axbbtm1j3rx5hc5g2bJl9OvXj65du1K7dm1OnDjB\n7NmzOe200+jcuXOhlydO8ertQ8Eg2dgL53xmzYKWLc0p/bx4NZtCH+FrrU9orTdrrZO01oOBH4GH\nLGb/FlDABfktt3379vh8vmyv5s2bM3/+/MKW6ClRUVH07duXH3/8keHDhzNgwAB+/fVXXnrpJR56\nyMTas2dP3n33XY4fP87jjz/OK6+8QufOnfnqq69y3YOf88h96NCh3HXXXcydO5fHHnsMrTWffvpp\nrnH3mzRpwtNPP81PP/1EXFwct9xyC3v27MlcZtZ5q1WrxqpVq2jbti1Tpkxh0KBBlClThgULFuDz\n+WzryWt6gwYNuO6661iwYAGPPPIII0aMoEKFCixatIhLL720CKkGVlJSEj6fj71792abPmzYMMaN\nG5dtWkpKCj6fL9c1v8mTJ+cavCM1NRWfz0diYmK26QkJCcTFxeWqo1u3brl+PpYsWZLrewDQt29f\nZsyYIdsh2yHbkc92bN0Kn38OV13lvu1ISEjIbBujo6Px+Xz0798/12esqOJ0CgNQSn0ObNNa53qc\nmlLqCuBLoIHW+meLzzcC1q5du5ZGjRrluY6kpCQaN27MqtWraP5J8zzncRt5PG5oyNj37PZPIUTo\nGDYMnn8edu3yxu14Gb+jgMZa6yS7eQt7H/4opdSVSqlz06/ljwFaAW8qpWoppYYopRqlv+8DXgdW\nWDX2QojAynkkIk6RbOyFYz4nT8LMmea593aNvVezKew1/OrAbMz99QeAn4C2WutlSqmzgWswp/fL\nYu7Zfw8Y5b9yhRCFkZqa6nQJriXZ2AvHfD7/HP74A+68034+r2ZTqAZfa32XzXt/YnrnB0zJiJIc\nGXwkkKvwm1IlSuU/kxABNmLECKdLcC3Jxl445jNjBtSrd+q591a8mo2nxtJXSsl1cSGEEH63bx/M\nnw+jR3v/ufdW5PG4Qgghwt5bb5nx80PhufdWpMEXIoTlvKVInCLZ2AunfLQ2p/N9PqhWLf/5vZqN\nNPhChLDevXPdLSvSSTb2wimfpCT46Sco6CZ7NRtp8IUIYcOHD3e6BNeSbOyFUz4zZkCNGtCuXcHm\n92o20uALEcJksCBrko29cMnnv//g7behVy84rYDd2L2ajad66WutPfMwlVKlShXqCXRCCCGCb948\nOHAA8hjVNuR4qsE/fvw4ZcqUcbqMAjly5AilS8sthEII4WavvQatWsEF+T7xxfvklL7DZs2aRURE\nBCkpKYX+7IoVK4iIiMj1+Nu8bNu2jYiICGbPnh3UGoWzcj7YRJwi2dgLh3w2b4Zly/IfWS8nr2Yj\nDb7Dcj6JriifzyohIYGJEycWaN7CrEMuT3hTUpLtszTCmmRjLxzymTkTKlSAwj6F26vZeOqUfii6\n44476NGjB6VKFX4o3latWvHff/9l++zbb7/N+vXrMx+tm+Hcc8/lv//+o2TJksWuWXjHiy++6HQJ\nriXZ2Av1fE6eNM+979EDoqIK91mvZiMNvsOUUkVq7DMU5rPFWY8QQoSSpUvhzz8Lfu99KJBT+g7L\n6/r4eeedh8/nY+XKlTRr1ozIyEjOP/983njjjWyfzXkN/6qrrmLhwoWZ1+sjIiKoVasWkPc1/HXr\n1hEXF8f5559PZGQkZ555JnfeeSd///13kbfnvffe46KLLiIyMpL69eszf/58evXqRc2aNS3rzmDV\nz2DTpk106dKFM844g8jISJo2bcrHH3+cbZ4TJ04wYsQIateuTWRkJFWqVKFFixZ8/vnnmfP89ddf\nxMXFcc4551CmTBlq1KhBp06dpG+CEGHotdfg4ouhaVOnKwkeOcJ3WF7Xx5VS/Prrr3Tt2pU777yT\nXr168dprrxEXF0eTJk2oW7dutnkzDBkyhAMHDrB9+3ZeeOEFtNaUK1fOct1Lly5ly5Yt9O7dm+jo\naNavX8+0adPYsGEDq1atKvS2LFy4kO7du9OgQQPGjh3L/v37ufPOOznrrLPy3MaCWL9+PVdeeSVn\nn302TzzxBGXLlmXOnDl06tSJefPm0bFjRwCGDRvG2LFjueeee2jatCkHDx5kzZo1JCUl0aZNGwBu\nuukmNm7cyIMPPsi5557L7t27Wbp0KSkpKcTExBR6e4UQ3rR7t3lQzvjxofugnDxprR19AY0AvXbt\nWm1l7dq1GtCrVq3SgCdeR44csdyerGbNmqUjIiL0tm3bMqedd955OiIiQq9cuTJz2p49e3SZMmV0\nfHx85rTly5friIgIvWLFisxpHTp00DVr1sy1nq1bt2qllH799dczp+VV4zvvvKMjIiJ0YmKibY15\nueSSS3RMTIxOTU3NnPbll19qpVS2mvKq26rGNm3a6IYNG+rjx49nm/eKK67QderUyfy6YcOGOjY2\n1rK2f/75Ryul9HPPPWe7DTll7Ht2+6eb2WUS7iQbe6Gcz/jxWpcurfW+fUX7vJuyyfgdBTTS+bS3\nckrfperVq8fll1+e+XWVKlWoU6cOmzdv9ts6so4TcPToUfbt20ezZs3QWhe6F+rOnTv5+eef6dmz\nJ5GRkZnTW7RowSWXXFKk+vbv388XX3xB165dOXDgAPv27ct8tW3bll9//ZWdO3cCULFiRdavX89v\nv/2W57IiIyMpVaoUy5cv559//ilSPV7Ur18/p0twLcnGXqjmozVMnw5du0LlykVbhlezkQbfpfI6\nxVypUiX279/vt3Xs37+fhx56iOjoaCIjI6latSq1atVCKcWBAwcKtaxt27YBcP755+d674Iijmjx\n22+/obXmySefpGrVqtleGWNZ7969G4CRI0fyzz//ULt2berXr89jjz3GunXrMpdVqlQpxo0bx6ef\nfkr16tVp1aoVzzzzDH/99VeRavOKtm3bOl2Ca0k29kI1n+XL4bff4J57ir4Mr2Yj1/BdqkSJEnlO\n1+YyiF907dqVb775hoEDB9KgQQPKlStHWloa7dq1Iy0tzW/rycnq+v3JkyezfZ1Rw6OPPko7i6da\nZPwx0aJFC37//Xc+/PBDlixZwquvvsrzzz/PtGnTMp9s9dBDD+Hz+Zg/fz6LFy9m6NChjBkzhi++\n+IIGDRr4a/OEEC42fTpceCFceaXTlQSfNPghpqCd4f755x+WLVvGU089xeDBgzOnW50Sz8+5555r\n+fmc0ypVqoTWOtep9a1bt2b7OuMOg5IlS3L11VfnW0PFihXp2bMnPXv2JDU1lRYtWjB8+PBsj7Ks\nWbMm/fv3p3///vz+++80aNCA5557rkgjEAohvGXPHjN2/tixYdZZL52c0g8xZcuWLdDp+IwzCDmP\n5CdMmFCkUfXOPPNMLr74YmbPnk1qamrm9BUrVmQ7tQ7mj4MSJUrkui1v6tSp2dZdtWpVWrduzbRp\n09i1a1eude7duzfz/zlvJYyKiuKCCy7IfNjSf//9l+vBSzVr1qR8+fKeeSBTUcyfP9/pElxLsrEX\nivlk/F1/xx3FW45Xs5EjfI/LeYq/cePGzJkzh0ceeYSmTZtSrlw5OnTokOtz5cuXp2XLlowfP55j\nx45x1llnsWTJErZs2VLkywajR4+mU6dOXH755cTFxfH333/z4osvcskll/Dvv/9mzlehQgW6du3K\npEmTAHPd/+OPP87WgGd48cUXMzv+3X333dSqVYu//vqLVatWsX37dr7//nvAdHJs3bo1jRs3pnLl\nynz33Xe8//77PPjggwD88ssvtGnThptvvpl69epx2mmnMW/ePHbv3k2PHj2KtL1ekJCQQKdOnZwu\nw5UkG3uhlk9GZ70uXeCMM4q3LM9mk183/kC/kNvyct3yVrNmTe3z+XLN27p1a3311Vdnfp3X7W2H\nDx/Wt912m65cubKOiIjIvB1u69atOiIiItstbzt27NCdO3fWlStX1pUqVdLdu3fXu3bt0hEREXrk\nyJG2NVqZM2eOrlevni5TpoyuX7++XrBgge7SpYuuV69etvn27t2ru3btqsuVK6fPOOMM3adPH71h\nw4ZcNWqt9ZYtW3SvXr10jRo1dOnSpfU555yjfT6fnjdvXuY8o0eP1pdddpmuXLmyLlu2rK5Xr54e\nO3asPnHihNZa63379ukHHnhA16tXT5cvX15XqlRJN2/eXM+dO9d2e7x+W54Qwli+XGsw/4aSwtyW\np7QfO4EVhVKqEbB27dq1NGrUKM95kpKSaNy4MatWraJ58+bBLbCI5PG4p/zvf/+jWrVqLF682OlS\nCi1j37PbP4UQ7nfrrbBmDSQnh9b1+4zfUUBjrbXt/dRyDV/4zcmTJ3P1CVi+fDk//vgjV111lUNV\nCSHC3b598P775la8UGrsC0uu4Qu/+fPPP7n22mu59dZbqVGjBhs3bmTatGnUqFGDe++91+nyhBBh\n6vXXzb89ezpbh9M81eCXLFmSI0eOOF1GgYTjk+kqVapE48aNmTFjBnv27KFs2bLExsYyZswYKlWq\n5HR5YSkuLo6ZM2c6XYYrSTb2QiWfjM56N90EVar4Z5lezcZTDb5SSq6Lu1iFChVISEhwugyRhVdH\nBAsGycZeqOTz1VewaRO89JL/lunVbOQavhAhLJRvOSwuycZeqOQzfTpccAG0bu2/ZXo1G2nwhRBC\nhCTprJedNPhCCCFC0htvQFqadNbLUKgGXyl1n1LqR6XUgfTX10qp67K8X1op9aJSaq9S6pBS6n2l\nVDX/ly2EKIjExESnS3Atycae1/PR2ly3v+kmqObnVsir2RS2094fwGNAxtNQegEfKqUaaq03Ai8A\n1wOdgYPAi8BcoIU/it24caM/FiNEgXl9nxs/fjxXhuNjwQpAsrHn9XyWLYNffoFXXvH/sr2aTbFH\n2lNK7QMexTTse4DuWusP0t+rA2wELtNar7b4fL4j7aWkpFC3bt1sD2URIliioqLYuHEjMTExTpdS\naKmpqURFRTldhitJNva8nk+XLmZUvXXr/H/93k3ZFGakvSLflqeUigBuBqKAVUDj9OV9njGP1nqT\nUioFaA7k2eAXRExMDBs3bszz4SpCBFqVKlU82dgDrvml5EaSjT0v57N9O8yfDxMnBqaznlezKXSD\nr5S6GNPAlwEOATdqrZOVUv8DjmmtD+b4yF9AdHELjYmJ8ewvXSGEEMHzyitQpgzcfrvTlbhLUY7w\nk4EGQEXMtfrZSqmWNvMrzJN8hBBCiIA6ftw0+LffDhUqOF2NuxT6tjyt9Qmt9WatdZLWejDwI/AQ\nsAsopZTKGXE1zFG+rfbt2+Pz+bK9mjdvzvz587PNt2TJEnw+X67P9+3blxkzZmSblpSUhM/ny3Up\nYNiwYYwbNy7btJSUFHw+H8nJydmmT548mfj4+GzTUlNT8fl8uXpqJiQkEBcXl6u2bt26FWs77rjj\njpDYjkB8P+Lj40NiOyAw34+MdXh9OzL4czuy1u3l7cjKn9uRdTle2o6PPoIdO2Dz5sB9P1q2zH6M\nG6yfj4SEhMy2MTo6Gp/PR//+/XN9xlJ+z8/N74W5Zv8aUAE4ijnFn/FebSANuNTm842Q543bmjRp\nktMluJZkY0/ysSbZ2PNqPm3aaH3FFYFdh5uyWbt2rcacRW+k82mvC9VLXyk1CvgUc3teeeBWIB5o\nq7VeppSairktLw5zfX8SkKa1trwtryC99IUQQoj8JCdD3brw1ltwyy1OVxMcgeylXx2YDZwJHAB+\nIr2xT3+/P3ASeB8oDSwC+hZyHUIIIUShvfwyVK0KnTs7XYk7FarB11rflc/7R4EH0l9CCCFEUBw+\nDLNmwf33gzxUNW8ylr4H5OwIIk6RbOxJPtYkG3tey+edd+DgQbj33sCvy2vZZJAG3wMGDhzodAmu\nJdnYk3ysSTb2vJSP1jB1KrRvD+edF/j1eSmbrIo80p4InilTpjhdgmtJNvYkH2uSjT0v5fPdd5CU\nBAsXBmd9XsomKznC9wAZYdCaZGNP8rEm2djzUj5Tp0LNmtCuXXDW56VsspIGXwghhGft22eu3997\nL5Qo4XQ17iYNvhBCCM/KGCCvd29n6/ACafA9IOdwlOIUycae5GNNsrHnhXxOnIAXX4QePcz998Hi\nhWzyIg2+B6SmpjpdgmtJNvYkH2uSjT0v5PPxx5CSAg8EeeQXL2STl0INrRuQAmRoXSGEEEVw9dVw\n7BjkeA4JGu9OAAAgAElEQVRNWAnk0LpCCCGE49atgy++MB32RMHIKX0hhBCeM2UK1KgBN93kdCXe\nIQ2+B+R8HrM4RbKxJ/lYk2zsuTmfv/+GN94w4+aXLBn89bs5GzvS4HtAb7nfxJJkY0/ysSbZ2HNz\nPq+9BidPwj33OLN+N2djRxp8Dxg+fLjTJbiWZGNP8rEm2dhzaz4nT5pb8bp3h2rVnKnBrdnkRzrt\neYDcvWBNsrEn+ViTbOy5NZ8FC2DrVnj/fedqcGs2+ZEjfCGEEJ4xaRI0bw7mTjRRGHKEL4QQwhPW\nr4dlyyAhwelKvEmO8D1gRsZg0SIXycae5GNNsrHnxnwmT4Yzz4TOnZ2tw43ZFIQ0+B6QlGQ7eFJY\nk2zsST7WJBt7bstn/35nb8XLym3ZFJQMrSuEEML1nnsOBg0yY+dXr+50Ne5RmKF15QhfCCGEq2Xc\nitetmzT2xSGd9oQQQrjahx/Cli3w3ntOV+JtcoSfw6pVsHKl01UIIYTIMGECtGwpt+IVlzT4OQwe\nDAMGgMNdG7Lx+XxOl+Bako09yceaZGPPLfmsXm0efztggNOVnOKWbApLGvwcBgwwO9iqVU5Xckq/\nfv2cLsG1JBt7ko81ycaeW/KZMAHOPx86dHC6klPckk1hSS/9HNLSoF49uOgimDvX6WqEECJ8paRA\nrVrwwgvg0TY24KSXfjFERED//vDBB/D7705XI4QQ4WvKFChfHnr1crqS0CANfh5uvx0qV4aJE52u\nRAghwtOhQzB9unkEbrlyTlcTGqTBz0NUFPTpY565vH+/09XA/PnznS7BtSQbe5KPNcnGntP5zJwJ\nhw/DAw84WkaenM6mqKTBt9CnDxw/Dq+84nQlkCBPirAk2diTfKxJNvaczOfkSXPd/uab4eyzHSvD\nklf3nUJ12lNKPQHcCFwI/Ad8DTymtf4lyzzLgZZZPqaBaVrrPhbLdFWnvax694YlS2DzZihVyulq\nhBAiPHzwAdx0E3z3HTRp4nQ17hbITnstgMlAM+AaoCSwRCkVmWUeDUwHqgPRwJnAwEKuxxUGDIDt\n22V0JyGECKbnn4cWLaSx97dCDa2rtW6f9WulVC9gN9AYSMzyVqrWek+xq3PYxRdD27bmoQ233AJK\nOV2REEKEtoyBdj74wOlKQk9xr+FXxBzR/51j+q1KqT1KqXVKqdE5zgB4yiOPwPffw4oVTlcihBCh\nb8IEc+99bKzTlYSeIjf4SikFvAAkaq03ZHnrLeA2oDUwGrgdeKMYNTrq2mvNIDzPP+9cDXFxcc6t\n3OUkG3uSjzXJxp4T+fzxh7mE+vDDUKJE0FdfYF7dd4rztLypQD3giqwTtdavZvlyvVJqF/CZUqqm\n1npLMdbnCKXMtfw774RNm6BOneDX0LZt2+Cv1CMkG3uSjzXJxp4T+UyebO65d3t76tV9p0hH+Eqp\nKUB7oLXWemc+s38LKOACu5nat2+Pz+fL9mrevHmu+x2XLFmS54ML+vbty4wZM7JNS0pKwufzsXfv\n3mzThw0bxrhx47JNS0lJwefzkZycnG365MmT+fnneKpXN7eJAKSmpuLz+UhMTMw2b0JCQp5/+XXr\n1q1Y21GnTh2/bEd8fHy2acHeDn99P7JuR48ePUJiOyAw348ePXqExHZk8Od2ZGTj9e3Iyp/bkTWf\nYGzHgQMwbRrcey/MnOnu70dKSorldmTl7/0qISEhs22Mjo7G5/PRv3//XJ+xUuix9NMb+45AK631\n5gLMfwXwJdBAa/1zHu+79ra8rJ56CsaMMWM7V6nidDVCCBFannkGhgwxz72vUcPparwjYLflKaWm\nArcCtwCHlVLV019l0t+vpZQaopRqpJQ6VynlA14HVuTV2HvJ/febR+ZOnep0JUIIEVqOHjVnUG+/\nXRr7QCrsKf37gArAcmBHltfN6e8fw9yfvxjYCDwDvAd48+HBWVSpYgbimTwZUlODu+6cp37EKZKN\nPcnHmmRjL5j5vP027NgBjz4atFUWi1f3nUI1+FrrCK11iTxes9Pf/1Nr3VprXVVrHaW1rqO1fkJr\n/W9gyg+uRx6Bv/82Y+wH0/jx44O7Qg+RbOxJPtYkG3vByictzZzO79gRLrwwKKssNq/uO4W+hu/3\nAjxyDT9Djx7wzTfw669wWnHucSiE1NRUoqKigrMyj5Fs7Ek+1iQbe8HK5+OPweeDlSvh8ssDvjq/\ncNO+E8ihdcPewIGwdWtwh9t1y47lRpKNPcnHmmRjL1j5jB8PV1zhncYevLvvSINfSP/7nxlud9w4\n04lPCCFE0Xz9tRlGd6Ann7biPdLgF8Fjj8GPP5on6QkhhCia8ePNdfsOHZyuJDxIg18EV10FjRub\nnTUYcg7cIE6RbOxJPtYkG3uBzic5GT78EOLjIcJjLZFX9x2PxewOSpmj/GXLYM2awK8vJiYm8Cvx\nKMnGnuRjTbKxF+h8nn0WzjwTbr01oKsJCK/uO9JLv4hOnjSnoho2DG4HPiGE8LodO6BmTTOCqVy/\nLx7ppR8EJUqYQSLmzjW36AkhhCiYSZOgdGkzbr4IHmnwi6FnT6hWDZ57zulKhBDCGw4cgJdeMo39\n6ac7XU14kQa/GMqUgYceglmzYNeuwK0n5xOYxCmSjT3Jx5pkYy9Q+UydCkeOmMeOe5VX9x1p8Ivp\n/vuhZEmYODFw6xgoF7ksSTb2JB9rko29QOSTmgoTJpjnkpx5pt8XHzRe3XekwS+mihVNoz91Kvzz\nT2DWMWXKlMAsOARINvYkH2uSjb1A5PPqq+Z5JB5tLzN5dd+RBt8PBgwwj3cM1D7g1VtAgkGysSf5\nWJNs7Pk7n2PHzENybrnF9ND3Mq/uO9Lg+0F0NNx9tzlV9W9IPBdQCCH8a/Zs2L4dnnjC6UrClzT4\nfhIfDwcPwrRpTlcihBDucuIEjB0LN94Ides6XU34kgbfT2Ji4I47zOhRR474d9njxo3z7wJDiGRj\nT/KxJtnY82c+770Hv/8Ogwb5bZGO8uq+Iw2+Hz3+OOzeDa+95t/lpqam+neBIUSysSf5WJNs7Pkr\nn7Q0GD0a2rUzzyAJBV7dd2RoXT+75RZYuRJ++83crieEEOHso4+gY0f48kto0cLpakKPDK3roEGD\nICUF3nzT6UqEEMJZWsOoUXDlldLYu4E0+H528cXQqROMGWMesCOEEOFq2TJYvRoGD3a6EgHS4AfE\n4MHmgTrvv++f5e3du9c/CwpBko09yceaZGPPH/mMHg2NGpnr96HEq/uONPgB0KSJ2cFHjTIdVoqr\nd+/exV9IiJJs7Ek+1iQbe8XNZ9Uqc4Q/aBAo5aeiXMKr+440+AEyeDCsWwcLFhR/WcOHDy/+QkKU\nZGNP8rEm2dgrbj4jRkC9eube+1Dj1X1HeukHUMuW5p78b78Nvb9whRDCyjffQPPm8M470K2b09WE\nNuml7xJPPgnffQeLFjldiRBCBE/G0X2XLk5XIrKSBj+ArrkGLr8chg83t6cIIUSo+/Zbc5Dz5JNQ\nooTT1YispMEPIKXMX7qrV8OnnxZ9OTNmzPBfUSFGsrEn+ViTbOwVNZ8RI8x4+V27+rkgF/HqviMN\nfoC1aQNXXFG8o/ykJNvLMmFNsrEn+ViTbOwVJZ+Mg5tQP7r36r4jnfaC4LPP4NprYeFCaN/e6WqE\nECIwbrgBNm+Gn38O7QbfTaTTnsu0aWOGlhw2TK7lCyFC03ffwSefhP7RvZcVqsFXSj2hlFqtlDqo\nlPpLKfWBUqp2jnlKK6VeVErtVUodUkq9r5Sq5t+yvUUpc0p/zRrzAyGEEKFmxAioU0duw3Ozwh7h\ntwAmA82Aa4CSwBKlVGSWeV4AbgA6Ay2BGsDc4pfqbVdfbR4eIT32hRChZs0ac8lSju7drVANvta6\nvdb6Da31Rq31OqAXEAM0BlBKVQB6A/211iu01t8DccAVSqlL/Vu6t2Q9yl+4sHCf9fl8AakpFEg2\n9iQfa5KNvcLkM2IE1K4N3bsHsCAX8eq+U9xr+BUBDfyd/nVj4DTg84wZtNabgBSgeTHX5XlXXVW0\no/x+/foFrCavk2zsST7WJBt7Bc1n7VozhHg4Hd17dd8pci99pZQCPgbKa61bpU/rAbymtY7MMe+3\nwDKt9RN5LCfke+ln9cUX5vT+Rx9BbKzT1QghRPF06GCeDrp+PZx2mtPVhJ9g9dKfCtQDehRgXoU5\nE2Cpffv2+Hy+bK/mzZszf/78bPMtWbIkz9Mpffv2zTUYQlJSEj6fL9ejDIcNG8a4ceOyTUtJScHn\n85GcnJxt+uTJk4mPj882LTU1FZ/PR2JiYrbpCQkJxMXF5aqtW7dumdvRurUZY3/AAG9vRwavfz9k\nO2Q7ZDuKvh2dO/dl4cIZDB9+qrH34nZ45fuRkJCQ2TZGR0fj8/no379/rs9YKdIRvlJqChALtNBa\np2SZfhXwGVBJa30wy/StwASt9cQ8lhVWR/gAy5eb0/vz50PHjk5XI4QQRXP11bB3L/zwA0TITd6O\nCOgRfnpj3xG4Kmtjn24tcAJok2X+2piOfasKu65Q1bq1+UEZMgTS0vKfP+dfqeIUycae5GNNsrGX\nXz7LlplLlE89FX6NvVf3ncLehz8VuBW4BTislKqe/ioDkH5UPwN4XinVWinVGJgJrNRar/Zz7Z42\napQZjeqdd/KfNyEhIfAFeZRkY0/ysSbZ2LPLR2sYPBiaNgWPdlgvFq/uO4U6pa+USiPva/FxWuvZ\n6fOUBp7FXNsvDSwC+mqtd1ssM+xO6Wfo2NF0dNm4EUqWdLoaIYQomAULTKfjxYuhbVunqwlvATul\nr7WO0FqXyOM1O8s8R7XWD2itq2ity2utu1o19uHuqafMuNMzZzpdiRBCFExamrkc2bKleUaI8I4w\nu/LiLvXrm4EqRo6EI0ecrkYIIfI3dy78+CM8/bQZUEx4hzT4DhsxAnbtgpdecroSIYSwd/IkDB0K\n7dqZQcSEt0iD77D/+z+Ii4PRo+HQobznyeveTGFINvYkH2uSjb288nnzTUhONkf34cyr+440+C4w\ndCgcPAgvvJD3+22lV4wlycae5GNNsrGXM59jx8yw4DfeCE2aOFOTW3h13yny0Lp+KyCMe+ln1b8/\nvPYabNkClSs7XY0QQmT30kvQty/89BNcfLHT1YgMwRpaV/jRE0+Y62PjxztdiRBCZJeaak7j9+gh\njb2XSYPvEtWqwcMPw6RJsHOn09UIIcQpEyfCnj3mVmLhXdLgu8ijj0KZMrl/qHI+ZEGcItnYk3ys\nSTb2MvLZuxfGjoX77oNatRwuyiW8uu9Ig+8iFSuaU/vTp8Mvv5yaPl7O81uSbOxJPtYkG3sZ+Ywe\nfWqwHWF4dd+RTnsuc+QI1K4Nl14K779vpqWmphIVFeVsYS4l2diTfKxJNvZSU1PZvTuKOnVMY//k\nk05X5B5u2nek056HZZzSnzsXvvnGTHPLjuVGko09yceaZGMvKiqKJ5+ESpXMXUTiFK/uO9Lgu9Bt\nt5lhd+PjzVOphBAi2H74Ad56C4YNg3LlnK5G+IM0+C5UogSMGweJifDxx05XI4QIR088ARdcAHfd\n5XQlwl+kwXepdu2gTRt47DF45JF4p8txrfh4ycaO5GNNsrG2bBksWhTP6NHy6O68eHXfkQbfpZQy\nR/nJybBjR4zT5bhWTIxkY0fysSbZ5C0tDQYONPl07ux0Ne7k1X1Heum73K23whdfwK+/QtmyTlcj\nhAh1c+ZAt26wfDm0auV0NSI/0ks/hDz9NOzbBxMmOF2JECLUHT1qrt23by+NfSiSBt/lataEPn3M\nGPu7dztdjRAilE2ZAtu2wTPPOF2JCARp8D2gW7dkIiJgxAinK3Gf5ORkp0twNcnHmmST3d69ZgyQ\ne+6BevUkHztezUYafA8YPXogQ4bAtGmwfr3T1bjLwIEDnS7B1SQfa5JNdiNGmHE/Mg4sJB9rXs1G\nOu15QEpKCtWrx3DRRea+2EWLnK7IPVJSUjzbYzYYJB9rks0pycnmsbdjxpgBv0DyseOmbKTTXoiJ\niYmhdGlzXW3xYvj0U6crcg+3/NC5leRjTbI5JT4eYmLggQdOTZN8rHk1G2nwPaRTJ2jdGgYMgOPH\nna5GCBEKPvsMFiww436UKeN0NSKQpMH3EKXM7XmbNpnr+UIIURwnT5oDiCuugC5dnK5GBJo0+B4w\nbty4zP83bAi9e5sHWuzf72BRLpE1G5Gb5GNNsoGZM2HdOnj+eXNAkZXkY82r2UiD7wGpqanZvn76\naTh2DEaOdKggF8mZjchO8rEW7tkcOmSec3/rrXDppbnfD/d87Hg1G+ml71FjxsDQoeY2vdq1na5G\nCOE1gwebI/tNm0yHPeFN0ks/DPTvD2edBY8+6nQlQgiv+f13ePZZ8/tDGvvwIQ2+R5UpY3rVfvwx\nLF3qdDVCCC8ZMACqVzfj5ovwIQ2+B+zduzfP6TffDC1awEMPhe9telbZCEPysRau2SxaBB99BM89\nB1FR1vOFaz4F4dVsCt3gK6VaKKU+UkptV0qlKaV8Od6fmT496+sT/5Ucfnr37p3ndKVg8mRzDW7y\n5CAX5RJW2QhD8rEWjtkcO2YOEK66Kv/b8MIxn4LyajZFOcIvC/wA9AWsevx9ClQHotNfPYpUnQBg\n+PDhlu81aGCepjd8OOzcGbSSXMMuGyH52AnHbCZONNfvJ07MfRteTuGYT0F5NZti9dJXSqUBnbTW\nH2WZNhM4XWt9UwGXIb30i2n/ftNT//rrYfZsp6sRQrjRzp3m90RcHEya5HQ1wl/c0Eu/tVLqL6VU\nslJqqlKqcoDWI4BKlcxtem+8AYmJTlcjhHCjxx4znX3lMdvhKxAN/qfAHcDVwECgFfCJUvmdQBLF\n0bs3NG0K/fqZ4TKFECLD11+bA4IxY8wBgghPfm/wtdZztNYLtNbr00/1dwAuBVrbfa59+/b4fL5s\nr+bNmzN//vxs8y1ZsgSfz5fr83379mXGjBnZpiUlJeHz+XL1qBw2bFiuoRFTUlLw+XwkJydnmz55\n8mTiM54XmS41NRWfz0dijsPphIQE4uLictXWrVu3Ym3H0KFD892OiAiYMgV+/DGF//3PndsRiO/H\njBkzQmI7IDDfj4zavb4dGfy5HVmX4+XtyCqv7Th0KJX27X3UqZNI1rLz246s9blhO9z0/eiSo8dj\nsLYjISEhs22Mjo7G5/PRv3//XJ+xpLUu8gtIA3wFmG83cLfFe40AvXbtWi3y1qdPnwLPe+edWleq\npPWePQEsyEUKk004knyshUs206ZpDVp//XXhPhcu+RSFm7JZu3atxnSgb6TzaYv93mkvj3nOBrYB\nHbXWC/J4Xzrt+dHu3aZjTrdu8kQ9IcLd7t1w4YXg88GsWU5XIwIhoJ32lFJllVINlFIN0yfVSv/6\nnPT3xiulmimlzlVKtQHmA78Aiwu7LlF41arBU0/BK6/Ad985XY0QwkkDB5p/n3nG2TqEOxTlGn4T\n4HtgLeY0wnNAEjACOAnUBz4ENgGvAN8BLbXWYToWXPDdf7+5P//ee+HECaerEUI4YflyeP11GD8e\nqlZ1uhrhBqcV9gNa6xXY/6FwXdHLEf5w2mnmdP5ll5mOfA8/7HRFQohgOnbMDMh1+eXmDh4hQMbS\n94S8epXm59JLzZH+k0/Cn38GoCiXKEo24UTysRbK2Tz3HPzyC7z0krmDpyhCOZ/i8mo20uB7QL9+\n/Yr0udGjoVw5ePBBPxfkIkXNJlxIPtZCNZvNm2HkSPMI7fr1i76cUM3HH7yaTbF66fulAOmlH1Dv\nvgvdu5unY8XGOl2NECKQtIYOHWDdOtiwwfzBL0KbG4bWFS5x883Qrp0Zge/wYaerEUIE0gcfwCef\nmLHypbEXOUmDH+KUgqlTzf24Hn3AkxCiAA4dMpfvYmOhY0enqxFuJA2+B+QcHrKwatWCoUNhwgT4\n8Uc/FeUSxc0m1Ek+1kItmyefhL//Nkf3/nhySajl409ezUYafA9ISEgo9jIeeQTq1DH35qel+aEo\nl/BHNqFM8rEWStl8841p6J96Cs47zz/LDKV8/M2r2UinvTCSmAgtWsDkyeaavhDC+44dg0aNIDIS\nVq0y43CI8CGd9kSerrzS3Jv/+OOwdavT1Qgh/GHMGNi0CWbMkMZe2JMGP8yMHQuVK5tT+w6f3BFC\nFNP69TBqFDz2WPHuuRfhQRr8MFOhArz8MixZArNnO12NEKKoTp6Eu+4ynXKHDHG6GuEF0uB7QFxc\nnF+X17493HabGYlr1y6/Ljro/J1NqJF8rHk9mxdfNJ31Xn0VypTx//K9nk8geTUbafA9oG3btn5f\n5gsvmOt9Xu+8F4hsQonkY83L2WzbBoMGmQfkXHllYNbh5XwCzavZSC/9MDZnDnTrBu+/D507O12N\nEKIgtDZn6X7+2VzDr1DB6YqEk6SXviiQrl2hUyfo29cM2CGEcL+ZM2HRItMXRxp7URjS4Icxpcx1\nwCNHzPV8IYS7paTAww9Dr15www1OVyO8Rhp8D0hMTAzYsmvUMNfzZ8+GDz8M2GoCJpDZhALJx5rX\nstEa7rwTTj/d/MwGmtfyCSavZiMNvgeMHz8+oMvv2RN8PrjnHtizJ6Cr8rtAZ+N1ko81r2Xz8svw\n2WdmgJ3TTw/8+ryWTzB5NRvptOcBqampREVFBXQdu3bBxRdDq1amE58/Hr4RDMHIxsskH2teymbz\nZjOwzm23mYY/GLyUT7C5KRvptBdigrFjRUebXyTz5sHbbwd8dX7jlh86t5J8rHklm7Q06N0bqlaF\nZ54J3nq9ko8TvJqNNPgiU5cu0KOHuTd/+3anqxFCgHnY1YoVpnd++fJOVyO8TBp8kc2UKeapW3fd\nJWPtC+G0X36BJ56ABx6A1q2drkZ4nTT4HhAfHx+0dVWubIbqXLQIXnklaKstsmBm40WSjzW3Z3P8\nuLlmf9ZZ5ol4web2fJzk1WykwfeAmJiYoK6vfXu4+24YMAB+/z2oqy60YGfjNZKPNbdnM3IkJCXB\nW29B2bLBX7/b83GSV7ORXvoiT4cOQcOGpqPQV19ByZJOVyRE+EhMNHfMjBwJgwc7XY1wM+mlL4qt\nfHnTW3/NGvNLRwgRHAcOmFP5l18Ojz/udDUilEiDLyw1awbDh8OoUfDll05XI0R46NsX9u+HN96A\nEiWcrkaEEmnwPSA5OdmxdT/xhHn85m23mV9CbuNkNl4g+VhzYzZvv22u2U+dCued52wtbszHLbya\njTT4HjBw4EDH1l2iBLz5prmmf9997rtVz8lsvEDysea2bLZuhfvvN2Nh3Hqr09W4Lx838Wo2hW7w\nlVItlFIfKaW2K6XSlFK+POYZqZTaoZRKVUotVUpd4J9yw9OUKVMcXX9MDEybBnPmwKxZjpaSi9PZ\nuJ3kY81N2Zw4AXfcARUrmqN7N3BTPm7j1WyKcoRfFvgB6AvkOt5TSj0G9APuBS4FDgOLlVKlilFn\nWHPDLSA332weyfnAA/Drr05Xc4obsnEzyceam7IZORJWrjRn0ypWdLoaw035uI1Xsyl0g6+1XqS1\nHqq1ng/k9YiVh4CntNYfa61/Bu4AagCdileqcNqkSWbM/R494OhRp6sRIjR89hk8/TSMGAEtWjhd\njQhlfr2Gr5SqCUQDn2dM01ofBL4FmvtzXSL4ypeHd9+Fdevg0UedrkYI79u1y3SIbdPGdJAVIpD8\n3WkvGnOa/68c0/9Kf08Uwbhx45wuIVPjxjBhghlz/733nK7GXdm4keRjzelsTp40nfOUMqfy3XYL\nntP5uJlXswlWL31FHtf7RcGkpqY6XUI2998P3brBnXfCb785W4vbsnEbycea09mMGgVffGFuw6te\n3dFS8uR0Pm7m2Wy01kV+AWmAL8vXNdOn1c8x33JggsUyGgG6evXqOjY2Ntvrsssu0x988IHOavHi\nxTo2Nlbn1KdPH/3qq69mm7Z27VodGxur9+zZk2360KFD9dixY7NN27Ztm46NjdUbN27MNn3SpEn6\n0UcfzTbt8OHDOjY2Vn/11VfZpr/99tu6V69euWq7+eabQ247DhzQ+v/+T+tatRbrG27w7nZk8Pr3\nQ7bDW9vRsWMfrdSretgwb29HqHw/vLIdb7/9dmbbmNFmtmzZUmMOqBvpfNrsYo2lr5RKAzpprT/K\nMm0H8IzWekL61xUwp/Tv0FrnOgksY+l71w8/wGWXQVwcvPSS09UI4Q27d5vnVFx4ISxd6r5T+cJb\nAjqWvlKqrFKqgVKqYfqkWulfn5P+9QvAEKVUrFLqEmA28CfwYWHXJdytYUPTc//llyEhwelqhHC/\nEyfgllvMv2+9JY29CK6iXMNvAnwPrMWcRngOSAJGAGitxwOTgWmY3vmRwPVa62P+KDgc7d271+kS\nLN19t+l4dM894MRok27Oxg0kH2tOZDN4sLlu/+67cOaZQV99oci+Y82r2RTlPvwVWusIrXWJHK/e\nWeYZrrWuobWO0lq301o73LXL23r37p3/TA5Ryhzhx8RAp05w8GBw1+/mbNxA8rEW7GzmzoXx42Hc\nOLjqqqCuukhk37Hm1WxkLH0PGD58uNMl2CpXDj74AHbuNMODpqUFb91uz8Zpko+1YGazcaMZqbJr\nV3jkkaCttlhk37Hm1WyK1WnPLwVIp72QsWABxMbCU0/BkCFOVyOEOxw6BJdeas6GffutGcBKCH8J\naKc9Iax06GCGBx06FBYudLoaIZyntTmy377dnAWTxl44SRp84VdDhpij/FtvdddDdoRwwrhxMG8e\nzJ4Ndeo4XY0Id9Lge8CMGTOcLqHAIiLML7fq1U0nvn//Dez6vJSNEyQfa4HO5uOPYdAg8+rkwUeH\nyb5jzavZSIPvAUlJtpdlXOf002H+fEhJgZ49A9uJz2vZBJvkYy2Q2fz8s7nf3uczfVq8SPYda17N\nRjrtiYD58EO48UZzhPP0005XI0Rw7N1rOumVL2+ecV+unNMViVAmnfaEK3TsCGPHmoeEvPGG09UI\nEVxdOPcAABQrSURBVHjHjkGXLuZS1kcfSWMv3OU0pwsQoS0+3ozAd9ddUKsWXHGF0xUJERhaQ79+\n8PXXZjS9c891uiIhspMjfBFQGSPxXXaZ6bi0ZYvTFQkRGJMnwyuvwPTp8oetcCdp8D3A5/M5XUKx\nlCplhhU9/XRzy54/h9/1ejaBJvlY82c2H34I/fubUfR69fLbYh0l+441r2YjDb4H9OvXz+kSiq1K\nFTMS359/ws03w/Hj/lluKGQTSJKPNX9ls3o19OhhOqiOH++XRbqC7DvWvJqN9NIXQfXZZ3D99XD7\n7TBjhjnlL4RXbd5sLlf93/+ZfTsy0umKRLiRXvrCta65Bl57DWbOhGHDnK5GiKLbt8/88Vqxojml\nL429cDvppS+C7vbbYccOePxxOPtsuOcepysSonCOHDG3ne7fD6tWmUtWQridHOF7wPz5850uwe8G\nDjS3MN1/vxmCtKhCMRt/knysFTWbkyfNY6CTksy+e/75fi7MJWTfsebVbKTB94CEhASnS/A7peCF\nF8ytet26wTffFG05oZiNP0k+1oqSjdbQt6+56yQhAZo1C0BhLiH7jjWvZiOd9oSj/vsP2raFjRvh\nyy+hXj2nKxLC2qBBMGaM6YcSF+d0NUJIpz3hIZGRZgjSs84yHfp+/93pioTI2zPPmMb++eelsRfe\nJA2+cFylSrBkiRl3/JprzL36QrjJq6+afieDB5sBdoTwImnwhStUr27uY05Lg2uvhd27na5ICGPu\nXLj3XtPB1KuPuhUCpMH3hLgwOX8YEwOffw7//APt2pl/8xMu2RSV5GOtINksWGBG0evWDaZMCa+B\nomTfsebVbKTB94C2bds6XULQXHABLF0KKSnQvj0cOmQ/fzhlUxSSj7X8svnkE+jcGTp0gNdfh4gw\n+20p+441r2YjvfSFK333nbmef8kl8OmnUL680xWJcLJ4sRlY57rrYM4c8wAoIdxIeukLz2va1HTk\nW7fO/NL15xP2hLCzdKlp7K+9Vhp7EVqkwReu1ayZ+eW7fr25pn/ggNMViVD3+efg80GbNvD++9LY\ni9AiDb4HJCYmOl2CYy691PTeT07Ou9EP52wKQvKxljObpUshNhZatzY980uXdqYut5B9x5pXs5EG\n3wPGh9JDtougSRPT6G/aZEbl27//1Hvhnk1+JB9rWbOZP990zrvqKpg3D8qUcbAwl5B9x5pXs5FO\nex6QmppKVFSU02U4LinJXFc96yxzfT86WrLJj+RjLSObt96Cnj3hppvgzTflNH4G2XesuSkb6bQX\nYtyyYzmtUSMz3v6+fXDllbBli2STH8nHWlRUFNOnm8c133GHeRiONPanyL5jzavZ+L3BV0oNU0ql\n5Xht8Pd6RHi66CJYudL8/8orTYc+IYriuefMCHr9+pmhc0uUcLoiIQIrUEf4PwPVgej015UBWo8I\nQ+edB4mJUKUKtGwJq1c7XZHwkrQ0eOQRePRR8/S7iRPDb1AdEZ4CtZuf0Frv0VrvTn/9HaD1hIX4\n+HinS3Cd6GhYsQLKlInn6qth4UKnK3In2XeyO3oUbrkFJkyANm3iGTUqvIbLLQzZd6x5NZtANfj/\np5TarpT6XSn1plLqnACtJyzExMQ4XYIrVawIAwbEcO215t7pl15yuiL3kX3nlIxnNMyfb+6x79hR\nsrEj+441r2bj9176Sql2QDlgE3AmMByoAVystT6cx/zSS18Uy8mT5hTtxInmNO24cXKKVmT3xx9w\n/fWwYwd89JHp/yFEKChML/3T/L1yrfXiLF/+rJRaDWwDbgZm+nt9QpQoAS+8ADVrmmeVb9tmHnYS\nGel0ZcINvv0WOnUyPfBXroS6dZ2uSAhnBPw4SGt9APgFuMBuvvbt2+Pz+bK9mjdvzvz587PNt2TJ\nEnw+X67P9+3blxkzZmSblpSUhM/nY+/evdmmDxs2jHHjxmWblpKSgs/nIzk5Odv0yZMn57pek5qa\nis/nyzXaUkJCQp6PTezWrZtsRxC2o3HjRObNM480veoqmDrVm9sRKt8PN2xH9+7jaNXK/DG4ejWU\nLevN7QiV74dsR/G2IyEhIbNtjI6Oxufz0b9//1yfsRLwgXeUUuUwR/jDtNZT8nhfTunnIzk5mQsv\nvNDpMlwpr2zWrDFHdGlpZtS0yy5zqDgXCNd9Jy0NhgyBMWPMPfbTp+ceKjdcsykoyceam7JxdOAd\npdQzSqmWSqlzlVKXAx8AJ4AEf68rXAwcONDpElwrr2yaNDGNfs2a0KoVzAzjC0nhuO8cOmRGzRs7\nFp55BmbNyntc/HDMpjAkH2tezcbv1/CBs4G3gTOAPUAicJnWel8A1hUWpkzJdWJEpLPKJjoali2D\nBx6A3r3h++/NQCslSwa5QIeF276zfj107nyqc16HDtbzhls2hSX5WPNqNoHotNfD38sMd169BSQY\n7LIpXRqmTYP//Q8efBDWroV33oFzwugm0XDad956C+65B2rVMmd4ate2nz+csikKyceaV7ORm5dE\nSFMK7r/fDNLzxx/QsCF88onTVQl/OnoU+vSB224zR/fffJN/Yy9EOJIGX4SFyy83p/UvvxxuuAEe\newyOH3e6KlFcv/1m7qmfMcOczXn9dShb1umqhHAnafA9IOctIOKUwmRzxhnmuu6zz8Lzz0Pr1uaJ\ne6EsVPcdrU0j37Ah7N8PX39tTucXZpjcUM3GXyQfa17NRhp8D0hNTXW6BNcqbDZKmVH5vvrKdOyq\nX980HAG+O9Uxobjv7NtnTt3fdRd07w4//ADmrqTCCcVs/EnysebVbAJ+H36+Bch9+MIhhw6Zkflm\nzDC9uV95xfTuF+61ZAnExcGRI+b7ddNNTlckhLMcvQ9fCK8oX948B/2jj8wobBdfbB6qEqpH+162\nf7+5vbJdO7joIli3Thp7IQpLGnwR9mJj4eefzSA9XbuaUfr++MPpqkSGuXPN+Pfz5pk/0BYvhho1\nnK5KCO+RBt8Dco7lLE7xVzZVq5qj+7lzzT3cdeuaB/KcPOmXxTvGy/vOjh3mWn2XLmZ45A0b4M47\n/ff8ei9nEwySjzWvZiMNvgf07t3b6RJcy5/ZKGVOE2/YAL16wYAB0KyZ+QPAq7y47xw7BuPHQ506\nkJgIc+bABx/4/6jei9kEk+RjzavZSIPvAcOHD3e6BNcKRDannw5TpphbvY4fh6ZNzR8AO3b4fVUB\n57V9Z9EiuOQSGDTIXLPftMlcZvHXUX1WXssm2CQfa17NRhp8D5C7F6wFMpvLLjPD8b70EixcaEZv\nGzUK/vsvYKv0O6/sOz//bPpSXH+9OZL/4QeYOBEqVgzcOr2SjVMkH2v/3969x3ZZ3XEcf39BtC3O\n1gg0lRkGQWTIUJmXKBcTa6aLG0RNwClImKBVqOguOFMvE3VanW4REaJWmMJcdXEG3YjEOSTUS4Fq\nh1BQEAQFEXFRwmWjcPbH+QEV1vKr66/nPDyfV/JL09On/X355uH3fZ5zeU5Sc6OCL9KCo46CsjL4\n4AP/9a67fFfzE0/oSX2t5ZzjpptuoulS4I8+8r0nAwb4oZTqar/pUf/+4eIUOVKp4ItkoajIP6Fv\n+XIYNMg/1a1vX3jmmeRP7GsvS5cuZdq0adTV1bFpk58j0acPzJsHU6dCQwOMGJGb7nsRUcFPhKqq\nqtAhRKu9c3PyyfDss1Bf7+9Kr77ajzk//bSfbBabmM6d6dOn09jYyOjR0+nZ0z/w6LbbYM0amDAB\njj66feOJKTcxUn6al9TctPn2uNL26urquOaaa0KHEaVQuRkwwM8cX7wYpkyBMWP8RLObb4bx4+G4\n43L7/u999h43zrvxkPYlG5ew7T/bDjT8FcZ9PA6AoT2G0tE6fu34nkU9qRqeuw+vO+64gxkzZpCX\nV8iWLT6ulStf5vjje9Ohw1fs3l3GscdOydn7t0T/r1qm/DQvqbnRo3VF2sDy5b7Lf84cKCjw68Wv\nuy5327TWrK9h8MzB//ff6d+tP8uuX9YGER2qsRFeeGE3FRUzWL36XmDz/p8VFxdTUVFBWVkZnTp1\nysn7i6SBHq0r0s5OPRVmzvS77117Lcya5Sf3lZbC88/H2d2fK+vXwz33QK9eMHJkJ4qLyykpKfza\nMYWFhZSXl6vYi7QjFXyRNtS9u39ozCefwOzZvtCPGOHbJ0yAmhrYuzd0lG1v2zZ/kXPBBdCjB9x3\nH1x4oV/WuGgRdO68l/z8fEpKSsjPz2fvkZgEkcip4IvkQF4eXHWV34Z32TK/9GzuXBg8GHr2hFtu\ngTffTPYM/y++8JMVL7sMiov9LnbgC/+nn8JTT8G+UbqBAwdSWVnJ6tWrqays1PCdSAAq+AkwbNiw\n0CFEKwm56d8fHnzQrzl//XX/cJmqKjjvPL8d7+jRfv351q05ePM/tvzj3Xuyf5jAnj1QV+fnKpSW\nQrdufrLipk1w552wbp1fQz9mjN+JsKnq6mrKy8spKCigvLyc6urq1v9b2lgSzp2QlJ/mJTU3mqWf\nABMnTgwdQrSSlJsOHWDoUP+aNg3efhteftk/xW/2bH9Mv34wZIh/DRrku8f/17r09V+uz+5Nz275\nx6u2rmr2Z9u3+yfeLV4MCxfCggV+m9r8fL+z4KOPwvDhUFKSXSixSdK5E4Ly07yk5kaz9EUisGGD\nv/tfuNAPA6xc6duLivwSwNNO81979/ZDAlNXTeahNx9sk/feMdmxZg28/75/rVzpx95XrPDzDY45\nxu8nUFrqX2ef7dtEJLzWzNLXHb5IBE46CUaN8i+ALVugttY/4Ke+HubP970C++a62ZUN0EZL/jp3\nhn3X/YWFfinhoEEwaRKceaZfgaDJ9CLJp4IvEqGuXeGSS/xrn127/DyAtWvhhqWfs7axbd6rqsoX\n+T59oEsXPdpW5EilSXsJ8OKLL4YOIVppyk1enl/bf/HFMOTULG/vGw5/yNix/o6+a9d0Ffs0nTvf\nhPLTvKTmRgU/ASorK0OHEK205qbvCX2zO7Amt3EkWVrPnWwpP81Lam5U8BOga9euoUOIVlpzc1b3\ns7I7sCC3cSRZWs+dbCk/zUtqblTwRRIo/6j8Nvk7/btp43mRtFDBFxERSQEVfBERkRSIYVleHkBD\nQxbTiVOqtraWuroWn6eQWmnNzapNq2BjFgd+QovH7dy1M5X5g/SeO9lSfpoXU26a1M68wx0bw5P2\nrgTmBA1CREQk2a5yzrW4e0YMBf8E4CJgHbAraDAiIiLJkgd8B3jFOdfiFlzBC76IiIjknibtiYiI\npIAKvoiISAqo4IuIiKSACr6IiEgKqOCLiIikQJQF38yONrN3zWyvmQ0IHU8MzKyHmT1pZh+a2Q4z\n+8DMfm1mnULHFoqZTTCztWa208zeMrMsd5Q5cpnZrWZWa2ZfmdlmM/uLmWW5l276ZPK118weDh1L\nDMzsRDN7xsw+z3zO1JvZwNBxxcDMOpjZ3U0+g1eb2W2h42qNKAs+8ADwMaA1gwf0BQwYD/QDbgbK\ngHtDBhWKmY0EHgLuBM4A6oFXzKxL0MDCGwJMBc4BLgQ6AfPNrG122zmCZC4Qx+PPndQzsyL8hsr/\nxj8b5bvAz4F/hYwrIr8CrgNuwH8eTwYmm9nEoFG1QnTr8M3sh8BvgcuBFcDpzrl/ho0qTmb2C6DM\nOdc7dCztzczeAt52zk3KfG/ABuAR59wDQYOLSOYC6DNgqHNuUeh4YmFmxwJLgeuB24F3nHM/CxtV\nWGZ2P3Cuc+780LHEyMxeAj51zo1v0vZnYIdz7upwkWUvqjt8MysGHgdGATsDh5MERcAXoYNob5lh\njO8Df9/X5vyV66vAuaHiilQRvqcsdefJYUwDXnLOvRY6kIj8GFhiZs9lhoPqzGxc6KAi8gZQamYn\nA5jZacAg4G9Bo2qFGDbPaWom8Jhz7h0z6xE6mJiZWW9gIpDGu5IuQEdg80Htm4FT2j+cOGV6PX4P\nLHLOrQgdTyzM7ArgdODM0LFEphe+x+Mh/FDhOcAjZrbLOTc7aGRxuB84DlhpZnvwN8wVzrk/hQ0r\nezm/wzez+zKTYpp77TGzPmZ2I/AtoHLfr+Y6thhkm5+Dfqc7MA+ods49FSbyKBma99HUY/j5HleE\nDiQWZvZt/EXQKOfc7tDxRKYDsNQ5d7tzrt459zjwBP4iQGAkcCX+/9MZwBjgl2Y2OmhUrZDzMfzM\n5jgnHOawtcBzwI8Oau8INAJznHNjcxBecFnm50PnXGPm+BOBfwBvHKk5OZxMl/4O4HLn3Nwm7bOA\nQufcpaFii4WZPYrvoh3inFsfOp5YmNlw4AVgDwduKjriLxT3AMe42CY2tRMzWwfMd85d26StDH8X\ne1KwwCJhZuuB3zjnZjRpq8DvUtcvXGTZy3mXfmb3nhZ38AEws3KgoknTicArwAigNjfRhZdtfmD/\nnf1rwGLgp7mMK2bOud1mthQoBebC/u7rUuCRkLHFIFPshwPnq9gf4lXgewe1zQIagPvTWuwzajh0\nSOwU4KMAscSogEN7EPcS2Vy4lkQzhu+c+7jp92a2HX8F/qFzbmOYqOJhZiXAAvw2wpOBbr7GgXPu\n4LHsNHgY+EOm8NfilykW4D+8U8vMHgN+AgwDtmcmwgJ86ZxL/fbTzrnt+NU/+2U+a7Y65xrCRBWN\n3wE1ZnYrvsf1HGAcfumiwEtAhZltAJYDA/GfO08GjaoVoin4zUjz1fbBfoCfVNMLv/wMDoxZdwwV\nVCjOuecyS86mAMXAu8BFzrktYSMLrgx/Tiw4qH0s8HS7R5MM+pwBnHNLzOxS/OS02/FDrZOSNCkt\nxyYCd+NXeHQDNgLTM22JEN06fBEREWl7iRl7EBERkW9OBV9ERCQFVPBFRERSQAVfREQkBVTwRURE\nUkAFX0REJAVU8EVERFJABV9ERCQFVPBFRERSQAVfREQkBVTwRUREUuC/7c+eS6Ik5SIAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112987f60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_mins(f,min_df,xmin=-5,xmax=5,ticks=1000,title='',xguess=3,legend_pos='upper left'):\n",
    "    x = np.linspace(xmin, xmax, ticks)\n",
    "    y = f(x)\n",
    "    \n",
    "    plt.plot(x, y ,color='b')\n",
    "    \n",
    "    plt.plot(min_df, f(min_df),'g',linewidth=13,marker=\"+\",label='solutions')\n",
    "    \n",
    "    plt.plot(xguess, f(xguess),'black',linewidth=13,marker=\"*\",label='initial guess')\n",
    "    \n",
    "    plt.axis([xmin-1, xmax+1, y.min()-1, y.max()+1])\n",
    "    \n",
    "    plt.title(title)\n",
    "    \n",
    "    plt.grid(True)\n",
    "    plt.legend(loc=legend_pos)\n",
    "    \n",
    "    plt.show()\n",
    "    \n",
    "    \n",
    "plot_mins(f=parab,min_df=min_df,xmin=-3,xmax=8,title='Parabola')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Non-Convex example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Nelder-Mead</th>\n",
       "      <td>[4.5]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Powell</th>\n",
       "      <td>7.087928961545842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CG</th>\n",
       "      <td>[4.5]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BFGS</th>\n",
       "      <td>[4.5]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>L-BFGS-B</th>\n",
       "      <td>[4.5]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TNC</th>\n",
       "      <td>[4.5]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>COBYLA</th>\n",
       "      <td>4.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[4.5]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           min\n",
       "Method                        \n",
       "Nelder-Mead              [4.5]\n",
       "Powell       7.087928961545842\n",
       "CG                       [4.5]\n",
       "BFGS                     [4.5]\n",
       "L-BFGS-B                 [4.5]\n",
       "TNC                      [4.5]\n",
       "COBYLA                     4.5\n",
       "SLSQP                    [4.5]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def non_convex_1(X):\n",
    "    if type(X) is float:\n",
    "        if X <=0 or X >=4:\n",
    "            return 5.5\n",
    "        else: \n",
    "            return (X - 2)**2 + 1.5 \n",
    "    else:\n",
    "        Y = (X - 2)**2 + 1.5 \n",
    "        Y[np.logical_or(X <= 0 , X >=4)] = 5.5\n",
    "        return Y\n",
    "\n",
    "initial_guess = 4.5\n",
    "mins = [spo.minimize(non_convex_1,initial_guess,method=om,) for om in opt_methods_no_Jacobian]\n",
    "\n",
    "min_df = pd.DataFrame({'Method' : opt_methods_no_Jacobian,\n",
    " 'min' : [m.x for m in mins]\n",
    "  }).set_index('Method')\n",
    "min_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfwAAAFyCAYAAAAQ6Gi7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xl8E3X+x/HXp5wth6ioiHJ5gKAcAsoPVLxRWTYqyiLq\nrhYVDxBFBXV1BdZVgdXVFbx2RfFYq6LIegMrCnIo0sohh6IgVVQQEETLzff3xzfFJE2mTZvMTDKf\n5+ORB3SSzHzy7jSfzMx3JmKMQSmllFLZLcfrApRSSimVftrwlVJKqQDQhq+UUkoFgDZ8pZRSKgC0\n4SullFIBoA1fKaWUCgBt+EoppVQAaMNXSimlAkAbvlJKKRUA2vCV8jkR2SMid3ldh1dEpFk4gz9V\n8vkVzk9EvhaRpyqxjCrVqJQbtOHHISKXhf94S0Tk4Dj3fyAii7yoLaKGw0TkCRH5SkS2ishmEZkl\nIoNFpLaXtSl/Cze1PSLyzzj3nRy+r3cF5nOtiLwsIqvDz0m6USahKtcAN5HPF5GuIjJcROrHeeye\nKi5LpZiInBhev3aLyH4VeHzpOhx72y0ix7tRs19V97oAn6sF3AbcEDPd0zcEEekJTAS2Ac8CnwE1\ngROBMUAb4BrPClR+V9oArxKRUcaY7+PcXxHDgLrAPKBRCuuLLsaY1SKSC+ys5CxygV0RP3cD7gKe\nBn6OeWwrbNNXPiAiAjwM/ALUSfLpDwHzY6Z9mYq6MpU2fGcLsG+K9xljfvC6GAARaQ68CKwCTjPG\nrIu4+zER+QvwOw9KU5llCba53QrcGHOfVHAe3Y0x3wCIyJYU1laGMWZHCp+b8PUZYyr7oUKlx9XA\nocCTlN3wKs8sY8yk1JeUuXSXfmIGuBf7oei28h4sItVE5C8i8qWIbBORVSLyNxGpGfO4r0XkdRE5\nQUQ+Du+O/0pE/ljBum7FftK9IqbZ26KNWWmMGZuOukSkc3jX2KVxXv/Z4fvOiZjWWESeEpEfwsv+\nTET6xzzvmfCyWsVMnyoiG0TEcctRrBvD894aXtbjItIg4jGnhnfnDY957iXhmq+OmJYvIu+JyNpw\nzUtEpMzekoi8ThaRT8KHfxaJyMnh+3uHf94qIvNFpEPM8yeIyBYRaSEiU0TkFxFZE/7AVq6KZFuO\nr7F7h64qL+NESpt9ukmc4+MR+TUWkcnh/68Tkb+Htwojn7/3GH54HRgTvqv00MZuEWkavj/qGL6I\n7Csi94d/l1vEHjp7W0TaVeH1tBORGeF15hsRuSO83u0prSO27pjnlxlnICL7iMhDIlIcXh9WiMiw\nOFlcFF4ffw6/lkUiMjji/upiD3d8EV5314vIhyJyemVfb2WF/4bvBv4CbK7kPOqKSLWUFpbJjDF6\ni7kBlwG7gY7YT5a/Ao0i7n8fWBTznAnYXYEvYnenPx3++dWYx60ClgHfYVfma4FPsLscW1egtm+A\nFUm8lpTWBawA3oiznKeA9UD18M8Hhmv9GvgzMAB4LbzswRHP2wcoBj4CJDzt6nD+/Srw+v4NbAce\nA67CfkjbEp5ftYjHjQ0/7tjwzweH6303Zn4fA+OBwcB1wDvhmq9NkNe32DekweHXuxm4OHz/LcBQ\n4Cfg85jnPx1erz4P/46uBf4bXtaImMfuAe6K+LlC2Tpktgp4HWgG7AAeirjv5PB8eif5N7MFeCpN\nf4/NwjX9KU5+i8PrwADg5fB6c3Wi/IC2wH/Cj7s+/Lu6GMiNyOapiOd2Ar4A7gGuBO4Ir68biX5P\nKFNjgtfSGNgArAvPawh2b8un4ZqaJvq9x/z+ImvMBRYCPwJ/xf4dPB2e3z8iHndmeJ5TsO8F1wD/\nBAoiHnNP+HmPAf2xe3+eB4aW87oE2L+Ct+oV/L0/AiwKz3t4uK79KvC80nV4c/jfncB0oFM61s9M\nunlegB9vRDf8FuE3xQcj7o9q+EC78Ir1eMx8xoTnc3LEtFXhad0ipjUEtgJjyqmrXng5kyr4OlJe\nV/gNYRvQIGJajfAb4L8ipj2JbYYNYpb9QvixtSKmlb4R3Q40xx5XfaUCr+/E8PP6xkwvnd9FEdNy\nsW/ci7DjHd7ANuJDYp5bK85y3iHmQ1ZEXl3iLPcX4NCI6VeFH9s9YlrpG/KDMfN9I5z5fhHTYht+\nhbNNkNsq4PXw/8djG+dB4Z8zqeHvBv4c89hCYF7MtNj8biamucZkE9lMa8R5TNPw7+gOpxoTvJaH\nsR+i20VMa4D98FnZhn9n+G/msJjH3Yt97zok/PODwMZy6vu0dN2o5O+ovFvU34HD/NphG/Xp4Z+T\nafhdsR/+Lgd6YcearAuv5+3TsY5myk136ZfDGLMKeA4YICIHJXhYT+whgAdjpj+A/XQae0x9qTFm\nTsQy1mO39A4rp5zSUcUVPV6ajrpewjbM8yOmnYXdUn8pYlpvbPOqJiL7l96AqeHHdoxYzjTgCewf\n9STsm+m1FXh9FwKbgPdilvEptumeGrGMrdgPcq2BmdhsbjTGrImcoTFme+n/RaR+eH4zgcNEpF7M\n8pcaYz6O+Ln0/+8ZY76NmS7E//0+EvPzOGy+ZyR+2RXPtgL+hv3AVu5hK596IubnDyn/76jCTMQx\nfRHJETtKvAT7d5FMzqXOAuYaY/ae5WOM2YTd61BZF2Jf9+aY9eE97CHJ7uHHbQLqishZDvPaBBwt\nIkckWcMP2HW2vNuZ2L0R5XkYeMsY816SdWCMmWuM+YMxZoIx5k1jzBjshwCA+5KdXzbRQXsV8zfg\nj9g3xSFx7i/9dBs1AtQYs1ZENoXvj1QcZx4/AfuCfWMBDoi5fwO/jSiObTyJpLSu8HMXicjnQF/s\nVhbh/6/H7vlARA7AbrUMwO6ej2Wwu6Uj3QKcC7QHLjbG/FjeiwOODC+nzFiGeMswxswVkceBgcAU\nY8wzsU8SkROAkcD/AXkx89uH6A9bUXkZY34OHzKNbPbw2/HHfWOm7wFWxkz7AvvhIPZ3U1pfZbJN\nyBizSkRKP9COqujzqiLB+r3RJD9gbpsxZkPMtKj1tarCx8BvxH4AbQGUHg822HU+Wc2AOXGmV2X0\n+JHYQxXx/mYi14dHgT7A2yLyHfYD4svGmCkRj78LmAx8ISKfYfduPW+MWexUQPiD8vQqvIa9RKQv\n9u/v6FTMD8AY85WI/Bc4X0TEhHcFBI02/AoIvyk+j31THB3nIaUDYyq6Eu1OML10Pk2wu+1MeJoB\nTjXGzAz/obat4HJSXVepl4Dbw1s7vwC/x74plJ7OVLrn6HmgTFMNi72OQUd+e2NqS/TegkRygLXY\nY7DxRl5HvQGKHah4MjaPw0SktjFmW8T9hwH/wx6bH4I9Tr4DuyfkRsoOck2UV0VzrMxjKpNtee7B\nfqC9FTuOIN3irt/YPSnJSJRzKt2BPS4+HrvrfCP2g9o/8W7Qc+wgtBxgGjCa+OvPFwDGmB/Dg0fP\nAs4J3/JF5BljTH74MR+KyOHYD989sOMWbhKRq40xCa+zkOBDXCLlfbgbgz3teJeIlH7wLf0Q11RE\napmyp5JWxDfYvWd1sO9bgaMNv+L+BlyKfVOM9TX2j+5I7K4+AETkQOzW2Ookl1W6eyxS6W6wN7Ej\nq7vE7E6OJ9V1lXoRuyVwAXbruh7RDfpH7JZwNWNMuZ/6RSQPu7dgCXbr51YRec0YU1jOU78CTgfm\nRO6Kd/BX7C79W7BvKqOIPiXt99g3hN9H7upP4wjlHOzu58itu5bhfxP9bpLKtiKMMSvDH2ivxp5T\nn25O67cbktm6uwCYboy5KnJieAR5RfZCxVoNxNtdfmScaT9h/04jl1sDO+A00ldAXWPM++Ut3Biz\nC3grfENEHsNuyNxtjFkZfswm7IfJZ8J/mx8CI7ADcxMp/RBXbgmU/+GuCfZD/CVx7ivCni5dmcMp\nh2P3CgWy2YOelldh4T+G0jfF2NOY3sZ+so49n/lm7Ar+VpLL2m6MmR5zK90tPAZ7DPHJcOOOIiKH\nR5xmk9K6Iupbjh0dfRF2d/4PxpgPI+7fA7wKXCAiZXbLiUjDmEljsH/kfwrX9jX2zaZGOaW8jP3Q\nGu/UpWoisk/Ez13C837QGPMg8HdgkIicFPG00i3GnIjn7YMd/JMug+L8vAN7/LWMSmRbUaXH8ocR\n0xBFJFdEWoWPC1dZOeu3G34N/9vA8VHWbmK2mkWkD3BIJZc9BegaeVpfeE/ZxXEe+xW/HX8vdQ1l\nt/BfDs+zR+wMwqfrVYtYTqzSXfW14j3GGFOC/UBaK9ELCkvlMfzzsGOEzou4vYRdLy8l4rBqeLxC\nK7EXZiqdVuZvQETaYz/QT4m9L0h0Cz+xeLvGSnd9tsJe3Q7Ye1z7Gewn5X2BGUAXbAObZIyZkaqi\nwltjF2O3speJSOSV9rphj9E97UJdL2G3mLdhR43Hug04BfhYRP4NLAX2w57mdBr2DABE5DTs8dHh\nxpiF4WmXh2v9G/H3qBB+fTNF5AngtvCuyqnYkb0tsQOZBgOTxF5q+BnsXo47w08fjn0DeFpE2oYH\n9ZU+/83wfOthd2muJT1XktsOnB3+HX2EHUh4DnBPnGPTkSqUbTLC69V/sAMbY7eAj8eOzxiB/Z0D\nICK9sGMuBPthob2I3BG++7/GmM/wp0JszfeKyIvY3/nr4XUg1pvAX8Se9z4He7jpEmwzrowx2Kb1\nnog8jP3wcSV2y39forN/EnhcRF7B7rJvj93NHrtn4e9ACLveTgi/vjrYke69sWe+bMRuJOyHPdb+\nbXj6IGCBMWZZeF5LReSD8Dw2Asdh/5YednpRqTyGb4x5PXaaiBwb/u+7xpiNEXddj/3Afwq/7TV4\nSUS2Yn9f67BjAa7C7sa/PRU1ZiyvTxPw442I0/Li3PdU+L6FMdNzsM3kS2wT/Bp7PnuNmMetxL4Z\nxs73fezo7orWeDjwOPaNZyt2dO1MbPOske66wsvfjT3FqGuCGhti3yi+Di97Dbap9g/fXxe7G3Ae\nkBPz3Aewb8THVyCLK8Lz+CWcwwLsKUkHRcxrBzHn4WJ3C24HxkVM+x12lP+v4Wxvxm7hx54ylSiv\n3cA/Y6Y1C08fEjHtaewgzObAu9jd9N8Bf0kwz7/ETHPMtpy8EtV+eDinXUSclocd9xCvhtJT4+Ld\nHE9PS/LvsVnsPMPL3hznscOBXRXI78/YQZc7I3+34WzGRzyuJrZJfxtev2ZgPwBNJ+LvIl6NDq+n\nHfABdk/dauy1GgaFn39AxOMkvB6vDa8fb2EHDkbVGH5sHvYD8ufY94O12F3xNxK+HgV2q/kd4Pvw\nY1ZhzxI5MGI+twNzsYOEf8EeZruViGtaeHEjwWl5EdMjT3kdFH4NP2L/vr/FXuviMDdr9uOt9EIn\nSikXicjTwAXGmHhf4KICRkQewm6F1jX6pqzSRI/hK6WUi0SkVszP+2N383+ozV6lkx7DV0opd80N\nHydfjh0b0h87XuRuL4tS2U8bvlLe0a25YHobOxBuAHYdKATyjTGzPa1KZT09hq+UUkoFQFq38MPH\nps7it5HESimllKqY2tgzeaYY51N1KyTdu/TPompfCqGUUkoF3SXYb8OsknQ3/K8Bnn/+eVq3bp3m\nRVXMkCFDePDB2C+PU6U0n8Q0G2eaT2KajTPNJ75ly5Zx6aWXQriXVlW6G/42gNatW9OxY2UufZx6\n++yzj29q8SPNJzHNxpnmk5hm40zzKVdKDokH7jz8H374wesSfE3zSUyzcab5JKbZONN83BG4hr9m\nzZryHxRgmk9imo0zzScxzcaZ5uOOwDX8Tp06eV2Cr2k+iWk2zjSfxDQbZ5qPOwLX8Pv16+d1Cb6m\n+SSm2TjTfBLTbJxpPu5I64V3RKQjUFhYWKgDMpRSSqkkFBUVle796GSMKarq/AK3ha+UUkoFUeAa\nfn5+vtcl+Jrmk5hm40zzSUyzcab5uCNwDb9Hjx5el+Brmk9imo0zzScxzcaZ5uMOPYavlFJK+ZAe\nw1dKKaVU0rThK6WUUgEQuIY/a9Ysr0vwNc0nMc3GmeaTmGbjTPNxR+Aa/pgxY7wuwdc0n8Q0G2ea\nT2KajTPNxx2BG7RXUlJCXl6e12X4luaTmGbjTPNJTLNxpvnEp4P2qkhXKmeaT2KajTPNJzHNxpnm\n447ANXyllFIqiLThK6WUUgEQuIY/dOhQr0vwNc0nMc3GmeaTmGbjTPNxR9INX0Qai8hzIrJeREpE\nZGF4cF5GaNq0qdcl+Jrmk5hm40zzSUyzcab5uCOpUfoi0gD4FHgPeAxYDxwJfGWMWRXn8b4bpa+U\nUkplglSP0q+e5ONvA4qNMVdGTFtd1SKUUkoplV7J7tL/PTBfRF4WkbUiUiQiV5b7LKWUUkp5KtmG\nfxhwLfA50AN4HHhYRC5NdWHpsnz5cq9L8DXNJzHNxpnmk5hm40zzcUeyDT8HKDTG/MUYs9AY8y/g\n39gPAQn17NmTUCgUdevatSuTJ0+OetzUqVMJhUJlnj9w4EDGjx8fNa2oqIhQKMT69eujpg8fPpzR\no0dHTSsuLiYUCrF8+XKGDRu2d/rYsWPLjA4tKSkhFAqVubZzQUEB+fn5ZWrr27evJ68jUipfR2Q+\nmfw6IqXqdfTu3TsrXke6fh/xvtM8E19HOn4fAwYMyIrXka7fx0033ZQVr6Mqv4+CgoK9vbFRo0aE\nQiGGDBlS5jlVkeygva+BqcaYARHTrgHuMMY0ifN43w3aKy4u1hGhDjSfxDQbZ5pPYpqNM80nPq8v\nrTsbaBUzrRUZNHBPVypnmk9imo0zzScxzcaZ5uOOZBv+g8D/icjtInK4iFwMXAmMS31pSimllEqV\npBq+MWY+cD7QD1gM3AHcYIx5MQ21KaWUUipFkr7SnjHmbWNMO2NMnjHmaGPMU+koLF1iB2SoaJpP\nYpqNM80nMc3GmebjjsBdS7+kpMTrEnxN80lMs3Gm+SSm2TjTfNyR1Cj9pGfuw1H6SimlVCbwepS+\nUkoppTKQNnyllFIqAALX8GOvrKSiaT6JaTbONJ/ENBtnmo87Atfw+/fv73UJvqb5JKbZONN8EtNs\nnGk+7ghcwx8xYoTXJfia5pOYZuNM80lMs3Gm+bhDR+krpZRSPqSj9JVSSimVNG34SimlVABU97oA\nt40fP54rrrjC6zJ8S/NJTLNxVl4+xhi27NjiYkX+8ezTz/Kn/D95XUZcdWvWJUe83fbTvy13BK7h\nFxUV6YrlQPNJTLNxVl4+W3ZsYZ9R+7hYkY+8Bdevvd7rKsr19zP/zi3dbnF9ufq35Q4dtKeUcsXP\n238ObsPPMBP7TOTCNhd6XUbg6aA9pZRSadVnYh8mLJjgdRkqxbThK6WUKiP/v/m8svQVr8tQKaQN\nXymlVFx9JvbxugSVQoFr+KFQyOsSfE3zSUyzcZaSfAzwTvjfbPKC1wVU3gNzHkj7MvRvyx2Ba/iD\nBg3yugRf03wS02ycpSSf74BPgO+rPitfOd7rAirvlmnpH7Wvf1vuCFzD79Gjh9cl+Jrmk5hm4ywl\n+cwH9mCbfjY5wusC/E3/ttwRqPPw334b3nrL6yr8q04dGDEC8vK8rkRlmueeg48+cn7MDgEOiHPH\ndGyjrwXsCE/7AvgnsB3oDJyWqkpVZVw3cA8SoO3Dyy+H447zuorUC1TD/+YbmDvX6yr8a9EiOPRQ\nGDzY60pUJjEGbrzRflA8IF5DD9tdHfhdnDtOBvKAD4Ffw9NK/+0OZOEbb6aZOxfE6yJc9Lt462kW\nCFTDv/pqOOigyZx33nlel+JLZ54J//nPZAYP1nzimTxZ1514vvsONm6EAQMmc999ifP5eTvsMyrO\nHdWA/8Puxv81Ynrt8PRssAxo7XURlfdpUXq37vVvyx3B2UcTVlBQ4HUJvtW+PSxZovkkoutOfIsW\n2X8XL65iPga7CVI3/G82jdT/zOsC/E3/ttwRqC18gJdeesnrEnyrbVv49deX2LIF6tXzuhr/0XUn\nvkWL7Pry+utVzKcRdjR7R6AIKK56bb6Rwaez33/m/Wlfhv5tuSNwW/gqsXbt7L+f6daISsKiRXbd\nyanqu8kfsLvwa4b//UOVS1MpcHO3m70uQaWINny1V+vWUK0aLF7sdSUqk5Q2fJV9JvaZ6HUJKoW0\n4au9ateGli1/OyarVHm2b4fly7XhZ6Onz31avzEvywSu4efn53tdgq9t3ZqvDT8BXXfKWr4cdu2y\nDV/zcTDZ6wKSM7HPRC7vcLlry9N1xx2Ba/h6RSdnxx/fg8WL7bnVKpquO2WVfjg85hjNx9HhXhdQ\nMfefeT9muHF9y17XHXeISeM7u4h0BAoLCwvp2LFj2pajUueNNyAUguJiaNLE62qU3w0dCq++CitX\nlv9YYwxbdmxJf1EqKXVr1iVHArftlxGKioro1KkTQCdjTFFV5xe40/KUs9JjsYsWacNX5UtmwJ6I\nUL9W/fQWpJRKSD/WqShNm0L9+jpSX1WMjtBXKnMEruHPmjXL6xJ8bfbsWbRtqyP149F1J9q6dfDD\nD781fM0nMc3GmebjjsA1/DFjxnhdgq+NGTOGdu204cej60600r1ApQ1f80lMs3Gm+bgjcA3/xRdf\n9LoEX3vxxRdp1w4+/9yeY61+o+tOtEWLIDcXDg+PQNd8EtNsnGk+7ghcw8/TL3t3lJeXR9u29tzq\n5cu9rsZfdN2JtmiRPR2vWjX7s+aTmGbjTPNxR+AavirfMcfYf3W3vnKyeLEO2FMqk2jDV2Xssw80\nb64j9VViu3bBkiXa8JXKJIFr+EOHDvW6BF8rzUdH6pel685vvvwStm2LbviaT2KajTPNxx2Ba/hN\nmzb1ugRfK81HR+qXpevOb0rXjbZtf5um+SSm2TjTfNyR1KV1RWQ4MDxm8nJjTJsEj9dL62aol16C\niy6CH3+Ehg29rkb5zZ13woQJ8O23XleiVPZK9aV1K7OF/xlwENAofDuxqkUo/yndVavH8VU8eoU9\npTJPZRr+LmPMj8aYdeHbxpRXpTx35JFQq5bu1lfxLVgQvTtfKeV/lWn4R4rIGhH5SkSeF5GM+oqV\n5XpyuaPSfKpXh6OPhoULPS7IR3TdsTZsgG++gWOPjZ6u+SSm2TjTfNyRbMP/CLgcOAu4BmgBzBSR\nOimuK22GDRvmdQm+FpnPscfaLTll6bpjlX4IjG34mk9imo0zzccdSTV8Y8wUY8yrxpjPjDHTgJ7A\nvsAf0lJdGowbN87rEnwtMp8OHeCzz2DHDg8L8hFdd6xPP4W8PDjiiOjpmk9imo0zzccdVTotzxiz\nGfgCOMLpcT179iQUCkXdunbtyuTJk6MeN3XqVEKhUJnnDxw4kPHjx0dNKyoqIhQKsX79+qjpw4cP\nZ/To0VHTiouLCYVCLF++POr0j7Fjx5Y5/7OkpIRQKFTm25sKCgrIz88vU1vfvn09eR2RUvk6IvPZ\nuXMqO3eGWLYs815HpFT9PgoKCrLidVT19/HggyGaN5+195K6pa9j+PDYE3j8/Trc/H2sX78+K15H\nun4fDRs2zIrXUZXfR0FBwd7e2KhRI0KhEEOGDCnznKpI6rS8Mk8WqQusBoYbY8p8RNPT8jLbli1Q\nv749/eqyy7yuRvlF27Zw4onw2GNeV6JUdvP0tDwR+buIdBeRZiLSDXgN2AUUlPNUlYHq1bO7bT/9\n1OtKlF9s2wbLlpU9fq+U8r9kd+kfCrwALAdeBH4E/s8YsyHVhaVL7O4aFS02Hx249xtdd+yYjt27\n7fiOWJpPYpqNM83HHdWTebAxpl+6CnFLSUmJ1yX4Wmw+HTrAmDFgDIh4VJRP6LpjP/zl5Pz2jYqR\nNJ/ENBtnmo87qnQMv9yZ6zH8jPfOO9CzJ6xcCS1aeF2N8tqgQTB9Oixd6nUlSmU/P1xaVwVI6a5b\nPY6vwG7h6/F7pTKTNnzl6OCD4aCD9Di+gj177EV34h2/V0r5X+Aafux5lypavHw6dNCGD7rufPUV\n/PJL4oYf9HycaDbONB93BK7h9+/f3+sSfC1ePh066C590HWn9ENfooYf9HycaDbONB93BK7hjxgx\nwusSfC1ePscea7/3POgfwoO+7ixYAIccAgccEP/+oOfjRLNxpvm4I3ANX88WcBYvn9ItuqDv1g/6\nuvPpp87H74OejxPNxpnm447ANXyVvCOOgDp1tOEH3YIFOmBPqUymDV+Vq1o1aNdOG36QrV0L33+v\nDV+pTBa4hh/7rUkqWqJ8jj1WB+4Fed0p/bDndA5+kPMpj2bjTPNxR+AaflFRlS9WlNUS5dOhAyxf\nDlu3ulyQjwR53VmwwH6ZktPVFoOcT3k0G2eajzv00rqqQj75BI4/Hj7+2P6rgqVfP3umxocfel2J\nUsGhl9ZVnjjmGHssP+i79YOqsFAvqatUptOGryokNxfatAHd8xY8mzfDihVgNzSUUplKG76qsE6d\n7JaeCpbSvTqdO3tbh1KqagLX8EOhkNcl+JpTPp07w6JFsH27iwX5SFDXncJCyMuDo45yflxQ86kI\nzcaZ5uOOwDX8QYMGeV2Crznl06kT7NwJixe7WJCPBHXdKSy0Z2lUq+b8uKDmUxGajTPNxx2Ba/g9\nevTwugRfc8qnfXv7ph/U3fpBXXfmz6/Y8fug5lMRmo0zzccdgWv4qvJyc+Hoo20DUMGgA/aUyh7a\n8FVSdOBesOiAPaWyR+Aa/uTJk70uwdfKy6dzZ3sMf9s2lwrykSCuOxUdsAfBzKeiNBtnmo87Atfw\nCwoKvC7B18rLp1Mn2LUrmAP3grjuVHTAHgQzn4rSbJxpPu7QS+uqpGzdCvXrw8MPw7XXel2NSrdW\nreCss+zvWynlLr20rvJU6cA9PY6f/TZvhi++0AF7SmULbfgqaZ0760j9INABe0plF234KmmdOsGS\nJcH+qtwgSGbAnlLK/wLX8PPz870uwdcqkk/nznbg3qJFLhTkI0Fbd5IZsAfByycZmo0zzccdgWv4\nekUnZxXJp21bqF49eMfxg7buFBYmd/w+aPkkQ7Nxpvm4Q0fpq0rp2NFu/T31lNeVqHTYvBkaNIAJ\nE+Cyy7yuRqlg0lH6yhf0invZrXTAno7QVyp7aMNXlaID97LbvHlQty60bu11JUqpVAlcw581a5bX\nJfhaRfMH+qQSAAAgAElEQVTp3Bl274aFC9NckI8Ead2ZN8/+jis6YA+ClU+yNBtnmo87Atfwx4wZ\n43UJvlbRfNq2hRo14JNP0lyQjwRp3Zk3D44/PrnnBCmfZGk2zjQfdwRu0F5JSQl5eXlel+FbyeRz\n/PH20qvPPZfmonwiKOvO999D48bwyitwwQUVf15Q8qkMzcaZ5hOfDtqrIl2pnCWTT5cu8PHHaSzG\nZ4Ky7sybZ//t0iW55wUln8rQbJxpPu4IXMNXqdOlC6xYARs3el2JSqV58+Dgg+GQQ7yuRCmVStrw\nVaWVHuMt3SJU2aH0+L2I15UopVIpcA1/6NChXpfga8nkc+SRsO++wWn4QVh39uyp3IA9CEY+laXZ\nONN83BG4ht+0aVOvS/C1ZPIRsY0hKMfxg7DufPEF/Pxz8sfvIRj5VJZm40zzcUfgRumr1Bo+HB55\nBH78UXcBZ4Nnn7WX0t20CfbZx+tqlAo2HaWvfKVLF9iwAVau9LoSlQoff2y/DlebvVLZp0oNX0Ru\nF5E9IvKPVBWkMosO3Msu8+ZVbne+Usr/Kt3wReQ44Cogoy6uunz5cq9L8LVk82nYEA47LBjH8bN9\n3dm2zV4quTID9iD786kKzcaZ5uOOSjV8EakLPA9cCWxKaUVpNmzYMK9L8LXK5BOUC/Bk+7qzYAHs\n3Fn5hp/t+VSFZuNM83FHZbfwHwHeMMZMT2Uxbhg3bpzXJfhaZfLp0sV+neqOHWkoyEeyfd2ZNw9q\n1YJ27Sr3/GzPpyo0G2eajzuSbvgichHQAbg99eWkn57+4awy+XTpAtu3w6JFaSjIR7J93Zk3D449\nFmrWrNzzsz2fqtBsnGk+7kiq4YvIocBDwKXGmJ3pKUllmg4d7DfnBWG3fjb76KPK785XSvlfslv4\nnYADgEIR2SkiO4GTgRtEZIdI/DOxe/bsSSgUirp17dqVyZMnRz1u6tSphEKhMs8fOHAg48ePj5pW\nVFREKBRi/fr1UdOHDx/O6NGjo6YVFxcTCoXKDAwZO3ZsmSs8lZSUEAqFynw/c0FBAfn5+WVq69u3\nb+BfR+3acOSRRYwaldmvA7Lj91GZ13HWWSG++mo53bpl9uvIlt+Hvo7gvY6CgoK9vbFRo0aEQiGG\nDBlS5jlVYoyp8A2oA7SJuc0DngFax3l8R8AUFhYavxg1apTXJfhaZfMZONCYli1TXIzPZPO6M3my\nMWBMcXHl55HN+VSVZuNM84mvsLDQAAboaJLo1YluSW3hG2N+NcYsjbwBvwIbjDHLUvUhJJ1KSkq8\nLsHXKptP1672sqwbNqS4IB/J5nVnzhz77XhNmlR+HtmcT1VpNs40H3dU+dK6IjIdWGCMuSnOfXpp\n3YBYtcqej//GG9Crl9fVqGSddJL9StyXX/a6EqVUKd9dWtcYc1q8Zq+CpXlz2zBmz/a6EpWsHTtg\n/nyijt8rpbKPXktfpYSIbRhz5nhdiUrWggX2Knva8JXKboFr+LGjMlW0quTTrZs9l3tnlp6wma3r\nzpw59kyLDh2qNp9szScVNBtnmo87Atfw+/fv73UJvlaVfE44wW4pfvppCgvykWxdd+bMgeOOq/wF\nd0plaz6poNk403zcEbiGP2LECK9L8LWq5HPssXZLMVt362fjumOMHXfRtWvV55WN+aSKZuNM83FH\n4Bq+ni3grCr51KxptxSzdeBeNq4733wD332XmuP32ZhPqmg2zjQfdwSu4av0Kh24V8WzPZVL5s61\n/6ZiC18p5W/a8FVKnXCC3WJcvdrrSlRFzJkDRxwBBx7odSVKqXQLXMOPvaayilbVfEq3FLPxOH42\nrjtz5qTudLxszCdVNBtnmo87Atfwi4qqfLGirFbVfBo2hJYts7PhZ9u68+uv9oyKVO3Oz7Z8Ukmz\ncab5uKPKl9Z1nLleWjeQ+ve3jSRbT8/LFjNmwCmnwMKF0K6d19UopWL57tK6SsXq1g0WLYItW7yu\nRDmZNQvq14ejj/a6EqWUG7Thq5Q74QTYswc+/tjrSpSTmTPhxBOhWjWvK1FKuUEbvkq5Vq1g333t\nFqTyp1277DiL7t29rkQp5ZbANfxQKOR1Cb6WinxycuzXrX74YQoK8pFsWncWLIBffrG/p1TJpnxS\nTbNxpvm4I3ANf9CgQV6X4Gupyqd7d3tRlx07UjI7X8imdWfmTHsZ5M6dUzfPbMon1TQbZ5qPO3SU\nvkqL+fN/u8yufu2q/5x3Hvz8M0yf7nUlSqlEdJS+yggdOkC9evbUL+Uve/bYwy16/F6pYNGGr9Ki\nenU7Wn/mTK8rUbGWLYONG7XhKxU0gWv4kydP9roEX0tlPt2725H6u3albJaeypZ1Z+ZM+4Hs//4v\ntfPNlnzSQbNxpvm4I3ANv6CgwOsSfC2V+Zx8sh0JvmBBymbpqWxZd2bOtIP18vJSO99syScdNBtn\nmo87dNCeSpsdO6BBA7j7brj5Zq+rUWC/tvjQQ+HSS2H0aK+rUUo50UF7KmPUrGm/mEWP4/vHypX2\n64v1+L1SwaMNX6VV9+52RPiePV5XosD+LkTsgEqlVLBow1dpdfLJ8NNP8NlnXleiwO5tadfOHmpR\nSgVL4Bp+fn6+1yX4Wqrz6dLF7trPhvPxs2HdmTEjtZfTjZQN+aSLZuNM83FH4Bp+jx49vC7B11Kd\nT24uHH98djT8TF93Vq+2x/BPPTU988/0fNJJs3Gm+bhDR+mrtLvjDvj3v2HtWnv8WHljwgTo3x/W\nr4f99vO6GqVUeXSUvso4p5wCP/4IS5Z4XUmwTZ8Oxx6rzV6poNKGr9LuhBPscfz33vO6kuAyxjb8\n007zuhKllFcC1/BnzZrldQm+lo588vLsN+ZlesPP5HVnxQpYsya9DT+T80k3zcaZ5uOOwDX8MWPG\neF2Cr6UrnzPOgA8+yOzr6mfyujN9ur1+/oknpm8ZmZxPumk2zjQfdwRu0F5JSQl5qb6IeBZJVz4f\nfWSvujdnjv03E2XyutOnj73C3uzZ6VtGJueTbpqNM80nPh20V0W6UjlLVz6dO0P9+pm9Wz9T1509\ne+D999N//D5T83GDZuNM83FH4Bq+8kb16vaqe5nc8DPV4sWwYYMO2FMq6LThK9eccYbdpV9S4nUl\nwTJ9OtSqlbmHUpRSqRG4hj906FCvS/C1dOZz+un2K3PTeRw5nTJ13Zk+3Z4aWbt2epeTqfm4QbNx\npvm4I3ANv2nTpl6X4GvpzKdNG2jUCP73v7QtIq0ycd3Ztcte1tiN3fmZmI9bNBtnmo87AjdKX3nr\nkkvg889h/nyvKwmG2bPtqXgffWS/yEgplTl0lL7KaGecAUVFsHGj15UEw5QpsO++9iwJpVSwacNX\nrjr9dHuZ1w8+8LqSYJgyxX7IqlbN60qUUl4LXMNfvny51yX4WrrzadoUjjwSpk1L62LSItPWnY0b\n4ZNP4Kyz3FlepuXjJs3GmebjjqQavohcIyILRWRz+DZHRM5OV3HpMGzYMK9L8DU38jnrLHj3Xbul\nn0kybd353/9sxm41/EzLx02ajTPNxx3JbuF/A9wKdArfpgP/FZHWqS4sXcaNG+d1Cb7mRj5nnw1f\nf22/0CWTZNq6M2WKPTPi0EPdWV6m5eMmzcaZ5uOOpBq+MeYtY8y7xpgvw7c7gV+A/0tPeamnp384\ncyOfU06xX5f7zjtpX1RKZdK6Y4xt+G5t3UNm5eM2zcaZ5uOOSh/DF5EcEbkIyAPmpq4kle3q1IHu\n3e1ufZUeS5far8N1s+Erpfwt6YYvIseIyBZgO/AocL4xRkdcqKScc44dqb91q9eVZKepU+3ldE86\nyetKlFJ+UZkt/OVAe6AL8BjwrIgc5fSEnj17EgqFom5du3Zl8uTJUY+bOnUqoVCozPMHDhzI+PHj\no6YVFRURCoVYv3591PThw4czevToqGnFxcWEQiGWL18edd/YsWPLXNKxpKSEUCjErFmzoqYXFBSQ\nn59fpra+fft68joipfJ1RC4zna/jjTdCbNu2nBkz0vM6IqXqdZxzzjllXke6fx+VfR0TJxZRr16I\nkhL31qvOcU72z7a/j8q+jsGDB2fF60jX7+Puu+/OitdRld9HQUHB3t7YqFEjQqEQQ4YMKfOcKjHG\nVOkGTAMeS3BfR8AUFhYav7jrrru8LsHX3Mpnzx5jmjQx5oYbXFlcSmTKulNSYkzt2sbcf7+7y82U\nfLyg2TjTfOIrLCw0gAE6mir2amNM1S+tKyLvAauNMf3j3KeX1lUJDRgAM2eCnoKbWlOm2DMhFi+G\nY47xuhqlVGV5emldEblHRE4UkWbhY/n3AScDz1e1EBU855xjr6u/apXXlWSXN9+EZs3g6KO9rkQp\n5SfJHsM/CHgWexz/f9hz8XsYY6anujCV/U47DapX19H6qWSMbfi9eoGI19Uopfwk2fPwrzTGHGaM\nyTXGNDLGZFyzjx2koaK5mc8++0C3bplzPn4mrDtLl9qLGvXq5f6yMyEfr2g2zjQfdwTuWvr9+5cZ\naqAiuJ1Pr172ErAlJa4utlIyYd15803Iy7MXN3JbJuTjFc3GmebjjsA1/BEjRnhdgq+5nU8oZM/F\nf+89VxdbKZmw7rz5Jpx5JtSu7f6yMyEfr2g2zjQfdwSu4evZAs7czqdVK2jZEl5/3dXFVorf150N\nG2DOHG9254P/8/GSZuNM83FH4Bq+8p9QCN54A/bs8bqSzPbuuzbDnj29rkQp5Ufa8JXnQiFYu9Z+\nd7uqvDffhE6doHFjrytRSvlR4Bp+7CUWVTQv8unaFfbf3/+79f287uzcabfwvdqdD/7Ox2uajTPN\nxx2Ba/hFRVW+WFFW8yKf6tXhd7+zu/X9zM/rzuzZsGmTtw3fz/l4TbNxpvm4o8qX1nWcuV5aV1XQ\nq6/ChRfCypXQooXX1WSeG26AV16Bb76BnMB9jFcqO3l6aV2l0qVHD6hZ0/9b+X5kDEyaBL17a7NX\nSiWmbw/KF+rVs5fa/e9/va4k88yfD99+axu+Ukolog1f+ca558KMGaBX2UzOq6/aQY8nneR1JUop\nPwtcww+FQl6X4Gte5nP++Xb3tF+38v247hhjG/5559nBj17yYz5+odk403zcEbiGP2jQIK9L8DUv\n8znoIOjeHSZO9KwER35cdz77DL780h+78/2Yj19oNs40H3foKH3lK488AjfeaC/Es99+XlfjfyNH\nwj/+AevWQa1aXlejlEolHaWvslrv3rB7t3936/vNq6/ac++12SulyqMNX/nKwQfDiSfac8qVsxUr\nYPFif+zOV0r5X+Aa/uTJk70uwdf8kE+fPjBtmr1ynJ/4IZtIkyZBbi6cfbbXlVh+y8dPNBtnmo87\nAtfwCwoKvC7B1/yQT+/e9trwfru2vh+yiVRQYHfn16njdSWW3/LxE83GmebjDh20p3zpxBPtoD2/\nNX2/WLYM2rSxW/nnn+91NUqpdNBBeyoQLrwQpkzx3259vygogPr14ZxzvK5EKZUptOErX/rDH2DX\nLh28F48xtuH37g21a3tdjVIqU2jDV77UuLG9tv5//uN1Jf5TWGgvtnPxxV5XopTKJIFr+Pn5+V6X\n4Gt+yufSS+GDD6C42OtKLL9kU1AABx4Ip57qdSXR/JKPH2k2zjQfdwSu4ffo0cPrEnzNT/mcf749\n7cwvA3j9kM3u3fDii/aQh9fXzo/lh3z8SrNxpvm4Q0fpK1+76CJYuhQWLfK6En94/317qGP2bOjW\nzetqlFLppKP0VaBceqm9mpw2fGvCBDjiCOja1etKlFKZRhu+8rWzzrLf9f78815X4r0tW+xZC5df\nDiJeV6OUyjSBa/izZs3yugRf81s+NWrY3frPP29P0/OS19lMnAhbt8Kf/uRpGQl5nY+faTbONB93\nBK7hjxkzxusSfM2P+VxxBXz/Pbz7rrd1eJ3N00/DGWdAkyaelpGQ1/n4mWbjTPNxR+AG7ZWUlJCX\nl+d1Gb7l13w6doSmTcHL79jwMpsVK6BlS3jhBejXz5MSyuXXdccPNBtnmk98OmivinSlcubXfK68\nEt58027pe8XLbJ55BvbZB847z7MSyuXXdccPNBtnmo87AtfwVWa6+GJ7PP/ZZ72uxH27d9uGf9FF\n9roESilVGdrwVUZo0MB+oc6TT9pryQfJ22/Dt9/asQxKKVVZgWv4Q4cO9boEX/NzPldeaa8hP3Om\nN8v3KpvHHoPOneG44zxZfIX5ed3xmmbjTPNxR+AaftOmTb0uwdf8nE/37nDkkfDEE94s34tsvvrK\nnp1w3XWuLzppfl53vKbZONN83BG4Ufoqsz30EAwdCqtX22/Uy3bDhsG//w1r1oCOa1IqWHSUvgq0\n/HyoVcu7rXw3bdsGTz1lX7M2e6VUVWnDVxlln33sleaeeAJ27PC6mvSaOBE2bIBrrvG6EqVUNghc\nw1++fLnXJfhaJuQzaBCsXWsbopvczMYYeOQROP10e8GdTJAJ645XNBtnmo87Atfwhw0b5nUJvpYJ\n+bRpYxvh2LHuLtfNbObMgY8/hhtvdG2RVZYJ645XNBtnmo87kmr4InK7iMwTkZ9FZK2IvCYiGbL9\nYY0bN87rEnwtU/K5/nrbED/+2L1lupnNAw9Aq1bQs6dri6yyTFl3vKDZONN83JHsFv5JwFigC3AG\nUAOYKiIZc/0vPf3DWabk06uXPUXPze/ccCubFSvsdwbcfDPkZNA+uExZd7yg2TjTfNxRPZkHG2Oi\ntjdE5HJgHdAJ0O83VK6pVs2esjZgACxfDkcd5XVFqfPQQ9CwIfzxj15XopTKJlXdfmgAGGBjCmpR\nKil//CMcfLC7W/nptn69/RrcgQOhdm2vq1FKZZNKN3wREeAhYJYxZmnqSkqv0aNHe12Cr2VSPrVq\nwU03wXPPwTffpH95bmTzj3/Y3fgDB6Z9USmXSeuO2zQbZ5qPO6qyhf8o0Aa4KEW1uKKkpMTrEnwt\n0/IZMADq1bONMt3Snc3GjfbMg4ED7S79TJNp646bNBtnmo87KtXwRWQc0BM4xRhT7jeU9+zZk1Ao\nFHXr2rUrkydPjnrc1KlTCYVCZZ4/cOBAxo8fHzWtqKiIUCjE+vXro6YPHz68zKfF4uJiQqEQy5cv\nZ+TIkXunjx07tsyXNpSUlBAKhZg1K3pIQkFBAfn5+WVq69u3ryevI1IqX0dkPpnwOurVs+fl/+tf\nsG5den8fsd/Znerfx0MP2a/CvfnmzFyviouLy9SWia8jHX/n5557bla8jnT9Pm699daseB1V+X0U\nFBTs7Y2NGjUiFAoxZMiQMs+piqSvpR9u9ucCJxtjVpbzWL2Wvkq7jRuhRQvo3x8efNDrairnp5+g\neXP7jYAPPOB1NUopP/D0Wvoi8ihwCXAx8KuIHBS+6fAi5Zn99rNbxY895s6x/HR48EF7qWD9llCl\nVLoku0v/GqA+8AHwXcTtD6ktK31id+GoaJmaz5Ah9lj+3Xenbxnpyub77+0YhOuvh0aN0rIIV2Tq\nuuMGzcaZ5uOOpBq+MSbHGFMtzu3ZdBWYav379/e6BF/L1Hzq1YPbbrPfLrdiRXqWka5sRo6EmjXh\n9tvTMnvXZOq64wbNxpnm444Muo5XaowYMcLrEnwtk/O57jq7hXzrremZfzqy+fxzePJJuOMO2Hff\nlM/eVZm87qSbZuNM83FH0oP2kpq5DtpTLnvhBbjkEvjf/+wX7Pjd+efDp5/aqwXqhXaUUpE8HbSn\nlN/16wfdutlvmdu1y+tqnL37rr1m/n33abNXSqWfNnyVVUTg4YdhyRJ44gmvq0ls2zZ7/YBTT4WL\nMurSVUqpTBW4hh97AQYVLRvy6dTJnpN/xx3w3Xepm28qsxkzBoqL4ZFH7IeUbJAN6066aDbONB93\nJPVtedmgqKiIK664wusyfCtb8hkzBt580w7ke+211DTVVGWzYgXcey/ccgu0bl31uvwiU9ed4uLi\ntJ8WNmXKFI499ti0LiOTBTmfhg0buvb1wDpoT2WtV1+FCy+El16CP/jkShG7dkH37vYywAsXQp06\nXlcUbMXFxbRu3Vqv5a48k5eXx7Jly+I2/VQP2gvcFr4KjgsusLdBg+CUU+DAA72uyO55+Phj+PBD\nbfZ+sH79ekpKSnj++edpnU27W1RGWLZsGZdeeinr1693ZStfG77Kao88Au3awR//CO+8Y7961isL\nFsCIEfY6Ad26eVeHKqt169a6F1JlvcAN2lPBctBB8PzzMG2a3br2yubN9rBCmzYwfLh3dSilgitw\nDT/e1yeq32RjPmeeaS9be+eddld6ZVU2G2Pg8svtcftXXoFatSpfg59l47qjVDYJ3C79QYMGeV2C\nr2VrPiNHwuzZ0Ls3fPQRHH548vOobDb33GMvsPPf/8IRR1RqFhkh09cdYww7du/wuoxy1axWE8mW\nczmVqwLX8Hv06OF1Cb6WrflUr25H7XftCr/7Hcydm/y16yuTzTPPwF/+Yj9wZPsGcKavOzt276D2\nPf6/5OG2O7ZRq3qW7iZSaRW4XfoquPbfH956C378EXr2hJ9/Tu/y3noLrrwSrrrKNn2llPKSNnwV\nKEceCVOn2i+rOfvs9DX9116zX4zTqxc8+mj2XE1PBdeIESPIScNpLs2bN9evx3VJ4Br+5MmTvS7B\n14KQT6dOtukvXWovgvPNNxV7XkWzmTAB+vSxDf/ll+3hhCAIwroTZCJS6bEDc+fOZeTIkfwc5xN2\nTk6OjklwSeAafkFBgdcl+FpQ8jnuOJg1C376Cbp0sQP6ylNeNjt3wuDBkJ9vr+X/wgtQo0aKCs4A\nQVl3VPLmzJnDX//6VzZt2lTmvs8//5x//etfHlQVPIFr+C+99JLXJfhakPI55hh71bvDDrNb+n/+\nM2zfnvjxTtl8+qn94PDYY3YX/hNPQLVqaSjax4K07qjkOF3CvUaNGlQL2h+LRwLX8JWK1KgRfPAB\n3H03/P3v0KoVPPusveZ9RRQXw7XX2j0Gu3fDnDn2Z91Dqfzil19+4cYbb6RFixbUrl2bgw46iB49\nerBgwYK9j5k4cSKdO3cmLy+PAw44gD/+8Y98V85XTa5evZqcnByeffbZMvfl5OTw17/+FYCRI0cy\nbNgwwB6vz8nJoVq1ahQXF++dFnsMf9WqVfTp04f999+fOnXq0LVrV95+++2ox8yYMYOcnBwmTpzI\nPffcQ5MmTcjNzeWMM87gq6++inrsl19+yQUXXMDBBx9Mbm4uTZo0oV+/fmzZsqWCKWaHgBxdVCqx\n6tXt1v0FF9iv1L3sMrjtNrjkEujRA4491o7wF7G77VetshfwmTQJpkyBBg3gvvvgxhuDtQtfZYar\nr76aSZMmcf3119O6dWs2bNjA7NmzWbZsGR06dGDChAn079+fLl26MGrUKNauXctDDz3EnDlz+PTT\nT6lfv36Vlt+7d2+++OILXnzxRf75z3+y//77A3DAAQcAlDl+v27dOrp27cq2bdu44YYb2G+//Xjm\nmWf4/e9/z6RJkzj33HOjHj9q1CiqVavG0KFD2bx5M6NHj+bSSy9l7ty5AOzcuZMePXqwc+dOBg8e\nTKNGjVizZg1vvvkmmzZtol69elV6fZlEG75SYa1a2SvhLVoE//oXPPcc3H+/vS831zbzX36BPXts\n8+/SBcaOtdfpr1vX29qVSuTtt9/mqquuYkzEtaVvueUWAHbt2sVtt91Gu3btmDFjBjVr1gTghBNO\noFevXjz44IMMr+K1oNu2bUvHjh158cUXOffcc8v9kpj77ruPH3/8kVmzZtG1a1cArrzyStq1a8dN\nN91UpuFv376dhQsX7j0s0KBBA2688UaWLl1KmzZtWLp0KV9//TWvvvoq559//t7n3XnnnVV6XZko\ncLv08/PzvS7B1zQf+2U748bB99/DkiX2Q8C990LLlvk8/rgd4b9xo714z7XXarMvpeuOPzVo0IB5\n8+bx/fffl7lv/vz5rFu3juuuu25vswfo2bMnRx11FG+99ZabpQLwzjvvcPzxx+9t9gB16tRhwIAB\nfP311yxdujTq8f37948aA3DSSSdhjGHlypUA7LPPPgC8++67bN261YVX4F+Ba/iZfjWwdNN8fiNi\nv+zmggvs7vqbburBVVfZa/M3aOB1df6j644/jRkzhsWLF9OkSRO6dOnCyJEjWbVqFWCPw4sILVu2\nLPO8o446itWrV7tdLqtXr6ZVq1Zlppd+fXFsTU2aNIn6ed/wJTR/+uknwI4RuPnmm3nyySdp2LAh\nZ599No8++mjcUwSzXeAafr9+/bwuwdc0n8Q0G2eajz/16dOHlStXMm7cOA455BDuv/9+jj76aKZM\nmeI4er48ic6d37NnT6XnWRmJRvhHvra///3vLFq0iDvuuINt27YxePBgjjnmmHIHJmabwDV8pZQK\nmoMOOohrrrmGSZMmsWrVKvbff3/uuecemjdvjjGGzz//vMxzPv/8c5o1a5ZwnqVb0rHn1sfbK5DM\nhXWaNWsWt55ly5btvb8yjj76aP785z/zwQcfMGvWLL799lsef/zxSs0rU2nDV0qpLLVnz54yu64b\nNmxI48aN2b59O8cddxwHHnggjz/+ODt37tz7mHfeeYdly5bRq1evhPOuV68eDRs2ZObMmVHTx40b\nV6bB16lTByj74SCenj17Mm/ePD7++OO903799Vf+9a9/0aJFC9q0aVPuPCJt2bKF3bt3R007+uij\nycnJYbvThTeyUOBG6c+aNYsTTzzR6zJ8S/NJTLNxpvn4z5YtWzj00EO58MILad++PXXr1mXatGnM\nnz+ff/zjH1SrVo3Ro0fTv39/unfvTr9+/fjhhx94+OGHOeyww7jxxhsd53/llVcyatQorrrqKjp3\n7szMmTNZsWJFmUMFnTp1whjDn//8Zy666CJq1KhBKBQiNze3zDxvu+02CgoKOPvssxk8eDD77bcf\nEyZMYPXq1UyaNCnpDKZPn86gQYPo06cPLVu2ZNeuXTz77LNUr16dCy64IOn5ZbLANfwxY8bom5ID\nzexEQHwAABJ9SURBVCcxzcaZ5uM/eXl5DBw4kKlTp/Laa6+xZ88ejjjiCB577DEGDBgAwGWXXUad\nOnUYNWoUt912G3Xq1OGCCy5g1KhRZc7Bj91yv+uuu1i/fj2vvPIKEydOpGfPnrzzzjsceOCBUY/t\n3Lkzf/vb33j88ceZMmUKe/bsYdWqVTRt2rTMNfoPPPBA5s6dy6233sq4cePYtm0b7dq148033+Ts\ns892rCfe9Pbt23P22Wfz5ptvsmbNGvLy8mjfvj3vvvsuxx9/fOWCzVBSlUEb5c5cpCNQWFhYSMeO\nHdO2nGSUlJSQl5fndRm+pfkkptk4y8R8ioqK6NSpE4WFhRzd7mhq31Pb65LKte2ObdSqXsvrMlQK\nRK5/8Xpk6f1AJ2NMUVWXF7hj+Jn2huQ2zScxzcaZ5qOUvwVul75SSsVTs1pNtt2xzesyylWzWs3y\nH6RUHNrwlVIKe9xXd5WrbBa4XfpDhw71ugRf03wS02ycaT5K+VvgGn55X9wQdJpPYpqNM81HKX8L\nXMO//vrrvS7B1zSfxDQbZ5qPUv4WuIavlFJKBZE2fKWUUioAAtfwly9f7nUJvqb5JKbZONN8lPK3\nwDX8YcOGeV2Cr2k+iWk2zjI9H2MM27dv9/0tnVdHVdktcOfhjxs3zusSfE3zSUyzcZbp+ezYsYPa\ntTPg0rrbtlGrll4vQCUvcFv4euqQM80nMc3GmebjPxMmTCAnJ4fi4uKknztjxgxycnLKfP1tPKtX\nryYnJ4dnn33W1RpVcpJu+CJykoi8LiJrRGSPiITSUZhSSqmqif0muso8P1JBQQH//Oc/K/TYZJZR\nlRpVxVVml34dYAHwFPBqastRSimVKn/605/o168fNWsmf/39k08+ma1bt0Y994UXXmDJkiXccMMN\nUY9t1qwZW7dupUaNGlWuWaVP0lv4xph3jTF3GWMmAxn3sWz06NFel+Brmk9imo0zzcd/RKRSzb5U\nMs+tWbOmbqn7XOCO4ZeUlHhdgq9pPolpNs40H/+Jd3y8efPmhEIhZs+eTZcuXcjNzeXwww/nueee\ni3pu7DH8U089lbfeemvv8fqcnBwOO+wwIP4x/MWLF5Ofn8/hhx9Obm4uBx98MFdccQUbN26s9OuZ\nOHEiRx99NLm5ubRr147Jkydz+eWX06JFi4R1l0o0zuDzzz/nwgsvZP/99yc3N5fjjjuON954I+ox\nu3btYuTIkbRs2ZLc3FwaNmzISSedxHvvvbf3MWvXriU/P58mTZpQu3ZtGjduzHnnneersQmBG6U/\ncuRIr0vwNc0nMc3GmebjP/GOj4sIK1asoE+fPlxxxRVcfvnlPPXUU+Tn59O5c2dat24d9dhSd955\nJ5s3b2bNmjU89NBDGGOoW7duwmVPmzaNVatW0b9/fxo1asSSJUt44oknWLp0KXPnzk36tbz11ltc\ndNFFtG/fnlGjRvHTTz9xxRVXcMghh8R9jRWxZMkSTjzxRA499FBuv/126tSpw8svv8x5553HpEmT\nOPfccwEYPnw4o0aNYsCAARx33HH8/PPPzJ8/n6KiIk4//XQAevfuzbJlyxg8eDDNmjVj3bp1TJs2\njeLiYt8MaA1cw1dKqaD74osv+PDDD+nWrRsAffr0oUmTJjz99NOMGTMm7nNOP/10DjnkEDZt2kS/\nfv3KXcbAgQO56aaboqZ16dKFiy++mNmzZ3PCCSckVfPtt9/OoYceyuzZs8nNzd1b08knn0zz5s2T\nmlepG264gebNm/PJJ59Qvbpth9deey0nnngit956696G//bbb/O73/2Oxx57LO58Nm/ezNy5c7n/\n/vujXvOtt95aqbrSxZVd+j179iQUCkXdunbtyuTJk6MeN3XqVEKhsoP+Bw4cyPjx46OmFRUVEQqF\nWL9+fdT04cOHlzmWWFxcTCgUKnMlsLFjx5b5Ss+SkhJCoRCzZs2Kml5QUEB+fn6Z2vr27auvQ1+H\nvo4MfR2ffvppmTqCoE2bNnubPUDDhg1p1aoVK1euTNkyIq8VsH37djZs2ECXLl0wxlBUVJTUvL7/\n/ns+++wzLrvssr3NHuCkk06ibdu2larvp59+4v3336dPnz5s3ryZDRs27L316NGDFStW8P333wPQ\noEEDlixZwpdffhl3Xrm5udSsWZMPPviATZs2VaqegoKCvb2xUaNGhEIhhgwZUql5JWSMqfQN2AOE\nHO7vCJjCwkLjFz/++KPXJfia5pOYZuMsE/MpLCw0pe9R27ZtM4Dvb9u2bavw65swYYLJyckxq1ev\n3jutefPmpmfPnmUee8opp5jTTjtt788ffPCBycnJMTNmzNg7rVevXqZFixZlnvv1118bETHPPPPM\n3mkbN240gwcPNgcddJARkb23nJwcc/fddzvWGGvu3LlGRMyECRPK3Ne7d++omuLVHa/GefPm7a0n\nsr7IOhcsWGCMMWbmzJlmv/32MyJi2rZta4YNG2YWLVoUNf+HHnrIVK9e3dSsWdN0797djBkzxvzw\nww8JX5Mx0euf0/1AR1OFXl16q8x5+HVEpL2IdAhPOiz8c5Nk5+WF/v37e12Cr2k+iWk2zjSfzFGt\nWrW4000KL9vbp08fxo8fz3XXXcdrr73GtGnTmDJlCsYY9uzZk7LlxEp0/H737t1RP5fWcMstt/C/\n//2vzG3atGkcccQRgN2T8NVXX/H000/Ttm1bnnzySTp27MhTTz21d3433HADX3zxBaNGjSI3N5e7\n7rqL1q1bs3DhwjS90uRV5hh+Z+B9fvvE+UB4+jOA7//iR4wY4XUJvqb5JKbZONN8sl9FB8Nt2rSJ\n6dOnc/fdd3PHHXfsnZ5ol3h5mjVrlvD5sdP23XdfjDFldq1//fXXUT+XnmFQo0YNTjvttHJraNCg\nAZdddhmXXXYZJSUlnHTSSYwYMSLqg26LFi0YMmQIQ4YM4auvvqJ9+/Y88MADlboCYTpU5jz8GcaY\nHGNMtZib75s9QMeOHb0uwdc0n8Q0G2eaT/arU6cOmzdvLvdxpXsQYrfkH3zwwUqdq3/wwQdzzDHH\n8Oyzz0ad/jljxgwWL14c9dhmzZpRrVq1MqflPfroo1HLPuCAAzjllFN44okn+OGHH8osM3LcSOyp\nhHl5eRxxxBFs374dgK1bt+79f6kWLVpQr169MtO9pKP0lVJKxRW7i79Tp068/PLL3HzzzRx33HHU\nrVuXXr16lXlevXr16N69O2PGjGHHjh0ccsghTJ06lVWrVlX6sMG9997LeeedR7du3cjPz2fjxo08\n8sgjtG3bll9++WXv4+rXr0+fPn14+OGHATj88MN54403ygz8BHjkkUf2Dvy76qqrOOyww1i7di1z\n585lzZo1ewd1tmnThlNOOYVOnTqx33778cknn/DKK68wePBgwJ71cPrpp/OHP/yBNm3aUL16dSZN\nmsS6desqdEaDW7ThK6VUgDhdu76889mvu+46Fi5cyIQJE3jooYdo1qzZ3oYf77r7119/PY8++ijG\nGM466yzeffddGjduXKmt/F69elFQUMCIESO47bbbaNmyJc888wwTJkxg6dKlUY8dO3Ysu3bt4okn\nnqBWrVr07duXBx54gGOOOSbqca1bt2b+/PmMHDmSZ555hg0bNnDggQdy7LHHctddd+193A033MDr\nr7/OtGnT2L59O82aNePee+/llltuAaBJkyZcfPHFvPfeezz//PNUr16do446iokTJ3Leeecl/VrT\nJhUj/xLd8OEo/SeffNLrEnxN80lMs3GWiflk+yj9IOjQoYPp0aOH12VUiu9H6We6ZM//DBrNJzHN\nxpnmo9Jp9+7dZcYEfPDBByxcuJBTTz3Vo6oyS+B26T/yyCNel+Brmk9imo0zzUel07fffsuZZ57J\nJZdcQuPGjVm2bBlPPPEEjRs35uqrr/a6vIwQuIavlFLx1KxZk23btnldRrmq8u13mWzfffelU6dO\njB8/nh9//JE6derw+9//nvvuu499993X6/IygjZ8pZTCDjqLvBys8pf69etTUFDgdRkZLXDH8JVS\nSqkgClzDj/elHeo3mk9imo0zzUcpfwtcwx80aJDXJfia5pOYZuNM81HK3wLX8Hv06OF1Cb6m+SSm\n2TjTfJTyNx20p5QKvGXLlnldggogt9c7bfhKqcBq2LAheXl5XHrppV6XogIqLy+Phg0burKswDX8\nyZMn++vaxj6j+SSm2TjLxHyaNm3KsmXL4n6xSiq9//77ejU4B0HOp2HDhjRt2tSVZYmp5DcXVWjm\nIh2BwsLCQt98dWbXrl2ZO3eu12X4luaTmGbjTPNJTLNxpvnEV1RURKdOnQA6GWOqfO3qwA3aO+CA\nA7wuwdc0n8Q0G2eaT2KajTPNxx2Ba/hKKaVUEGnDV0oppQJAG75SSikVAOkepV8b/HWO67x58/R7\nux1oPolpNs40n8Q0G2eaT3wRvbN2KuaX7lH6FwP/SdsClFJKqex3iTHmharOJN0Nf3/gLOBrwP9f\nNK2UUkr5R22gOTDFGLOhqjNLa8NXSimllD/ooD2llFIqALThK6WUUgGgDV8ppZQKAG34SimlVABo\nw1dKKaUCIJANX0SaiciTIrJSREpEZIWIjBCRGl7X5hURGSgiq0Rkq4h8JCLHeV2TH4jI7SIyT0R+\nFpG1IvKaiLT0ui4/Cme1R0T+4XUtfiEijUXkORFZH36vWRj+FtFAE5EcEbk74j34SxG50+u6vCIi\nJ4nI6yKyJvw3FIrzmL+KyHfhvKaJyBHJLieQDR84ChDgKqANMAS4BrjHy6K8IiJ9gQeA4cCxwEJg\niog09LQwfzgJGAt0Ac4AagBTRSTX06p8JvwB8SrsuqMAEWkAzAa2Y69H0hq4GfjJy7p84jbgauA6\n7PvxMGCYiAz6//buLjTLMo7j+PeXUKAhHtS2Xg5KDImIIiMRyR2MToReT7IOksReNYKIREwIBjUs\ns0Z5FJUVUauTHNSJrSCLNKcWaBGS1GLOKKFgEZj+Orjuje3ZsD0H23Vv1/9ztPvm2fP8eNiu//XK\nnTVVPguAw8AGYMJZeUmbgI2k7+wmYJjURp/fzIfEOfyKpCeBh2033Wua7SR9Deyz/Xh1LWAA6La9\nLWu4mqk6Qb8Bq2zvzZ2nDiRdCPQDjwBbgUO2n8ibKj9JXcAK2+25s9SNpF5gyPYDY+59CPxt+758\nyfKTdBa4w/buMfcGgedt76iuFwIngbW2e6b63qWO8CezCDiVO8RMq5YxlgGfjtxz6gXuAVbkylVj\ni0g98OL+Vs7hVaDXdl/uIDVzK3BAUk+1HHRQ0vrcoWriK6BD0lUAkq4DVgIfZ01VQ5KuBNoY30b/\nBeyjyTZ6uh+eMytUayEbgRJHJRcB80i9xbFOAktnPk59VTMfLwF7bR/NnacOJK0BrgduzJ2lhhaT\nZj22k5YLlwPdkv6x/U7WZPl1AQuBHySdIQ0+t9h+L2+sWmojDTIma6PbmnmjOVXwJT0HbDrHSwxc\nbfvHMb9zGfAJ8L7t16c54mwiJllLKtxO0p6PlbmD1IGky0kdoFtsn86dp4bOA/bb3lpdfyvpGlIn\noPSCfzdwL7AGOErqNL4sadD221mTzR5Nt9FzquADLwBv/M9rfhr5QdKlQB9pxPbQdAarsd+BM0Br\nw/0WJvYoiyXpFWA1cLPtE7nz1MQy4GKgv5r9gDRbtKrafHWBy94kdAJofDb498BdGbLUzTbgWdsf\nVNdHJF0BbAai4I83RCrurYxvk1uAQ8280Zwq+NXThKb0RKFqZN8HfAOsm85cdWb7tKR+oAPYDaNT\n1x1Ad85sdVEV+9uBdtu/5M5TI3uAaxvuvUkqal2FF3tIO/Qbl8WWAj9nyFI385k4Oj1L7CubwPZx\nSUOkNvk7GN20t5y0f2bK5lTBnypJlwCfkx7b+xTQMjJAsV3iqPZFYFdV+PeTjinOJzXeRZO0E7gH\nuA0YljQyE/Kn7aIf+Wx7mDQdO0rSMPCH7caRbYl2AF9K2gz0kBro9aTji6XrBbZIGgCOADeQ2p3X\nsqbKRNICYAlpJA+wuNrIeMr2AGnp7GlJx0h1qxP4Ffioqc8psRMuaS3QuF4v0gb1eRkiZSfpUVLn\np5V0HvQx2wfypsqvOiIz2T/J/bbfmuk8dSepDzgcx/ISSatJG9SWAMeB7bFXaLTAdQJ3kqamB4F3\ngU7b/+bMloOkduAzJrY1u2yvq17zDPAg6aTQF8AG28ea+pwSC34IIYRQmlgvCSGEEAoQBT+EEEIo\nQBT8EEIIoQBR8EMIIYQCRMEPIYQQChAFP4QQQihAFPwQQgihAFHwQwghhAJEwQ8hhBAKEAU/hBBC\nKEAU/BBCCKEA/wFa5RD0ULD0+QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112a35710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_mins(f=non_convex_1,min_df=min_df,xmin=-2,xmax=9,title='Non-Convex example N.1 - initial guess = 4.5',\n",
    "          xguess=4.5,legend_pos='lower right')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Nelder-Mead</th>\n",
       "      <td>[1.99998779297]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Powell</th>\n",
       "      <td>1.9999999999683606</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CG</th>\n",
       "      <td>[1.99999984958]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BFGS</th>\n",
       "      <td>[2.00000003039]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>L-BFGS-B</th>\n",
       "      <td>[1.99999996669]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TNC</th>\n",
       "      <td>[1.99999999867]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>COBYLA</th>\n",
       "      <td>1.99989765625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[2.0]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            min\n",
       "Method                         \n",
       "Nelder-Mead     [1.99998779297]\n",
       "Powell       1.9999999999683606\n",
       "CG              [1.99999984958]\n",
       "BFGS            [2.00000003039]\n",
       "L-BFGS-B        [1.99999996669]\n",
       "TNC             [1.99999999867]\n",
       "COBYLA            1.99989765625\n",
       "SLSQP                     [2.0]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "initial_guess = 3.9\n",
    "mins = [spo.minimize(non_convex_1,initial_guess,method=om,) for om in opt_methods_no_Jacobian]\n",
    "\n",
    "min_df = pd.DataFrame({'Method' : opt_methods_no_Jacobian,\n",
    " 'min' : [m.x for m in mins]\n",
    "  }).set_index('Method')\n",
    "min_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfwAAAFyCAYAAAAQ6Gi7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xl8FPX9x/HXJ5wJcqgIiHIpooAgAkpBxbNoka6KUkRt\nNahoBVFU8MAK1FqB1moFvCoK6s94IlU8AEVADkWIHAooyhGlihyCR7j5/v74buJmszvJJrszszuf\n5+OxD8hkd/az73yTz87Md2bFGINSSimlMluW1wUopZRSKvW04SullFIBoA1fKaWUCgBt+EoppVQA\naMNXSimlAkAbvlJKKRUA2vCVUkqpANCGr5RSSgWANnyllFIqALThK+VzInJARO7xug6viEizcAZ/\nquDjy52fiKwXkacq8ByVqlEpN2jDj0FErgz/8haKyOExvj9bRJZ7UVtEDUeJyOMi8pWI7BSRHSIy\nT0QGi0hNL2tT/hZuagdE5N8xvnd6+Hu9y7GeP4vISyKyIfyYhBtlAipzDXAT+XgR6SoiI0SkToz7\nHqjkc6lKEpGaIjJRRFaIyHYR+UlElob/tlUt5zqOFpFXRGSbiPwiIh+IyBkpLt33yhVegNUA7gBu\nilru6R8EEekJvAzsAp4BPgWqA6cCY4E2wPWeFaj8rqgBXisio40x38b4fnkMAw4CFgGNklhfyWKM\n2SAi2cDeCq4iG9gX8XU34B7gaeDHqPsei236yjvZQGvgTWA99ufRDXgQOBm4wunBInIk8CF2vIwB\nCoFcYIaInGWMmZeyyn1OG76zpdg/ivcbY77zuhgAEWkOvACsA84yxnwf8e1HReQvwPkelKbSy2fY\n5nY7cHPU96Sc6+hujPkaQER+SmJtpRhj9iTxsXFfnzGmom8qVJIYY37ANvhIT4jIj8BAEbnVGLPJ\nYRV3AnWAtsaYLwFE5ElgNfZNw0kpKDst6C79+Azwd+ybojvKurOIVBGRv4jIlyKyS0TWicjfRKR6\n1P3Wi8jrInKKiHwU3h3/lYj8sZx13Q7UAq6Oava2aGPWGmPGpaIuEekc3nVb6h22iJwX/t7vIpY1\nFpGnROS78HN/KiL9ox43Ofxcx0YtnyEiW0XEcctRrJvD694Zfq7HRKRexH3OFJH9IjIi6rGXh2u+\nLmJZroi8JyKbwjV/JiKl9pZE5HW6iHws9vDPchE5Pfz93uGvd4rIYhHpEPX4SeFdlS1EZLqI/Cwi\nG8Nv2MpUnmzLsB67d+jasjKOp6jZp5rEOD4ekV9jEZka/v/3IvIPEZGoxxcfww+PgbHhbxUd2tgv\nIk3D3y9xDF9EDhaRf4Z/lj+JPXT2loi0r8TraS8ic8Jj5msRGR4edweK6oiuO+rxpeYZiEhdEXlI\nRArC42GNiAyLkcWl4fH4Y/i1LBeRwRHfryr2cMcX4bG7Rezu8LMr+nqTaEP437pl3O9U4JOiZg9g\njNkJvA50FJGWKarP97ThO1tH+f8oTgRGAYuxW0yzgbuAvKj7GeAY7C75GcAtwDbgaRFpXY6aegFr\njTEflfM1JK0uY8xi4Cugb4zn+UP4/jMBRKQB8BFwFvAwMBhYAzwZ+QcmvHwzMLnoj1O4AZ8NDCrH\nnpUnsLvtPgiv6yngcuAdEakSrvt94BHgLhE5MfwchwP/BmYYYx6PWN/12GZ4XziDAuAREflznLz+\nD/uH5A7gYOB1EbkMeAA7du4BjgZejPH4LOAd4FtgKPZnNEpERjq94ASyLcvfgGqU4w2tDxXlNx07\nfm7Fju1bgAEOj5vCr2P/Juzu4T+G11G03khHASHgDWAI9s3C8cDsirxREpHGwPvYXdb3Af8CLsP+\nDMt7KKXE/cQe7piLHfeTgBuBecD92HFYdL/fAs8DW7GHY24P19I1YnWjsGP2PWAgdoxsADqW8bpE\nRA4t5628x+Grhe9/pIhchP0Zrwe+dH4kNYCdMZYXhv91fC0ZzRijt6gbcCWwHzswWgB7gAcjvv8+\nsDzi6/bY40yPRa1nbHg9p0csWxde1i1iWX3sAB1bRl21w88zpZyvI+l1Yf9I7QLqRSyrhm32T0Qs\nexL4JvJ+4eXPh+9bI2LZb8N13gk0xx5XfaUcr+/U8OP6Ri0vWt+lEcuygS+A5dj5Dm8APwBHRD22\nRozneRtYE7WsKK8uMZ73Z+DIiOXXhu/bPWLZ0+FlD0at941w5odELDsA3FORbOPktg54Pfz/icAv\nQMPw16eHn693gr8zPwFPpej3sVm4pj/FyO+uqPsuARZFLYvO79bwY5vGyeapiK+rxbhP0/DPaLhT\njXFey8PY+QTtI5bVA7ZE1xRdt0ONd4d/Z46Kut/fsX+7jgh//SCwrYz6PikaGxX8GZV1K/F7UMY6\n+0Y99iPsbvqyHvdf7JuaWlHLF4Sff0gqxmk63HQLvwzGmHXAs8AAEWkY5249se+6H4xa/gD2eGH0\nMfWVxpgFEc+xBfgcuzXhpGhWcXmPl6airhexDfOiiGXnYnezRW7F9sY2ryqR7+6xew/qEvEu2xgz\nE3gcGIHdAtsJRG9Rx3IJsB14L+o5PsE23TMjnmMn9o1ca+zWUE/gZmPMxsgVGmN2F/1fROqE1zcX\nOEpEakc9/0pTck9L0f/fM8Z8E7VciP3znRD19XhsvufEf9nlz7Yc0nkrH+y4ifQBZf8elZuJOKYv\nIlkicgh2S/FzKraleC6w0BhTfJaPMWY7dk9RRV2Cfd07osbDe9hDkt3D99sOHCQi5zqsazvQtgK7\nvb/Djtmybr8FlpVznbPCj7kEeBQ7Ce+gcjzuUezetpdEpIOIHCMiDwGdwt/PLufzZxydtFc+f8Pu\n9rsDu1svWtG72xK7mowxm0Rke/j7kQpirOMH7CBFRLKAw6K+v5VfZxRHN554klpX+LHLReRz7Lvv\np8OL+2K3UN4P138YdqtlAHAdpRmgQdSy24ALgBOAy4wxm0s9qrRjws9Tai5DrOcwxiwUkcewuyqn\nG2MmRz9IRE7B7tb8DZATtb66lHyzVSIvY8yP4aMSkc0eYEf434Ojlh8A1kYt+wL75iD6Z1NUX0Wy\njcsYs05Eit7Qji7v4yojzvjeZhKfMLfLGLM1almJ8VpZ4cNMN2PfgLYAqoS/ZbBjPlHNsFua0cra\nTe3kGKAdvx6WiBQ5Hh4B+gBvicj/sG8QXzLGTI+4/z3AVOALEfkUu3frOWPMCqcCwm+UZ1XiNcRa\n5+aIdU4RkTuBmSLS0sSYvxTxuHdEZBAwGrvHR7CHvO4C/oHdGAgkbfjlEP6j+Bz2j+KYGHcpmhhT\n3mNw++MsL1pPE+xuOxNeZoAzjTFzw7+o7cr5PMmuq8iLwJ3hrZ2fgd9j/ygUnc5UtOfoOaBUUw2L\nvo5BR379w9SO0se8Y8kCNmGPgcaaeV3iD6DYiYqnY/M4SkRqGmN2RXz/KOBdYBX2jd3X2F2i52P/\n6EfvEYuXV3lzrMh9KpJtWe7DvqG9Hbs7NNVijm/snpRExMs5mYYDf8Ue+rgbe8jkAHb+h1d7SKtE\nfZ2FnTszhtjj5wuwDVTs5NFzgd+Fb7kiMtkYkxu+zwcicjT2zXcP4BrgFhG5zhgT9zoLcd7ExVOR\nN3cAr2DH6gXAf5zuaIx5RESexh7W3IM94+oa7Fj7ogLPnRG04Zff37ATfG6P8b312F+6Y7C7+oDi\nyVX1+HV2aXkV7R6LVLQbbBp2EmEXU/bEvWTXVeQF7JbAxdit69qUbNCbsVvCVYwxZb7rF5Ec7N6C\nz7BbP7eLyGvGmCVlPPQr7OS+BZG74h38FbtL/zbsPIbRlDwl7ffY3em/j9zVn8IZylnY3c+RW3et\nwv/G+9kklG15GGPWht/QXoc9pz7VnMa3GxK5jsbFwCxjzLWRC8WeBVKevVDRNgCxdpcfE2PZD9jf\n08jnrQZEXwzsK+AgYyenOjLG7MOe3/5meH2PYjdk7jXGrA3fZzv2zeTk8O/mB8BI7ITYeIrexJVZ\nAhV7cwe/7oova5a+fSJ7GK/4b2R40uJOYH4Fnjsj6DH8cgr/MhT9UYyenfsW9p119PnMt2IH+JsJ\nPtduY8ysqFvRbuGx2GOIT4YbdwlirzBVNFM7qXVF1LcaWAFcit2d/50x5oOI7x8AXgUuFpG2MWqs\nH7VoLPYPxp/4dSbu5PAfNycvYd+0xjp1qYqI1I34ukt43Q8aYx7E7tobJCKnRTysaIsxK+JxdYGr\nyqijMgbF+HoP9vhrKRXItryKjuUPI8YscBE5NnxcuNLKGN9u+CX8bz3He1n7idpqFpE+wBEVfO7p\nQFeJOK0vvKfsshj3/Ypfj78XuZ7SW/gvhdfZI3oFYk/XqxLxPNGKdtXXiHUfY0wh9g1pjXgvKCxp\nx/Adxtm12LG5OOK+dcJjM9ZVEyPX2Q077+hJY0xKrxnhZ7qFH1+sXWNFuz6PxV7dDig+rj0Z+075\nYGAO0AXbwKYYY+Ykq6jw1thl2K3sVSISeaW9bthjdE+7UNeL2C3mXdhZ49HuAM4APhKR/wArgUOw\nE2fOwp4BgIichT0+OsIYsyy87KpwrX8j9h4Vwq9vrog8DtwR3lU5AzuxpxV2os9g7LG/mtgtls+x\nu2XBThD8Pfa0w3bhrYGix08Lr7c2djfgJlJzJbndwHnhn9GH2ImEvwPui3FsOlK5sk1EeFz9H3Zi\nY/QW8MnY+RkjsT9zAESkF3bOhWDfLJwgIsPD3/6vMeZT/KnouO7fReQF7M/89fAYiDYN+IvY894X\nYA83XY5txhUxFrun8D0ReRj75uMa7Jb/wZTM/kngMRF5BbvL/gTsbvboPQv/wJ46OE1EJoVfXy3s\n7uze2DNftmE3Eg7BHhf/Jrx8ELDUGLMqvK6VIjI7vI5t2IvUXII9uyCuJB/Dv0LstS+mYue41MYe\nhjgH+3OaHXHfi7B/767CngaL2GsZvIQ9XfY77GmU12F36w8nyLw+TcCPNyJOy4vxvafC31sWtTwL\n20y+xDbB9cC9RJ3Wgx3A/42x3vexs7vLW+PRwGPYPzw7sbNr52KbZ7VU1xV+/v3YU4y6xqmxPvYP\nxfrwc2/ENtX+4e8fhN0NuAjIinrsA9g/xCeXI4urw+v4OZzDUuwpSQ0j1rUH6BT1uI7Ypjs+Ytn5\n2Fn+v4SzvRX7xyT6lKl4ee0H/h21rBlRpwPx62Vdm2PPxf8J+B/wlzjr/EvUMsdsy8grXu1Hh3Pa\nR8Rpedh5D7FqKDo1LtbN8fS0BH8fm0WvM/zcO2LcdwSwrxz53YWddLk38mcbzmZixP2qY5v0N+Hx\nNQf7BmgWEb8XsWp0eD3tsdcMKMQ2+qHYxrsfOCzifhIex5vC4+NN7MTBEjWG75uDfYP8OfbvwSbs\nrvibsYd/wDbHt7HXfdiJ/d2bADSIWM+dwELsJOGfsYfZbi9ahxs37BvXoquJ7gz/nnyMfQMf/Xei\n6G915Niohz3bZ2P48V9hN9ZqufUa/HqTcEBKKReFJxRdbIxx3BWpgiF82ti12GPx+kdZpYQew1dK\nKReJSI2orw/F7ub/QJu9SiU9hq+UUu5aGD5Ovho7N6Q/9jj1vV4WpTKfNnylvKNbc8H0FnYi3ADs\nGFgC5BpjAnu6mHKHHsNXSimlAiClW/jhY1Pn8utMYqWUUkqVT03smTzTjfOpuuWS6l3651K5D4VQ\nSimlgu5y7KdhVkqqG/56gOeee47WrcvzUe+pN2TIEB58MPrD41QRzSc+zcaZ5hOfZuNM84lt1apV\nXHHFFRDupZWV6oa/C6B169Z07FiRT5JMvrp16/qmFj/SfOLTbJxpPvFpNs40nzIl5ZB44M7D/+67\n77wuwdc0n/g0G2eaT3yajTPNxx2Ba/gbN24s+04BpvnEp9k403zi02ycaT7uCFzD79Spk9cl+Jrm\nE59m40zziU+zcab5uCNwDb9fv35el+Brmk98mo0zzSc+zcaZ5uOOlF54R0Q6AkuWLFmiEzKUUkqp\nBOTn5xft/ehkjMmv7PoCt4WvlFJKBVHgGn5ubq7XJfia5hOfZuNM84lPs3Gm+bgjcA2/R48eXpfg\na5pPfJqNM80nPs3GmebjDj2Gr5RSSvmQHsNXSimlVMK04SullFIBELiGP2/ePK9L8DXNJz7Nxpnm\nE59m40zzcUfgGv7YsWO9LsHXNJ/4NBtnmk98mo0zzccdgZu0V1hYSE5Ojtdl+JbmE59m40zziU+z\ncab5xKaT9ipJB5UzzSc+zcaZ5hOfZuNM83FH4Bq+UkopFUTa8JVSSqkACFzDHzp0qNcl+JrmE59m\n40zziU+zcab5uCPhhi8ijUXkWRHZIiKFIrIsPDkvLTRt2tTrEnxN84lPs3Gm+cSn2TjTfNyR0Cx9\nEakHfAK8BzwKbAGOAb4yxqyLcX/fzdJXSiml0kGyZ+lXTfD+dwAFxphrIpZtqGwRSimllEqtRHfp\n/x5YLCIvicgmEckXkWvKfJRSSimlPJVowz8K+DPwOdADeAx4WESuSHZhqbJ69WqvS/A1zSc+zcaZ\n5hOfZuNM83FHog0/C1hijPmLMWaZMeYJ4D/YNwFx9ezZk1AoVOLWtWtXpk6dWuJ+M2bMIBQKlXr8\nwIEDmThxYoll+fn5hEIhtmzZUmL5iBEjGDNmTIllBQUFhEIhVq9ezbBhw4qXjxs3rtTs0MLCQkKh\nUKlrO+fl5ZGbm1uqtr59+3ryOiIl83VE5pPOryNSsl5H7969M+J1pOrnEeszzdPxdaTi5zFgwICM\neB2p+nnccsstGfE6KvPzyMvLK+6NjRo1IhQKMWTIkFKPqYxEJ+2tB2YYYwZELLseGG6MaRLj/r6b\ntFdQUKAzQh1oPvFpNs40n/g0G2eaT2xeX1p3PnBs1LJjSaOJezqonGk+8Wk2zjSf+DQbZ5qPOxJt\n+A8CvxGRO0XkaBG5DLgGGJ/80pRSSimVLAk1fGPMYuAioB+wAhgO3GSMeSEFtSmllFIqSRK+0p4x\n5i1jTHtjTI4xpq0x5qlUFJYq0RMyVEmaT3yajTPNJz7Nxpnm447AXUu/sLDQ6xJ8TfOJT7NxpvnE\np9k403zckdAs/YRX7sNZ+koppVQ68HqWvlJKKaXSkDZ8pZRSKgAC1/Cjr6ykStJ84tNsnGk+8Wk2\nzjQfdwSu4ffv39/rEnxN84lPs3Gm+cSn2TjTfNwRuIY/cuRIr0vwNc0nPs3GmeYTn2bjTPNxh87S\nV0oppXxIZ+krpZRSKmHa8JVSSqkACFzDj/5cZFWS5hOfZuNM84lPs3Gm+bgjcA0/P7/Sh0EymuYT\nn2bjTPOJT7Nxpvm4QyftKaWUUj6kk/aUUkoplTBt+EoppVQAaMNXSimlAiBwDT8UCnldgq9pPvFp\nNs40n/g0G2eajzsC1/AHDRrkdQm+pvnEp9k403zi02ycaT7u0Fn6SimllA8le5Z+1cqXlD7eegve\nfNPrKvyrVi0YORJycryuRKWbZ5+FDz/0ugqlkuOqq+Ckk7yuIvkC1fC//hoWLvS6Cv9avhyOPBIG\nD/a6EpVOjIGbb7ZvFA87zOtqlKq888/3uoLUCFTDv+46aNhwKhdeeKHXpfjSb38L//d/Uxk8WPOJ\nZepUHTux/O9/sG0bDBgwlfvv13xi0bHjTPNxR+Am7eXl5Xldgm+dcAJ89pnmE4+OndiWL7f/rlih\n+cSjY8eZ5uOOwDX8F1980esSfKtdO/jllxf56SevK/EnHTuxLV8OtWvD669rPvHo2HGm+bgjcA1f\nxde+vf3300+9rUOll+XL7djJ0r8mSvma/oqqYq1bQ5UqsGKF15WodFLU8JVS/qYNXxWrWRNatfr1\nmKxSZdm9G1av1oavVDoIXMPPzc31ugRf27kzVxt+HDp2Slu9Gvbtsw1f84lPs3Gm+bgjcA2/R48e\nXpfgayef3IMVK+y51aokHTulFb05PP54zceJZuNM83GHXlpXlfDGGxAKQUEBNGnidTXK74YOhVdf\nhbVrva5EqcyT7EvrBm4LXzkrOharu/VVeeiEPaXShzZ8VULTplCnjs7UV+WjDV+p9BG4hj9v3jyv\nS/C1+fPn0a6dbuHHomOnpO+/h++++7Xhaz7xaTbONB93BK7hjx071usSfG3s2LG0b68NPxYdOyUV\n7QUqaviaT3yajTPNxx2Ba/gvvPCC1yX42gsvvED79vD55/Yca/UrHTslLV8O2dlw9NH2a80nPs3G\nmebjjsA1/Bz9sHdHOTk5tGtnz61evdrravxFx05Jy5fb0/GqVLFfaz7xaTbONB93BK7hq7Idf7z9\nV3frq3iMMbz55s20a6cXbFAqXWjDV6XUrQvNm+tMfRXfRx8tYfPmCRxySKVPDVZKuSRwDX/o0KFe\nl+BrRfnoTP3SdOz86h//eBTYx8qVjxYv03zi02ycaT7uCFzDb9q0qdcl+FpRPjpTv7Sgj5177rmH\nBg0acMwxx/Dee28C8PHH02jZsiUNGjRghe4SiivoY6csmo87Erq0roiMAEZELV5tjGkT5/56ad00\n9eKLcOmlsHkz1K/vdTXKD/bu3ctjjz3Gfffdx6ZNm4qXN2zYkOHDh3P99ddTrVo1DytUKrP44dK6\nnwINgUbh26mVLUL5T9G51brRpopUq1aNG2+8kbp165ZYXrduXW688UZt9kr5XEUa/j5jzGZjzPfh\n27akV6U8d8wxUKOG7tZXpR04cADIplatw8nOzg5/rZTyu4o0/GNEZKOIfCUiz4lIWn2m2mo9udxR\nUT5Vq0LbtrBsmccF+YiOHatt247AGCZM+JIxY8YUH67TfOLTbJxpPu5ItOF/CFwFnAtcD7QA5opI\nrSTXlTLDhg3zugRfi8znxBNh6VIPi/EZHTvW4MEvAjfym9/kcOONN/Liiy8Cmo8TzcaZ5uOOhBq+\nMWa6MeZVY8ynxpiZQE/gYOAPKakuBcaPH+91Cb4WmU+HDvDpp7Bnj4cF+YiOHeuTTyAnB1q2LLlc\n84lPs3Gm+bijUqflGWN2AF8ALZ3u17NnT0KhUIlb165dmTp1aon7zZgxg1AoVOrxAwcOZOLEiSWW\n5efnEwqF2LJlS4nlI0aMYMyYMSWWFRQUEAqFWL16dYnTP8aNG1fq/M/CwkJCoVCpT2/Ky8sjNze3\nVG19+/b15HVESubriMxn794Z7N0bYtWq9HsdkZL188jLy8uI11HZn8eDD4Zo3nxe8SV1i17HiBHR\nJ/D4+3W4+fPYsmVLRryOVP086tevnxGvozI/j7y8vOLe2KhRI0KhEEOGDCn1mMpI6LS8Ug8WOQjY\nAIwwxpR6i6an5aW3n36COnVg0iS48kqvq1F+0a4dnHoqPPpo2fdVSlWcp6flicg/RKS7iDQTkW7A\na8A+IK+Mh6o0VLu23W37ySdeV6L8YtcuWLXKzu9QSqWXRHfpHwk8D6wGXgA2A78xxmxNdmGpEr27\nRpUUnY9O3PuVjh07p2P/fju/I5rmE59m40zzcUfVRO5sjOmXqkLcUlhY6HUJvhadT4cOMHYsGAMi\nHhXlEzp27Ju/rKxfP1ExkuYTn2bjTPNxR6WO4Ze5cj2Gn/befht69oS1a6FFC6+rUV4bNAhmzYKV\nK72uRKnM54dL66oAKdp1q8fxFdgtfD1+r1R60oavHB1+ODRsqMfxFRw4YK+8GOv4vVLK/wLX8KPP\nu1QlxcqnQwdt+KBj56uv4Oef4zf8oOfjRLNxpvm4I3ANv3///l6X4Gux8unQQXfpg46dojd98Rp+\n0PNxotk403zcEbiGP3LkSK9L8LVY+Zx4InzzDQT9TXjQx87SpXDEEXDYYbG/H/R8nGg2zjQfdwSu\n4evZAs5i5VO0RRf03fpBHzuffOJ8/D7o+TjRbJxpPu4IXMNXiWvZEmrV0oYfdEuX6oQ9pdKZNnxV\npipVoH17bfhBtmkTfPutNnyl0lngGn70pyapkuLlc+KJOnEvyGOn6M2e0zn4Qc6nLJqNM83HHYFr\n+Pn5lb5YUUaLl0+HDrB6Nezc6XJBPhLksbN0qf0wJaerLQY5n7JoNs40H3fopXVVuXz8MZx8Mnz0\nkf1XBUu/fvZMjQ8+8LoSpYJDL62rPHH88fZYftB36wfVkiV6SV2l0p02fFUu2dnQpg3onrfg2bED\n1qwBu6GhlEpX2vBVuXXqZLf0VLAU7dXp3NnbOpRSlRO4hh8Khbwuwdec8uncGZYvh927XSzIR4I6\ndpYsgZwcOO445/sFNZ/y0GycaT7uCFzDHzRokNcl+JpTPp06wd69sGKFiwX5SFDHzpIl9iyNKlWc\n7xfUfMpDs3Gm+bgjcA2/R48eXpfga075nHCC/aMf1N36QR07ixeX7/h9UPMpD83GmebjjsA1fFVx\n2dnQtq1tACoYdMKeUplDG75KiE7cCxadsKdU5ghcw586darXJfhaWfl07myP4e/a5VJBPhLEsVPe\nCXsQzHzKS7Nxpvm4I3ANPy8vz+sSfK2sfDp1gn37gjlxL4hjp7wT9iCY+ZSXZuNM83GHXlpXJWTn\nTqhTBx5+GP78Z6+rUal27LFw7rn2562UcpdeWld5qmjinh7Hz3w7dsAXX+iEPaUyhTZ8lbDOnXWm\nfhDohD2lMos2fJWwTp3gs8+C/VG5QZDIhD2llP8FruHn5uZ6XYKvlSefzp3txL3ly10oyEeCNnYS\nmbAHwcsnEZqNM83HHYFr+HpFJ2flyaddO6haNXjH8YM2dpYsSez4fdDySYRm40zzcYfO0lcV0rGj\n3fp76imvK1GpsGMH1KsHkybBlVd6XY1SwaSz9JUv6BX3MlvRhD2doa9U5tCGrypEJ+5ltkWL4KCD\noHVrrytRSiVL4Br+vHnzvC7B18qbT+fOsH8/LFuW4oJ8JEhjZ9Ei+zMu74Q9CFY+idJsnGk+7ghc\nwx87dqzXJfhaefNp1w6qVYOPP05xQT4SpLGzaBGcfHJijwlSPonSbJxpPu4I3KS9wsJCcnJyvC7D\ntxLJ5+ST7aVXn302xUX5RFDGzrffQuPG8MorcPHF5X9cUPKpCM3GmeYTm07aqyQdVM4SyadLF/jo\noxQW4zNBGTuLFtl/u3RJ7HFByaciNBtnmo87AtfwVfJ06QJr1sC2bV5XopJp0SI4/HA44givK1FK\nJZM2fFUDuLsFAAAgAElEQVRhRcd4i7YIVWYoOn4v4nUlSqlkClzDHzp0qNcl+Foi+RxzDBx8cHAa\nfhDGzoEDFZuwB8HIp6I0G2eajzsC1/CbNm3qdQm+lkg+IrYxBOU4fhDGzhdfwI8/Jn78HoKRT0Vp\nNs40H3cEbpa+Sq4RI2DCBNi8WXcBZ4JnnrGX0t2+HerW9boapYJNZ+krX+nSBbZuhbVrva5EJcNH\nH9mPw9Vmr1TmqVTDF5E7ReSAiPwrWQWp9KIT9zLLokUV252vlPK/Cjd8ETkJuBZIq4urrl692usS\nfC3RfOrXh6OOCsZx/EwfO7t22UslV2TCHmR+PpWh2TjTfNxRoYYvIgcBzwHXANuTWlGKDRs2zOsS\nfK0i+QTlAjyZPnaWLoW9eyve8DM9n8rQbJxpPu6o6Bb+BOANY8ysZBbjhvHjx3tdgq9VJJ8uXezH\nqe7Zk4KCfCTTx86iRVCjBrRvX7HHZ3o+laHZONN83JFwwxeRS4EOwJ3JLyf19PQPZxXJp0sX2L0b\nli9PQUE+kuljZ9EiOPFEqF69Yo/P9HwqQ7Nxpvm4I6GGLyJHAg8BVxhj9qamJJVuOnSwn5wXhN36\nmezDDyu+O18p5X+JbuF3Ag4DlojIXhHZC5wO3CQie0Rin4nds2dPQqFQiVvXrl2ZOnVqifvNmDGD\nUChU6vEDBw5k4sSJJZbl5+cTCoXYsmVLieUjRoxgzJgxJZYVFBQQCoVKTQwZN25cqSs8FRYWEgqF\nSn0+c15eHrm5uaVq69u3b+BfR82acMwx+Ywend6vAzLj51GR13HuuSG++mo13bql9+vIlJ+Hvo7g\nvY68vLzi3tioUSNCoRBDhgwp9ZhKMcaU+wbUAtpE3RYBk4HWMe7fETBLliwxfjF69GivS/C1iuYz\ncKAxrVoluRifyeSxM3WqMWBMQUHF15HJ+VSWZuNM84ltyZIlBjBAR5NAr453S2gL3xjzizFmZeQN\n+AXYaoxZlaw3IalUWFjodQm+VtF8una1l2XdujXJBflIJo+dBQvsp+M1aVLxdWRyPpWl2TjTfNxR\n6UvrisgsYKkx5pYY39NL6wbEunX2fPw33oBevbyuRiXqtNPsR+K+9JLXlSilivju0rrGmLNiNXsV\nLM2b24Yxf77XlahE7dkDixdT4vi9Uirz6LX0VVKI2IaxYIHXlahELV1qr7KnDV+pzBa4hh89K1OV\nVJl8unWz53LvzdATNjN17CxYYM+06NChcuvJ1HySQbNxpvm4I3ANv3///l6X4GuVyeeUU+yW4ief\nJLEgH8nUsbNgAZx0UsUvuFMkU/NJBs3GmebjjsA1/JEjR3pdgq9VJp8TT7Rbipm6Wz8Tx44xdt5F\n166VX1cm5pMsmo0zzccdgWv4eraAs8rkU7263VLM1Il7mTh2vv4a/ve/5By/z8R8kkWzcab5uCNw\nDV+lVtHEvUqe7alcsnCh/TcZW/hKKX/Thq+S6pRT7Bbjhg1eV6LKY8ECaNkSGjTwuhKlVKoFruFH\nX1NZlVTZfIq2FDPxOH4mjp0FC5J3Ol4m5pMsmo0zzccdgWv4+fmVvlhRRqtsPvXrQ6tWmdnwM23s\n/PKLPaMiWbvzMy2fZNJsnGk+7qj0pXUdV66X1g2k/v1tI8nU0/MyxZw5cMYZsGwZtG/vdTVKqWi+\nu7SuUtG6dYPly+Gnn7yuRDmZNw/q1IG2bb2uRCnlBm34KulOOQUOHICPPvK6EuVk7lw49VSoUsXr\nSpRSbtCGr5Lu2GPh4IPtFqTyp3377DyL7t29rkQp5ZbANfxQKOR1Cb6WjHyysuzHrX7wQRIK8pFM\nGjtLl8LPP9ufU7JkUj7Jptk403zcEbiGP2jQIK9L8LVk5dO9u72oy549SVmdL2TS2Jk7114GuXPn\n5K0zk/JJNs3GmebjDp2lr1Ji8eJfL7OrH7vqPxdeCD/+CLNmeV2JUioenaWv0kKHDlC7tj31S/nL\ngQP2cIsev1cqWLThq5SoWtXO1p871+tKVLRVq2DbNm34SgVN4Br+1KlTvS7B15KZT/fudqb+vn1J\nW6WnMmXszJ1r35D95jfJXW+m5JMKmo0zzccdgWv4eXl5Xpfga8nM5/TT7UzwpUuTtkpPZcrYmTvX\nTtbLyUnuejMln1TQbJxpPu7QSXsqZfbsgXr14N574dZbva5Ggf3Y4iOPhCuugDFjvK5GKeVEJ+2p\ntFG9uv1gFj2O7x9r19qPL9bj90oFjzZ8lVLdu9sZ4QcOeF2JAvuzELETKpVSwaINX6XU6afDDz/A\np596XYkCu7elfXt7qEUpFSyBa/i5ublel+Bryc6nSxe7az8TzsfPhLEzZ05yL6cbKRPySRXNxpnm\n447ANfwePXp4XYKvJTuf7Gw4+eTMaPjpPnY2bLDH8M88MzXrT/d8Ukmzcab5uENn6auUGz4c/vMf\n2LTJHj9W3pg0Cfr3hy1b4JBDvK5GKVUWnaWv0s4ZZ8DmzfDZZ15XEmyzZsGJJ2qzVyqotOGrlDvl\nFHsc/733vK4kuIyxDf+ss7yuRCnllcA1/Hnz5nldgq+lIp+cHPuJeene8NN57KxZAxs3prbhp3M+\nqabZONN83BG4hj927FivS/C1VOVzzjkwe3Z6X1c/ncfOrFn2+vmnnpq650jnfFJNs3Gm+bgjcJP2\nCgsLyUn2RcQzSKry+fBDe9W9BQvsv+koncdOnz72Cnvz56fuOdI5n1TTbJxpPrHppL1K0kHlLFX5\ndO4Mdeqk9279dB07Bw7A+++n/vh9uubjBs3GmebjjsA1fOWNqlXtVffSueGnqxUrYOtWnbCnVNBp\nw1euOeccu0u/sNDrSoJl1iyoUSN9D6UopZIjcA1/6NChXpfga6nM5+yz7UfmpvI4ciql69iZNcue\nGlmzZmqfJ13zcYNm40zzcUfgGn7Tpk29LsHXUplPmzbQqBG8+27KniKl0nHs7NtnL2vsxu78dMzH\nLZqNM83HHYGbpa+8dfnl8PnnsHix15UEw/z59lS8Dz+0H2SklEofOktfpbVzzoH8fNi2zetKgmH6\ndDj4YHuWhFIq2LThK1edfba9zOvs2V5XEgzTp9s3WVWqeF2JUsprgWv4q1ev9roEX0t1Pk2bwjHH\nwMyZKX2alEi3sbNtG3z8MZx7rjvPl275uEmzcab5uCOhhi8i14vIMhHZEb4tEJHzUlVcKgwbNszr\nEnzNjXzOPRfeecdu6aeTdBs7775rM3ar4adbPm7SbJxpPu5IdAv/a+B2oFP4Ngv4r4i0TnZhqTJ+\n/HivS/A1N/I57zxYv95+oEs6SbexM326PTPiyCPdeb50y8dNmo0zzccdCTV8Y8ybxph3jDFfhm93\nAz8Dv0lNecmnp384cyOfM86wH5f79tspf6qkSqexY4xt+G5t3UN65eM2zcaZ5uOOCh/DF5EsEbkU\nyAEWJq8klelq1YLu3e1ufZUaK1faj8N1s+Erpfwt4YYvIseLyE/AbuAR4CJjjM64UAn53e/sTP2d\nO72uJDPNmGEvp3vaaV5XopTyi4ps4a8GTgC6AI8Cz4jIcU4P6NmzJ6FQqMSta9euTJ06tcT9ZsyY\nQSgUKvX4gQMHMnHixBLL8vPzCYVCbNmypcTyESNGMGbMmBLLCgoKCIVCrF69usT3xo0bV+qSjoWF\nhYRCIebNm1dieV5eHrm5uaVq69u3ryevI1IyX0fkc6bydbzxRohdu1YzZ05qXkekZL2O3/3ud6Ve\nR6p/HhV9HS+/nE/t2iEKC90bV51jnOyfab8fFX0dgwcPzojXkaqfx7333psRr6MyP4+8vLzi3tio\nUSNCoRBDhgwp9ZhKMcZU6gbMBB6N872OgFmyZInxi3vuucfrEnzNrXwOHDCmSRNjbrrJladLinQZ\nO4WFxtSsacw//+nu86ZLPl7QbJxpPrEtWbLEAAboaCrZq40xlb+0roi8B2wwxvSP8T29tK6Ka8AA\nmDsX9BTc5Jo+3Z4JsWIFHH+819UopSrK00vrish9InKqiDQLH8u/HzgdeK6yhajg+d3v7HX1163z\nupLMMm0aNGsGbdt6XYlSyk8SPYbfEHgGexz/Xey5+D2MMbOSXZjKfGedBVWr6mz9ZDLGNvxevUDE\n62qUUn6S6Hn41xhjjjLGZBtjGhlj0q7ZR0/SUCW5mU/dutCtW/qcj58OY2flSntRo1693H/udMjH\nK5qNM83HHYG7ln7//qWmGqgIbufTq5e9BGxhoatPWyHpMHamTYOcHHtxI7elQz5e0WycaT7uCFzD\nHzlypNcl+Jrb+YRC9lz8995z9WkrJB3GzrRp8NvfQs2a7j93OuTjFc3GmebjjsA1fD1bwJnb+Rx7\nLLRqBa+/7urTVojfx87WrbBggTe788H/+XhJs3Gm+bgjcA1f+U8oBG+8AQcOeF1JenvnHZthz55e\nV6KU8iNt+MpzoRBs2mQ/u11V3LRp0KkTNG7sdSVKKT8KXMOPvsSiKsmLfLp2hUMP9f9ufT+Pnb17\n7Ra+V7vzwd/5eE2zcab5uCNwDT8/v9IXK8poXuRTtSqcf77dre9nfh478+fD9u3eNnw/5+M1zcaZ\n5uOOSl9a13HlemldVU6vvgqXXAJr10KLFl5Xk35uugleeQW+/hqyAvc2XqnM5OmldZVKlR49oHp1\n/2/l+5ExMGUK9O6tzV4pFZ/+eVC+ULu2vdTuf//rdSXpZ/Fi+OYb2/CVUioebfjKNy64AObMAb3K\nZmJefdVOejztNK8rUUr5WeAafigU8roEX/Myn4susrun/bqV78exY4xt+BdeaCc/esmP+fiFZuNM\n83FH4Br+oEGDvC7B17zMp2FD6N4dXn7ZsxIc+XHsfPopfPmlP3bn+zEfv9BsnGk+7tBZ+spXJkyA\nm2+2F+I55BCvq/G/UaPgX/+C77+HGjW8rkYplUw6S19ltN69Yf9+/+7W95tXX7Xn3muzV0qVRRu+\n8pXDD4dTT7XnlCtna9bAihX+2J2vlPK/wDX8qVOnel2Cr/khnz59YOZMe+U4P/FDNpGmTIHsbDjv\nPK8rsfyWj59oNs40H3cEruHn5eV5XYKv+SGf3r3tteH9dm19P2QTKS/P7s6vVcvrSiy/5eMnmo0z\nzccdOmlP+dKpp9pJe35r+n6xahW0aWO38i+6yOtqlFKpoJP2VCBccglMn+6/3fp+kZcHderA737n\ndSVKqXShDV/50h/+APv26eS9WIyxDb93b6hZ0+tqlFLpQhu+8qXGje219f/v/7yuxH+WLLEX27ns\nMq8rUUqlk8A1/NzcXK9L8DU/5XPFFTB7NhQUeF2J5Zds8vKgQQM480yvKynJL/n4kWbjTPNxR+Aa\nfo8ePbwuwdf8lM9FF9nTzvwygdcP2ezfDy+8YA95eH3t/Gh+yMevNBtnmo87dJa+8rVLL4WVK2H5\ncq8r8Yf337eHOubPh27dvK5GKZVKOktfBcoVV9iryWnDtyZNgpYtoWtXrytRSqUbbfjK1849137W\n+3PPeV2J9376yZ61cNVVIOJ1NUqpdBO4hj9v3jyvS/A1v+VTrZrdrf/cc/Y0PS95nc3LL8POnfCn\nP3laRlxe5+Nnmo0zzccdgWv4Y8eO9boEX/NjPldfDd9+C++8420dXmfz9NNwzjnQpImnZcTldT5+\nptk403zcEbhJe4WFheTk5Hhdhm/5NZ+OHaFpU/DyMza8zGbNGmjVCp5/Hvr186SEMvl17PiBZuNM\n84lNJ+1Vkg4qZ37N55prYNo0u6XvFS+zmTwZ6taFCy/0rIQy+XXs+IFm40zzcUfgGr5KT5ddZo/n\nP/OM15W4b/9+2/AvvdRel0AppSpCG75KC/Xq2Q/UefJJey35IHnrLfjmGzuXQSmlKipwDX/o0KFe\nl+Brfs7nmmvsNeTnzvXm+b3K5tFHoXNnOOkkT56+3Pw8drym2TjTfNwRuIbftGlTr0vwNT/n0707\nHHMMPP64N8/vRTZffWXPTrjhBtefOmF+Hjte02ycaT7uCNwsfZXeHnoIhg6FDRvsJ+plumHD4D//\ngY0bQec1KRUsOktfBVpuLtSo4d1Wvpt27YKnnrKvWZu9UqqytOGrtFK3rr3S3OOPw549XleTWi+/\nDFu3wvXXe12JUioTBK7hr1692usSfC0d8hk0CDZtsg3RTW5mYwxMmABnn20vuJMO0mHseEWzcab5\nuCNwDX/YsGFel+Br6ZBPmza2EY4b5+7zupnNggXw0Udw882uPWWlpcPY8Ypm40zzcUdCDV9E7hSR\nRSLyo4hsEpHXRCRNtj+s8ePHe12Cr6VLPjfeaBviRx+595xuZvPAA3DssdCzp2tPWWnpMna8oNk4\n03zckegW/mnAOKALcA5QDZghImlz/S89/cNZuuTTq5c9Rc/Nz9xwK5s1a+xnBtx6K2Sl0T64dBk7\nXtBsnGk+7qiayJ2NMSW2N0TkKuB7oBOgn2+oXFOlij1lbcAAWL0ajjvO64qS56GHoH59+OMfva5E\nKZVJKrv9UA8wwLYk1KJUQv74Rzj8cHe38lNtyxb7MbgDB0LNml5Xo5TKJBVu+CIiwEPAPGPMyuSV\nlFpjxozxugRfS6d8atSAW26BZ5+Fr79O/fO5kc2//mV34w8cmPKnSrp0Gjtu02ycaT7uqMwW/iNA\nG+DSJNXiisLCQq9L8LV0y2fAAKhd2zbKVEt1Ntu22TMPBg60u/TTTbqNHTdpNs40H3dUqOGLyHig\nJ3CGMabMTyjv2bMnoVCoxK1r165MnTq1xP1mzJhBKBQq9fiBAwcyceLEEsvy8/MJhUJs2bKlxPIR\nI0aUerdYUFBAKBRi9erVjBo1qnj5uHHjSn1oQ2FhIaFQiHnzSk5JyMvLIzc3t1Rtffv29eR1RErm\n64jMJx1eR+3a9rz8J56A779P7c8j+jO7k/3zeOgh+1G4t96anuOqoKCgVG3p+DpS8Xt+wQUXZMTr\nSNXP4/bbb8+I11GZn0deXl5xb2zUqBGhUIghQ4aUekxlJHwt/XCzvwA43Riztoz76rX0Vcpt2wYt\nWkD//vDgg15XUzE//ADNm9tPBHzgAa+rUUr5gafX0heRR4DLgcuAX0SkYfim04uUZw45xG4VP/qo\nO8fyU+HBB+2lgvVTQpVSqZLoLv3rgTrAbOB/Ebc/JLes1InehaNKStd8hgyxx/LvvTd1z5GqbL79\n1s5BuPFGaNQoJU/hinQdO27QbJxpPu5IqOEbY7KMMVVi3J5JVYHJ1r9/f69L8LV0zad2bbjjDvvp\ncmvWpOY5UpXNqFFQvTrceWdKVu+adB07btBsnGk+7kij63glx8iRI70uwdfSOZ8bbrBbyLffnpr1\npyKbzz+HJ5+E4cPh4IOTvnpXpfPYSTXNxpnm446EJ+0ltHKdtKdc9vzzcPnl8O679gN2/O6ii+CT\nT+zVAvVCO0qpSJ5O2lPK7/r1g27d7KfM7dvndTXO3nnHXjP//vu12SulUk8bvsooIvDww/DZZ/D4\n415XE9+uXfb6AWeeCZem1aWrlFLpKnANP/oCDKqkTMinUyd7Tv7w4fC//yVvvcnMZuxYKCiACRPs\nm5RMkAljJ1U0G2eajzsS+rS8TJCfn8/VV1/tdRm+lSn5jB0L06bZiXyvvZacppqsbNasgb//HW67\nDVq3rnxdfpGuY6egoCDlp4VNnz6dE088MaXPkc6CnE/9+vVd+3hgnbSnMtarr8Ill8CLL8IffHKl\niH37oHt3exngZcugVi2vKwq2goICWrdurddyV57Jyclh1apVMZt+siftBW4LXwXHxRfb26BBcMYZ\n0KCB1xXZPQ8ffQQffKDN3g+2bNlCYWEhzz33HK0zaXeLSgurVq3iiiuuYMuWLa5s5WvDVxltwgRo\n3x7++Ed4+2370bNeWboURo601wno1s27OlRprVu31r2QKuMFbtKeCpaGDeG552DmTLt17ZUdO+xh\nhTZtYMQI7+pQSgVX4Bp+rI9PVL/KxHx++1t72dq777a70iuqotkYA1ddZY/bv/IK1KhR8Rr8LBPH\njlKZJHC79AcNGuR1Cb6WqfmMGgXz50Pv3vDhh3D00Ymvo6LZ3HefvcDOf/8LLVtWaBVpId3HjjGG\nPfv3eF1GmapXqY5kyrmcylWBa/g9evTwugRfy9R8qla1s/a7doXzz4eFCxO/dn1Fspk8Gf7yF/uG\nI9M3gNN97OzZv4ea9/n/koe7hu+iRtUM3U2kUipwu/RVcB16KLz5JmzeDD17wo8/pvb53nwTrrkG\nrr3WNn2llPKSNnwVKMccAzNm2A+rOe+81DX9116zH4zTqxc88kjmXE1PBdfIkSPJSsFpLs2bN9eP\nx3VJ4Br+1KlTvS7B14KQT6dOtumvXGkvgvP11+V7XHmzmTQJ+vSxDf+ll+zhhCAIwtgJMhGp8NyB\nhQsXMmrUKH6M8Q47KytL5yS4JHANPy8vz+sSfC0o+Zx0EsybBz/8AF262Al9ZSkrm717YfBgyM21\n1/J//nmoVi1JBaeBoIwdlbgFCxbw17/+le3bt5f63ueff84TTzzhQVXBE7iG/+KLL3pdgq8FKZ/j\nj7dXvTvqKLulf9ddsHt3/Pv/t91/437vk0/sG4dHH7W78B9/HKpUSUHRPhaksaMS43QJ92rVqlEl\naL8sHglcw1cqUqNGMHs23Hsv/OMfcOyx8Mwz9pr30XbvL/1uoKAA/vxnu8dg/35YsMB+rXsolV/8\n/PPP3HzzzbRo0YKaNWvSsGFDevTowdKlS4vv8/LLL9O5c2dycnI47LDD+OMf/8j/yvioyQ0bNpCV\nlcUzzzxT6ntZWVn89a9/BWDUqFEMGzYMsMfrs7KyqFKlCgUFBcXLoo/hr1u3jj59+nDooYdSq1Yt\nunbtyltvvVXiPnPmzCErK4uXX36Z++67jyZNmpCdnc0555zDV199VeK+X375JRdffDGHH3442dnZ\nNGnShH79+vHTTz+VM8XMEJCji0rFV7Wq3bq/+GL7kbpXXgl33AGXXw49esCJJ9oZ/mB3269bZy/g\nM2UKTJ8O9erB/ffDzTcHaxe+Sg/XXXcdU6ZM4cYbb6R169Zs3bqV+fPns2rVKjp06MCkSZPo378/\nXbp0YfTo0WzatImHHnqIBQsW8Mknn1CnTp1KPX/v3r354osveOGFF/j3v//NoeFfpsMOOwyg1PH7\n77//nq5du7Jr1y5uuukmDjnkECZPnszvf/97pkyZwgUXXFDi/qNHj6ZKlSoMHTqUHTt2MGbMGK64\n4goWLlwIwN69e+nRowd79+5l8ODBNGrUiI0bNzJt2jS2b99O7dq1K/X60ok2fKXCjj3WXglv+XJ4\n4gl49ln45z/t97KzgWFQsyYcOGC34Lt0gXHj7HX6DzrI09KViuutt97i2muvZWzEtaVvu+02APbt\n28cdd9xB+/btmTNnDtWrVwfglFNOoVevXjz44IOMqOS1oNu1a0fHjh154YUXuOCCC8r8kJj777+f\nzZs3M2/ePLp27QrANddcQ/v27bnllltKNfzdu3ezbNmy4sMC9erV4+abb2blypW0adOGlStXsn79\nel599VUuuuii4sfdfffdlXpd6Shwu/Rzc3O9LsHXNB/7YTvjx8P2QTVhpMBIYeftAv+FA/fYr80I\n4ZPza/LnP2uzL6Jjx5/q1avHokWL+Pbbb0t9b/HixXz//ffccMMNxc0eoGfPnhx33HG8+eabbpYK\nwNtvv83JJ59c3OwBatWqxYABA1i/fj0rV64scf/+/fuXmANw2mmnYYxh7dq1ANStWxeAd955h507\nd7rwCvwrcA0/3a8Glmqaz6923b0LM8IU3ziaEl/vunuX1yX6io4dfxo7diwrVqygSZMmdOnShVGj\nRrFu3TrAHocXEVq1alXqcccddxwbNmxwu1w2bNjAscceW2p50ccXR9fUpEmTEl8fHL6E5g8//ADY\nOQK33norTz75JPXr1+e8887jkUceiXmKYKYLXMPv16+f1yX4mubjoJ3XBfibjh1/6tOnD2vXrmX8\n+PEcccQR/POf/6Rt27ZMnz7dcfZ8WeKdO3/gwIEKr7Mi4s3wj3xt//jHP1i+fDnDhw9n165dDB48\nmOOPP77MiYmZJnANXymlgqZhw4Zcf/31TJkyhXXr1nHooYdy33330bx5c4wxfP7556Ue8/nnn9Os\nWbO46yzako4+tz7WXoFELqzTrFmzmPWsWrWq+PsV0bZtW+666y5mz57NvHnz+Oabb3jssccqtK50\npQ1fqXKqUUU/sESllwMHDpTadV2/fn0aN27M7t27Oemkk2jQoAGPPfYYe/fuLb7P22+/zapVq+jV\nq1fcddeuXZv69eszd+7cEsvHjx9fqsHXqlULKP3mIJaePXuyaNEiPvroo+Jlv/zyC0888QQtWrSg\nTZs2Za4j0k8//cT+/ftLLGvbti1ZWVnsdrrwRgYK3Cz9efPmceqpp3pdhm9pPvG9e8a7Xpfgazp2\n/Oenn37iyCOP5JJLLuGEE07goIMOYubMmSxevJh//etfVKlShTFjxtC/f3+6d+9Ov379+O6773j4\n4Yc56qijuPnmmx3Xf8011zB69GiuvfZaOnfuzNy5c1mzZk2pQwWdOnXCGMNdd93FpZdeSrVq1QiF\nQmRnZ5da5x133EFeXh7nnXcegwcP5pBDDmHSpEls2LCBKVOmJJzBrFmzGDRoEH369KFVq1bs27eP\nZ555hqpVq3LxxRcnvL50FriGP3bsWP2j5EDziU+zcab5+E9OTg4DBw5kxowZvPbaaxw4cICWLVvy\n6KOPMmDAAACuvPJKatWqxejRo7njjjuoVasWF198MaNHjy51Dn70lvs999zDli1beOWVV3j55Zfp\n2bMnb7/9Ng0aNChx386dO/O3v/2Nxx57jOnTp3PgwAHWrVtH06ZNS12jv0GDBixcuJDbb7+d8ePH\ns2vXLtq3b8+0adM477zzHOuJtfyEE07gvPPOY9q0aWzcuJGcnBxOOOEE3nnnHU4++eSKBZumpDKT\nNoTZCFkAABGySURBVMpcuUhHYMmSJUvo2LFjyp4nEYWFheTk5Hhdhm9pPvFpNs7SMZ/8/Hw6derE\nkiVLaNu+LTXvq+l1SWXaNXwXNarq4aVMEDn+YvXIou8DnYwx+ZV9vsAdw0+3P0hu03zi02ycaT5K\n+VvgdukrpVQs1atUZ9dw/19boXqV6mXfSakYtOErpRT2uK/uKleZLHC79IcOHep1Cb6m+cSn2TjT\nfJTyt8A1/LI+uCHoNJ/4NBtnmo9S/ha4hn/jjTd6XYKvaT7xaTbONB+l/C1wDV8ppZQKIm34Siml\nVAAEruGvXr3a6xJ8TfOJT7Nxpvko5W+Ba/jDhg3zugRf03zi02ycpXs+xhh2797t+1sqr46qMlvg\nzsMfP3681yX4muYTn2bjLN3z2bNnDzVrpsGldXftokYNvV6ASlzgtvD11CFnmk98mo0zzcd/Jk2a\nRFZWFgUFBQk/ds6cOWRlZZX6+NtYNmzYQFZWFs8884yrNarEJNzwReQ0EXldRDaKyAERCaWiMKWU\nUpUT/Ul0FXl8pLy8PP7973+X676JPEdlalTlV5Fd+rWApcBTwKvJLUcppVSy/OlPf6Jfv35Ur574\n9fdPP/10du7cWeKxzz//PJ999hk33XRTifs2a9aMnTt3Uq1atUrXrFIn4S18Y8w7xph7jDFTgbR7\nWzZmzBivS/A1zSc+zcaZ5uM/IlKhZl8kkcdWr15dt9R9LnDH8AsLC70uwdc0n/g0G2eaj//EOj7e\nvHlzQqEQ8+fPp0uXLmRnZ3P00Ufz7LPPlnhs9DH8M888kzfffLP4eH1WVhZHHXUUEPsY/ooVK8jN\nzeXoo48mOzubww8/nKuvvppt27ZV+PW8/PLLtG3bluzsbNq3b8/UqVO56qqraNGiRdy6i8SbZ/D5\n559zySWXcOihh5Kdnc1JJ53EG2+8UeI++/btY9SoUbRq1Yrs7Gzq16/PaaedxnvvvVd8n02bNpGb\nm0uTJk2oWbMmjRs35sILL/TV3ITAzdIfNWqU1yX4muYTn2bjTPPxn1jHx0WENWvW0KdPH66++mqu\nuuoqnnrqKXJzc+ncuTOtW7cucd8id999Nzt27GDjxo089NBDGGM46KCD4j73zJkzWbduHf3796dR\no0Z89tlnPP7446xcuZKFCxcm/FrefPNNLr30Uk444QRGjx7NDz/8wNVXX80RRxwR8zWWx2effcap\np57KkUceyZ133kmtWrV46aWXuPDCC5kyZQoXXHABACNGjGD06NEMGDCAk046iR9//JHFixeTn5/P\n2WefDUDv3r1ZtWoVgwcPplmzZnz//ffMnDmTgoIC30xoDVzDV0qpoPviiy/44IMP6NatGwB9+vSh\nSZMmPP3004wdOzbmY84++2yOOOIItm/fTr9+/cp8joEDB3LLLbeUWNalSxcuu+wy5s+fzymnnJJQ\nzXfeeSdHHnkk8+fPJzs7u7im008/nebNmye0riI33XQTzZs35+OPP6ZqVdsO//znP3Pqqady++23\nFzf8t956i/PPP59HH3005np27NjBwoUL+ec//1niNd9+++0VqitVXNml37NnT0KhUIlb165dmTp1\naon7zZgxg1Co9KT/gQMHMnHixBLL8vPzCYVCbNmypcTyESNGlDqWWFBQQCgUKnUlsHHjxpX6SM/C\nwkJCoRDz5s0rsTwvL4/c3NxStfXt21dfh74OfR1p+jo++eSTUnUEQZs2bYqbPUD9+vU59thjWbt2\nbdKeI/JaAbt372br1q106dIFYwz5+fkJrevbb7/l008/5corryxu9gCnnXYa7dq1q1B9P/zwA++/\n/z59+vRhx44dbN26tfjWo0cP1qxZw7fffgtAvXr1+Oyzz/jyyy9jris7O5vq1asze/Zstm/fXqF6\n8vLyintjo0aNCIVCDBkypELrissYU+EbcAAIOXy/I2CWLFli/GLz5s1el+Brmk98mo2zdMxnyZIl\npuhv1K5duwzg+9uuXbvK/fomTZpksrKyzIYNG4qXNW/e3PTs2bPUfc844wxz1llnFX89e/Zsk5WV\nZebMmVO8rFevXqZFixalHrt+/XojImby5MnFy7Zt22YGDx5sGjZsaESk+JaVlWXuvfdexxqjLVy4\n0IiImTRpUqnv9e7du0RNseqOVeOiRYuK64msL7LOpUuXGmOMmTt3rjnkkEOMiJh27dqZYcOGmeXL\nl5dY/0MPPWSqVq1qqlevbrp3727Gjh1rvvvuu7ivyZiS48/p+0BHU4leXXSryHn4tUTkBBHpEF50\nVPjrJomuywv9+/f3ugRf03zi02ycaT7po0qVKjGXmyRetrdPnz5MnDiRG264gddee42ZM2cyffp0\njDEcOHAgac8TLd7x+/3795f4uqiG2267jXfffbfUbebMmbRs2RKwexK++uornn76adq1a8eTTz5J\nx44deeqpp4rXd9NNN/HFF18wevRosrOzueeee2jdujXLli1L0StNXEWO4XcG3ufXd5wPhJdPBnz/\nGz9y5EivS/A1zSc+zcaZ5pP5yjsZbvv27cyaNYt7772X4cOHFy+Pt0u8LM2aNYv7+OhlBx98MMaY\nUrvW169fX+LrojMMqlWrxllnnVVmDfXq1ePKK6/kyiuvpLCwkNNOO42RI0eWeKPbokULhgwZwpAh\nQ/jqq6844YQTeOCBByp0BcJUqMh5+HOMMVnGmCpRN983e4COHTt6XYKvaT7xaTbONJ/MV6tWLXbs\n2FHm/Yr2IERvyT/44IMVOlf/8MMP5/jjj+eZZ54pcfrnnDlzWLFiRYn7NmvWjCpVqpQ6Le+RRx4p\n8dyHHXYYZ5xxBo8//jjfffddqeeMnDcSfSphTk4OLVu2ZPfu3QDs3Lmz+P9FWrRoQe3atUst95LO\n0ldKKRVT9C7+Tp068dJLL3Hrrbdy0kkncdBBB9GrV69Sj6tduzbdu3dn7Nix7NmzhyOOOIIZM2aw\nbt26Ch82+Pvf/86FF15It27dyM3NZdu2bUyYMIF27drx888/F9+vTp069OnTh4cffhiAo48+mjfe\neKPUxE+ACRMmFE/8u/baaznqqKPYtGkTCxcuZOPGjcWTOtu0acMZZ5xBp06dOOSQQ/j444955ZVX\nGDx4MGDPejj77LP5wx/+QJs2bahatSpTpkzh+++/L9cZDW7Rhq+UUgHidO36ss5nv+GGG1i2bBmT\nJk3ioYceolmzZsUNP9Z192+88UYeeeQRjDGce+65vPPOOzRu3LhCW/m9evUiLy+PkSNHcscdd9Cq\nVSsmT57MpEmTWLlyZYn7jhs3jn379vH4449To0YN+vbtywMPPMDxxx9f4n6tW7dm8eLFjBo1ismT\nJ7N161YaNGjAiSeeyD333FN8v5tuuonXX3+dmTNnsnv3bpo1a8bf//53brvtNgCaNGnCZZddxnvv\nvcdzzz1H1apVOe6443j55Ze58MILE36tKZOMmX/xbvhwlv6TTz7pdQm+pvnEp9k4S8d8Mn2WfhB0\n6NDB9OjRw+syKsT3s/TTXaLnfwaN5hOfZuNM81GptH///lJzAmbPns2yZcs488wzPaoqvQRul/6E\nCRO8LsHXNJ/4NBtnmo9KpW+++Ybf/va3XH755TRu3JhVq1bx+OOP07hxY6677jqvy0sLgWv4SikV\nS/Xq1dm1a5fXZZSpMp9+l84OPvhgOnXqxMSJE9m8eTO1atXi97//Pffffz8HH3yw1+WlBW34SimF\nnXQWeTlY5S916tQhLy/P6zLSWuCO4SullFJBFLiGH+tDO9SvNJ/4NBtnmo9S/ha4hj9o0CCvS/A1\nzSc+zcaZ5qOUvwWu4ffo0cPrEnxN84lPs3Gm+SjlbzppTykVeKtWrfK6BBVAbo87bfhKqcCqX78+\nOTk5XHHFFV6XogIqJyeH+vXru/JcgWv4U6dO9de1jX1G84lPs3GWjvk0bdqUVatWxfxglWR6//33\n9WpwDoKcT/369WnatKkrzyWmgp9cVK6Vi3QElixZssQ3H53ZtWtXFi5c6HUZvqX5xKfZONN84tNs\nnGk+seXn59OpUyeATsaYSl+7OnCT9g477DCvS/A1zSc+zcaZ5hOfZuNM83FH4Bq+UkopFUTa8JVS\nSqkA0IavlFJKBUCqZ+nXBH+d47po0SL93G4Hmk98mo0zzSc+zcaZ5hNbRO+smYz1pXqW/mXA/6Xs\nCZRSSqnMd7kx5vnKriTVDf9Q4FxgPeD/D5pWSiml/KMm0ByYbozZWtmVpbThK6WUUsofdNKeUkop\nFQDa8JVSSqkA0IavlFJKBYA2fKWUUioAtOErpZRSARDIhi8izUTkSRFZKyKFIrJGREaKSDWva/OK\niAwUkXUislNEPhSRk7yuyQ9E5E4RWSQiP4rIJhF5TURaeV2XH4WzOiAi//K6Fr8QkcYi8qyIbAn/\nrVkW/hTRQBORLBG5N+Jv8JcicrfXdXlFRE4TkddFZGP4dygU4z5/FZH/hfOaKSItE32eQDZ84DhA\ngGuBNsAQ4HrgPi+L8oqI9AUeAEYAJwLLgOkiUt/TwvzhNGAc0AU4B6gGzBCRbE+r8pnwG8RrsWNH\nASJSD5gP7MZej6Q1cCvwg5d1+cQdwHXADdi/x8OAYSIyyNOqvFMLWAoMBEqdKy8itwODsJmdDPyC\n/RtdPZEn0fPww0TkNuB68//t3V1olmUcx/HvLw8CC/GgNns5KDEkIooMRCR3MDoRej3JOkgSK3oj\n6KAQE4JBDcusUR5FZUXU6iQHdWIryCLNZQVahCS1mDNKKFgEpr8Orntjb9ieg+26tuv/Odp9c+95\nfgfb9b9eue2We03znaQvgf22H22uBQwCPba3Zw1XmKYT9Buwzva+3HlKIOl8YAB4ANgGHLL9WN5U\n+UnqBtbY7sidpTSS+oBh2/eOu/c+8Lftu/Mly0/SGeBW23vG3RsCnrW9s7leApwANtruneln1zrC\nn85S4GTuEHOtWcZYBXw8es+pF7gXWJMrV8GWknrg1f2tnMXLQJ/t/txBCnMTcFBSb7Mc9LWkzblD\nFeILoFPSFQCSrgHWAh9mTVUgSZcDy5jYRv8F7KfFNnq2X54zLzRrIQ8DNY5KLgAWkXqL450AVs59\nnHI1Mx8vAPtsH8mdpwSSNgDXAtfnzlKg5aRZjx2k5cLVQI+kf2y/lTVZft3AEuAHSadJg8+ttt/J\nG6tIy0iDjOna6GWtfNCCKviSngGeOMsjBq60/eO437kE+Ah41/arsxxxPhHTrCVVbhdpz8fa3EFK\nIOlSUgfoRtuncucp0DnAAdvbmutvJV1F6gTUXvDvAO4CNgBHSJ3GFyUN2X4za7L5o+U2ekEVfOA5\n4LX/eean0R8kXQz0k0Zs989msIL9DpwG2ifdb2Nqj7Jakl4C1gM32D6eO08hVgEXAgPN7Aek2aJ1\nzearc133JqHjwOR3g38P3J4hS2m2A0/bfq+5PizpMmALEAV/omFScW9nYpvcBhxq5YMWVMFv3iY0\nozcKNSP7fuArYNNs5iqZ7VOSBoBOYA+MTV13Aj05s5WiKfa3AB22f8mdpyB7gasn3XudVNS6Ky/2\nkHboT14WWwn8nCFLaRYzdXR6hthXNoXtY5KGSW3ydzC2aW81af/MjC2ogj9Tki4CPiW9tvdxoG10\ngGK7xlHt88DupvAfIB1TXExqvKsmaRdwJ3AzMCJpdCbkT9tVv/LZ9ghpOnaMpBHgD9uTR7Y12gl8\nLmkL0EtqoDeTji/Wrg/YKmkQOAxcR2p3XsmaKhNJ5wErSCN5gOXNRsaTtgdJS2dPSjpKqltdwK/A\nBy19T42dcEkbgcnr9SJtUF+UIVJ2kh4kdX7aSedBH7F9MG+q/JojMtP9k9xj+425zlM6Sf3AN3Es\nL5G0nrRBbQVwDNgRe4XGClwXcBtpanoIeBvosv1vzmw5SOoAPmFqW7Pb9qbmmaeA+0gnhT4DHrJ9\ntKXvqbHghxBCCLWJ9ZIQQgihAlHwQwghhApEwQ8hhBAqEAU/hBBCqEAU/BBCCKECUfBDCCGECkTB\nDyGEECoQBT+EEEKoQBT8EEIIoQJR8EMIIYQKRMEPIYQQKvAfLbB/0zkE5KcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114bdb198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_mins(f=non_convex_1,min_df=min_df,xmin=-2,xmax=9,title='Non-Convex example N.1 - initial guess = 3.9',\n",
    "          xguess=3.9,legend_pos='lower right')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The problem with such __function__ is the fact it is __not convex__. Hence, starting from initial guess \n",
    "x = 3.9 the above optimizers are able to find the global minimum. On the contrary, starting from initial guess x = 4.5 \n",
    "no optmizer is able to find the global minimum. \n",
    "\n",
    "### Basin Hopping and Simulated Annealing\n",
    "For solving this optimization problem we need something more than the above optmiziation methods. \n",
    "For instance, __simulated annealing__ is one of the most widely used algorithms for finding the global minimum \n",
    "of a multivariable function for different complex systems. In __scipy.optimize__ __simulated annealing__ has been \n",
    "replaced by __Basin Hopping__ that is a global optimization framework  \n",
    "particularly suited for multivariable multimodal optimization problems "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from scipy.optimize import basinhopping\n",
    "res = basinhopping(non_convex_1, 4.5, minimizer_kwargs={\"method\": \"BFGS\"},niter=200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 2.00000001])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res.x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that Basin Hopping is able to find the global minimum instead of previous methods. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More dimensions, contraints and bounds  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us consider the problem of minimizing the Rosenbrock function. \n",
    "This function (and its respective derivatives) is implemented in rosen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from scipy.optimize import minimize, rosen, rosen_der"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "      <th>min_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Powell</th>\n",
       "      <td>[1.0, 1.0, 1.0, 1.0, 1.0]</td>\n",
       "      <td>1.696763e-22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>L-BFGS-B</th>\n",
       "      <td>[0.999999569191, 0.999999146161, 0.99999833610...</td>\n",
       "      <td>1.504062e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BFGS</th>\n",
       "      <td>[0.999999245546, 0.999998516729, 0.99999706458...</td>\n",
       "      <td>4.582239e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CG</th>\n",
       "      <td>[0.999998258204, 0.99999652201, 0.99999303492,...</td>\n",
       "      <td>2.587047e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[0.999996799614, 0.999995480491, 0.99999335775...</td>\n",
       "      <td>2.434160e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TNC</th>\n",
       "      <td>[0.999971153807, 0.999943162109, 0.9998872987,...</td>\n",
       "      <td>6.736061e-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Nelder-Mead</th>\n",
       "      <td>[0.999101151259, 0.998209232287, 0.99646345959...</td>\n",
       "      <td>6.617482e-05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>COBYLA</th>\n",
       "      <td>[1.03328356885, 1.06781943615, 1.14060138059, ...</td>\n",
       "      <td>1.166637e-01</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                           min     min_value\n",
       "Method                                                                      \n",
       "Powell                               [1.0, 1.0, 1.0, 1.0, 1.0]  1.696763e-22\n",
       "L-BFGS-B     [0.999999569191, 0.999999146161, 0.99999833610...  1.504062e-11\n",
       "BFGS         [0.999999245546, 0.999998516729, 0.99999706458...  4.582239e-11\n",
       "CG           [0.999998258204, 0.99999652201, 0.99999303492,...  2.587047e-10\n",
       "SLSQP        [0.999996799614, 0.999995480491, 0.99999335775...  2.434160e-09\n",
       "TNC          [0.999971153807, 0.999943162109, 0.9998872987,...  6.736061e-08\n",
       "Nelder-Mead  [0.999101151259, 0.998209232287, 0.99646345959...  6.617482e-05\n",
       "COBYLA       [1.03328356885, 1.06781943615, 1.14060138059, ...  1.166637e-01"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mins = [spo.minimize(rosen,[1.3, 0.7, 0.8, 1.9, 1.2],method=om) for om in opt_methods_no_Jacobian]\n",
    "\n",
    "min_df = pd.DataFrame({'Method' : opt_methods_no_Jacobian,\n",
    " 'min' : [m.x for m in mins], \n",
    "  'min_value': [rosen(m.x) for m in mins]\n",
    "  }).set_index('Method')\n",
    "min_df.sort_values(by = 'min_value' , inplace=True)\n",
    "min_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that all optimizers provide close minima although __Powell__ makes a better job this time. \n",
    "Let's test __Basin Hopping__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>> basinhopping min: [ 0.99999967  0.9999993   0.99999876  0.99999751  0.99999518] - min_value:1.35681803995e-11\n"
     ]
    }
   ],
   "source": [
    "res = basinhopping(rosen, [1.3, 0.7, 0.8, 1.9, 1.2], minimizer_kwargs={\"method\": \"BFGS\"},niter=200)\n",
    "print(\">>> basinhopping min: \"+str(res.x)+\" - min_value:\"+str(rosen(res.x)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pretty close the other ones. \n",
    "Let's concatenate results. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "      <th>min_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Powell</th>\n",
       "      <td>[1.0, 1.0, 1.0, 1.0, 1.0]</td>\n",
       "      <td>1.696763e-22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Basin Hopping</th>\n",
       "      <td>[0.999999671333, 0.999999298452, 0.99999876249...</td>\n",
       "      <td>1.356818e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>L-BFGS-B</th>\n",
       "      <td>[0.999999569191, 0.999999146161, 0.99999833610...</td>\n",
       "      <td>1.504062e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BFGS</th>\n",
       "      <td>[0.999999245546, 0.999998516729, 0.99999706458...</td>\n",
       "      <td>4.582239e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CG</th>\n",
       "      <td>[0.999998258204, 0.99999652201, 0.99999303492,...</td>\n",
       "      <td>2.587047e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[0.999996799614, 0.999995480491, 0.99999335775...</td>\n",
       "      <td>2.434160e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TNC</th>\n",
       "      <td>[0.999971153807, 0.999943162109, 0.9998872987,...</td>\n",
       "      <td>6.736061e-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Nelder-Mead</th>\n",
       "      <td>[0.999101151259, 0.998209232287, 0.99646345959...</td>\n",
       "      <td>6.617482e-05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>COBYLA</th>\n",
       "      <td>[1.03328356885, 1.06781943615, 1.14060138059, ...</td>\n",
       "      <td>1.166637e-01</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                             min     min_value\n",
       "Method                                                                        \n",
       "Powell                                 [1.0, 1.0, 1.0, 1.0, 1.0]  1.696763e-22\n",
       "Basin Hopping  [0.999999671333, 0.999999298452, 0.99999876249...  1.356818e-11\n",
       "L-BFGS-B       [0.999999569191, 0.999999146161, 0.99999833610...  1.504062e-11\n",
       "BFGS           [0.999999245546, 0.999998516729, 0.99999706458...  4.582239e-11\n",
       "CG             [0.999998258204, 0.99999652201, 0.99999303492,...  2.587047e-10\n",
       "SLSQP          [0.999996799614, 0.999995480491, 0.99999335775...  2.434160e-09\n",
       "TNC            [0.999971153807, 0.999943162109, 0.9998872987,...  6.736061e-08\n",
       "Nelder-Mead    [0.999101151259, 0.998209232287, 0.99646345959...  6.617482e-05\n",
       "COBYLA         [1.03328356885, 1.06781943615, 1.14060138059, ...  1.166637e-01"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "min_df = pd.DataFrame({'Method' : opt_methods_no_Jacobian+['Basin Hopping'],\n",
    " 'min' : [m.x for m in mins]+[res.x], \n",
    "  'min_value': [rosen(m.x) for m in mins]+[rosen(res.x)]\n",
    "  }).set_index('Method')\n",
    "min_df.sort_values(by = 'min_value' , inplace=True)\n",
    "min_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bounds and contraints\n",
    "\n",
    "Let's say variables must be not negative. \n",
    "As there are methods (e.g. Nelder-Mead) that cannot handle constraints nor bounds we use a different approach. \n",
    "We modify the cost function so that if constraints or bounds don't hold a very large value is returned. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def my_rosen(x,out_value=10e10):\n",
    "    \n",
    "    ## no negative values \n",
    "    if np.any(x<0.0):\n",
    "        return out_value\n",
    "    \n",
    "    return rosen(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "      <th>min_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Powell</th>\n",
       "      <td>[1.0, 1.0, 1.0, 1.0, 1.0]</td>\n",
       "      <td>3.319898e-22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CG</th>\n",
       "      <td>[1.00000000075, 1.00000001398, 1.00000003705, ...</td>\n",
       "      <td>1.111023e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Basin Hopping</th>\n",
       "      <td>[1.00000000633, 1.00000004562, 1.00000011665, ...</td>\n",
       "      <td>6.196018e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>L-BFGS-B</th>\n",
       "      <td>[0.999999569191, 0.999999146161, 0.99999833610...</td>\n",
       "      <td>1.504062e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BFGS</th>\n",
       "      <td>[0.999999305001, 0.999998588025, 0.99999718981...</td>\n",
       "      <td>4.205330e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TNC</th>\n",
       "      <td>[0.999971153807, 0.999943162109, 0.9998872987,...</td>\n",
       "      <td>6.736061e-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[0.999981003037, 1.00000561327, 1.00001633397,...</td>\n",
       "      <td>2.008566e-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Nelder-Mead</th>\n",
       "      <td>[0.999101151259, 0.998209232287, 0.99646345959...</td>\n",
       "      <td>6.617482e-05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>COBYLA</th>\n",
       "      <td>[1.03328356885, 1.06781943615, 1.14060138059, ...</td>\n",
       "      <td>1.166637e-01</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                             min     min_value\n",
       "Method                                                                        \n",
       "Powell                                 [1.0, 1.0, 1.0, 1.0, 1.0]  3.319898e-22\n",
       "CG             [1.00000000075, 1.00000001398, 1.00000003705, ...  1.111023e-13\n",
       "Basin Hopping  [1.00000000633, 1.00000004562, 1.00000011665, ...  6.196018e-13\n",
       "L-BFGS-B       [0.999999569191, 0.999999146161, 0.99999833610...  1.504062e-11\n",
       "BFGS           [0.999999305001, 0.999998588025, 0.99999718981...  4.205330e-11\n",
       "TNC            [0.999971153807, 0.999943162109, 0.9998872987,...  6.736061e-08\n",
       "SLSQP          [0.999981003037, 1.00000561327, 1.00001633397,...  2.008566e-07\n",
       "Nelder-Mead    [0.999101151259, 0.998209232287, 0.99646345959...  6.617482e-05\n",
       "COBYLA         [1.03328356885, 1.06781943615, 1.14060138059, ...  1.166637e-01"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mins = [spo.minimize(my_rosen,[1.3, 0.7, 0.8, 1.9, 1.2],method=om) for om in opt_methods_no_Jacobian]\n",
    "res = basinhopping(my_rosen, [1.3, 0.7, 0.8, 1.9, 1.2], minimizer_kwargs={\"method\": \"BFGS\"},niter=200)\n",
    "min_df = pd.DataFrame({'Method' : opt_methods_no_Jacobian+['Basin Hopping'],\n",
    " 'min' : [m.x for m in mins]+[res.x], \n",
    "  'min_value': [rosen(m.x) for m in mins]+[rosen(res.x)]\n",
    "  }).set_index('Method')\n",
    "min_df.sort_values(by = 'min_value' , inplace=True)\n",
    "min_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We know that the global minima is (1,1,1,1) so we should not have sub-optimal solution requiring the constraints that \n",
    "the points must lie on the hyperplane identified by all vector components with same value, i.e. $x_1=x_2=x_3=x_4$ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def my_rosen(x,out_value=10e10,approx = 0.1):\n",
    "    \n",
    "    ## no negative values \n",
    "    if np.any(x<0.0):\n",
    "        return out_value\n",
    "    \n",
    "    ## x1 = x2 = x3 = x4 \n",
    "    if type(x) is list:\n",
    "        x = np.array(x)\n",
    "    if  np.absolute(x[0] - x[1])<approx and np.absolute(x[0] - x[2])<approx and np.absolute(x[0] - x[3])<approx and np.absolute(x[1] - x[2])<approx and np.absolute(x[1] - x[3])<approx and np.absolute(x[2] - x[3])<approx:\n",
    "        rosen(x)\n",
    "    else:\n",
    "        return out_value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "      <th>min_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Nelder-Mead</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CG</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BFGS</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>L-BFGS-B</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TNC</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>COBYLA</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Basin Hopping</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Powell</th>\n",
       "      <td>[3.88792896155, 3.28792896155, 3.38792896155, ...</td>\n",
       "      <td>51161.317214</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                             min     min_value\n",
       "Method                                                                        \n",
       "Nelder-Mead                            [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "CG                                     [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "BFGS                                   [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "L-BFGS-B                               [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "TNC                                    [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "COBYLA                                 [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "SLSQP                                  [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "Basin Hopping                          [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "Powell         [3.88792896155, 3.28792896155, 3.38792896155, ...  51161.317214"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mins = [spo.minimize(my_rosen,[1.3, 0.7, 0.8, 1.9, 1.2],method=om) for om in opt_methods_no_Jacobian]\n",
    "res = basinhopping(my_rosen, [1.3, 0.7, 0.8, 1.9, 1.2], minimizer_kwargs={\"method\": \"BFGS\"},niter=200)\n",
    "min_df = pd.DataFrame({'Method' : opt_methods_no_Jacobian+['Basin Hopping'],\n",
    " 'min' : [m.x for m in mins]+[res.x], \n",
    "  'min_value': [rosen(m.x) for m in mins]+[rosen(res.x)]\n",
    "  }).set_index('Method')\n",
    "min_df.sort_values(by = 'min_value' , inplace=True)\n",
    "min_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__We can see that no one handled this constraint properly__.\n",
    "\n",
    "We need to change approach restricting the list of methods to the ones able to handle constraints and bounds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "      <th>min_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>L-BFGS-B</th>\n",
       "      <td>[1.0, 1.0, 1.0, 1.0, 1.0]</td>\n",
       "      <td>0.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TNC</th>\n",
       "      <td>[1.0, 1.0, 0.999999984865, 0.999999955943, 0.9...</td>\n",
       "      <td>4.725746e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[0.999992213548, 0.999987923136, 0.99999852996...</td>\n",
       "      <td>1.599157e-07</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                        min     min_value\n",
       "Method                                                                   \n",
       "L-BFGS-B                          [1.0, 1.0, 1.0, 1.0, 1.0]  0.000000e+00\n",
       "TNC       [1.0, 1.0, 0.999999984865, 0.999999955943, 0.9...  4.725746e-14\n",
       "SLSQP     [0.999992213548, 0.999987923136, 0.99999852996...  1.599157e-07"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "opt_methods_no_Jacobian_bounds = ['SLSQP','L-BFGS-B','TNC']\n",
    "\n",
    "## handling bounds for SLSQP, L-BFGS-B, TNC\n",
    "bnds = ((0,1),(0,1),(0,1),(0,1),(0,1))\n",
    "mins = [spo.minimize(rosen,[1.3, 0.7, 0.8, 1.9, 1.2],method=om,bounds=bnds) for om in opt_methods_no_Jacobian_bounds]\n",
    "min_df = pd.DataFrame({'Method' : opt_methods_no_Jacobian_bounds,\n",
    " 'min' : [m.x for m in mins],\n",
    "  'min_value': [rosen(m.x) for m in mins]\n",
    "  }).set_index('Method')\n",
    "min_df.sort_values(by = 'min_value' , inplace=True)\n",
    "min_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "      <th>min_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[0.5, 0.7, 0.8, 0.7, 1.2]</td>\n",
       "      <td>81.1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                              min  min_value\n",
       "Method                                      \n",
       "SLSQP   [0.5, 0.7, 0.8, 0.7, 1.2]       81.1"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## handling constraints for SLSQP (the only that seems to support constraints)\n",
    "cons = ({'type': 'eq','fun' : lambda x: np.array([x[0]- x[1]]),'jac' : lambda x: np.array([1,-1,0,0,0])},\n",
    "        {'type': 'eq','fun' : lambda x: np.array([x[0]- x[2]]),'jac' : lambda x: np.array([1,0,-1,0,0])}, \n",
    "        {'type': 'eq','fun' : lambda x: np.array([x[0]- x[3]]),'jac' : lambda x: np.array([1,0,0,-1,0])},\n",
    "        {'type': 'eq','fun' : lambda x: np.array([x[0]- x[4]]),'jac' : lambda x: np.array([1,0,0,0,-1])},\n",
    "        {'type': 'eq','fun' : lambda x: np.array([x[1]- x[2]]),'jac' : lambda x: np.array([0,1,-1,0,0])},\n",
    "        {'type': 'eq','fun' : lambda x: np.array([x[1]- x[3]]),'jac' : lambda x: np.array([0,1,0,-1,0])},\n",
    "        {'type': 'eq','fun' : lambda x: np.array([x[1]- x[4]]),'jac' : lambda x: np.array([0,1,0,0,-1])},\n",
    "        {'type': 'eq','fun' : lambda x: np.array([x[2]- x[3]]),'jac' : lambda x: np.array([0,0,1,-1,0])},\n",
    "        {'type': 'eq','fun' : lambda x: np.array([x[2]- x[4]]),'jac' : lambda x: np.array([0,0,1,0,-1])},\n",
    "        {'type': 'eq','fun' : lambda x: np.array([x[3]- x[4]]),'jac' : lambda x: np.array([0,0,0,1,-1])})\n",
    "\n",
    "bnds = ((0,1),(0,1),(0,1),(0,1),(0,1))\n",
    "mins = spo.minimize(rosen,[0.5, 0.7, 0.8, 0.7, 1.2],method='SLSQP',bounds=bnds,constraints=cons) \n",
    "min_df = pd.DataFrame({'Method' : 'SLSQP',\n",
    " 'min' : [mins.x] ,\n",
    " 'min_value': rosen(mins.x)\n",
    "  }).set_index('Method')\n",
    "min_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a very interesting and disastrous results. __The constraints leads us to a sub-optimal solution even \n",
    "if we know the optimal solution satisfies such contraint__.\n",
    "Let's try to release the contrant transorming it formn equalities to 1 disequality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "      <th>min_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[0.961048879385, 0.923988082931, 0.85415761252...</td>\n",
       "      <td>0.101835</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                      min  min_value\n",
       "Method                                                              \n",
       "SLSQP   [0.961048879385, 0.923988082931, 0.85415761252...   0.101835"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cons = ({'type': 'ineq','fun' : lambda x: np.array(4-np.sum(np.array(x))),'jac' : lambda x: np.array([-1,-1,-1,-1,-1])})\n",
    "\n",
    "bnds = ((0,1),(0,1),(0,1),(0,1),(0,1))\n",
    "mins = spo.minimize(rosen,[0.5, 0.7, 0.8, 0.7, 1.2],method='SLSQP',bounds=bnds,constraints=cons) \n",
    "min_df = pd.DataFrame({'Method' : 'SLSQP',\n",
    " 'min' : [mins.x] ,\n",
    " 'min_value': rosen(mins.x)\n",
    "  }).set_index('Method')\n",
    "min_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Better ... even if not the optimal solution (1,1,1,1,1). This is pretty important for the use case of \n",
    "optimizing a portfolio. __We know that the sum of allocation percentages must be equal to 1, $\\sum{AP_i}=1$ \n",
    "Hence, it seems better to express such constraint as $\\sum{AP_i} \\leq 1$__\n",
    "\n",
    "We can see anyway that such result is much better than the one we get injecting such constraint \n",
    "directly inside the cost function as it often happens in literature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>min</th>\n",
       "      <th>min_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Method</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Nelder-Mead</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CG</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BFGS</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>L-BFGS-B</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TNC</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>COBYLA</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SLSQP</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Basin Hopping</th>\n",
       "      <td>[1.3, 0.7, 0.8, 1.9, 1.2]</td>\n",
       "      <td>848.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Powell</th>\n",
       "      <td>[3.88792896155, 3.28792896155, 3.38792896155, ...</td>\n",
       "      <td>51161.317214</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                             min     min_value\n",
       "Method                                                                        \n",
       "Nelder-Mead                            [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "CG                                     [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "BFGS                                   [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "L-BFGS-B                               [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "TNC                                    [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "COBYLA                                 [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "SLSQP                                  [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "Basin Hopping                          [1.3, 0.7, 0.8, 1.9, 1.2]    848.220000\n",
       "Powell         [3.88792896155, 3.28792896155, 3.38792896155, ...  51161.317214"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def my_rosen(x,out_value=10e10,approx = 0.1):\n",
    "    \n",
    "    ## no negative values \n",
    "    if np.any(x<0.0):\n",
    "        return out_value\n",
    "    \n",
    "    \n",
    "    if type(x) is list:\n",
    "        x = np.array(x)\n",
    "        \n",
    "    ## x1 = x2 = x3 = x4 \n",
    "    if  np.absolute(np.sum(x)-4)<approx:\n",
    "        rosen(x)\n",
    "    else:\n",
    "        return out_value\n",
    "    \n",
    "mins = [spo.minimize(my_rosen,[1.3, 0.7, 0.8, 1.9, 1.2],method=om) for om in opt_methods_no_Jacobian]\n",
    "res = basinhopping(my_rosen, [1.3, 0.7, 0.8, 1.9, 1.2], minimizer_kwargs={\"method\": \"BFGS\"},niter=200)\n",
    "min_df = pd.DataFrame({'Method' : opt_methods_no_Jacobian+['Basin Hopping'],\n",
    " 'min' : [m.x for m in mins]+[res.x], \n",
    "  'min_value': [rosen(m.x) for m in mins]+[rosen(res.x)]\n",
    "  }).set_index('Method')\n",
    "min_df.sort_values(by = 'min_value' , inplace=True)\n",
    "min_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's see how __Basin Hopping behaves best working with bonds and constraints__.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">>> basinhopping min: [ 0.99999925  0.99999852  0.99999706  0.99999416  0.99998833] - min_value:4.58223880115e-11\n"
     ]
    }
   ],
   "source": [
    " class MyBounds(object):\n",
    "        def __init__(self, xmax=[1.0,1.0,1.0,1.0,1.0], xmin=[0,0,0,0,0] ):\n",
    "            self.xmax = np.array(xmax)\n",
    "            self.xmin = np.array(xmin)\n",
    "            \n",
    "        def __call__(self, **kwargs):\n",
    "            x = kwargs[\"x_new\"]\n",
    "            tmax = bool(np.all(x <= self.xmax))\n",
    "            tmin = bool(np.all(x >= self.xmin))\n",
    "            cons = bool(np.sum(x) <= 4)\n",
    "            return tmax and tmin and cons\n",
    "        \n",
    "mybounds = MyBounds()\n",
    "\n",
    "res = basinhopping(rosen, [1.3, 0.7, 0.8, 1.9, 1.2], minimizer_kwargs={\"method\": \"BFGS\"},\n",
    "                   niter=200,accept_test=mybounds)\n",
    "\n",
    "print(\">>> basinhopping min: \"+str(res.x)+\" - min_value:\"+str(rosen(res.x)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimizing a portfolio \n",
    "\n",
    "For evaluating the performance of a portfolio key factors to focus on are \n",
    "1. __Cumulative return__ \n",
    "2. __Average daily return__ \n",
    "3. __Rsk__ (Standard deviation of daily return)\n",
    "4. __Sharpe ratio__ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas.io.data as web\n",
    "\n",
    "def get_data(symbols, \n",
    "             add_ref=True,\n",
    "             data_source='yahoo',\n",
    "             price='Adj Close',\n",
    "             start='1/21/2010', \n",
    "             end='4/15/2016'):\n",
    "    \"\"\"Read stock data (adjusted close) for given symbols from.\"\"\"\n",
    "    \n",
    "    if add_ref and 'SPY' not in symbols:  # add SPY for reference, if absent\n",
    "        symbols.insert(0, 'SPY')\n",
    "\n",
    "    df = web.DataReader(symbols, \n",
    "                        data_source=data_source,\n",
    "                        start=start, \n",
    "                        end=end)\n",
    "    \n",
    "    return df[price,:,:]\n",
    "\n",
    "def compute_daily_returns(df):\n",
    "    \"\"\"Compute and return the daily return values.\"\"\"\n",
    "    # Note: Returned DataFrame must have the same number of rows\n",
    "    daily_returns = (df / df.shift(1)) - 1 \n",
    "    daily_returns.ix[0,:] = 0 \n",
    "    return daily_returns\n",
    "\n",
    "def fill_missing_values(df_data):\n",
    "    \"\"\"Fill missing values in data frame, in place.\"\"\"\n",
    "    df_data.fillna(method='ffill',inplace=True)\n",
    "    df_data.fillna(method='backfill',inplace=True)\n",
    "    return df_data\n",
    "\n",
    "def cumulative_returns(df):\n",
    "    return df/df.ix[0,:] - 1 \n",
    "\n",
    "\n",
    "df = fill_missing_values(get_data(symbols=['GOOG','SPY','IBM','GLD'],\n",
    "                             start='4/21/2015', \n",
    "                             end='7/15/2016'))\n",
    "\n",
    "dr = compute_daily_returns(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Cumulative return "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def cumulative_returns_obj(alloc,df=df):\n",
    "    \n",
    "    if type(alloc) is list:\n",
    "        alloc = np.array(alloc)\n",
    "    \n",
    "    cr = cumulative_returns(df).ix[-1,:]\n",
    "    \n",
    "    return -1 * np.dot(cr , alloc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>allocation</th>\n",
       "      <th>portfolio cumul. ret</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Asset</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>GLD</th>\n",
       "      <td>1.443290e-15</td>\n",
       "      <td>0.348103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>0.348103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>IBM</th>\n",
       "      <td>1.421426e-16</td>\n",
       "      <td>0.348103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SPY</th>\n",
       "      <td>4.077524e-16</td>\n",
       "      <td>0.348103</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         allocation  portfolio cumul. ret\n",
       "Asset                                    \n",
       "GLD    1.443290e-15              0.348103\n",
       "GOOG   1.000000e+00              0.348103\n",
       "IBM    1.421426e-16              0.348103\n",
       "SPY    4.077524e-16              0.348103"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cons = ({'type': 'ineq','fun' : lambda x: np.array(1-np.sum(np.array(x))),'jac' : lambda x: np.array([-1,-1,-1,-1])})\n",
    "\n",
    "bnds = ((0,1),(0,1),(0,1),(0,1))\n",
    "mins = spo.minimize(cumulative_returns_obj,[0.25, 0.25, 0.25, 0.25],method='SLSQP',bounds=bnds,constraints=cons) \n",
    "min_df = pd.DataFrame({'Asset' : df.columns,\n",
    " 'allocation' : mins.x ,\n",
    " 'portfolio cumul. ret': -1 * cumulative_returns_obj(mins.x)\n",
    "  }).set_index('Asset')\n",
    "min_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "According to this allocation __all the money should be put on GOOG__ as it is expected as GOOG has the \n",
    "highest cumulative return. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GLD     0.099324\n",
       "GOOG    0.348103\n",
       "IBM     0.017407\n",
       "SPY     0.057167\n",
       "Name: 2016-07-15 00:00:00, dtype: float64"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cumulative_returns(df).ix[-1,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As excercise, let's check if __Basin Hopping gets the same result__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class MyBoundsP(object):\n",
    "        def __init__(self, xmax=[1,1,1,1], xmin=[0,0,0,0] ):\n",
    "            self.xmax = np.array(xmax)\n",
    "            self.xmin = np.array(xmin)\n",
    "            \n",
    "        def __call__(self, **kwargs):\n",
    "            x = kwargs[\"x_new\"]\n",
    "            tmax = bool(np.all(np.array(x) <= self.xmax))\n",
    "            tmin = bool(np.all(np.array(x) >= self.xmin))\n",
    "            conss = bool( np.sum(np.array(x)) <= 1)\n",
    "            return tmax and tmin and conss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "at minimum -192692159.4246 accepted 0\n",
      "at minimum -192664467.7904 accepted 0\n",
      "at minimum -192644003.2324 accepted 0\n",
      "at minimum -192774921.6759 accepted 0\n",
      "at minimum -192633110.3483 accepted 0\n",
      "at minimum -192855427.2053 accepted 0\n",
      "at minimum -192715750.8904 accepted 0\n",
      "at minimum -192559382.1396 accepted 0\n",
      "at minimum -192685611.9210 accepted 0\n",
      "at minimum -192799277.9159 accepted 0\n",
      ">>> basinhopping max: [  1.42197728e+08   4.98364107e+08   2.49208056e+07   8.18440642e+07] - max_value:192718446.154\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "                        fun: -192718446.1542715\n",
       " lowest_optimization_result:       fun: -192718446.1542715\n",
       " hess_inv: array([[1, 0, 0, 0],\n",
       "       [0, 1, 0, 0],\n",
       "       [0, 0, 1, 0],\n",
       "       [0, 0, 0, 1]])\n",
       "      jac: array([ 0.,  0.,  0.,  0.])\n",
       "  message: 'Optimization terminated successfully.'\n",
       "     nfev: 102\n",
       "      nit: 1\n",
       "     njev: 17\n",
       "   status: 0\n",
       "  success: True\n",
       "        x: array([  1.42197728e+08,   4.98364107e+08,   2.49208056e+07,\n",
       "         8.18440642e+07])\n",
       "                    message: ['requested number of basinhopping iterations completed successfully']\n",
       "      minimization_failures: 0\n",
       "                       nfev: 162\n",
       "                        nit: 10\n",
       "                       njev: 27\n",
       "                          x: array([  1.42197728e+08,   4.98364107e+08,   2.49208056e+07,\n",
       "         8.18440642e+07])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "myboundsp = MyBoundsP()\n",
    "\n",
    "def print_fun(x, f, accepted):\n",
    "    print(\"at minimum %.4f accepted %d\" % (f, int(accepted)))\n",
    "\n",
    "res = basinhopping(cumulative_returns_obj, [.25,0.25,0.25,0.25], minimizer_kwargs={\"method\": \"BFGS\"},\n",
    "                   niter=10,accept_test=myboundsp,callback=print_fun , T=-1.0, stepsize=-500000)\n",
    "\n",
    "print(\">>> basinhopping max: \"+str(res.x)+\" - max_value:\"+str(-1 * cumulative_returns_obj(res.x)))\n",
    "res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This behaviour is __very disappointing__. It turns out that each iteration of basinhopping rejected the \n",
    "step but the __botton line is a crazy portfolio allocation__. In the following we discard this \n",
    "optimization method. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Average daily return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def average_daily_return_obj(alloc,dr=dr):\n",
    "    \n",
    "    if type(alloc) is list:\n",
    "        alloc = np.array(alloc)\n",
    "    \n",
    "    return -100 * np.dot(dr.mean(),alloc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>allocation</th>\n",
       "      <th>portfolio avg daily ret</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Asset</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>GLD</th>\n",
       "      <td>5.967449e-16</td>\n",
       "      <td>0.001113</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>0.001113</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>IBM</th>\n",
       "      <td>2.432834e-16</td>\n",
       "      <td>0.001113</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SPY</th>\n",
       "      <td>1.387779e-16</td>\n",
       "      <td>0.001113</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         allocation  portfolio avg daily ret\n",
       "Asset                                       \n",
       "GLD    5.967449e-16                 0.001113\n",
       "GOOG   1.000000e+00                 0.001113\n",
       "IBM    2.432834e-16                 0.001113\n",
       "SPY    1.387779e-16                 0.001113"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cons = ({'type': 'ineq','fun' : lambda x: np.array(1-np.sum(np.array(x))),'jac' : lambda x: np.array([-1,-1,-1,-1])})\n",
    "\n",
    "bnds = ((0,1),(0,1),(0,1),(0,1))\n",
    "mins = spo.minimize(average_daily_return_obj,[0.25, 0.25, 0.25, 0.25],method='SLSQP',bounds=bnds,constraints=cons) \n",
    "min_df = pd.DataFrame({'Asset' : df.columns,\n",
    " 'allocation' : mins.x ,\n",
    " 'portfolio avg daily ret': -1 * average_daily_return_obj(mins.x)/100\n",
    "  }).set_index('Asset')\n",
    "min_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "According to this allocation all the money should be put on GOOG as it is expected as \n",
    "GOOG has the highest average daily return. Notice that __we needed to multiply for 100 for handling \n",
    " numerical problems__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GLD     0.000352\n",
       "GOOG    0.001113\n",
       "IBM     0.000155\n",
       "SPY     0.000229\n",
       "dtype: float64"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dr.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Rsk (Standard deviation of daily return)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def std_daily_return_obj(alloc,dr=dr):\n",
    "    \n",
    "    if type(alloc) is list:\n",
    "        alloc = np.array(alloc)\n",
    "    \n",
    "    return 10000000 * np.dot(dr.std(),alloc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>allocation</th>\n",
       "      <th>portfolio risk</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Asset</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>GLD</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.010003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.010003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>IBM</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.010003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SPY</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.010003</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       allocation  portfolio risk\n",
       "Asset                            \n",
       "GLD           1.0        0.010003\n",
       "GOOG          0.0        0.010003\n",
       "IBM           0.0        0.010003\n",
       "SPY           0.0        0.010003"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cons = ({'type': 'eq','fun' : lambda x: np.array(1-np.sum(np.array(x))),'jac' : lambda x: np.array([-1,-1,-1,-1])})\n",
    "bnds = ((0,1),(0,1),(0,1),(0,1))\n",
    "mins = spo.minimize(std_daily_return_obj,[0.25, 0.25, 0.25, 0.25],method='SLSQP',bounds=bnds,constraints=cons) \n",
    "min_df = pd.DataFrame({'Asset' : df.columns,\n",
    " 'allocation' : mins.x ,\n",
    " 'portfolio risk': std_daily_return_obj(mins.x) / 10000000\n",
    "  }).set_index('Asset')\n",
    "min_df "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__In order to make the optmizer get the rigth solution we had to multiply \n",
    "by 10000000, i.e. all the money on GLD.   \n",
    "Otherwise ~50% of the money should have gone on GLD and 50% on SPY. This is a sub-optimal solution \n",
    "beacuse we know that GLD has the minimum risk.__ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GLD     0.010003\n",
       "GOOG    0.018065\n",
       "IBM     0.014062\n",
       "SPY     0.010117\n",
       "dtype: float64"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dr.std()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Sharpe ratio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sharpe_ratio_obj(alloc,dr=dr,sample_freq='d',risk_free_rate=0.0):\n",
    "    \n",
    "    if type(alloc) is list:\n",
    "        alloc = np.array(alloc)\n",
    "    \n",
    "    sr = ( np.sum(alloc * dr,axis=1) - risk_free_rate).mean() / np.sum(alloc * dr,axis=1).std()\n",
    "    \n",
    "    if sample_freq == 'd':\n",
    "        sr = sr * np.sqrt(252)\n",
    "    elif sample_freq == 'w':\n",
    "        sr = sr * np.sqrt(52)\n",
    "    elif sample_freq == 'm':\n",
    "        sr = sr * np.sqrt(12)\n",
    "    else:\n",
    "        raise Exception('unkown sample frequency :'+str(sample_freq))\n",
    "            \n",
    "    return -1*sr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>max_point</th>\n",
       "      <th>max_value</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Asset</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>GLD</th>\n",
       "      <td>5.543488e-01</td>\n",
       "      <td>0.06883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>GOOG</th>\n",
       "      <td>4.429360e-01</td>\n",
       "      <td>0.06883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>IBM</th>\n",
       "      <td>2.499626e-17</td>\n",
       "      <td>0.06883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SPY</th>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.06883</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          max_point  max_value\n",
       "Asset                         \n",
       "GLD    5.543488e-01    0.06883\n",
       "GOOG   4.429360e-01    0.06883\n",
       "IBM    2.499626e-17    0.06883\n",
       "SPY    0.000000e+00    0.06883"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cons = ({'type': 'ineq','fun' : lambda x: np.array(1-np.sum(np.array(x))),'jac' : lambda x: np.array([-1,-1,-1,-1])})\n",
    "\n",
    "bnds = ((0,1),(0,1),(0,1),(0,1))\n",
    "mins = spo.minimize(sharpe_ratio_obj,[0.25, 0.25, 0.25, 0.25],method='SLSQP',bounds=bnds,constraints=cons) \n",
    "min_df = pd.DataFrame({'Asset' : df.columns,\n",
    " 'max_point' : mins.x,\n",
    " 'max_value': -1 * average_daily_return_obj(mins.x)\n",
    "  }).set_index('Asset')\n",
    "min_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__According to this allocation ~50% of the money should be put on GLD and 50% on GOOG__"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}