joint_control/scikit-learn-intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to scikit-learn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Classification of Handwritten Digits** the task is to predict, given an image, which digit it represents. We are given samples of each of the 10 possible classes (the digits zero through nine) on which we fit an estimator to be able to predict the classes to which unseen samples belong."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Data collection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Data preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A dataset is a dictionary-like object that holds all the data and some metadata about the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "digits = datasets.load_digits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/xu/miniconda3/envs/waymo_2d_detection/lib/python3.7/site-packages/IPython/core/magics/pylab.py:160: UserWarning: pylab import has clobbered these variables: ['clf']\n",
      "`%matplotlib` prevents importing * from pylab and numpy\n",
      "  \"\\n`%matplotlib` prevents importing * from pylab and numpy\"\n"
     ]
    }
   ],
   "source": [
    "%pylab inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  5., ...,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0., ..., 10.,  0.,  0.],\n",
       "       [ 0.,  0.,  0., ..., 16.,  9.,  0.],\n",
       "       ...,\n",
       "       [ 0.,  0.,  1., ...,  6.,  0.,  0.],\n",
       "       [ 0.,  0.,  2., ..., 12.,  0.,  0.],\n",
       "       [ 0.,  0., 10., ..., 12.,  1.,  0.]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "digits.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1797, 64)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "digits.data.shape  # n_samples, n_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## digits.images.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2, ..., 8, 9, 8])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "digits.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1797,)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "digits.target.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcEAAAHBCAYAAAARuwDoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dfXBUVZo/8G8vISzhTZAEhABJiBNiAgnhJeiiCQqIiGhIdIjohBAXa4rZirhWMVXzB5OyStAqKkRlaxlrl6F0JTtj6cCgUAMEwYlgFvJSUjgZXBIkwUGSBXlJMEnbvz9mkl8Y7/OQvi99k77fT9VUjac53c+59/Q96e7nPNcXCARARETkRf/gdgBERERu4SJIRESexUWQiIg8i4sgERF5FhdBIiLyrIhg/vHYsWMDcXFxQb3A5cuXDdubmprEPiNHjjRsj42NFfsMGjQoqLgaGxvR0tLiA8yNS1JfXy8+5vf7DdsnTJgg9rnjjjuCjuHkyZMtgUAg2s5xXbt2TXzsf//3fw3bhw4dKvZJSkoKOobucQHmztlf/vIXw/bm5maxT2RkpGH7PffcI/bpL3NRmm8A0NDQYNiemJhoy2t3szIXpffSkCFDxD52HbvbsToXJWauH9pcDJbVuXjx4kXDdm0uXrlyxbC9vb1d7CO9x6ZPny72qa2t7TlnvQW1CMbFxeHEiRPBdMFvf/tbw/YNGzaIfRYtWmTYvnnzZrHP6NGjg4pr9uzZPf/fzLgk2dnZ4mPSyS4pKRH7PP7440HH4PP5zgH2juvjjz8WH3viiScM29PT0009n6R7XIC5sb366quG7T//+c/FPhMnTjRsr6ioEPv0l7kozTcAWL16tWH77373O1teu5uVuSi9l7QL869//eugXsMsq3NRYub6YddrA9bn4tatWw3btbkozbm6ujqxz/Dhww3bDx8+LPYZPXr0OaN2fh1KRESexUWQiIg8i4sgERF5FhdBIiLyrKASY8yQEmCk7DRAzigdM2aM2Oc3v/mNYfuTTz6pRGc/LZvzyJEjhu3aj7lmEmOsqK2tNWxfsGCB2GfUqFGG7Y2NjXaEFBQtyUWaI9u3bxf7PP/884btJ0+eFPssXLhQfCyUtCQRLWmpv5Dmj/Q+AoCdO3catk+ZMiXo13HS7t27Ddu1sW3cuNGpcBynXRelZBqpHZATbcxk0/OTIBEReRYXQSIi8iwugkRE5FlcBImIyLO4CBIRkWdxESQiIs+yZYuEli4ubYWQii4DQEJCgmG7VFNUi8GpLRLSVgIzNTH7U7q6VMcvLS1N7CPVDtVqojpl7dq14mPSdp1Zs2aJfeLj4w3b+8s2CEBOF9e2SLzwwguG7Wa2CzhVtFpKdz93zrAEJAB5u46Zmpxm0u37ysx2B+l91p9I80rzy1/+0rBdm4tmrrMSfhIkIiLP4iJIRESexUWQiIg8i4sgERF5FhdBIiLyLFuyQ6WC1wCQkZFh2C5lgGq0LD4naAVcpYymb7/9NujX0TLXQk3K7tIyAKU+oS7+Dejz6uzZs4btWjF3KQtUm/PB3lneKikLVMuuk+4sr2X3SdmS0nvBKmnOaXccl95/Wga2k1mgEikjVcvC7i9Z5FpmppmsTe06K5Gy2KV5reEnQSIi8iwugkRE5FlcBImIyLO4CBIRkWdxESQiIs/iIkhERJ7l+BYJrei1na/jRFq6li4upeKaiUNKl3aK9npSurKUkqzRCji7Qdo+8X//939iH2mLhFZA++DBg4btVubo7t27xcfWr19v2F5QUBD065SVlYmP7dixI+jns0Kac1oavlTYXjpGGjPFoPtKeg9qW5Gk96ZWWNuJ4ubac9p5YwHtmmPntjJ+EiQiIs/iIkhERJ7FRZCIiDyLiyAREXkWF0EiIvIsW7JDtay3kydPBv18UhboiRMnxD5PPfVU0K/TX0gZVYAzRXO1gsdadqBEyuJyozCxGdr8lTI9n3/+ebHPq6++ati+efPm4ALrZdSoUUE/tnPnTrGPNuckWhZiKNldcF4rNO4UKcPyyJEjYh8po1TLfK2pqTFst3Jd0bJDpWuBz+cLuk+obizAT4JERORZXASJiMizuAgSEZFncREkIiLP4iJIRESexUWQiIg8y5YtElJhYkDe1vDb3/5W7KM9JtmwYUPQfbxKKv4NyIVu6+rqxD5S6vzjjz8u9iksLAy6j1U///nPDdu1YtjSdp0DBw6IfZzYrqOli0up89o2COn5tKLbod7yIhUN17aLaNt/JG5s/ZDeg9p2B2lrgrbFQ9p+4MTWK0AuOq6ds6ysLEdi6St+EiQiIs/iIkhERJ5leRHcv38/Fi5ciAULFuDf//3f7YjJdWvWrEFMTAxSU1PdDsVW58+fx4IFC7BixQrk5ubi3XffdTsk29y8eRNz585FWloaUlJSsHHjRrdDspXf78fMmTOxbNkyt0OxTVxcHKZPn4709HTMnj3b7XBsdeXKFeTl5WHatGlITk7GsWPH3A7Jsvr6eqSnp/f8b+TIkeI9DgcSS78J+v1+rFu3Dv/5n/+J8ePHIycnBw899BDuvvtuu+JzxerVq/Gzn/0MP/nJT9wOxVYRERHYsmUL/uEf/gE3btzA008/jczMTEydOtXt0CwbMmQIKioqMHz4cHR2dmL+/Pl45JFHMG/ePLdDs0VZWRmSk5Nx9epVt0Ox1eHDhzF27Fi3w7BdcXExlixZgvfeew8dHR1oa2tzOyTLkpKSen5n9vv9mDhxInJyclyOyjpLnwSrqqqQmJiIyZMnIzIyEsuWLRNrLQ4kDzzwAMaMGeN2GLa76667kJGRAQAYNmwY4uPjcenSJZejsofP58Pw4cMBAJ2dnejs7FTrFQ4kTU1N+PDDD/Hcc8+5HQr1wdWrV3H06FEUFRUBACIjIwdMHd2+OnToEKZOnYopU6a4HYpllj4JNjc3Y9KkST3ZoTNmzMBnn312S7aoVExYy+aUvhoxU4zbKdKk1rIbpWw3KSMT0DM5zUpPT0djYyPOnj2Lp59+GiNHjux5TMoo1DINpYw8abyAnOlmJTvU7/dj1qxZ+PLLL7Fu3TpkZmbe8rhUKHvt2rVBv5aWAbp9+/agn0/zwgsv4LXXXsO1a9eC6qddeL/99lvDdifmm8Tn82Hx4sXw+Xx4/vnnf3AeDh8+bNjPTJF3LevV7kLNZ8+eRXR0NAoLC1FXV4dZs2ahrKwMw4YN6/k30nHWMj1//etfG7Zr8TuV+VpeXo78/PwftEvXMq2Yu9t/IFj6JBgIBH7QFi5/fYez69evIzc3F1u3br1lARzoBg0ahNraWjQ1NaGqqgqnTp1yOyTL9u7di5iYGMyaNcvtUGxXWVmJ6upq7Nu3D9u2bcPRo0fdDskWXV1dqK6uxk9/+lPU1NRg2LBhlu4g0t90dHRgz549ePLJJ90OxRaWFsHY2FicP3++57+bmpowYcIEy0GRczo7O5Gbm4tVq1ZhxYoVbofjiDvuuAPZ2dnYv3+/26FYVllZiT179iAuLg4rV65ERUUFnnnmGbfDskX3tSImJgY5OTmoqqpyOSJ7xMbGIjY2tuebiLy8PFRXV7sclX327duHjIwMjBs3zu1QbGFpEZwzZw7OnDmDhoYGdHR0oLy8HMuXL7crNrJZIBBAUVERkpOT8eKLL7odjq0uXbrUs2m8vb0dBw8exLRp01yOyrpNmzahqakJjY2NKC8vx4MPPoh33nnH7bAsu3HjRs/Xuzdu3MAf/vCHsMnGHj9+PCZNmoT6+noAf/397J577nE5Kvvs2rXL8KvQgcrSb4IRERF488038fDDD8Pv92PNmjVISUmxKzbX5Ofn4+OPP0ZLSwtiY2NRUlLS8yP3QFZZWYm33367Jy0dAF555RUsXbrU5cis+/rrr1FQUAC/34/vv/8eTz31VFhtJwg3Fy9e7Mks7OrqwtNPP40lS5a4HJV93njjDaxatQodHR1ISEjAjh073A7JFm1tbThw4IDtv3u7yXLZtKVLl4bFRbS3Xbt2uR2CI+bPn2/4O244mDFjhngX7XCRnZ0dsrttOy0hIUEtxTfQpaeniyUjB7KoqCi0tra6HYatWDGGiIg8yxfMJwOfz3cJwDnnwgmpKYFAIBoIu3EBfxtbuI4LCLtzFq7jAjgXB5pwHRfQa2y9BbUIEhERhRN+HUpERJ4VVGLM2LFjA1KlD0nvfYS9SfdAA4A777zTsF3blzJo0KCg4mpsbERLS4sPMDeuL7/80rDd7/eLfZKSkoJ6DbNOnjzZEggEos2MS4r/woULYh/ph/LuMmZGEhMTg4oL+P/jAsydMzM+//xzw3ZtvknnWepjdS5K76WLFy+KfaTjH+z76HZuNxc7OjrEvlL8WmKGFL9WlUS63kRFRYl9nJqL2vvsm2++MWyfPn262MeJ66J27ZbOmXZdbG9vDypGQB5zZGSk2Kf3OestqEUwLi4u6Iwn6SaL0s0eAbmkkPRcQPCld3qXZjMzLqkckTZBtPJodvL5fOcAc+OS4tduVmqmnJN2/iXd4wLMjc0M6eKmzTep3JfUx+pclMrTlZaWin2k4293CavbzUWtTJh0hwJpvgFy/Fr5MOl6o9141qm5qL3PpOMhzTfAmeuiVg5RmnPaddFMlvCePXsM27U/Rnqfs974dSgREXkWF0EiIvIsLoJERORZlivG3I52Cx6J9J2/9puaE7+3ab9XaN+LS6Q7bKSlpYl9zBw/K6TfR7TxSndx1367kR4L5a18umljO3fOeJuU1A7Iv384dcsY6TZB2utJx1/73d0J2ntMek9rMUrHXrv9knSctN8ErZLi1N4zZpJvnJiLWgm4I0eOGLaPGjVK7CNdP7ScAjuT4vhJkIiIPIuLIBEReRYXQSIi8iwugkRE5FlcBImIyLO4CBIRkWc5vkVCSjPWUlylNGEtrVdKp7ZyE1Kt1I8kKytLfEwac6jKqXUzs/VDSsMH5FJP2vEL9dYPTXFxcdB9zJxnp5iZV1IZsVBvkdDen9Ic0bYRSHNRS9HXSqo5RTrO2ntGKnWnzTfp+JopW9hN2zoinTOtj3QsnNpS9Pf4SZCIiDyLiyAREXkWF0EiIvIsLoJERORZXASJiMizHM8OlQoiz5w5U+wjZS9q2UJOZOSZeU4t68rMjXidYCbrykxh61Bld/WmHUspC00rht1faBm9Uuaddvy15+vvzGQ2atnITmXzSjfBBYCdO3catms3Qpbi/Pbbb8U+ThYBN2Km4LwUY6jmKD8JEhGRZ3ERJCIiz+IiSEREnsVFkIiIPIuLIBEReRYXQSIi8izHt0iYSf8/cuSIYXtDQ4PYx4k0Zy3FPC0tzbB99OjRYh+pULOWvi2lCVsZb38qXm03La1aemzKlCliHym1O9Sp59r5lopGa6Rxae9XN7a8GNG2HkjnRSsMbqWYtMZMir9WHFwbt0TbimaWdizNXJcKCwstRGMdPwkSEZFncREkIiLP4iJIRESexUWQiIg8i4sgERF5li3ZoVq24YIFCwzbN27cKPaRsqqkAtSAnOHlVHFcaczasTCTUShlYlnJaDMTh1akV8oo1I6FmYzGvtDG9vHHHxu27969W+wjzTm7s/iskOa4FuOoUaMM2/tLBqhGe09Lc87MvMjOzg4iqh/S5rj0ntHe19J7UMtufvzxx8XHzNLmiFRoX3uPSey+lkr4SZCIiDyLiyAREXkWF0EiIvIsLoJERORZXASJiMizuAgSEZFn2bJFQktZllKxtSKs0hYJrRislA7uVCq+REvdlcaspbI7UdxXS3HOysoybC8tLRX7fPDBB0G/TqgLUGukOarpT1sJpHlVVlYm9jHzvpTGLKXF94VWsFsqpH/58mWxj7Q9RdviY6bQdV9oc0R6z2vHQyrOb3UrR7DMnLOCggKxj3QzglBdI/hJkIiIPIuLIBEReZblRbC0tBT33nsv7r33XhQVFeHmzZt2xOW6srIypKamIiUlJeQVQJy0f/9+JCUlITExEZs3b3Y7HNusWbMGMTExSE1NdTsUW50/fx4LFixAcnIyUlJS1K84B5KbN29i7ty5mD9/Pu69915s2rTJ7ZBs5ff7MXPmTCxbtsztUGwVFxeH++67D/fff79YDWygsbQINjc34/XXX0dFRQWOHTuG77//Hu+//75dsbnm1KlTeOutt1BVVYW6ujrs3bsXZ86ccTssy/x+P9atW4d9+/bh9OnT2LVrF06fPu12WLZYvXo19u/f73YYtouIiMCWLVvwxRdf4Pjx49i2bVtYnLMhQ4agoqICf/zjH3H06FEcOnQI//M//+N2WLYpKytDcnKy22E44ve//z0++eQTHD582O1QbGH5k2BXVxdu3ryJrq4utLW1Yfz48XbE5aovvvgC8+bNQ1RUFCIiIpCVlSUmfwwkVVVVSExMREJCAiIjI7Fy5UpTNf36owceeABjxoxxOwzb3XXXXcjIyAAAjBgxAsnJyWhubnY5Kut8Ph+GDx8OAOjs7ERnZyd8Pp/LUdmjqakJH374IZ577jm3Q6E+sJQdOnHiRLz00kuYPn06hg4disWLF2PFihW3/Bspc0nKdALkzDWtGKyW1Ras1NRU/OIXv0BrayuGDh2Kjz76CLNnz+7T62lFX6WsKqmAL2BvhlRzczMmTZrU89+xsbH47LPPbvk3UjaqdnylMWtZr/2JdoylzLW6ujqxj3SerWaUNjY2oqamBpmZmbe0S9mZWtajNGYtG1mK32x2ot/vx3333Ydz587h2WefRWJi4i3HTstIDpZ27bCS3WrkhRdewGuvvYZr166Z6iuRrot2x6/x+Xx45JFHAACPPfYYHnvssVsel7JAtexcJzLgg2Hpk+Dly5exe/duNDQ04MKFC7hx4wbeeecdu2JzTXJyMjZs2IBFixZhyZIlSEtLQ0SELbtJXBUIBH7QFi5/fYe769evIzc3F1u3bsXIkSPdDscWgwYNwkcffYRjx46hrq4O9fX1bodk2d69exETE4NZs2a5HYojKisr8atf/Qqvvvoqfve736l/DA4UlhbBgwcPIj4+HtHR0Rg8eDBWrFiBTz/91K7YXFVUVITq6mocPXoUY8aMwd133+12SJbFxsbi/PnzPf/d1NSECRMmuBgR9UVnZydyc3OxatWqH3zTEg5GjhyJefPmiXvMBpLKykrs2bMHcXFxWLlyJSoqKvDMM8+4HZZtuq8Xo0ePxv33348//elPLkdknaVFcPLkyTh+/Dja2toQCARw6NChsPkx+JtvvgEAfPXVV3j//feRn5/vckTWzZkzB2fOnEFDQwM6OjpQXl6O5cuXux0WKQKBAIqKipCcnIwXX3zR7XBsc+nSpZ6vPm/evIk//vGPmDp1qstRWbdp0yY0NTWhsbER5eXlePDBB8Pi2zEAuHHjRs9XvO3t7Thx4gTi4+Ndjso6S9/xZWZmIi8vDxkZGYiIiMDMmTOxdu1au2JzVW5uLlpbWzF48GBs27ZN/Q1zoIiIiMCbb76Jhx9+GH6/H2vWrEFKSorbYdkiPz8fH3/8MVpaWhAbG4uSkhIUFRW5HZZllZWVePvttzF9+vSe3/FeeeUVLF261OXIrPn6669RUFCA9vZ2BAIBPProo3jooYfcDosUFy9eRE5ODq5fvw6/34+FCxdi7ty5bodlmeUfukpKSlBSUmJHLP3KJ5984nYIjli6dOmAv4Aa2bVrl9shOGL+/PmGv+UOdDNmzEBNTY1jJcv6g+zs7JCXNHNSQkIC6urq1ES+gYgVY4iIyLN8wfyV6fP5LgE451w4ITUlEAhEA2E3LuBvYwvXcQFhd87CdVwA5+JAE67jAnqNrbegFkEiIqJwwq9DiYjIs4JKjBk7dmxAu3dgMC5cuCA+1r094e9Nnz5d7DNo0KCgXr+xsREtLS0+wNy4/H6/YfvFixfFPtK4tEoiZo73yZMnWwKBQLSZcUmJCkOHDhX7tLa2GraPGDFC7NO7ck1fdY8LMHfOpDi1uSi9hja2YFmdi21tbeLzSiIjIw3btXGNGzcuqLgAa3NR0tHRIT72+eefB/180nVFOkaA9bkozbmvv/5a7CNtIbHz3pZ9mYvStQ8A/vKXvxi2X716VewjzV/tmp6QkGDYrhWS6H3OegtqEYyLi8OJEyeC6SLSbnYr3bVBK9ga7EToXQbNzLik0ljaHSekx7RK82ZKj/l8vnOAuXFJJZi00mJSjFpmnJk7c3SPCzA3NjM3Xt6+fbthu51Zf1bnolS2TiunJV20tXGZKU1oZS5KtMXdzL61PXv2GLZrC5vVuSjNOS3TfsuWLYbtWkm4YPVlLmo3NX711VcN2w8cOCD2qa6uNmzX/iD7t3/7N8P2hQsXin16n7Pe+HUoERF5FhdBIiLyLC6CRETkWVwEiYjIs1y7P5BWekdKcrEzC6ovtHsDmrmHmxR/fypDJMWoHQupj5bUIyVZ2JVBaES6b9m5c/J+YDNJP6EmJVlot7mRHtNusvzEE08Ytjt5zowMlFJrUvIcIM9FLclFOv6h3ut99uxZ8bGTJ08ati9atEjsIz2mJdNs2LAhqNfX8JMgERF5FhdBIiLyLC6CRETkWVwEiYjIs7gIEhGRZ3ERJCIiz3J8i4SUWn/kyBGxT2lpqVPhBEVLnZdqadq9rSLUpDRsrdanlCJvd2Fwq8ycs507dxq2a/VGnRibto1G2tZQXFws9pHi12rEOkHbRiCdF+3YS7KyssTHnJqL2vyXjrO2rcjM/HXifM6aNUt8TNvWIJG2XPzmN78R+zz//PNBv46EnwSJiMizuAgSEZFncREkIiLP4iJIRESexUWQiIg8y7XsUI2UoRhqWjHbKVOmGLZrBYilornaeKXMUacy2qRsMm1cBQUFhu1appsbpKLdWualdJy1u6xL5znUtIxeiZYR7QRtjqxfvz50gYSYNH+0bFmp6LkbmdbB0opuT5061bA9IyND7LN27VrLMXXjJ0EiIvIsLoJERORZXASJiMizuAgSEZFncREkIiLP4iJIRESe5fgWCS3lVxIfH2/YnpaWJvYpKSkxbNe2OVgxc+ZM255LKtIMyFsktLR+K6TtGtqxlwqDawWE3SDFY+ZYattapG1BVooZZ2dnB91He+9Jx0IrNC1tZzBT0LqbttVEGrO29UN6L/WnIvWAPBe04yGdm/72PjOSkJAgPiZd73/+85+LfUaPHm05pm78JEhERJ7FRZCIiDyLiyAREXkWF0EiIvIsLoJERORZjmeHmskcKy4utq2PlexQLbtu48aNhu1apqGUoSZlVwL9p5i4Ni4pRqcyWPuDwsJC8TFpzjtVWHvUqFFBxQHIGZbanA91oWYpg9JMHP2tyLSUQawVFNeyyAeyRYsWGbZv2LBB7PPkk0/a9vr8JEhERJ7FRZCIiDyLiyAREXkWF0EiIvIsLoJERORZXASJiMizHN8iIaXJm0n914rLlpWVGbZL2xI6Ojpu+3paYVop/Vzb7iCln1spQGyGmTR4rU9/K04skcYgpatrGhoaxMd2795t2G5lLmqkOWdmS4Z2nvvLdh0z2x2OHDkiPiadFye3VZg5ljU1NUG1a69jpZi75tVXXzVsv3z5stjnN7/5jWG7mZsvmMFPgkRE5FmWF8ErV64gLy8P06ZNQ3JyMo4dO2ZHXK6qr69Henp6z/9Gjhyp3r5lICktLcW9996Le++9F0VFRbh586bbIdmmrKwMqampSElJCZvzBQD79+9HUlISEhMTsXnzZrfDsc2aNWsQExOD1NRUt0Ox1fnz57FgwQIkJycjJSVF/JZqoLl58ybmzp2LtLQ0pKSkiAVDBhrLi2BxcTGWLFmCP/3pT6irq0NycrIdcbkqKSkJtbW1qK2txcmTJxEVFYWcnBy3w7KsubkZr7/+OioqKnDs2DF8//33eP/9990OyxanTp3CW2+9haqqKtTV1WHv3r04c+aM22FZ5vf7sW7dOuzbtw+nT5/Grl27cPr0abfDssXq1auxf/9+t8OwXUREBLZs2YIvvvgCx48fx7Zt28LinA0ZMgQVFRWoq6tDbW0t9u/fj+PHj7sdlmWWFsGrV6/i6NGjKCoqAgBERkYOiBs8BuPQoUOYOnUqpkyZ4nYotujq6sLNmzfR1dWFtrY2jB8/3u2QbPHFF19g3rx5iIqKQkREBLKysvDBBx+4HZZlVVVVSExMREJCAiIjI7Fy5UrxN8eB5oEHHsCYMWPcDsN2d911FzIyMgAAI0aMQHJyMpqbm12Oyjqfz4fhw4cDADo7O9HZ2Qmfz+dyVNZZWgTPnj2L6OhoFBYWYubMmXjuuedw48YNu2LrF8rLy5Gfn+92GLaYOHEiXnrpJUyfPh3Tpk3DyJEj8eCDD7odli1SU1Nx9OhRtLa2oq2tDR999BHOnz/vdliWNTc3Y9KkST3/HRsbGxYXVK9obGxETU0NMjMz3Q7FFn6/H+np6YiJicGiRYvCYlyWskO7urpQXV2NN954A5mZmSguLsbmzZvx8ssv9/wbKQtJy/SUsiW179alQtlShldkZKT4XN06OjqwZ88ebNq06bb/tpuW0ZSdnd3n53HC5cuXsXv3bjQ0NOCOO+7Ak08+ib179+KZZ57p+TdS/Frs/eE3j+TkZGzYsAGLFi3C8OHDkZaWhoiIW6e3VJx4/fr1Qb9eWlqa+Jg0F6VvSQYNGiQ+VyAQ+EHb3//1Lb2XtKxXKWtbK+DcX77l0eZiVlaWYbt2LJzKDr1+/Tpyc3OxdetWjBw58pbHpHOmZfSaKUgvPZ/0XH6/X32+QYMGoba2FleuXEFOTg5OnTp1y2+60m/W2nVx4cKFhu3bt29XY7GLpU+CsbGxiI2N7flrIC8vD9XV1bYE1h/s27cPGRkZGDdunNuh2OLgwYOIj49HdHQ0Bg8ejBUrVuDTTz91OyzbFBUVobq6GkePHsWYMWNw9913ux2SZbGxsbd8om1qasKECRNcjIj6orOzE7m5uVi1ahVWrFjhdji2u+OOO5CdnR0Wv+laWgTHjx+PSZMmob6+HsBffz+75557bAmsP9i1a1fYfBUKAJMnT8bx48fR1taGQCCAQ4cOhUUiU7dvvvkGAPDVV1/h/fffD4tzN2fOHJw5cwYNDQ3o6NXmkyAAACAASURBVOhAeXk5li9f7nZYpAgEAigqKkJycjJefPFFt8OxzaVLl3o+0bW3t+PgwYOYNm2ay1FZZ3mz/BtvvIFVq1aho6MDCQkJ2LFjhx1xua6trQ0HDhwI2UfyUMjMzEReXh4yMjIQERGBmTNnYu3atW6HZZvc3Fy0trZi8ODB2LZtG0aPHu12SJZFRETgzTffxMMPPwy/3481a9YgJSXF7bBskZ+fj48//hgtLS2IjY1FSUlJT5LdQFZZWYm3334b06dP7/k56JVXXsHSpUtdjsyar7/+GgUFBfD7/fj+++/x1FNPYdmyZW6HZZnlRTA9PR0nTpywI5Z+JSoqCq2trW6HYbuSkhKUlJS4HYYjPvnkE7dDcMTSpUsH/AXUyK5du9wOwRHz5883/C13oJsxY4ZanWagYsUYIiLyLC6CRETkWb5gPrb7fL5LAM45F05ITQkEAtFA2I0L+NvYwnVcQNids3AdF8C5ONCE67iAXmPrLahFkIiIKJwElRgzduzYgJO3F+nWveXi78XHx4t9+rL5vbfGxka0tLT4AHPjkmIcMWJEUM8D6JulzexRPHnyZEsgEIg2My5ps6w0Xq3P1KlTxT5RUVFBxQX8/3EB9s7FCxcuiI+ZSY5KSkoybJfmqNW5KG327urqEvtI88rM/NVYmYtSxZ9r166Jfe68807Ddrv3+lqdi9I508Y2dOhQw3Zt32iw7zOrc1G6LdiXX34p9pGKMNi9H7b3OestqEUwLi4uJJmgUkUIrZpFsCdr9uzZt/QNdlxSjGaqwmiVOLTKOhKfz3cOMDcuMxVjpD7vvvuu2MfM/cy6xwXYOxe1+zlqc06yZ88ew3Zpjlqdi9L9BLUqHdK8sruqkZW5KMWoVU6RjoWZ95HG6lyU4tTGJr1ntPkb7PvM6lyUFnft/onSY3bfZ7X3OeuNiTFERORZXASJiMizuAgSEZFncREkIiLPslw2zSwt4UD6cTXUt3LRbr9y5MiRoNoB+RY7bt9iqbetW7cattfV1Yl9pNsK9Zdb79yOllQlnRvtljdmbpPjBC3JQnqPmXk+p86z9P7T5qJ0WywtMcOpjHft+O/cudOwXbtFlzQGbWzSMXTqnElj1s6Z9Jh2XqTEIjP4SZCIiDyLiyAREXkWF0EiIvIsLoJERORZXASJiMizuAgSEZFnOb5FQkqZLSwsFPuUlpYatkvp+4D9deYAPY14ypQphu3ator+smVAS483c9d5abtLKIqt20FLt5Ye08YW6vMsxaJtvZG2a2jjkuZ2qLf4aNsIpHR7M/VG3aBto5HOjdZHem/aXUu12+jRow3bR40aJfYxMy5ukSAiIrIBF0EiIvIsLoJERORZXASJiMizuAgSEZFnOZ4dKmUhFRcXB93H5/OJfaQMIytZRFrRV4mZAtqhpt1xXJKVlSU+1p+yQLXMVylTTsvolY7VuXOGN6kGEPrjIWVGa3cVlzJYzRQTd4r03tUyyyVa9rhT2aFaRqrEzNzRspHj4+ODfj4rpGucdvyloudmirybwU+CRETkWVwEiYjIs7gIEhGRZ3ERJCIiz+IiSEREnsVFkIiIPMuWLRJS6jkgbzPQ0refeOKJoGNwIs1ZK9Qrpc5rsUvbQrTC4E7Q0vslWrqytC3EjS0h2lw0UxzcDCcKaGvbWqS5b2aLj7ZdJNSkcWnvdWmealsFpDFr16j+ZCAUB9cKdkuPmSnmbuac8ZMgERF5FhdBIiLyLC6CRETkWVwEiYjIs7gIEhGRZ9mSHaplII0ePdqw/YMPPhD7hKpw6u1o2UlStp4Wu5ShpmVOOVGMecqUKUH30TJKzWTz7tixw7DdajabVqhXe0wija0/ZeRJsdTU1Ih9pCxaLXYt87a/kN4vWsF+KTu7v43XTDH3gZLhakS7Lkrv5d/97ndBvw4/CRIRkWdxESQiIs/iIkhERJ7FRZCIiDyLiyAREXkWF0EiIvIsW7ZIaKQiylpxZSk1ubCw0I6QbCGlpWup8xJtW4UTWyS055S2T5gpuq0xk6LvFC0Ve/fu3YbtpaWlYh8nCmhrzyk9phXDlo6/me0uTpHi195jUoq89h6T5raZLTW9ZWdni49JxdzNFEofNWqU2MeJuWiGNhelMWvbO9avX2/YbmZ7HT8JEhGRZ3ERJCIiz7K8CPr9fsycORPLli2zI55+Iy4uDtOnT0d6ejpmz57tdji2uXLlCgoKCjB37lxkZmaiqqrK7ZBsUV9fj/T09J7/jRw5MuT3aXRKaWkpUlJSkJqaivz8fNy8edPtkGxRVlaG1NRUpKSkhM25AoD9+/cjKSkJq1atwrvvvut2OLYKx3NmeREsKytDcnKyHbH0O4cPH0ZtbS1OnDjhdii2KS4uxkMPPYSqqip88sknSEpKcjskWyQlJaG2tha1tbU4efIkoqKikJOT43ZYljU3N+P111/HiRMncOrUKfj9fpSXl7sdlmWnTp3CW2+9haqqKtTV1WHv3r04c+aM22FZ5vf7sW7dOuzbtw+//vWvcejQoX5TBtKqcD1nlhbBpqYmfPjhh3juuefsioccdPXqVRw9ehTPPvssACAyMlL9UX2gOnToEKZOnWqqRmp/1NXVhfb2dnR1daGtrQ0TJkxwOyTLvvjiC8ybNw9RUVGIiIhAVlaWWk94oKiqqkJiYiISEhIwePBgPPjgg6isrHQ7LFuE6zmzlB36wgsv4LXXXsO1a9fsigeAnLm2ceNGW19H4/P5sHjxYvh8Pjz//PNYu3btLY9LXwVoWVBSEV8ti8xOZ8+eRXR0NNavX4+6ujrMmjULZWVlGDZsWM+/kbLrtAxKacxaZpqTWYjl5eXIz8/v87/XzllaWpphe6iyWCdOnIiXXnoJkydPxtChQ7F48WIsXry4T7GYKSYeqnGlpqbiF7/4BVpbWzF06FB89NFHP/jZQcoCNVMkWcuIlrIQzWRmNzc3Y9KkSQD++r4+f/48Pvvssx+8x6XseOmGAwCQlZVl2G4mI92Mvpwz6VOvdo2TjrOWKSu9L80w/Ulw7969iImJwaxZs2wLpj+prKxEdXU19u3bh23btuHo0aNuh2RZV1cXqqur8dOf/hQ1NTUYNmwYNm/e7HZYturo6MCePXvw5JNPuh2KLS5fvozdu3ejoaEBFy5cwI0bN/DOO++4HZZlycnJ2LBhAxYtWoQlS5YgLS0NERGO79hyXCAQ+EGbz+dzIRL7hes5M70IVlZWYs+ePYiLi8PKlStRUVGBZ555xs7YXNX9lVNMTAxycnLCIoEkNjYWsbGxyMzMBADk5eWhurra5ajstW/fPmRkZGDcuHFuh2KLgwcPIj4+HtHR0Rg8eDBWrFiBTz/91O2wbFFUVITq6mocPXoUY8aMwd133+12SJbFxsbi/PnzPf/d1NQUFl9fdwvHc2Z6Edy0aROamprQ2NiI8vJyPPjgg2HxFyoA3Lhxo+cr3hs3buAPf/gDUlNTXY7KuvHjx2PSpEmor68H8Nffzu655x6Xo7LXrl27gvoqtL+bPHkyjh8/jra2NgQCARw6dChsEtG++eYbAMBXX32F999/PyzO25w5c3DmzBk0NDSgo6MD5eXlWL58udth2SYcz9nA/yzrgIsXL/ZkFnZ1deHpp5/GkiVLXI7KHm+88QZWrVqFjo4OJCQkiDe3HYja2tpw4MABbN++3e1QbJOZmYm8vDxkZGQgIiICM2fO/MHv0wNVbm4uWltbMXjwYGzbtk39PWygiIiIwJtvvomHH34Yfr8fa9asQUpKitth2SYsz5kdT5KdnR2y5I5QSEhIQF1dndthOCI9PT2stnz0FhUVhdbWVrfDsF1JSYlYZmsg++STT9wOwRFLly7F0qVL3Q7DEeF4zlgxhoiIPMtnlM0k/mOf7xIAeyspu2dKIBCIBsJuXMDfxhau4wLC7pyF67gAzsWBJlzHBfQaW29BLYJEREThhF+HEhGRZwWVGDN27NiA0e7+jo4Osc+XX35p2N7e3h7MS9+WVP4rMTHRsL2xsREtLS0+QB6XlmRx8eJFw3ZtT1BbW5v4mETa7zZo0CCxz8mTJ1sCgUC0NC4ztOoNvfdF9abVJY2MjAw6hu5xAfI5047xn//8Z8P2mJiYoGMZMmSI+Nidd94Z1HP1ZS5qpHl64cIFsc+IESMM27X5a+WcmRnX6dOnDdujoqLEPt3VWv6e9n4xoy9zUSNdPzTSedaupT/60Y8M26Xz35e56Pf7xdeT5lz31gojQ4cONWzX3kdm9gH3Pme9BbUIxsXFGWYWagVipfJYdmdfStmpUoml3uV+pHFJ5dsAuWyalsVXU1MjPiaRypVpJcl8Pt85QB6XGdLNZQG5HNyePXvEPmYW5+5xdfc3GptWAk2aI2a2HGjxB1t6rC9zUSPNU61smnQstD5WzpmZcUnlzLSbrUrvS7tvLtuXuagxcwcG6Txr11Jpu5B0/vsyF7U/iKX5U1ZWJvaRFmrtfaSVcZT0Pme98etQIiLyLC6CRETkWVwEiYjIs2ypGKN9Jy09VlBQIPaRboaq3ftO+53ALO23Tmlcdt8iSPodxqlb3kjf92u3sbLzVjRWab9XfPvtt4btZqqxaLdykX5vcep4mPl9RPrtVHsfSbfssfLe035rlt5j2jmWfjczc4ycpOUbSKQxaM8lnWcrFb6015NyMA4fPhz082m3zLLzfPKTIBEReRYXQSIi8iwugkRE5FlcBImIyLO4CBIRkWdxESQiIs+yZYvE5cuXg+6jpVVPmTIl6D5OMJPSLpUQA8xtJQj1zYqPHDli2K5tg9FSmUPNTHks7ZxJqdih3v6hbdeRtn5o25CktHTtPSb1MVMCrJu27UmibQ+SYulvWySk46yNTTr+2px3YiuV9nrS9hXtGrFz507D9scffzy4wEziJ0EiIvIsLoJERORZXASJiMizuAgSEZFncREkIiLPsiU7VLuRqWT9+vVB99mxY4f4mFMFpYOl3TxSyoSTsqPcIGX6all8UuaoGwW0zWSHaudMysiTikkDzmQxmxmXVIje7OssWLAg6Oe7HW2OSFniZgqea0Wf3bh2SOPWjrGU7Rvq7GzteElrgZbdXFpaathuJes4GPwkSEREnsVFkIiIPIuLIBEReRYXQSIi8iwugkRE5FlcBImIyLNs2SKhpcxKKddaYWgzRXCdSHPWnlNKcZaKGQNyirOUIgw4U0RWKnILyMfYzLi0c2ymGHBfaCn30nHWXlOKU0u5dyK128xxkbYYaLS5YaZQ/u1o50tKt9e2ZEnbBbRz8sQTTxi2W52LWpza/LGzT6jZOfd/+ctfio9JWy7MbMviJ0EiIvIsLoJERORZXASJiMizuAgSEZFncREkIiLPsiU7VCsYLD2mZaFpj/UXUuajlhEmZVg6UXBZox1fKbtK6yONWcq6A+TMLyeL5kqZr9rYpDhDXRxci1Eqbn7u3Dmxj5mi99r5dIKZzHLpMe09JmWUWs041wpbS8+tnZfdu3cbtjuRQd4faOdZKmBv5pzxkyAREXkWF0EiIvIsLoJERORZXASJiMizuAgSEZFncREkIiLPsmWLhEZKZdWKYdfV1Rm279ixw46Q+kxLS5dS+bW0aCmV3Yni3xotvV8a14IFC8Q+UmHf/rbVRUoxLy4uFvtIY9CK+zpBK+Ysbb3R3i9SKr6Wlm61oHSwpPNVU1Mj9pk5c6ZhuzYu6VxafV+aKSCvvTelcYd6i4S2jUM6Ztr2Gumcaa9TWFgoPhYsfhIkIiLP4iJIRESeZXkRjIuLw/Tp05Geno7Zs2fbEVO/EBcXh/vuuw/333+/+lXgQHPlyhXk5eVh2rRpSE5OxrFjx9wOyTbhOhf379+PpKQkJCYmYvPmzW6HY5uysjKkpqYiJSXF0UpBoVZaWoqUlBQUFhbi5ZdfRkdHh9sh2aasrAx5eXnIzc3Ff/3Xf7kdji1s+U3w8OHDGDt2rB1P1a/8/ve/x5133ul2GLYqLi7GkiVL8N5776GjowNtbW1uh2SrcJuLfr8f69atw4EDBxAbG4s5c+Zg+fLluOeee9wOzZJTp07hrbfeQlVVFSIjI7FkyRI8+uijuPvuu90OzZLm5ma8/vrrOH36ND777DP88pe/REVFBZYsWeJ2aJZ1n7O3334bgwcPxrp16zB//nxTN2/uT/h1qIdcvXoVR48eRVFREQAgMjIy5MkOFJyqqiokJiYiISEBkZGRWLlypZgwMpB88cUXmDdvHqKiohAREYGsrCx88MEHbodli66uLrS3t8Pv9+O7774Lmz+ku8/Z0KFDERERgVmzZuHw4cNuh2WZ5U+CPp8P2dnZ8Pl8yM/Px9NPP33L49JXiVKmJABs3LjRsD2UWZQ+nw+PPPIIAOCxxx7DY489dsvjJSUlhv20cUmZsqEqoH327FlER0ejsLAQdXV1mDVrFsrKyjBs2LCefyNltGlZr1Lml3YsnDiXPp8PDz30EHw+H1avXv2D1ygoKDDsp/0h0NjYGHQfOzU3N2PSpEk9/x0bG4vPPvvsln9TWlpq2Hf9+vXi80oZhaH6WjI1NRW/+MUv0NraiqFDh+Kjjz76wVfY0nVAI8UvZdACQFpaWtCvI5k4cSJeeuklTJ48Gf/4j/+IBQsW4J//+Z9/8O+krOOdO3eKzx3q7Pi/133OXn31VQwdOhQ1NTWYPXv2Ldcv6fohZcMC8rVFy67NysrqS8h9YnkRrKysREdHB1paWvDss89i6tSpyMzMtCM2V1VWVuLPf/4zLl++3DOp7XyzuKGrqwvV1dV44403kJmZieLiYmzevBkvv/yy26HZorKyElFRUbh06RJycnJw991345/+6Z/cDsuSQCDwgzafz+dCJPZKTk7Ghg0bsGjRIgwfPhxpaWmIiHB8x5bjLl++jN27d6OhoQHAX//Y++///m/8+Mc/djky68L1nFn+OnTChAkAgLFjx+Lhhx8W9/gNNN3jGj16NO6//3786U9/cjki62JjYxEbG9vzR0peXh6qq6tdjso+3ecsOjoay5YtC4uxxcbG4vz58z3/3dTU1DPOga6oqAjV1dU4evQoxowZM+B/DwSAgwcPIj4+HtHR0Rg8eDAee+wxVFVVuR2WbcLxnFlaBG/cuIFr164BANra2vDJJ58gKSnJlsDc1Htc7e3tOHHiBOLj412Oyrrx48dj0qRJqK+vBwAcOnRowCdYdOt9zm7cuIGKigokJye7HJV1c+bMwZkzZ9DQ0ICOjg6Ul5dj+fLlbodli2+++QYA8NVXX+H9999Hfn6+yxFZN3nyZBw/fhxtbW0IBAI4cuRIWFwTu4XjObP0WfbixYvIyclBR0cH/H4/li9fbut3tW7pHtf169fh9/uxcOFCzJ071+2wbPHGG29g1apV6OjoQEJCguu/M9il+5z5/X74/X7k5uZi4cKFbodlWUREBN588008/PDD8Pv9WLNmDVJSUtwOyxa5ublobW3F4MGDsW3bNowePdrtkCzLzMxEXl4eMjIy4PP5MGPGDPG36IEoHM+ZpUUwISEBdXV1YvLAQNU9LimRZSBLT0/HiRMn3A7Ddt3nrL+VarPD0qVLsXTpUrfDsN0nn3zidgiOKCkpQUlJSVjOxXA8Z9wiQUREnuUzyj4T/7HPdwnAOefCCakpgUAgGgi7cQF/G1u4jgsIu3MWruMCOBcHmnAdF9BrbL0FtQgSERGFE34dSkREnhVUYszYsWMD2v2ujHz55ZfGL6xssvzuu+8M27X9USNGjAgqrsbGRrS0tPgAc+OS9N7T9fdaW1sN26dPny72GTRoUNAxnDx5siUQCETbOS5N95aLvzdu3Dixj5mKK93jAtw/Z0OHDhX7BJsSb3UuSvF3p7PbZerUqYbt2rm0MhfNjEs6L9pcNFPWrC9z0e/3i/0///xzw3bpGAPBX+PMcOq6qB2L7sICf2/IkCFin95VlPqq9znrLahFMC4uLujMQqmklplSVdqNTLUSO0Z6l2gyMy6JdrNgqXSQVn/PzGLh8/nOAfaOSyMdezNluzTd4wLcP2daqbtgs4qtzkUp/rKysqCe53a2bNli2K6dSytz0cy4fvSjHwX1XIC5En59mYtadqi0uGzfvl3sE+w1zgynrovasZCOv5mbf2t6n7Pe+HUoERF5FhdBIiLyLC6CRETkWVwEiYjIsxy/D4b0g6hWak36AVi6NyHw11uYGHHqvm9S8oP2o71UV3Ug3NhWO19HjhwJ+vnMJMZYJZ0z7Qd4KRkrVPfd66YlFkj3Y9NqVkpjlu6TCQA1NTWG7U6dSyn5yMy9LQsLC8U+Tt2nVDtn0v0NtWucRLuzu5k57wTtfoLSTaJDdes6fhIkIiLP4iJIRESexUWQiIg8i4sgERF5FhdBIiLyLC6CRETkWY5vkZBSlrWUWTN3qg/1NgNpXFq6sjRm6bkAOUVfq11phZTWbSaNvL9t/ZC23pipyajN39raWsN2K+fMTK1djZktHto8dYI057QawqNGjTJs37lzpw0RBSfYGrKAvt3EzPwJ9d3tpTGbmW+hqJUK8JMgERF5GBdBIiLyLC6CRETkWVwEiYjIs7gIEhGRZzmeHSpleGl3HZcymrQ7sDtBy+6SCuBqWZRSFp9UQBaQswK17MTb0bIJpfjNFMl2IztUO2dS4WW7My+dKE4sZZwC8ri0PmYyF6WsTK2gtRNmzpwpPiadSy1r2ymjR4+29fmkcYe6GL12jZOycLUYz50zvOF7yK4f/CRIRESexUWQiIg8i4sgERF5FhdBIiLyLC6CRETkWVwEiYjIsxzfIvHCCy8E3UdKjQ1VQdVuZlK/tZR6M8dCKxZslpYeLx177VgUFBQYtof6fN1OWVmZYbtUdBmQt8JopGNlpgj57Z4TAEpKSoJ+PmnMWiq7E3PRDC1GaRuVNhelrSRWt7pocUqPadtaiouLDduzsrLEPk5sM9C2Z0mPaeOStog5sdXICD8JEhGRZ3ERJCIiz+IiSEREnsVFkIiIPIuLIBEReZbj2aFSJmJpaanYR8qE27p1q9jHTObl7WivJ5EyEDVacV8nMqS0LEXpMamoOSBnUGoZYU7RsgADgUDQzycdfy1bMz09PejXuR0tM1M6Z/Hx8UE/nxPvo1CS3rNmisabKTLeV9L80YrpP/HEE4btWrbmQD6fZorXm8FPgkRE5FlcBImIyLO4CBIRkWdxESQiIs/iIkhERJ7FRZCIiDzLli0SZlKJtTRyKX24P6UCS6nYWsFaqdBxfylMrNG2SEhCleJslTZ3pC0STmyDMEt7X0isFPMOFem6ol1vpG05Wh83zqV0zgoLC4N+rv40FyXnzp0Lug+3SBARETnM8iJYWlqK1atXo7CwEC+//DI6OjrsiKtf8Pv9mDlzJpYtW+Z2KLZZs2YNYmJikJqa6nYotqqvr0d6enrP/0aOHGmq2EF/E67junnzJubOnYu0tDSkpKRg48aNbodkq7i4OEyfPh3p6emYPXu22+HYIlyvHZYWwebmZrz++uvYvn07duzYAb/fj4qKCrtic11ZWRmSk5PdDsNWq1evxv79+90Ow3ZJSUmora1FbW0tTp48iaioKOTk5LgdlmXhOq4hQ4agoqICdXV1qK2txf79+3H8+HG3w7LV4cOHUVtbixMnTrgdii3C9dph+ZNgV1cXvvvuO/j9fnz33Xe488477YjLdU1NTfjwww/x3HPPuR2KrR544AGMGTPG7TAcdejQIUydOlUtRzcQhdO4fD4fhg8fDgDo7OxEZ2cnfD6fy1GRJlyvHZYWwYkTJ+Kll17Cj3/8Y+Tm5mLYsGGYM2eOXbG56oUXXsBrr72Gf/gH/mw60JSXlyM/P9/tMGwXbuPy+/1IT09HTEwMFi1ahMzMTLdDso3P58PixYsxa9Ys/OpXv3I7HFJYyg69fPkydu/ejc8//xyjRo3C6tWrUV9fjx//+Mc9/0bKXNKy06SMvFD9FrJ3717ExMRg1qxZQWe+msmi1Io+9xdmxuVG1lpHRwf27NmDTZs29bmPloUmFS0ONW1cUjZ1QUGB+HxaFnOoDBo0CLW1tbhy5QpycnJw6tSpW35vkt7vWmF26dqhZQA7kZ1dWVmJCRMm4JtvvsGiRYswbdo0PPDAA7d9Te1TvpRROhCuH2lpaeJj0phDNS5LH3MOHjyI+Ph4jB07FoMHD8Zjjz2Gqqoqu2JzTWVlJfbs2YO4uDisXLkSFRUVeOaZZ9wOi/pg3759yMjIwLhx49wOxVbhOi7grwtydnZ2WP3eNGHCBABATEwMcnJywuK6GK4sLYKTJ0/G8ePH0dbWhkAggCNHjiApKcmu2FyzadMmNDU1obGxEeXl5XjwwQfxzjvvuB0W9cGuXbvC6ivDbuE2rkuXLvV8u9De3o6DBw9i2rRpLkdljxs3buDatWs9//8Pf/hD2GVUhhNLi2BmZiby8vKQnZ2N++67D99//736FQy5Lz8/H/feey/q6+sRGxuL//iP/3A7JNu0tbXhwIEDWLFihduh2Cocx/X1119jwYIFmDFjBubMmYNFixaFzVakixcvYv78+UhLS8PcuXPx6KOPYsmSJW6HZVm4XjssV4wpKSnB+vXr7YilX8rOzh4Q37n31a5du9wOwTFRUVFobW11OwzbheO4ZsyYgZqaGrfDcERCQgLq6urcDsN24XrtYOojERF5FhdBIiLyLF8gEOj7P/b5LgEIvhJq/zQlEAhEA2E3LuBvYwvXcQFhd87CdVwA5+JAE67jAnqNrbegFkEiIqJwElRizNixYwPSZlSJVFD79OnTYp/IyEjDdu21o6KigoqrsbERLS0tPsDcuPx+v2H7559/LvaRxqVtKxk0aFBQcQHAyZMnWwKBQLSZcUm0zfLnz583bJfGCwDx8fFB9+keF2DunEna2trEx/785z8btmubzSdNmmTYLp1Lq3NRil86LwBw/fr1oF4DkN9/WqlEK3OxnB12dwAAEmpJREFUe5tBX9uBv2adGpk6darYx0zhAKfmonRdAfRrpkQat3S9tDoXpet9fX292Ecas/baVs9Zb0EtgnFxcUEXg5WqcWjVRKTBa/dNC7Y6Se/K7mbGJS0K2omTHjt8+LDYx8zJ9vl857pfz67ivbt37xYfKy4uNmzXjoV0LrU+3ePq/nd2jU2rQCJlBmvp/MHea9LqXJTi16qkHDlyJKjXACDe6UGr/mRlLpq5n6B0z84tW7aIfR5//PFgwgLg3FzU/tg0U4Hp3XffDeq5rM5F6XqvZdhLY3bynPXGxBgiIvIsLoJERORZXASJiMizuAgSEZFnWS6bdjtmblMiJUeY+XHVKVJix7fffiv2kWLUbhHlxG1eNFIsWhxSAoaWyCT9gG5Xll0wtLFJt1LauXOn2EdKFHGq/F6wiTgAUFpaatiulUCUbtmkJcZYIb1eWVmZ2EdK3tmxY4fYx0yShVO0ZCYpmUW7FZh0DJ26zZn0Xjp3LvjthlodamnMZhIJ+UmQiIg8i4sgERF5FhdBIiLyLC6CRETkWVwEiYjIs7gIEhGRZ9myRUKrvSilkmspy1LKtVOp2BItdV6qXyjV0QTk9GcpDR+Qx+zUVgIpxVg7x9LWDyk9G3AuRdsMbXuNtM1DG5uWsu4EbSuKRIpR265jJv3cigULFhi2a+dLes9q860/bdfR4pSuH6F+L5m53mvbHYJ9Li0GM9uQ+EmQiIg8i4sgERF5FhdBIiLyLC6CRETkWVwEiYjIs2zJDjVTvNpMBp1WXFbKCrNSgFrLhpMysrTXk55PG5eUhepUpqz0vNo5lrJb+1OmoUaLU8uEkziRVbh7927xMSnTWssalM6nVuhYm6dOkApbS0WyAfn9EuqMXbO0YyzNU21sTpwzM8WwzWQwS+cSsPd88pMgERF5FhdBIiLyLC6CRETkWVwEiYjIs7gIEhGRZ3ERJCIiz7Jli4SWyjplyhTDdq1otMTMVgwr4uPjxcekFF0zqf9aKruZ1GIrpGOsbcmQitaaKWbrBm0bhJRirm2rcGLchw8fFh+Ttk9o2yrMkOaidiycoB1fqei2tq3CqULZ2nVRekzrI81TrZh7f9mKpF3HzNwkwM61gJ8EiYjIs7gIEhGRZ3ERJCIiz+IiSEREnsVFkIiIPMuW7FAt01PKHDOTkaVlOjmRBSUV8AWAgoICw3atYK00Zi27y0wB59vRinyXlJQYtqelpYl9tPhDTcuuk+bpt99+K/YpLi42bHeqgLlEO2fSuLTzUlZWZtguFeMG+s+YtUxDKRt95syZNkQUHK3Is/Q+00jnJtRZ2FlZWeJjo0aNMmzXMoil66KWAWrn9Z6fBImIyLO4CBIRkWdxESQiIs/iIkhERJ7FRZCIiDyLiyAREXmWLVsktALQUiqrlm4tpXxraeJSyrdTpPi1YyGlAtfV1Yl9tJR1s7R0cSnFXItROl9m0qKt0tLFzRx/MwXMzcyN29FSwqUxa8dfOs+h3gahkbYRaFs/pC0tUuq+k8wUndf6SHNRey9Jj1l5/5mZi1oxd6nouXbO7Jyn/CRIRESexUWQiIg8y/IiuH//fiQlJSExMRGbN2+2I6Z+IRzHdf78eSxYsAA/+9nP8C//8i/4/e9/73ZItukeW3JyMlJSUkL+9bhT6uvrkZ6e3vO/kSNHhvz+fU4pLS1FSkoKUlNTkZ+fj5s3b7odki3CdS4C4XldtPSboN/vx7p163DgwAHExsZizpw5WL58Oe655x674nNFuI4rIiICW7Zswfnz59He3o5//dd/RXp6OiZNmuR2aJZ1jy0jIwPXrl3DrFmzsGjRogF/zpKSknpK5/n9fkycOBE5OTkuR2Vdc3MzXn/9dZw+fRpDhw7FU089hfLy8n71m6RZ4ToXw/W6aOmTYFVVFRITE5GQkIDIyEisXLnS9rtZuyFcx3XXXXchIyMDADB06FDExsaitbXV5ajs0XtsI0aMQHJyMpqbm12Oyl6HDh3C1KlTxaSWgaarqwvt7e3o6upCW1sbJkyY4HZItgjXuRiu10VLnwSbm5tv+RQRGxuLzz777JZ/I2U0aZmeUhFq7Wsgrdh1sPoyLikWrYCzVBB248aNYh8n/jJ+/PHH0djYiK+//hrr16/HyJEjb3nMiJYlKT2mFVaX+ljJoOzW2NiImpoaZGZm3tIuFSPXzpmUiahlKEqZd1bHVl5ejvz8/B+0S/FrFyi3i55PnDgRL730EiZPnoyhQ4di8eLFWLx48S3/RsqM1uaVVNzZjnllhjQXzRTTlzIvpexK7fnMZIdauS5qpELjofra39InwUAg8IM2n89n5Sn7hXAdV7fr168jNzcXW7duvWUBDAfhOraOjg7s2bMHTz75pNuh2OLy5cvYvXs3GhoacOHCBdy4cQPvvPOO22HZKtzmYrheFy0tgrGxsTh//nzPfzc1NYXFVxrhOi4A6OzsRG5uLlatWoUVK1a4HY6twnls+/btQ0ZGBsaNG+d2KLY4ePAg4uPjER0djcGDB2PFihX49NNP3Q7LNuE4F8P1umhpEZwzZw7OnDmDhoYGdHR0oLy8HMuXL7crNteE67gCgQCKioqQnJyMF1980e1wbBXOYwOAXbt2GX4VOlBNnjwZx48fR1tbGwKBAA4dOoTk5GS3w7JFuM7FcL0uWloEIyIi8Oabb+Lhhx9GcnIynnrqKaSkpNgVm2vCdVyVlZV4++23UVFR0ZNy/9FHH7kdli3CeWxtbW04cOBA2HyiAIDMzEzk5eUhIyMD06dPx/fff4+1a9e6HZYtwnUuhut10XLZtKVLl2Lp0qV2xNKvhOO45s+fb/i9fjgI57FFRUWFTRZvbyUlJabusN7fhfNcDMfrIivGEBGRZ/mC+YvF5/NdAnDOuXBCakogEIgGwm5cwN/GFq7jAsLunIXruADOxYEmXMcF9Bpbb0EtgkREROGEX4cSEZFnBZUYM3bs2IBT94Hrrb6+3rDd7/eLfZKSkgzbBw0aZNje2NiIlpYWH2DvuLQYL1y4YNj+3XffiX0SExODjuHkyZMtgUAg2s5xSVUdgL9mLxqRzgkgnxdN97gA+Zxpx7/3HqfepEo+ADB8+HDD9vj4eLFPsGOzOhelc6ONS4pRe+0RI0YEExYAa3NRSga6ePGi2Ke9vd2wXXvtO++8M6i4gL7NRc3nn39u2N7R0SH2iYyMNGzX9uoFO7a+zEXp/Q4AX3zxRVCvB8jjiomJEfuMHTvWsF177/U+Z70FtQjGxcXhxIkTwXQxRSoPpL2pDx8+bNgu3QBy9uzZPf/fznFpMUql4rQFxkx5K5/Pdw6wd1xa+TapHJl0TgD9xpyS7nEB8ti04y+V4zNTqkorIxfs2KzORencaOOSYty+fbvYR7thscTKXJSOsVZOS7pBst2lCfsyFzXSonnunPwT3F133WXYbufY+jIXpfc7oN+wWyKNS3q/AvK4tPde73PWG78OJSIiz+IiSEREnsVFkIiIPMtyxRiztFspHTlyxLB91KhRYh/ptyAzvz1Zof1uIv1eEWra72bSd+1m+oT62AP676vS7zDabw/S7x/a7Xy0WzM5QTr+2lyUflfTbsvT0NBg2G4l+Ur7famwsNCwXbufonSNkJ4LkM+lk/NXmnPa8di5c6dhuzY26RZSVm4tpZ3v4uLioJ9P+u13/fr1Yh8pfjO/W/OTIBEReRYXQSIi8iwugkRE5FlcBImIyLO4CBIRkWdxESQiIs9yfIuElL4tpftqtLTeUNQ07Qstrbq0tNSwXSsD5QRtu8Pu3bsN27OyssQ+2haDUNPmiPSYNGZATt/WjmGot+tIaeFaur20XUdLcXfiPaadL6ns2+OPPy72kbZeaTfvdWN7lfSe0eaimWumE+dMOy7StUy7xn377beG7do1x8oWj7/HT4JERORZXASJiMizuAgSEZFncREkIiLP4iJIRESeZUt2qFYw2ExGk0TLyOsvtGNhpoCzE7Qi0xI3imGHyo4dO8THpDmnZV72l2NlJjPQzqw7q7QsUDv1p+tKWlpa0H20m+r2l7lo5ppj542rNfwkSEREnsVFkIiIPIuLIBEReRYXQSIi8iwugkRE5FlcBImIyLMcL6AtFY3WUrEXLFhg2N6fCjVLhW6l8QL9Z4uEGdrWD6losVTY+XaPhZp2/LVx93daGnlBQYFhu5aWLhXDt0LbaiLNEangslnSmENd2B7Qt7VIBaWlQuOAPLdDvXVCmzvS8TczLjP4SZCIiDyLiyAREXkWF0EiIvIsLoJERORZXASJiMizbMkONZMFaKZorZkirFZo2WHr168P+vm0Qs39nZaRJ2VxlZSUiH2kY+FEBuLtaPNXymKWMmIBPcOyv5Dm9ujRo8U+UqaslUzfc+fOiY9JmZLatUN6Pq0YtxtzzgxpzknZ9IA8F0Odka7tBpCOv/YeY3YoERGRDbgIEhGRZ3ERJCIiz+IiSEREnsVFkIiIPIuLIBEReZbjBbQHMi2tt7i42LBdK7hcWFho2G6mUKyVtHStr1QAXNsSIqU4a1sFpPRnJ9PVpWLNWsq9lKa/c+dOsY+0/cBK0WItRjOp85cvXw46Bun4WZmL2tYF6THt9aTjpM3FUBeTBuQC/IcPHxb7aNeJUNLmopmtC9p7SSLNRe2aLeEnQSIi8iwugkRE5FmWFsE1a9YgJiYGqampdsXTL5w/fx4LFixAQUEBVq9ejffee8/tkGxz5coV5OXlYdq0aUhOTsaxY8fcDskWN2/exNy5c5GWloaUlBRs3LjR7ZBs0T2u+fPn495778WmTZvcDskW9fX1SE9P7/nfyJEjXbl/nxM4FwcWS78Jrl69Gj/72c/wk5/8xK54+oWIiAhs2bIFV69eRVtbG55//nnMnj1bveHlQFFcXIwlS5bgvffeQ0dHB9ra2twOyRZDhgxBRUUFhg8fjs7OTsyfPx+PPPII5s2b53ZolnSPq6urC52dnXjkkUewcOFCzJkzx+3QLElKSur5Xcfv92PixInIyclxOSp7cC4OLJY+CT7wwAMYM2aMXbH0G3fddRcyMjIAAFFRUZg8eTJaWlpcjsq6q1ev4ujRoygqKgIAREZGupIU4ASfz4fhw4cDADo7O9HZ2Qmfz+dyVNaF67h6O3ToEKZOnYopU6a4HYotwvWcheu4XMsO1S6+WVlZhu1a5qUTsrOz0djYiPPnz2Pt2rUYOXLkLY8FS8qc0sZl5yJ19uxZREdHo7CwEHV1dZg1axbKysowbNiw28ao0QrdSsxkhN2O3+/HrFmz8OWXX2LdunXIzMy85XEp87Surk58zlGjRhm2FxQUiH3s/sPC7/cjOzu7Z1wLFy685XEpa1DKoNMe07I1n3jiiT5EG7zy8nLk5+f/oF16Xxw5ckR8Lim7OdR/7N1uLkpfkWpzUaLNRbuzrf1+P+677z6cO3cOzz77LBITE2/JFpXmlTYu6XqvZaSbyQKVMDFGcf36deTm5mLr1q23LIADVVdXF6qrq/HTn/4UNTU1GDZsGDZv3ux2WLYZNGgQamtr0dTUhKqqKpw6dcrtkGwRruMCgI6ODuzZswdPPvmk26HYKlzP2aBBg/DRRx/h2LFjqKurQ319vdshWcZFUNDZ2Ync3FysWrUKK1ascDscW8TGxiI2Nrbnr9K8vDxUV1e7HJX97rjjDmRnZ2P//v1uh2KrcBzXvn37kJGRgXHjxrkdiiPC8ZwBwMiRIzFv3jz1U/lAwUXQQCAQQFFREZKTk/Hiiy+6HY5txo8fj0mTJvX89Xbo0CHcc889Lkdlj0uXLvV8LdPe3o6DBw9i2rRpLkdlXbiOq9uuXbsMvwodyML1nPUe182bN/HHP/4RU6dOdTkq6yz9Jpifn4+PP/4YLS0tiI2NRUlJSU/SxUBWWVmJt99+G9OnT+/57vmVV17B0qVLXY7MujfeeAOrVq1CR0cHEhISBvSNfnv7+uuvUVBQAL/fj++//x5PPfUUli1b5nZYloXruACgra0NBw4cwPbt290OxVbhes66x9Xe3o5AIIBHH30UDz30kNthWWZpEdy1a5ddcfQr8+fPRyAQcDsMR6Snp+PEiRNuh2G7GTNmoKamxu0wbBeu4wL+mnnd2trqdhi2C9dz1j2uxsZGt0OxFb8OJSIiz/IF84nH5/NdAnDOuXBCakogEIgGwm5cwN/GFq7jAsLunIXruADOxYEmXMcF9Bpbb0EtgkREROGEX4cSEZFncREkIiLP4iJIRESexUWQiIg8i4sgERF5FhdBIiLyLC6CRETkWVwEiYjIs7gIEhGRZ/0/uG11N1SL3m0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x432 with 64 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# show images\n",
    "fig = plt.figure(figsize=(6, 6))  # figure size in inches\n",
    "fig.subplots_adjust(left=0, right=1, bottom=0, top=1, hspace=0.05, wspace=0.05)\n",
    "\n",
    "# plot the digits: each image is 8x8 pixels\n",
    "for i in range(64):\n",
    "    ax = fig.add_subplot(8, 8, i + 1, xticks=[], yticks=[])\n",
    "    ax.imshow(digits.images[i], cmap=plt.cm.binary)\n",
    "    \n",
    "    # label the image with the target value\n",
    "    ax.text(0, 7, str(digits.target[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Build a model on training data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In scikit-learn, an estimator for classification is a Python object that implements the methods fit(X, y) and predict(T).\n",
    "An example of an estimator is the class sklearn.svm.SVC that implements support vector classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import svm\n",
    "clf = svm.SVC(gamma=0.001, C=100.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=100.0, break_ties=False, cache_size=200, class_weight=None, coef0=0.0,\n",
       "    decision_function_shape='ovr', degree=3, gamma=0.001, kernel='rbf',\n",
       "    max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "    tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.fit(digits.data[:-500], digits.target[:-500]) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### predicting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([8]), array([8]))"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.predict(digits.data[-1:]), digits.target[-1:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Evaluate the model on the test data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### learning dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(clf.predict(digits.data[:-500]) == digits.target[:-500]).sum() / float(len(digits.target[:-500]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### test dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.97"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(clf.predict(digits.data[-500:]) == digits.target[-500:]).sum() / 500.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### evaluation metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification report:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      0.98      0.99        50\n",
      "           1       0.98      1.00      0.99        51\n",
      "           2       1.00      0.98      0.99        49\n",
      "           3       0.98      0.88      0.93        51\n",
      "           4       0.98      0.94      0.96        51\n",
      "           5       0.94      0.98      0.96        51\n",
      "           6       0.98      1.00      0.99        51\n",
      "           7       1.00      1.00      1.00        50\n",
      "           8       0.90      0.98      0.94        46\n",
      "           9       0.94      0.96      0.95        50\n",
      "\n",
      "    accuracy                           0.97       500\n",
      "   macro avg       0.97      0.97      0.97       500\n",
      "weighted avg       0.97      0.97      0.97       500\n",
      "\n",
      "\n",
      "Confusion matrix:\n",
      "[[49  0  0  0  1  0  0  0  0  0]\n",
      " [ 0 51  0  0  0  0  0  0  0  0]\n",
      " [ 0  0 48  1  0  0  0  0  0  0]\n",
      " [ 0  0  0 45  0  2  0  0  4  0]\n",
      " [ 0  0  0  0 48  0  0  0  0  3]\n",
      " [ 0  0  0  0  0 50  1  0  0  0]\n",
      " [ 0  0  0  0  0  0 51  0  0  0]\n",
      " [ 0  0  0  0  0  0  0 50  0  0]\n",
      " [ 0  1  0  0  0  0  0  0 45  0]\n",
      " [ 0  0  0  0  0  1  0  0  1 48]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "\n",
    "def evaluate(expected, predicted):\n",
    "    print(\"Classification report:\\n%s\\n\" % metrics.classification_report(expected, predicted))\n",
    "\n",
    "    print(\"Confusion matrix:\\n%s\" % metrics.confusion_matrix(expected, predicted))\n",
    "\n",
    "predicted = clf.predict(digits.data[-500:])\n",
    "evaluate(digits.target[-500:], predicted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Deploy to the real system"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "s = pickle.dumps(clf)\n",
    "clf2 = pickle.loads(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([8]), array([8]))"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf2.predict(digits.data[-1:]), digits.target[-1:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* http://scikit-learn.org/stable/tutorial/basic/tutorial.html#introduction\n",
    "* http://nbviewer.ipython.org/urls/raw2.github.com/yhat/DataGotham2013/master/notebooks/4%20-%20scikit-learn%20basics.ipynb?create=1"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:waymo_2d_detection]",
   "language": "python",
   "name": "conda-env-waymo_2d_detection-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}