K-Nearest Neighbors (KNN).ipynb

{
    "cells": [
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# import required libraries\nimport itertools\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.ticker import NullFormatter\nimport pandas as pd\nimport numpy as np\nimport matplotlib.ticker as ticker\nfrom sklearn import preprocessing\n%matplotlib inline\n\nprint('Imported')",
            "execution_count": 37,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "Imported\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "## Given the dataset, with predefined labels, we need to build a model to be used to predict class of a new or unknown case. Our objective is to build a classifier, to predict the class of unknown cases. A specific type of classification called K nearest neighbour is used."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# download the dataset\n!wget -O teleCust1000t.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/teleCust1000t.csv",
            "execution_count": 38,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "--2020-10-11 00:01:31--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/teleCust1000t.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 37048 (36K) [text/csv]\nSaving to: \u2018teleCust1000t.csv\u2019\n\n100%[======================================>] 37,048      --.-K/s   in 0.001s  \n\n2020-10-11 00:01:31 (25.2 MB/s) - \u2018teleCust1000t.csv\u2019 saved [37048/37048]\n\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# load data from file and check data\ndf = pd.read_csv('teleCust1000t.csv')\ndf.head()",
            "execution_count": 39,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 39,
                    "data": {
                        "text/plain": "   region  tenure  age  marital  address  income  ed  employ  retire  gender  \\\n0       2      13   44        1        9    64.0   4       5     0.0       0   \n1       3      11   33        1        7   136.0   5       5     0.0       0   \n2       3      68   52        1       24   116.0   1      29     0.0       1   \n3       2      33   33        0       12    33.0   2       0     0.0       1   \n4       2      23   30        1        9    30.0   1       2     0.0       0   \n\n   reside  custcat  \n0       2        1  \n1       6        4  \n2       2        3  \n3       1        1  \n4       4        3  ",
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>region</th>\n      <th>tenure</th>\n      <th>age</th>\n      <th>marital</th>\n      <th>address</th>\n      <th>income</th>\n      <th>ed</th>\n      <th>employ</th>\n      <th>retire</th>\n      <th>gender</th>\n      <th>reside</th>\n      <th>custcat</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>2</td>\n      <td>13</td>\n      <td>44</td>\n      <td>1</td>\n      <td>9</td>\n      <td>64.0</td>\n      <td>4</td>\n      <td>5</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>2</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>3</td>\n      <td>11</td>\n      <td>33</td>\n      <td>1</td>\n      <td>7</td>\n      <td>136.0</td>\n      <td>5</td>\n      <td>5</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>6</td>\n      <td>4</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>68</td>\n      <td>52</td>\n      <td>1</td>\n      <td>24</td>\n      <td>116.0</td>\n      <td>1</td>\n      <td>29</td>\n      <td>0.0</td>\n      <td>1</td>\n      <td>2</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>2</td>\n      <td>33</td>\n      <td>33</td>\n      <td>0</td>\n      <td>12</td>\n      <td>33.0</td>\n      <td>2</td>\n      <td>0</td>\n      <td>0.0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>2</td>\n      <td>23</td>\n      <td>30</td>\n      <td>1</td>\n      <td>9</td>\n      <td>30.0</td>\n      <td>1</td>\n      <td>2</td>\n      <td>0.0</td>\n      <td>0</td>\n      <td>4</td>\n      <td>3</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Check data type of variables \ndf.dtypes",
            "execution_count": 40,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 40,
                    "data": {
                        "text/plain": "region       int64\ntenure       int64\nage          int64\nmarital      int64\naddress      int64\nincome     float64\ned           int64\nemploy       int64\nretire     float64\ngender       int64\nreside       int64\ncustcat      int64\ndtype: object"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# target (y) field is 'custcat' and has four possible values. \n# 1 - Basic Service\n# 2 - E-Service\n# 3 - Plus Service\n# 4 - Total Service\nvalue = df['custcat']\nvalue.value_counts()",
            "execution_count": 41,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 41,
                    "data": {
                        "text/plain": "3    281\n1    266\n4    236\n2    217\nName: custcat, dtype: int64"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# visualize the data\ndf.hist(column='custcat', bins=50)",
            "execution_count": 42,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 42,
                    "data": {
                        "text/plain": "array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7f3fe5f60240>]],\n      dtype=object)"
                    },
                    "metadata": {}
                },
                {
                    "output_type": "display_data",
                    "data": {
                        "text/plain": "<Figure size 432x288 with 1 Axes>",
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAFEZJREFUeJzt3X+QXfV53/H3xxKxqcQIXJyNLOSKtrQNRg2BHYrLTLqq3UTGjXGmdoNLbWQ7o7TBrTOldbD/qN2mdGgncjrGiV25MOCaWKY2CQRwMwRbZZgG2xKlCKy4UWwZSxAp/BIIU3cET//Yc8tqc6W9ururu/fb92vmzt5zzvec+3324X726NwfpKqQJLXrVaOegCRpcRn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvXQMSTYluX+BjnVTkn+zEMeSTpRBL0mNM+jVlCRrk9yW5E+TPJXkU0k+nuTzM8asS1JJlnfLm5J8J8nzSb6b5IokPw58BnhTksNJnu3GnppkS5LvJTmU5P4kp3bb/kuSP+nW35fkjd36zcAVwIe7Y/3uyf696P9vBr2akWQZcCfwPWAdsAbYNsc+K4BPAm+tqtOAvwk8VFW7gX8E/EFVrayq07tdfg24sBv3WuDDwMvdtq8A5wA/CjwI3AJQVVu7+/++O9bPLkjB0oCWj3oC0gK6CHg98C+q6ki37v4kb5ljv5eB85I8VlVPAE/0G5TkVcD7gYuran+3+r/3tlfVjTPGfhx4Jsmqqjo0VDXSAvGMXi1ZC3xvRsjPqapeAH6e6bP3J5LcleSvHWP4mcBrgD+evSHJsiTXJfnjJM8Be2fsI42UQa+WfB94Q+/a+wwvAH9uxvKPzdxYVb9XVX8HWA38IfDZ3qZZx3kS+N/AX+rz2P8AuAx4C7CK6UtHADnGsaSTxqBXS77B9GWX65KsSPKaJJcADwE/leQNSVYBH+ntkGQiydu7a/U/BA4DL3WbDwBnJfkRgKp6GbgR+ESS13dn8W9K8mrgtG7/p5j+o/JvZ83tAPAXF6lu6bgMejWjql4Cfhb4y8BjwD7g56vqHuCLwMPATqZfsO15FXA18DjwNPC3gF/qtn0VeBT4kyRPduv+ObAL+GY3/t91x/gc0y8C7we+BTwwa3o3AOcmeTbJ7yxQydJA4v94RJLa5hm9JDXOoJekxhn0ktQ4g16SGrckPhl75pln1rp164ba94UXXmDFihULO6ERsZalqZVaWqkDrKVn586dT1bV6+YatySCft26dezYsWOofbdv387U1NTCTmhErGVpaqWWVuoAa+lJ8r1BxnnpRpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGrckPhkrafGsu+YuAK5ef4RN3X2Avde9bVRT0knmGb0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxo39VyDs2n/oqI91z+RHvCXJM3pJap5BL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekho3Z9AnWZvka0l2J3k0yYe69R9Psj/JQ93t0hn7fCTJniTfTvIzi1mAJOn4BvnA1BHg6qp6MMlpwM4k93Tbfr2qfm3m4CTnApcDbwReD/x+kr9SVS8t5MQlSYOZ84y+qp6oqge7+88Du4E1x9nlMmBbVf2wqr4L7AEuWojJSpJO3Aldo0+yDvhJ4Ovdqg8meTjJjUnO6NatAb4/Y7d9HP8PgyRpEaWqBhuYrAT+G3BtVd2WZAJ4EijgV4HVVfX+JL8B/EFVfb7b7wbg7qr68qzjbQY2A0xMTFy4bdu2oQo4+PQhDrzYf9v6NauGOuaoHD58mJUrV456GgvCWpaOXfsPATBxKkc9V8bt+THTuPdkpvnUsmHDhp1VNTnXuIG+1CzJKcCXgVuq6jaAqjowY/tngTu7xX3A2hm7nwU8PvuYVbUV2AowOTlZU1NTg0zlz7j+ltvZsqt/GXuvGO6Yo7J9+3aG/T0sNdaydPS+9O/q9UeOeq6M2/NjpnHvyUwno5ZB3nUT4AZgd1V9Ysb61TOG/RzwSHf/DuDyJK9OcjZwDvCNhZuyJOlEDHJGfwnwHmBXkoe6dR8F3p3kfKYv3ewFfhGgqh5NcivwLabfsXOV77iRpNGZM+ir6n4gfTbdfZx9rgWunce8JEkLxE/GSlLjDHpJapxBL0mNM+glqXEGvSQ1bqAPTEmS5mdd98G12W7auGLRH9szeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMb57ZVaELO/me/q9UfYdM1d7L3ubSOakaQez+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjZsz6JOsTfK1JLuTPJrkQ9361ya5J8kfdT/P6NYnySeT7EnycJILFrsISdKxDXJGfwS4uqp+HLgYuCrJucA1wL1VdQ5wb7cM8FbgnO62Gfj0gs9akjSwOYO+qp6oqge7+88Du4E1wGXAzd2wm4F3dPcvAz5X0x4ATk+yesFnLkkaSKpq8MHJOuA+4Dzgsao6fca2Z6rqjCR3AtdV1f3d+nuBX6mqHbOOtZnpM34mJiYu3LZt21AFHHz6EAde7L9t/ZpVQx1zVA4fPszKlStHPY2h7Np/6KjliVPhwIvj14N+xrkv8Epvej3pGefejGNPZj9Hes5etWzoWjZs2LCzqibnGjfwl5olWQl8GfjlqnouyTGH9ln3Z/6aVNVWYCvA5ORkTU1NDTqVo1x/y+1s2dW/jL1XDHfMUdm+fTvD/h5GbVOfLzXbsmv52PWgn3HuC7zSm15Pesa5N+PYk9nPkZ6bNq5Y9FoGetdNklOYDvlbquq2bvWB3iWZ7ufBbv0+YO2M3c8CHl+Y6UqSTtQg77oJcAOwu6o+MWPTHcCV3f0rgdtnrH9v9+6bi4FDVfXEAs5ZknQCBrl0cwnwHmBXkoe6dR8FrgNuTfIB4DHgXd22u4FLgT3AD4D3LeiMJUknZM6g715UPdYF+Tf3GV/AVfOclyRpgfjJWElqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDVuzqBPcmOSg0kembHu40n2J3mou106Y9tHkuxJ8u0kP7NYE5ckDWaQM/qbgI191v96VZ3f3e4GSHIucDnwxm6f30yybKEmK0k6cXMGfVXdBzw94PEuA7ZV1Q+r6rvAHuCiecxPkjRPqaq5ByXrgDur6rxu+ePAJuA5YAdwdVU9k+RTwANV9flu3A3AV6rqS32OuRnYDDAxMXHhtm3bhirg4NOHOPBi/23r16wa6pijcvjwYVauXDnqaQxl1/5DRy1PnAoHXhy/HvQzzn2BV3rT60nPOPdmHHsy+znSc/aqZUPXsmHDhp1VNTnXuOVDHR0+DfwqUN3PLcD7gfQZ2/cvSVVtBbYCTE5O1tTU1FATuf6W29myq38Ze68Y7pijsn37dob9PYzapmvuOmr56vVH2LJr+dj1oJ9x7gu80pteT3rGuTfj2JPZz5GemzauWPRahnrXTVUdqKqXqupl4LO8cnlmH7B2xtCzgMfnN0VJ0nwMFfRJVs9Y/Dmg946cO4DLk7w6ydnAOcA35jdFSdJ8zHnpJskXgCngzCT7gI8BU0nOZ/qyzF7gFwGq6tEktwLfAo4AV1XVS4szdUnSIOYM+qp6d5/VNxxn/LXAtfOZlCRp4fjJWElqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDVuzqBPcmOSg0kembHutUnuSfJH3c8zuvVJ8skke5I8nOSCxZy8JGlug5zR3wRsnLXuGuDeqjoHuLdbBngrcE532wx8emGmKUka1pxBX1X3AU/PWn0ZcHN3/2bgHTPWf66mPQCcnmT1Qk1WknTiUlVzD0rWAXdW1Xnd8rNVdfqM7c9U1RlJ7gSuq6r7u/X3Ar9SVTv6HHMz02f9TExMXLht27ahCjj49CEOvNh/2/o1q4Y65qgcPnyYlStXjnoaQ9m1/9BRyxOnwoEXx68H/YxzX+CV3vR60jPOvRnHnsx+jvScvWrZ0LVs2LBhZ1VNzjVu+VBHP7b0Wdf3L0lVbQW2AkxOTtbU1NRQD3j9LbezZVf/MvZeMdwxR2X79u0M+3sYtU3X3HXU8tXrj7Bl1/Kx60E/49wXeKU3vZ70jHNvxrEns58jPTdtXLHotQz7rpsDvUsy3c+D3fp9wNoZ484CHh9+epKk+Ro26O8AruzuXwncPmP9e7t331wMHKqqJ+Y5R0nSPMx56SbJF4Ap4Mwk+4CPAdcBtyb5APAY8K5u+N3ApcAe4AfA+xZhzpKkEzBn0FfVu4+x6c19xhZw1XwnJUlaOH4yVpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY1bPp+dk+wFngdeAo5U1WSS1wJfBNYBe4G/X1XPzG+akqRhLcQZ/YaqOr+qJrvla4B7q+oc4N5uWZI0Iotx6eYy4Obu/s3AOxbhMSRJA0pVDb9z8l3gGaCA/1hVW5M8W1WnzxjzTFWd0WffzcBmgImJiQu3bds21BwOPn2IAy/237Z+zaqhjjkqhw8fZuXKlaOexlB27T901PLEqXDgxfHrQT/j3Bd4pTe9nvSMc2/GsSeznyM9Z69aNnQtGzZs2DnjasoxzTfoX19Vjyf5UeAe4J8AdwwS9DNNTk7Wjh07hprD9bfczpZd/V9q2Hvd24Y65qhs376dqampUU9jKOuuueuo5avXH2HLruVj14N+xrkv8Epvej3pGefejGNPZj9Hem7auGLoWpIMFPTzunRTVY93Pw8Cvw1cBBxIsrqbxGrg4HweQ5I0P0MHfZIVSU7r3Qd+GngEuAO4sht2JXD7fCcpSRrefN5eOQH8dpLecX6rqv5rkm8Ctyb5APAY8K75T1OSNKyhg76qvgP8RJ/1TwFvns+kJEkLx0/GSlLjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXGLFvRJNib5dpI9Sa5ZrMeRJB3fogR9kmXAbwBvBc4F3p3k3MV4LEnS8S3WGf1FwJ6q+k5V/R9gG3DZIj2WJOk4UlULf9DkncDGqvqFbvk9wN+oqg/OGLMZ2Nwt/lXg20M+3JnAk/OY7lJiLUtTK7W0UgdYS89fqKrXzTVo+ZAHn0v6rDvqL0pVbQW2zvuBkh1VNTnf4ywF1rI0tVJLK3WAtZyoxbp0sw9YO2P5LODxRXosSdJxLFbQfxM4J8nZSX4EuBy4Y5EeS5J0HIty6aaqjiT5IPB7wDLgxqp6dDEeiwW4/LOEWMvS1EotrdQB1nJCFuXFWEnS0uEnYyWpcQa9JDVubII+yY1JDiZ55Bjbk+ST3VcuPJzkgpM9x0EMUMdUkkNJHupu//Jkz3FQSdYm+VqS3UkeTfKhPmOWfF8GrGMs+pLkNUm+keR/drX8qz5jXp3ki11Pvp5k3cmf6dwGrGVTkj+d0ZdfGMVcB5FkWZL/keTOPtsWtydVNRY34KeAC4BHjrH9UuArTL+H/2Lg66Oe85B1TAF3jnqeA9ayGrigu38a8L+Ac8etLwPWMRZ96X7PK7v7pwBfBy6eNeaXgM909y8Hvjjqec+jlk3Ap0Y91wHr+WfAb/X772ixezI2Z/RVdR/w9HGGXAZ8rqY9AJyeZPXJmd3gBqhjbFTVE1X1YHf/eWA3sGbWsCXflwHrGAvd7/lwt3hKd5v9jovLgJu7+18C3pyk34ccR2rAWsZCkrOAtwH/6RhDFrUnYxP0A1gDfH/G8j7G9MkKvKn75+pXkrxx1JMZRPdPzZ9k+qxrprHqy3HqgDHpS3eJ4CHgIHBPVR2zJ1V1BDgE/PmTO8vBDFALwN/rLgt+KcnaPtuXgv8AfBh4+RjbF7UnLQX9nF+7MCYeZPr7K34CuB74nRHPZ05JVgJfBn65qp6bvbnPLkuyL3PUMTZ9qaqXqup8pj+RflGS82YNGZueDFDL7wLrquqvA7/PK2fFS0aSvwscrKqdxxvWZ92C9aSloG/iaxeq6rneP1er6m7glCRnjnhax5TkFKbD8Zaquq3PkLHoy1x1jFtfAKrqWWA7sHHWpv/XkyTLgVUs8cuJx6qlqp6qqh92i58FLjzJUxvEJcDbk+xl+pt8/3aSz88as6g9aSno7wDe273L42LgUFU9MepJnagkP9a7NpfkIqZ79NRoZ9VfN88bgN1V9YljDFvyfRmkjnHpS5LXJTm9u38q8BbgD2cNuwO4srv/TuCr1b0KuJQMUsus13vezvTrK0tKVX2kqs6qqnVMv9D61ar6h7OGLWpPFuvbKxdcki8w/c6HM5PsAz7G9IszVNVngLuZfofHHuAHwPtGM9PjG6COdwL/OMkR4EXg8qX4JOxcArwH2NVdRwX4KPAGGKu+DFLHuPRlNXBzpv/nP68Cbq2qO5P8a2BHVd3B9B+1/5xkD9NnjZePbrrHNUgt/zTJ24EjTNeyaWSzPUEnsyd+BYIkNa6lSzeSpD4MeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktS4/wsn7CiVG1mRGAAAAABJRU5ErkJggg==\n"
                    },
                    "metadata": {
                        "needs_background": "light"
                    }
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# define feature set (X)\ndf.columns",
            "execution_count": 43,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 43,
                    "data": {
                        "text/plain": "Index(['region', 'tenure', 'age', 'marital', 'address', 'income', 'ed',\n       'employ', 'retire', 'gender', 'reside', 'custcat'],\n      dtype='object')"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "## KNN use scikit-learn library, we have to convert the Pandas data frame to a Numpy array:"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# convert to int or we can use  '.astype(float)' or float\nX = df[['region', 'tenure','age', 'marital', 'address', 'income', 'ed', 'employ','retire', 'gender', 'reside']] .values  #.astype(float)\nX[0:5]",
            "execution_count": 44,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 44,
                    "data": {
                        "text/plain": "array([[  2.,  13.,  44.,   1.,   9.,  64.,   4.,   5.,   0.,   0.,   2.],\n       [  3.,  11.,  33.,   1.,   7., 136.,   5.,   5.,   0.,   0.,   6.],\n       [  3.,  68.,  52.,   1.,  24., 116.,   1.,  29.,   0.,   1.,   2.],\n       [  2.,  33.,  33.,   0.,  12.,  33.,   2.,   0.,   0.,   1.,   1.],\n       [  2.,  23.,  30.,   1.,   9.,  30.,   1.,   2.,   0.,   0.,   4.]])"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# check labels of target \ny = df['custcat'].values\ny[0:5]",
            "execution_count": 45,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 45,
                    "data": {
                        "text/plain": "array([1, 4, 3, 1, 3])"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "## Normalize Data"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "### Data Standardization give data zero mean and unit variance, it is good practice, especially for algorithms such as KNN which is based on distance of cases:"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "X = preprocessing.StandardScaler().fit(X).transform(X.astype(float))\nX[0:5]",
            "execution_count": 46,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 46,
                    "data": {
                        "text/plain": "array([[-0.02696767, -1.055125  ,  0.18450456,  1.0100505 , -0.25303431,\n        -0.12650641,  1.0877526 , -0.5941226 , -0.22207644, -1.03459817,\n        -0.23065004],\n       [ 1.19883553, -1.14880563, -0.69181243,  1.0100505 , -0.4514148 ,\n         0.54644972,  1.9062271 , -0.5941226 , -0.22207644, -1.03459817,\n         2.55666158],\n       [ 1.19883553,  1.52109247,  0.82182601,  1.0100505 ,  1.23481934,\n         0.35951747, -1.36767088,  1.78752803, -0.22207644,  0.96655883,\n        -0.23065004],\n       [-0.02696767, -0.11831864, -0.69181243, -0.9900495 ,  0.04453642,\n        -0.41625141, -0.54919639, -1.09029981, -0.22207644,  0.96655883,\n        -0.92747794],\n       [-0.02696767, -0.58672182, -0.93080797,  1.0100505 , -0.25303431,\n        -0.44429125, -1.36767088, -0.89182893, -0.22207644, -1.03459817,\n         1.16300577]])"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# spliting the dataset into Train/Test sets mutually exclusive, which will provide a more accurate evaluation on out-of-sample accuracy\nfrom sklearn.model_selection import train_test_split\nX_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.3, random_state=4)\n\nprint ('Train set:', X_train.shape,  y_train.shape)\nprint ('Test set:', X_test.shape,  y_test.shape)",
            "execution_count": 47,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "Train set: (700, 11) (700,)\nTest set: (300, 11) (300,)\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "## Classification"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Import K-Nearest Neighbor (KNN) library\nfrom sklearn.neighbors import KNeighborsClassifier\n\nprint('Imported')",
            "execution_count": 48,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "Imported\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "## Training"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Lets start the algorithm w/ k=4, where k is the number of nearest neighbors to examine.\nk = 1\n\n#Train Model and Predict  \nneigh = KNeighborsClassifier(n_neighbors = k).fit(X_train,y_train)\nneigh",
            "execution_count": 49,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 49,
                    "data": {
                        "text/plain": "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n           metric_params=None, n_jobs=None, n_neighbors=1, p=2,\n           weights='uniform')"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "## Predicting"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "### So, how can we choose right value for K? The general solution is to reserve a part of your data for testing the accuracy of the model. Then chose k =1 use the training part for modeling, and calculate the accuracy of prediction using all samples in your test set. Repeat this process, increasing the k, and see which k is the best for your model."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# predict the test set\nyhat = neigh.predict(X_test)\nyhat[0:5]",
            "execution_count": 50,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 50,
                    "data": {
                        "text/plain": "array([3, 3, 1, 4, 4])"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "## Accuracy evaluation"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "### In multilabel classification, accuracy classification score is a function that computes subset accuracy. This function is equal to the jaccard_similarity_score function. Essentially, it calculates how closely the actual labels and predicted labels are matched in the test set."
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "from sklearn import metrics\nprint(\"Train set Accuracy: \", metrics.accuracy_score(y_train, neigh.predict(X_train)))\nprint(\"Test set Accuracy: \", metrics.accuracy_score(y_test, yhat))",
            "execution_count": 51,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "Train set Accuracy:  1.0\nTest set Accuracy:  0.28\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# calculate the accurancy of KNN for different Ks\nKs = 12\nmean_acc = np.zeros((Ks-1))\nstd_acc = np.zeros((Ks-1))\nConfustionMx = [];\nfor n in range(1,Ks):\n    \n    #Train Model and Predict  \n    neigh = KNeighborsClassifier(n_neighbors = n).fit(X_train,y_train)\n    yhat=neigh.predict(X_test)\n    mean_acc[n-1] = metrics.accuracy_score(y_test, yhat)\n\n    \n    std_acc[n-1]=np.std(yhat==y_test)/np.sqrt(yhat.shape[0])\n\nmean_acc",
            "execution_count": 52,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 52,
                    "data": {
                        "text/plain": "array([0.28      , 0.27333333, 0.30666667, 0.33      , 0.34      ,\n       0.31666667, 0.34      , 0.31666667, 0.32666667, 0.33666667,\n       0.30666667])"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "### Plot model accurancy for different K's"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "plt.plot(range(1,Ks),mean_acc,'g')\nplt.fill_between(range(1,Ks),mean_acc - 1 * std_acc,mean_acc + 1 * std_acc, alpha=0.10)\nplt.legend(('Accuracy ', '+/- 3xstd'))\nplt.ylabel('Accuracy ')\nplt.xlabel('Number of Nabors (K)')\nplt.tight_layout()\nplt.show()",
            "execution_count": 53,
            "outputs": [
                {
                    "output_type": "display_data",
                    "data": {
                        "text/plain": "<Figure size 432x288 with 1 Axes>",
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3XucVPP/wPHXe247e+1OiMolKdVWSxQpSveQrwo/VG4ht75f5BYil9zCN5SkkJKob6UkKkRiu+iiUpEk6d7e5/r5/TGzWdna2d0558zMfp6Ph4ed2XPm8+40zXvOOZ/P+y1KKTRN0zQt1tisDkDTNE3TSqMTlKZpmhaTdILSNE3TYpJOUJqmaVpM0glK0zRNi0k6QWmapmkxSScoTdM0LSbpBKVpmqbFJJ2gNE3TtJjksDqAaKldu7Zq0KCB1WFomqZpZVi+fPkepVSdsrZLmATVoEEDsrOzrQ5D0zRNK4OI/BrJdvoSn6ZpmhaTdILSNE3TYpJOUJqmaVpMSph7UJqmaZXl8/nYvn07RUVFVoeSENxuN/Xq1cPpdFZof52gNE3TwrZv3056ejoNGjRARKwOJ64ppdi7dy/bt2+nYcOGFXoNfYlP0zQtrKioiFq1aunkFAUiQq1atSp1NqoTlKZpWgk6OUVPZY+lTlCapmlaTNIJStNK4QsE8fqDVocRE/RxMN+MGTMQETZs2GB1KJbSCUrTSggEFQcLfezL93Kg0EsgqKwOyVJ5Hj8HCrz4AjpJmWnKlCmcd955TJ061dBxAoGAoa9fWXoWn6YBwaAi3+un0BugOCUpBQcKvNRMdVXJ+xJFvgD5Hj8ABwp81Ep1YbNVneNw1yd3sWrnqqi+ZmbdTEZ3HX3UbfLy8vj6669ZtGgRvXv35tFHHz30u1GjRvHOO+9gs9no1q0bTz/9NJs3b2bw4MHs3r0bu93OBx98wG+//cZzzz3HnDlzABgyZAhZWVkMGDCABg0aMGjQID799FOGDBlCbm4u48aNw+v1cuqpp/LOO++QkpLCn3/+yeDBg/n5558BeO2115g3bx61a9fmzjvvBODBBx/k2GOP5Y477ojqcSqmE5RWpSmlKPAGyPf6UaWcLPmDipwiP9WSK7aOI175A0FyinyHHgdV6MyyRqrLwqiqhpkzZ9K1a1caNWpEzZo1WbFiBa1atWLevHnMnDmTZcuWkZKSwr59+wC4+uqrGTZsGJdddhlFRUUEg0F+++23o47hdrtZsmQJAHv37uXGG28E4KGHHuLNN9/k9ttv54477uCCCy5gxowZBAIB8vLyOP744+nTpw933nknwWCQqVOn8t133xl2LHSC0qqsQm+API+fYGmZqYQiXwCnXUhxVY1/LiqcjA4/LN5AkNwiH+nuqpGsyzrTMcqUKVO46667AOjfvz9TpkyhVatWfPbZZwwcOJCUlBQAatasSW5uLr///juXXXYZEEo8kejXr9+hn9euXctDDz3EgQMHyMvLo0uXLgAsXLiQt99+GwC73U61atWoVq0atWrVYuXKlfz555+0bNmSWrVqRe3Pfriq8S9O00rw+APkFfnxl+P+Ul6RH4fNhsuR+LdtcwqPfGwKvAGcdhtup93kqKqGvXv3snDhQtauXYuIEAgEEBFGjRqFUuofl5rVEb5cORwOgsG/7hsevhYpNTX10M8DBgxg5syZtGjRgokTJ7J48eKjxnjDDTcwceJEdu7cyaBBg8r5JyyfxP/XpmlhvkCQ/fleDhT4ypWcABRwsNBHMMEnTeR7/BT5j37jPKfQh19PmjDE9OnTufbaa/n111/ZunUrv/32Gw0bNmTJkiV07tyZCRMmUFBQAMC+ffvIyMigXr16zJw5EwCPx0NBQQH169fnxx9/xOPxcPDgQT7//PMjjpmbm8txxx2Hz+dj8uTJh56/6KKLeO2114DQZIqcnBwALrvsMj755BO+//77Q2dbRtEJSkt4gaDiYEFoZp63Eh+sQaU4UOgre8M45fUHyQtPijgaBRwo9B3x27tWcVOmTDl0ua5Ynz59mDx5Mp0u7kKPnr3IysoiMzOT5557DoB33nmHl19+mebNm9O2bVt27tzJiSeeSN++fWnevDlXX301LVu2POKYjz/+OG3atKFz5840btz40PMvvfQSixYtolmzZrRu3Zp169YB4HK56NixI3379sVuN/ZMWhLlTZaVlaV0w0KtpGBQkef1U1RiZl40JLvsZCTYfZhAULE331PqRJEjSXLYqJ6SWJMm1q9fzxlnnGF1GIcEg4rAYX8pAjjs1p1bBINBWrVqxQcffMBpp51W5valHVMRWa6UyiprX30GpSUcpRT5Hj978jx/mzYeLYXeAEW+2F4/Uh5HmhRRFo8/eGgauhZdwaDCFwj+IzlB6AzWqkvNP/74I6eeeioXXXRRRMmpsgydJCEiXYGXADswXin19GG/HwzcBgSAPOAmpdSP4d81B8YCGUAQOEsppWvga0cV6cy8ysop9GG3CU4Lv8lGS67HX+GFuHkePw67kOTQkyaiIagUwaAq80tVQClEmV83sEmTJofWRZnBsH9dImIHxgDdgCbAlSLS5LDN3lNKNVNKZQKjgBfC+zqAd4HBSqmmQAcgcS/+a5VW5AuwJ89DTpHP8OQE4fswBfE/aaLQG6DQW7mzwYOFvipfcaOylFL4A0ECESSnYqWdXSUaI7/+nQ1sVkr9rJTyAlOBS0puoJTKKfEwFQ793VwMrFZK/RDebq9SKnGuqWhR4/UH2ZfvteRDMqjU3xazxhtfeF1TZRVX3EiU+9lmUkoRCAbxlyMx/bUvpnwZs5KRCeoEoORy5u3h5/5GRG4TkS2EzqCK62U0ApSIzBeRFSJyb2kDiMhNIpItItm7d++OcvhaLPMHghwo8LLf4jpxnghnvsWaYFBxoMAXtftzxRU3tMiEEpPCH1RU5ntVvJ/Bl8XIBFXaxdF/HE2l1Bil1CnAfcBD4acdwHnA1eH/XyYiF5Wy7zilVJZSKqtOnTrRi1yLWcFg6Kxlb74XT4xU2c73+ONu0sTBwuhfCi3yBSjw6iRVlr8SU+WPvwq/XqIyMkFtB04s8bgesOMo208FLi2x7xdKqT1KqQJgLtDKkCi1uKCUIq/EzLxYk1MUP4tXc4t8lVoPdjR5Rf6Eas/xZ05R1P7740AhvkAwKonp26VLGXzzTUDoMl9pl1dff/11mjVrRmZmJueddx4//vhjucfZunUr77333hF/36FDB4xc3mNkgvoeOE1EGoqIC+gPzCq5gYiUnKfYA9gU/nk+0FxEUsITJi4Ayn90tYRQ4PWzO89Dvscf9Snj0aJUfCxeDZ3lGJfgq0rFjfIIFk+AKOd744vFi7l+0MBSfzd//nwuLlHFobSzqKuuuoo1a9awatUq7r33XoYOHVq+wCk7QRnNsASllPIDQwglm/XANKXUOhEZISK9w5sNEZF1IrIKGApcF953P6EZfd8Dq4AVSqmPjYpVi03FM/Nyi0qvNB5rAkFFTmHsXuLyB4LkmFAJI9ErbkSqIjPzIrVo4UIuuqjTX2Pxz/tRGRkZh37Oz88/NCX9hRdeOFRDb82aNZx55pkUFBTwxRdfkJmZSWZmJi1btiQ3N5dhw4bx1VdfkZmZyYsvvkhhYSH9+/enefPm9OvXj8LCwij/yf7O0HVQSqm5hC7PlXxueImf7zzKvu8SmmquVTHBYOgDLh6b5BX5Azg8QmpSbNVhVuGkYVae94XbdSRaxY1IKBWq/mDUl6o9e/bgdDqpVq3a354vbW3UmDFjeOGFF/B6vSxcuBCAu+66iw4dOjBjxgxGjhzJ2LFjSUlJ4bnnnmPMmDG0a9eOvLw83G43Tz/99N/6Sr3wwgukpKSwevVqVq9eTatWxt55if9VhlrCySmKz+RULM/jx1NGwVWzWTENP9EqbpTlb1PGK3Go2517LlmtWzH45puYM3s2Wa1bkdW6FZ/Onw/AggWf0qlz51L3Pfyv+LbbbmPLli0888wzPPHEEwDYbDYmTpzINddcwwUXXEC7du1C47Zrx9ChQ3n55Zc5cOAADsc/v2R9+eWX/N///R8AzZs3p3nz5hX/g0ZAJygtpoQ+3OM3ORWLpcWr+RYe05w4PRMuj2hNGS/29dKlZC9fwetjx9GzVy+yl68ge/mKQ/ec5n/yyaGfb7h+EFmtW9G7Zw/gyBMm+vfvf6jiOcCmTZtIS0tjx46/5q0NGzaM8ePHU1hYyDnnnMOGDRtKjc/M6hU6QWkxo2SL8XinFOH6dtYmKY8/YOk6rUSpuHEkwShOGY+EUoo1a9aQmZkJwPg3J5C9fAWz5vx1i774i9GmTZsOPffxxx8fqp138OBB7rzzTr788kv27t3L9OnTAdiyZQvNmjXjvvvuIysriw0bNpCenk5ubu6h12nfvv2hlhxr165l9erVhv55Y+tCuVZlHd5iPBGE7sNY1y4+EAwVgbVaPLeLPzaj9A61kdbMi7YVy5eTmZl51LOY4gkT//3vf/nss89wOp3UqFGDSZMmAXD33Xdz66230qhRI9588006duxI+/btGT16NIsWLcJut9OkSRO6deuGzWbD4XDQokULBgwYwC233MLAgQNp3rw5mZmZnH322Yb+eXW7Dc1ySin25XvL3UQwXmS4nSS7zC2mGovHNDXJQVqMTR45XFntNqxKTMWeHDmSU049hX79+pe5rcMmpheTLU1l2m3E9rtFqxKO1mI8EeQW+XDYza18nlPOlvZmyPf4cdgkLtvFGz0zL1IPPPhgxNsGlcIeAwmqMvQ9KM1SBd6yW4zHO7PvwxR4Y7f0UjxV3IDozcyzQjABisnqBKVZxusPkldFCowW34cxWqwf03iouKHCM+GiOTPPKlZPTqns37NOUJolAkHFgUJvzJYuMoI3Su0tjiQYnhQR68c0lituuN1udu/ZE7WaeVazspisUoq9e/fidpc+0SQS+h6UZrqKthhPBAXeAE67Ler3YYorRcTLh2osVtzw+AOk1jiGfbt3smtXYrXvsYlgxe0ot9tNvXr1Krx/7Lw7tCqjMi3GE0FOoQ+HTXBEcdJEPB7TWGkX7wuELot6A0Gw2alx7D/a1sU9t8NOtZT4KzulL/FppopGi/F4p4jufZgiX/weUysrbgSCioMFPvblew1rPxIrivyBmCu/FQmdoDTTRKvFeCKI1iJan0kVyo1iRcWNQ00v8zwJP4O0pFBXgPi4BFxMJyjNFNFuMZ4IKtsuPlGOaXHFDaMppcj3+NmTH2p6Ge/HrbwCQWVoLzAj6ASlmcKIFuOJIL8Slc8T6ZgafZmy0BtgT56XPE989BYzSr7HHzNFjCOhE5RmuDyPP+Gv8VdGRe7DJOIxzTWgzUpx08ucosRJ5pWhIK4us+sEpRkqkSqUG0UpOFDgjfj+QKIe02hW3PAFguzP98ZU25NY4fEHY7bSyOF0gtIMY1aL8UTgD6qI7sMk+jGtbMUNfyBYZWbmVUa8TJjQCUozhNktxhNBkS9AgffISaqqHNOKVNwonpm3L99bpWbmVVRQKUv7hEVKL9TVDJFTGF83Y2NFXpEfh82Gy/HP745V6ZhGWnFDKUW+N0CBx5/wiTvaCrwBkp32qC4Yj7bYjUyLW/mexK9QbhRFeHbeYYmoKh7TnMKjVz4vnpmXr5NThZkxvb8ydILSosrqFuOJIFh8KS98j6CqHtMjVdzQM/OixxcIxnQVEp2gtKiJlRbjicAXCJIbXrNSlY9pyT+/1x9kn56ZF3W5HvN6lZWXvgelRUVVrlBulEJvAI8vWOWPqccfZG+eJ+Y6BCcKpULFhqslx14xWX0GpUVFTlH8VdOOB/oSVohOTsYq8gXw+mPv369OUFqlFXoDcbPwT9O00sVihQlDE5SIdBWRjSKyWUSGlfL7wSKyRkRWicgSEWly2O9PEpE8EfmPkXFqFacrlGtaYvAH1VHX4VnBsAQlInZgDNANaAJceXgCAt5TSjVTSmUCo4AXDvv9i8A8o2LUKidRqmlrmhaSVxRba+2MPIM6G9islPpZKeUFpgKXlNxAKZVT4mEq/PVZJyKXAj8D6wyMUauERKqmfThfwMfBogNWhxETcj05ZW+kJQRFKEnFCiMT1AnAbyUebw8/9zcicpuIbCF0BnVH+LlU4D7gMQPj0yoht8iXkLXOgirIjA3vc/6kTLLePJ2Ne9dbHZKlHvniPpqOPZFvtn9ldSiaSWKp+66RCUpKee4fX7eVUmOUUqcQSkgPhZ9+DHhRKZV31AFEbhKRbBHJ3r17d6UD1iITqhkXG2/gaFFKsXDrp1w8uS23zBtAqiuNJIebgbP6VtkzqWk/TmbsipexYePGOVezPWeb1SFpJomVYrJGJqjtwIklHtcDdhxl+6nApeGf2wCjRGQrcBfwgIgMOXwHpdQ4pVSWUiqrTp060YlaO6pErKa9/I9l9JnehatmXEKuN4dXu73FgquX8mbP99iWs5Vb5g0gEEyshFyWVTuXc89nt9HuxAv45Kqv8QY8DJrdn0J/odWhaSYIBEM1Dq1mZIL6HjhNRBqKiAvoD8wquYGInFbiYQ9gE4BS6nylVAOlVANgNPCkUuq/BsaqRSDRqmlv3LuegbP60mNqBzbv+4mnLhzNV9etok/j/tjERpsT2jGywwss3DqfUUtHWB2uaXbn/8mg2f2ok3osY7u/Q+PaTXi120TW7FrFPQtui4lv1prxCjz+o9ZCNINhlSSUUv7wWc98wA5MUEqtE5ERQLZSahYwREQ6AT5gP3CdUfFolZcoJWa252zj2aVP8MH6yaQ60xjW9lFubDWEVGfqP7a9tvkNrNm1ipe+G0XTOs3p3ehyCyI2jzfg5YY5V7G/aB+z+i2kdkroykTnk7txX9tHePqbR2l2bCY3t7rD4kg1o4W67/qpkeqyLAZJlG9DWVlZKjs72+owEla+xx/3BUv3Fu7h5e9G8dYPYxGEgZmDueOse6iZXOuo+3n8Hi6f3oV1u9fwcf/FNKnTzKSIzTds4V1M/GEsr3WbyGWN+/3td0opbphzFfO2zGLqZbNpX/9Ci6KsmvxBP2t3/UCLY1shUtotfmNUS3aW2fakvERkuVIqq6ztdCUJrUzxXk0735vH898+SZsJTXhj5Rj+dcZVfDNwDY+2f7rM5ASQ5EjizZ5TyEjKYMCsvuwv2mdC1OabvOYtJv4wlltb3/2P5AQgIrzc5Q0a1TyDm+dew68Ht5ofZBWklGLOphl0eLs1Xaecx4QfXjd1fCsnTOgEpR1VPFfT9ga8jF/5Km3easqzSx/ngpMuZPE1y3mh82uckH5i2S9QwrFpxzGh11R25u9g8MfX4g/Gb8IuTfaOb7l/0V10qN+JB897/IjbpbrSmNh7GkEVZOCsvuT78k2MsupZsm0x3aaczw1zrkLERuaxrXnq60f4I+9302IIKkWuRV9QdYLSjkgpxYECb9xV0w4EA0xfP4V2E5vz0OJ/c3qtM5jb/wve7DWVRrUaV/h1Wx/XhqcvfJkvtn3OyCUPRzFia+3M28H1c67kuLQTeK37JOy2o1/OaVD9ZF7v/jYb9q7j7k9v1pMmDPDDnyvo92FP/vVhN3YX/Mnoi8ex+JpsXu/+Nv6Aj4cWmVv9rdAbsKQYtE5Q2hHlFPnjqoq0UooFP8+j0+RzGPLJIKq7azC1z2ymXz6PVsedHZUxrjrzOga2uJnXlo/mow1To/KaVvL4PVw/50pyvblM7D2NGu6aEe3XsUFnHjzvcWb99CH/zX7e4Cirji37N3HTx/9Hl/fasWbXKh674Bm+HrCG/k2vwW6z06D6yQw95wE+3jyTT7d8bGpsuRZUmNCTJLRSFXj9lrwhK+q7379h5JKHWbbjGxpWP4X72j5C70aXY5PofwfzBXxc8WF3Vu3MZla/hTQ/tmXUxzCDUop/f3Yr762dyPie79HztMvKvf8t867jfxunM/mymVzY4GKDIk18O/N28MK3TzF57VskOdzc3OoObm19F+lJGf/Y1hvw0nnyueR5c/ny2hWkutJMizPD7STZVfkJE3qShFZhXn8wpupxHc36PWu59n//ove0i9h68GdGXfQKX167kktPv8KQ5ATgtDt5o+dkaibXYuDsfuwpiM8qJpNWv8F7aydy19n3lTs5QWjSxAudX6dpnebcMvc6ft6/2YAoE9uBov088dVDnPvWmUxZN4nrWtzIsoHruK/t8FKTE4DL7uLZi17h99zfGLX0yPcLjWB29119BqX9jVKKPXnemC8Cu+3gr4xaOoIP108hPSmD27P+zfUtbyPFmWJaDD/8uYJL3r+IVsedzft95uC0x15H0iP5dvsS/vVhNzrU78yk3h+Ued/paLYd/JWu77WjdsoxzL3yC9Jc6VGMNDEV+Ap4c9Wr/Pf758nxHOTyxv2559yHqV+9YcSvce9nt/Pu2gnMv+prmh2TaWC0f5fispPurtx7PdIzKJ2gtL/J8/jJj+Ep5bsLdvHSsmeYtPoN7GLn+pa3MuSsf0d87yTaPvjxPW6ffz03ZN7KEx3j417M77m/0eW986iWVI25/b+kmrt6pV9zybbF9PuoJxef3J03e0017Ow13vkCPqaue5vnvx3Jzvw/6NSwGw+0e6xCa+sOFO3n/EktOT79BOb2/7JSXzLKI9llJ8OkBKXfRdohgaCiIEaTU64nh2eXPsE5E5ry1g9j6d/0WpYOWsvD54+0LDkBXNHkKm5udTvjV73K1HXvWBZHpAr9hQyc1Z8ifyFv9ZoWleQEcN5JHXi0/dPM2zKb0cueicprJpKgCvK/jdO54O1W3PP5EE7MqM/MKxbw7qUfVXjhd3V3DR7vMIof/lzBWyavjTKLYaWOtPiT5/HHXJ09j9/DpNXjGP3dKPYV7qHXaX24r+0jnFqzkdWhHfLw+U+ybvca7vv8dhrVakyrumdZHVKplFLc+9ntrN61gkm9P6jUlPvS3NDyNlbvWsWopSNoWqcZXU7pGdXXj1df/Po5I5cMZ/WuFTSu1ZS3e0+n88ndo1IN4pJGV/D+und56utH6X7qJRyfXi8KEccOfQalAaEq5UU+66sXFwsEA7y/7l3aTmzG8C/u5cw6zfnkyiW80XNyTCUnAIfNwdge73BMal2un92fXfk7rQ6pVONXjuGD9ZO559yHDUkeIsKoTq/Q4thW3PbJIDbt2xj1MeLJyp3ZXDG9O/0+6sm+oj283GU8n//fMi4+pUfUShWJCE9f9DJBFeChxf+OymvGEp2gNICYKmW0/I9lXPju2dz56Y3UTqnDtD4fM+3yj8ms29rq0I6oVnJtJvaexoGi/dww5yq8Aa/VIf3Nkm2LefTLYXQ7pRd3txlm2DjJjmQm9JqK25HMgFlXkOM5aNhYsWrTvo1cP/tKuk05nx/3rOWJDs/x9XWr6dvkakPuE9Wv1oCh5zzA3M2z+GTL7Ki/vpX0JAkNrz/I/oLY+ED99eBWur7XjjRXOsPPf5Kep11mamHMypq58QMGz72Wa5vfyKiLXrY6HMCaWXYlZwm+fcn0KjFpYkfudp7/9kmmrJtEsiOFW1rfxeDWd5hyvH0BH50nn0uO5wBfXrfS0DH1JAnNVLlFsVFrL9+Xz8BZfQmqIB9cPpdejfrEVXICuPT0KxiS9W/eXv0G76x+0+pwKPAVMHB2X/zKz8Te00ybAn5OvfN4osPzfPbLPJ41ea2O2fYX7WPElw/Q9q1mfLB+Mjdk3sqyQT/yn3MfNO14O+1Onuv0X/7I28GobxKnd5meJFHFFfkCMVHOSCnF3Z/ezIa965h86UwaVD/Z6pAq7P52j7Fu92oeWHQ3jWs34azjz7UkDqUUQxcM5sfda3j30hmcXONUU8e/rvmNrNm1iheXPU3TOs0rtBg4luX78hm/cgxjsl8g15PDFU2u5j/nPMRJ1epbEk/W8edwbfMbGb/qVS4/40paHNvKkjiiSZ9BVWFKqZgpZ/Tf7OeZ9dOHPHje43Rs0NnqcCrFbrPzWvdJnJB+ItfPvsrUytMlvbr8RWZu/IAH2j3GRQ27mD6+iPBkxxdpfdzZ3DH/RtbvWWd6DEbwBXxM/GEc5044k6e+foRzTzifhdd8z8td3rAsORV7oN1j1E4+hns+G5IQFfd1gqrCCryBmKgYsXDrpzy5ZDiXnn4Ft7a+2+pwoqK6uwYTe39Ani+XQbP7U+QvMnX8RVsXMHLJw/Q6rQ9DzjK38nVJxb200l3pDJh1RVz30gqqIDM3TuP8SZkMW3gnJ9c4hdn9FjLpkg84o3ZTq8MDoJq7Oo93eJbVu1YyYdVrVodTaTpBVVHBoCLfa/03rJ/3b+aWudfRtE5zXuj8etzdczqaxrWb8EqX8azcmc39C+8yrS3FLwe2MHjutZxeqwmju4yz/JjWTTueCb2m8kfe73HZS0spxcKtn3Lx5LYMnnsdKc5U3r10BjOuWGDZ5duj6d3oci5s0IWnv3mM33N/szqcStEJqorK9/ot7/OU581lwKy+2MXOhF7vm1pHzyw9TruUu9vcz5R1k3jrh7GGj1d8TG1iY2KvaaQ6Uw0fMxKtj2vDUx1H88W2z3lyyXCrw4nY8j+W0Wd6F66acQk5noOM6TqBz/7vWzo17Gp54j8SEeHpC19CqSAPLhpqdTiVohNUFRQIKgq91i7KDaogt39yPVv2/8S4Hu9afu3eSPec+xCdG3Zn+Bf38M32rwwbJ6iC3DH/Bjbt28DY7u+Uq/CoGa5uNpABLW7m1eUvMmPD+1aHc1Qb965n4Ky+9JjagU37NvJkxxdZMuAHLj/jyriYMn9Stfr859yH+GTLHOZtnmV1OBUW+0dai7q8IutLGo1e9gzztszm0fZPc95JHSyOxlg2sTGm2wQaVDuZG+dczfacbYaM89J3o5i7eRbDz3+S9vUvNGSMyhpxwSjOOaEdQxfcwppdq6wO5x+252zjrk9vpuM7WXz122Lua/sIywauY1DmYFx2l9XhlctNLW+nSe1mPLDobnI9OVaHUyE6QVUxvkCQIr+1Z0/zt8xh1NIRXHHG1dzQ8jZLYzFLRlI1Jvb+AG/Aw6DZ/Sn0F0b19T/d8jGjvhnB5Y37c3OrO6L62tHksrt4o8dkarhrMmBW35g19AwAAAAgAElEQVTppbW3cA+PfHEvbSc2Y8aG97mx5RCWDfqRu9sMM7UhYDQ57U6e7fQKO/P+4JlvHrM6nArRCaqKsboR4aZ9G7ntk0G0OLYVozq9ErPX8Y1was1GjOn2Fmt2reKeBbdFbdJE8TFtdkwmz3V+NeaPaZ3UY5nQ6332FOzi5rnX4AtYt1A835vHC98+RZsJTXhj5Rgub3wlXw9YzWMXPEOt5NqWxRUtrY9rw3UtbuLNVa+xcmf8VdrRCaoK8fgDeANBy8bP8RxkwKwrcIfrtSU7ki2LxSoXn9yde9sOZ/qGKYxb+UqlXy/Hc5CBs/risifF1THNrNuaZzuN4evfvmDEV/ebPr434OXNVa/R5q2mjFo6gvYndWTRNdm8ePHr1Ms4yfR4jPRAu8c4JvVY7v389ribQakTVBVi5dlTUAW5dd5Afj34C+N7TOaE9BMti8Vqd559Lz1OvYTHvryfL39dWOHXCaogt80bxC8HtvBGz8lx98Hat8nV3NRyCG+sHMP76941ZcygCjJ9/RTOm9iCBxcNpVHNxnzcfzETer3P6bXOMCUGs2UkVeOJDs+zZtcqxq8cY3U45aITVBVR6LW2pNGzSx/ns1/m8USH5zmn3nmWxRELbGLjpS5vcFrNxtw89xp+Pbi1Qq/z7NInWPDLXEZ0eJa29c6PbpAmGd7+Kc47sQP3fj7E0EtQSikW/DyPTu+ew5BPBlHNXZ0pl83iw399Quvj2hg2bqzoedpldGrYjWe+GcFvOb9aHU7EdIKqApRSlrbTmLNpBi8ue5qrzxzIdc1vtCyOWJLmSmdi72kEVZCBs/qS78sv1/4fb5rJi8ueon/TaxnUYrBBURqvuJdWndRjGTS7H7vz/4z6GN/9/g2XTuvENf/rQ6G/gNe7v838q76mY4POMX+/LlqKy04BPLBwqGmLxivL0AQlIl1FZKOIbBaRfzShEZHBIrJGRFaJyBIRaRJ+vrOILA//brmIxOac2ThhZUmj9XvWcsf8G2l93Nk82fHFKvOBEImG1U/h9e5vs2HvOu7+9OaIPzTW71nH7fNvoGXdLJ6+8KW4P6a1kmszsVf0e2mt37OWa//3L3pPu4hfDvzMMxe+zJfXruTS06+Ii7VM0XZStfrc2/ZhFvwyl7mb/2d1OBEx7G9JROzAGKAb0AS4sjgBlfCeUqqZUioTGAW8EH5+D9BLKdUMuA54x6g4E10wqMi36Oxpf9E+BszqS7ornTd7TiHJkWRJHLGsY4POPNBuBLN++pD/Zj9f5vYHivYzcFZfUp1p4caAbhOiNN6Zx7TgxYtfZ9mOb3h48T2Veq1tB3/l9k9u4MJ3zubb35fwQLsRLB20luta3IjTXrk+RvHuxpZDaFqnOQ8uGhoXzSSN/BpxNrBZKfWzUsoLTAUuKbmBUqrk6rFUCK0fVUqtVErtCD+/DnCLiP50q4A8rzWLcv1BP4M/vpY/8n5nQq+p1E073oIo4sNtWUO5pNG/eHLJcBZu/fSI2wWCAW6Zex2/5/7Gmz3f47i0E0yM0niXnt6X27KGMmn1ON5dM6Hc++8u2MVDi/5Nu4nNmP3Th9yadTfLBv3IHWffEzMln6zmsDl4ttN/+TN/J09/86jV4ZTJyAR1AlCyUuH28HN/IyK3icgWQmdQpa0wvBxYqZTylLLvTSKSLSLZu3fHxoK/WOIPBCmyqKTRk0uG88W2z3mq4+gqcRO6MkSEFy5+nSZ1mnHL3Ov4ef/mUrd76utHWPTrAp7s+CJnn9DW5CjN8UC7EXSo34n7F97F9zuWRrRPrieHZ5c+wTkTmvLWD2Pp1+Qavhm4hofPH0kNd02DI44/reqexcDMm3lr1VhW7Pze6nCOysgEVdqF8X98mVdKjVFKnQLcBzz0txcQaQo8A9xc2gBKqXFKqSylVFadOnWiEHJiyfNYc/Y0Y8P7vLr8RQa0uJmrmw20IIL4k+pM5a1e07CLnQGz+pLnzf3b72du/ID/Zj/Ptc1u4Jrm11sUpfEO76W1M2/HEbf1+D2MW/EKbd5qyvPfjqRjg84svnY5z3Uew/Hp9UyMOv7c3/Yxjk2tG/N9o8pMUCIyRERqVOC1twMlF7vUA478bgtdAry0xLj1gBnAtUqpLRUYv0rz+oN4/OYvyl2zaxVDF9zCOSe0Y8QFo0wfP56dVK0+Y3u8w5b9P3H7J9cTVKG/v3W7V3P3pzdz9vHn8kTHsu9Txbsa7pq81XvaoV5aHv/fL54EggHeX/cubSc2Y/gX99K0djPmXfkV43u+x2k1T7co6viSnpTByI4vsG73asatqPyCcaNEcgZVF/heRKaFZ+VFOmXoe+A0EWkoIi6gP/C3sroiclqJhz2ATeHnqwMfA/crpb6OcDytBCumle8p2M2AWX2p4a7JGz0mx11xzVhw/kkdeaT9U8zbMpvRy55hb+EeBszqS7Wk6ozv+V6VOaZn1G7KK13Gs2Ln9wxbeCdKKZRSzN8yhwvfPZs7P72RWsl1mNbnYz7411xa1s2yOuS40/3US7j45B48u/QJth2MzbVREsnU1nBSuhgYCGQB04A3yzqzEZHuwGjADkxQSo0UkRFAtlJqloi8BHQCfMB+YIhSap2IPATcTzhhhV2slNp1pLGysrJUdnb81ZoyQpEvwMFCc+ub+QI++s/oRfaOb/lf38/JrNva1PETiVKK2+ffwPT173F6rSZsPbCFGX0X0KruWVaHZrpnvnmMF5c9zeDWd5K941uy/1jGydVPZVi7R+l52mVVcrp4NG3P2Ub7t1txbr3zefeSjyJaspDsspPhrtxsSBFZrpQq81tFRAkq/IItCCWorsAi4BxggVLq3soEGi06Qf1lT56HgMlVIx5e/B/eWDmGl7uMp2+Tq00dOxEV+gu5dFonfvhzBaMvHkf/ptdYHZIlgirIdf+7ggW/zKVu6nH859yH6Nfkmio/XTyaxq54mUe+uI83ekymV6M+ZW4fUwlKRO4gtBZpDzAemKmU8omIDdgUnuBgOZ2gQgq8fnJNrrn3/rp3ufPTG7mp5RBGdHjW1LET2b7CvazZtYoL6l9kdSiWyvfm8dkvn9D55O4J2XXZav6gn25TzmdX/k6+um4VGUnVjrq9mQkqkvPj2kAfpVQXpdQHSikfgFIqCPSsVJRaVFlR0mjlzmzu/XwI553YgeHtnzJ17ERXM7lWlU9OAKmuNC45/V86ORnEYXPw7EX/ZXfBLp78+hGrw/mbSBLUXGBf8QMRSReRNgBKqfVGBaaVX743gJkVjXbn/8mg2f2ok3osY3u8g8PmMG9wTdOiJrNuawa1GMykH8ax/I9lVodzSCQJ6jUgr8Tj/PBzWgwJBBUFJp49eQNebphzFQeK9jOx17SEaO6maVXZsHaPUjftOO757HZLm0iWFEmCElXiRlX40p7+qhxjzF6U+/Die1i24xtevPh1zjymhYkja5pmhDRXOk92fJEf96xh7IqXrQ4HiCxB/Swid4iIM/zfncDPRgemRc4fCFLkM6+k0btrJjBp9ThuyxrKpaf3NW1cTdOM1e3U3nQ9pSfPfzuywn3KoimSBDUYaAv8Tqg6RBvgJiOD0srHzIkR3+9Yyv0L76JD/U480G6EaeOaTYA472IRNfowVC0jO76AiO3QAmkrlZmglFK7lFL9lVLHKKWOVUpddbQFs5q5PP6AaSWNdubt4PrZV3FC+om81n0SdpvdlHHN5nbYqZWWRI0UV5X/cE5y2KiVlqSTdRVyQvqJDGv7CIu2fsqsnz60NJYy7yWJiBu4HmgKHGo+o5QaZGBcWoTyTFrz5PF7GDS7P3m+XN6/fE5CVolOcthIS3LgsIe+t9kRMpKdplfliBV2m1At2YmIUD3ZxYECryXFhzXzXZ95K9PXT+XhxffQoX4nqrmrWxJHJJf43iFUj68L8AWhoq+5R91DM0WRL4DfhIoRSimGLbyTFTu/55Uu4zmjdlPDxzSTy26jZqqL6imuQ8mpmNtpJ8WVmGeKRyMC1cPJCcDlsJFeycWZWvyw2+w82+kV9hTuYuTXwy2LI5IEdapS6mEgXyk1iVBR12bGhqWVRSllWsWIiavHMWXdJO5uM4wep11a9g5xwmm3USPFRY1UF077kf8ppLudJDmqVs23asnOfyTrZJed5CqYrKuqFse24obMW3l79Rtk7/jWkhgi+VdXfH3jgIicCVQDGhgWkRaRAm+AoAk3MJduX8LDi/9D54bduefchw0fzwwOm1A9xUnNVBeuCBNPtWQndlvVuBGT7naQ5Cg9EWW4nbiOksy1xHJv2+Ecn3YC//lsiCVroyJ5p40L94N6iFC7jB8JNRHULBIMKvK9xp89bc/Zxg1zrqJBtZMZ021C3FeOLr6nUist6YgfwEciIqFJEwmeo0KXNI9+a7oqJetYImD6l4M0VzpPXTiaDXvX8fry0aaODWUkqHBB2Byl1H6l1JdKqZPDs/nGmhSfVop8r9/wkkaF/sJQs7hAEW/1nlZmAclYZhMhw+2kdloSbmfFL1HZbaHJAon60ey028hwl70G32aT0P0pE2LSQokpxWWndloS1VOcOEz+ctDllJ50P7U3z3/7JL8e+MXUsY+aoMJVI4aYFIsWgUBQUeg1dlGuUop7FtzG6l0rebXrW3HbpdQmQrrbQe00V9TunbgcNtIi+BCPNzaRv02KKIvDbiMjWU+aMJIQuu9XKy2JdLcTm00QEUsmqzzR4XnsNofpa6MiOV9cICL/EZETRaRm8X+GR6aVKq/I+JJG41a+wvQNU7j33OFcfEoPg0eLPhFISwolphSXI+IP3UiluBwJNVlAgOopoQ/A8nA77aQlJV6yjgVuZygxZbj/eTnV5bCZ/v47Pr0e97d7lEW/LuDD9dNMGzeSflClndMppdTJxoRUMVWhH5QvEGRfvtfQMZZsW0zfj3rQ9ZSejO85Ja7uOwmQkuQg1WWPelIqzf58L96AOYukjVQt2VmpS58HC3wU+c0rtZXI3A47qUn2f8ygPJxSij15XlMmShULBAN0n9qeP/J+Z8Nt66mRXKPCrxW1flBKqYal/BdTyamqMHpRrsfvYeiCWzi5+qm83GV83CSnktfo05Kif8Z0JNWSndjifNZEapKjUskJICPZcdRp+lrZitfiVUv55/T+0ogIGcnmnr3abXae6zSG/YX7mL9lviljRlJJ4trSnldKvR39cKzl9QcjnnZstiJfwPBv62+uepVtOVt5v88c0lzpho4VDQK4XXZSXQ5LZpXZbEKNFCf78uOzwoLbEZ1LdCKh2ZH78s39Rp8InPZQ9ZKKfO4kOey4HUFTz16bHZPJDzdv5IxjGpoyXiTvzrNK/OwGLgJWAAmVoJRS7C/wYrcJbqedZKc9pqbSGl0Qdk/Bbl5c9jSdGnaLiy6uboedNLc1iamk4skC8VYOyWGL7jdwe3ht2f44TdZmc9iEtKOsN4tUutuBJ9/cRqUnZNQzbawy36FKqdtLPhaRaoTKHyWkQFCR7/GT7/HjtNtIdtpJctjKfQM5mgq9AQIGlzR67tuRFPjyGX7+k4aOU1mH18uLBW6nnUBQmVpVvjJEoHqKK+qXQp1xmqzNZLcJaVG4rFrMZgstoUjUY16Rr1AFwGnRDiQW+QJBfIEgQuh0OslpI8lhM+0eB4TO7Iz+4Nu4dz3vrB7Ptc1voFGtxoaOVVEue2h6d6ze60hNcuAPqJifLCBA9WSXYWeebqcdXyBIgcFLIeJN8ZKHaCWmktxOO4Ve428BWCGSe1Cz4dBZuw1oApg3zzAGKKDIH6DIH0CEQ5cAzfiwzDehpNGIrx4g1ZXGf859yNBxKqIy1+jNlpHswJ8fNKWAb0Wlu52GH8t0t5NAUJnWBiaW2URITQp9Xhj5xTYj2cnePE/CXV6N5AzquRI/+4FflVLbDYon5ikVuuRW6A1gtwnJTjtug+5XBYOKAoPPnhb/+hmf//IJj7R/ilrJtQ0dqzwcNonKDDMziQjVU1wxO1kgxcRir9WSnezN9xp+aTpWiUCqy0GKSUse7OF/L/FymTlSkSSobcAfSqkiABFJFpEGSqmthkYWB4rvO+R5/LjsNtxOO25n9C4B5nmNXZTrD/p55Iv7qF+tIYNa3GLgSOXjstuokeqyOowKKa73F2u9k1x2c9tlSLgyxb4Cr6k38K0mElrIbdZavJJSkxymteAxSyTn+h8AJc/VA+HntBK8gSA5RT5253o4WODDU8l7Ef5A0PCSRlPWTmLj3h95+PyRJDmSDB0rUsUf8PHM5YitMkBWHVOH3Rb3f5eROrQWL9XctXiHi6X3XTREkqAcSqlD5QvCP0f09VZEuorIRhHZLCLDSvn9YBFZIyKrRGSJiDQp8bv7w/ttFJEukYwXC4rvVx0o8LErt4jcIh++Cty8NPpUPdeTwzNLR9Dm+Lb0ODU2ejwVN8mzcsZktMRKo0Orj2mSw056AtYuLFZcL692iXp5VnLabTHxvouWSBLUbhHpXfxARC4B9pS1k4jYgTFAN0ITK64smYDC3lNKNVNKZQKjgBfC+zYB+hNqM98VeDX8enFFqVDfpn35Xvbmecj3+CO6Ju/1Bw2/wfzK98+xp2AXj17wjGXf9g5XWpO8eBYLjQ5j4ZimuOLrXmKkStbLszoxlZSWZP36wGiJ5J07GHhARLaJyDbgPuDmCPY7G9islPo5fNY1Fbik5AZKqZwSD1P5a7bgJcBUpZRHKfULsDn8enHLH75ftSfPw/58L0W+wBGrAht99rTt4K+MXfEy/zrjKlrWLbMclinSkiq/aDEWWdk7KZaOaUYMLxEoL7fDTq1UV8z2xZLwlPZEEMlC3S3AOSKSRqi4bG6Er30C8FuJx9uBNodvJCK3AUMJXTa8sMS+JXsMbw8/d/i+NwE3AZx00kkRhmU9byCIt/Cv9VVul+3QB0mRL1ChS4Ll8eTXD2MTG/e3e9TQcSLldtpJTdCq2FZNFoi1Y1p8HPbG6AzHSCQ5bKQmxUeiTXLYcTuDFPniez1amUdaRJ4UkepKqTylVK6I1BCRJyJ47dK+WvzjnamUGqOUOoXQmVnxQpxI9x2nlMpSSmXVqVMngpBiS8n7VbtzPeQW+Qw/e8re8S0zN37A4NZ3cUL6iYaOFYlIm+TFM7MnC8TqMbWFyyHF3jnH0bnsNmqkuKie4oqL5FQsPckR98WMIzna3ZRSB4ofKKX2A90j2G87UPITsB6w4yjbTwWK79aXd9+4F1SKAoNLGimleOTL+zg2tS5DsoYaNk6kytskL56ZNVkg1o+pM44aHTrCCbVGqisuFoofzmaL/0t9kRx1u4gcmoMsIslAJHOSvwdOE5GGIuIiNOlhVskNRKRkyaQewKbwz7OA/iKSJCINCZVW+i6CMbWj+N9P01n+x3cMa/cYqa40S2OpaJO8eGb0ZIF4OaaxdvnxcI7wtPxaaUkxcw+votzhWqLxKpJ3ybvA5yLyFqGrUoOIoJK5UsovIkOA+YAdmKCUWiciI4BspdQsYIiIdAJ8wH7guvC+60RkGvAjoeoVtyml4vtiqsWK/EU8seQhzqzTgr5nXG11OGQkO+Pqckm0ZLgdBILKkPuM8XRM05IcBGKsdmG0C7nGinS3E2++Jy4XTEcySWKUiKwGOhH6kva4UiqiblVKqbnA3MOeG17i5zuPsu9IYGQk42hle2PFK2zP2cboy8dit1n7DzDeShhFk1GTBeLxmMZK7UKbhBKT2W3UzVKceHMNbnhqhIi+bimlPlFK/Ucp9W8gT0TGGByXFkW78//kpe+fpcvJPTnvpA6WxlLcLqMqi/ZkgXg9psW1C626XSYS6qdUO82VsMmpWIorPmYfHi6iiEUkU0SeEZGtwBPABkOj0qLq2aVPUOQvZHh7a3s9ORKgjFG0RGuyQLwfU7tNqJ7sMnVmn0joEmOdtCRSXNaVJTJbhtsRdzMoj/i1S0QaEZrYcCWwF3if0DqojibFpkXB+j3reHftBK7PvIVTaljXxsuoJnnxzO204w83yKyIRDmmLkeokG1OkbFN94rLEqW6HDE/kcQIDruNlCRHhd9vVjjadYENwFdAL6XUZgARuduUqLSoUErx2JfDyHBVY+g5D1gWh9FN8uJZRScLJNoxTXbZ8QeNaXQogNtlJ62KJqaSUl12PHFU8fxol/guB3YCi0TkDRG5iNIX0GoxauHWT1n862cMPed+arhrWhaHGU3y4llGsgNHOT84E/GYpruduKJ8nyRW6+VZJVQGKX4uCR/x3aCUmqGU6gc0BhYDdwPHishrInKxSfFpFeQP+nn0y2E0rH4KA1pEUjrRGMkmNsmLV+WdLJDIxzRa9e1ivV6elVwOW9y8f8r8uqKUyldKTVZK9SRU0WEV8I/WGVpseXfNBDbt28Dw85/EZbem+Z/LbiMjjr6tWSnSyQKJfkxttuJKGBXbP8lhCyWmFOuruMeyeCmDVK6/QaXUPqXUWKXUhWVvrVnlYNEBRi19nLb12tP1lF6WxJAIjQfNVjxZ4EiqyjGtSO3CkvXydGIqm4iQkRz7SxNiP0Kt3F76bhT7C/fyaPunLZnhZXWTvHiW7LLjC/6zm3JoUkTVOaZJDjtpSarM4slOe2gNWKLdjzNDksOO2xGMqWoeh9MJKsH8euAXxq8aQ98m/0fzY1taEkMsNMmLZxluJ4GAwluiHFJGFTymqUkO/EFVassIh03isnpGrEl3O/DkB2K2DFLVesdXAU8seQi7OCzr9RRLTfLiWcmb+4lYHy5Shzc6tJco5FpVj0k02WwS0/c0dYJKIN/9/g2zN33EkLP+Td20400fP9arVMeT4skCya6qfUyLaxc6w5NDauvEFHVupz3q0/ujJTaj0sotqIIM/+Jejks7nsGtj1iD1zCx2iQvnjkSfMZepGw2oWZq4tfLs1JGcmw2ktSfKAlixob3WfXncl7uMp5UZ6qpY8d6kzxN047OHr6nZ3RH7/LSZ1AJoMBXwMglD9P8mJb864wrTR07XprkaZp2dKlJ5a9oYjSdoBLAuBWvsCPvdx67YBQ2MfevNJ6a5GmadnTRqLAfTfqTJc7tyt/Jy98/S49TL+HceueZOrae5qtpicVpt5ESQ/f6dIKKc09/8xi+gJeHzje3+XC8NsnTNO3o0pIcMVO/UCeoOLZu92qmrJ3EoMxbaFj9FNPGjfcmeZqmHZlI7KyN0gkqTimleOSL+6jhrsndbcyr3ZsoTfI0TTsyl8MWE5fvdYKKUwt+mceS3xbz73MfpLq7hiljJlqTPE3TjiwWKp7rBBWHfAEfI768n1NrNOLaZjeYNm4iNsnTNK10NpuQbvHie32XOw5NWv0Gm/f/xNuXfIjTbs614kRukqdpWuncTjtFvgAef7DsjQ2gvw7HmQNF+3n+25Gcf2JHOjfsZsqYid4kT9O0I0t3V7yBZGXpBBVnXlz2NAeK9vPoBeb0eqoqTfI0TSud3SaWLSnRCSqO/HJgCxNWvcZVZw6gaZ3mho+nGw9qmgaQ4nJYUjHG0BFFpKuIbBSRzSLyj7nQIjJURH4UkdUi8rmI1C/xu1Eisk5E1ovIy6LnNfP4Vw/gtLu4r+1wU8bTjQc1TSuW4XaYXvHcsE8fEbEDY4BuQBPgShFpcthmK4EspVRzYDowKrxvW6Ad0Bw4EzgLuMCoWOPBN9u/Yu7mWdxx1j0ck1rX8PF040FN00py2G2kmHypz8ivx2cDm5VSPyulvMBU4JKSGyilFimlCsIPvwXqFf8KcAMuIAlwAn8aGCuvZ7/O5DVvcbDogJHDVEhQBXn0i/s4Ib0eN7e+w/DxdONBTdNKk+qy47SZd1XFyJFOAH4r8Xh7+LkjuR6YB6CUWgosAv4I/zdfKbX+8B1E5CYRyRaR7N27d1cq2Klrp/Lvz26l2bj6DJrdjzmbZlDkL6rUa0bL9PVTWL1rJQ+e9zjJjmRDx7LbRDce1DStVCJi6nITIz+JSrtcqUrdUOT/gCzCl/FE5FTgDP46o1ogIu2VUl/+7cWUGgeMA8jKyir1tSO16LpFLNi8lI82TGXmxunM3TyLdFcGPU67lMsb96dtvfbYbeZf8sr35fPU18NpWTeLS0/va/h4aUkOXcZI07SYYGSC2g6cWOJxPWDH4RuJSCfgQeACpZQn/PRlwLdKqbzwNvOAc4AvD98/WkSEzLqtyazbmuHtn+Lr377gww1TmbNpBlPXvU3d1OO45PQruLxxf5odk2nah/hr2aP5I28HY7u/Y3ivJ5c9NupvaZqmAYhSlTrxOPILiziAn4CLgN+B74GrlFLrSmzTktDkiK5KqU0lnu8H3Ah0JXQm9gkwWik1+0jjZWVlqezs7ArHq5RiV67nH88X+gv5dMvHzNj4Pp//Mh9f0MdpNU+nT+P+9Dm9H/WrN6zwmGX5I+932r7VnE4Nu/JGz8mGjVOsVqpLz9rTNM1wIrJcKZVV5nZGJahwEN2B0YAdmKCUGikiI4BspdQsEfkMaEboPhPANqVU7/AMwFeB9oQuC36ilBp6tLGMSlAl7S/ax5yfZvDhhql8+/uS0LjHtaFP4/70bnQ5tVPqVHj80tw5/yZmbHyfr65dZWgihFApI10tQtM0M8REgjKTGQmqpO0525ixcRofbXif9XvWYhc7F9TvxOWN+9H11N6kOlMrHAvA6j9X0uW9dtyadTcPG9yMUARqpybpBbmapplCJ6hyKm+CKmn9nrV8uH4qMza+z++520l2pNDt1F70adyfC066qNwFXZVSXD69Kxv2/si3A9eSkVStQnFFKt3tIMWlZ+5pmmYOnaDKqTIJqlhQBfnu92/4aMP7zPrpQw549lMzuTaXNLqcyxv3p/VxbSKaXDFv8ywGzu7H0xe+xIAWN1UqprI4bEKttCRDx9A0TStJJ6hyikaCKskb8LJo66d8uGEqn275mKJAESdlNKBP4370adyfRrUaH3G/C95uhdPmYm0PNBUAAAzvSURBVOE13+GwGXtmUz3FqStGaJpmqkgTlL6uYxCX3UWXU3rS5ZSe5HpymLt5Fh9tmMrL3z/L6O+eodkxmfRp3I9LT7+C49L+Wr888Yex/HJgC+9d9j/Dk1OSw6aTk6ZpMUufQYVF+wzqSHbl7+R/G6fz4YaprPpzOYLQ7sQL6NO4P+3qtafLe+1oWTeLKX1mGRqHALXSknT7dk3TTKcv8ZWTWQmqpC37N/HRhvf5aMNUfjmwBQCb2Pj8/77jjNpNDR07xWUnXU8r1zTNAjpBlZMVCark2Cv/zGbmhmmcVK0hN7S81dDxbCLUTnPpkkaapllC34OKIyJCq7pn0aruWaaMl+7W9fY0TYt9uq5NFePU9fY0TYsTOkFVMWm6z5OmaXFCJ6gqxO2043Lov3JN0+KD/rSqIgR99qRpWnzRCaqKSE1y6DVPmqbFFZ2gqgC7TUgxsU2zpmlaNOgEVQXoNu6apsUjnaASnG7jrmlavNIJKsGlu/XECE3T4pNOUAks2WXHYdd/xZqmxSf96ZWgRCBNd8nVNC2O6QSVoNKSHNj0tHJN0+KYTlAJKDStXJ89aZoW33SCSkB6YoSmaYlAJ6gEo9u4a5qWKHSCSiACukuupmkJQyeoBJLssut6e5qmJQydoBKETURXK9c0LaEYmqBEpKuIbBSRzSIyrJTfDxWRH0VktYh8LiL1S/zuJBH5VETWh7dpYGSs8U7X29M0LdEYlqBExA6MAboBTYArRaTJYZutBLKUUs2B6cCoEr97G3hWKXUGcDawy6hY453TbiNZVyvXNC3BGHkGdTawWSn1s1LKC0wFLim5gVJqkVKqIPzwW6AeQDiROZRSC8Lb5ZXYTjuMvrSnaVoiMjJBnQD8VuLx9vBzR3I9MC/8cyPggIh8JCIrReTZ8BnZ34jITSKSLSLZu3fvjlrg8US3cdc0LVEZ+clW2g0RVeqGIv8HZAHPhp9yAOcD/wHOAk4GBvzjxZQap5TKUkpl1alTJxoxxxXdxl3TtERmZILaDpxY4nE9YMfhG4lIJ+BBoLdSylNi35Xhy4N+YCbQysBY45Ju465pWiIzMkF9D5wmIg1FxAX0B2aV3EBEWgJjCSWnXYftW0NEik+LLgR+NDDWuGMT3cZd07TEZliCCp/5DAHmA+uBaUqpdSIyQkR6hzd7FkgDPhCRVSIyK7xvgNDlvc9FZA2hq1lvGBVrPEp362nlmqYlNkNvYCil5gJzD3tueImfOx1l3wVAc+Oii1+6jbumaVWBnv4Vh3S1ck3TqgKdoOKMbuOuaVpVoT/p4ohu465pWlWiE1Qc0W3cNU2rSnSCChORUlcWxwrdxl3TtKpGJ6gSaqclkeF24orBezx6YoSmaVWN/tQrwWYTkl12kl12gkFFkT9AkS+ILxC0NC7dxl3TtKpIJ6gjsIUvqaW4IBBUFPkCFPkC+IOllhM0jG7jrmlaVaUTVATsNiE1yUFqkgN/IEiRP0iRL0DAhGSl27hrmlZV6QRVTg67jTS7jbQkB75AKFF5/EFDkpVu465pWlWmP/0qwWm34bTbSIdDyarIFySoopOsdBt3TdOqMp2gouRQsnKD1x8MT7AIUNFcpdu4a5pW1ekEZQCXw4bLYSPD7cQTngno8ZcvWelLe5qmVXX6U9BgSQ47SQ47Sjnw+IN4ipPVUfZxO3Qbd03TNJ2gTCIiuJ123M6/klWRL4DXH/xbshIgTS/K1TRN0wnKCiWTVTCo/kpWgaBu465pmhamE5TFDq9eoSftaZqmhegEFUN0pXJN07S/6DvxmqZpWkzSCUrTNE2LSTpBaZqmaTFJJyhN0zQtJukEpWmapsUknaA0TdO0mKQTlKZpmhaTdILSNE3TYpJOUJqmaVpMEhWl5npWE5HdwK9WxxEFtYE9VgcRA/RxCNHHIUQfh78kwrGor5SqU9ZGCZOgEoWIZCulsqyOw2r6OITo4xCij8NfqtKx0Jf4NE3TtJikE5SmaZoWk3SCij3jrA4gRujjEKKPQ4g+Dn+pMsdC34PSNE3TYpI+g9I0TdNikk5QmqZpWkzSCSpGiMiJIrJIRNaLyDoRudPqmKwiInYRWSkic6yOxUoiUl1EpovIhvD74lyrY7KCiNwd/jexVkSmiIjb6pjMICITRGSXiKwt8VxNEVkgIpvC/69hZYxG0wkqdviBfyulzgDOAW4TkSYWx2SVO4H1VgcRA14CPlFKNQZaUAWPiYicANwBZCmlzgTsQH9rozLNRKDrYc8NAz5XSp0GfB5+nLD+v717jZGzquM4/v3ZYqB4eUEDKTbaVl0wUG0li0oReiH6gsYStCFlgZZU4gUaMKIGjSgkmBLAQMLNAIGCBSyl4RIgXEKh+qZbbIvbFNBISylW24R4I1oo+/PFOQvPzs52p9Cd8zj9f5LNPHPm7DO/abrzn2eemfOPAlUTtnfYXp+3/0V6MvpY2VTtJ2kicCpwa+ksJUn6CHAScBuA7Tdt/71sqmLGAodIGguMA/5SOE9b2F4DvN4wPA9YlreXAae1NVSbRYGqIUmTgOnA2rJJirgW+CHQXzpIYVOAXcDt+e3OWyUdWjpUu9l+Dbga2AbsAP5h+4myqYo6wvYOSC9qgcML5xlVUaBqRtKHgPuBi2z/s3SedpI0F9hp+/els9TAWODzwE22pwNv0OFv5zSTz7HMAyYDRwKHSjqrbKrQLlGgakTSQaTitNz2qtJ5CpgBfE3SVuBeYLakX5eNVMx2YLvtgaPolaSCdaA5Bdhie5ftt4BVwAmFM5X0N0kTAPLlzsJ5RlUUqJqQJNL5hhds/7J0nhJsX2J7ou1JpBPhT9s+IF8t2/4r8Kqko/LQHGBzwUilbAO+KGlc/huZwwH4YZGKh4CFeXsh8GDBLKNubOkA4R0zgLOBPkkb89iPbT9aMFMoawmwXNIHgZeBcwvnaTvbayWtBNaTPum6gQNkqR9J9wAzgfGStgM/A5YCKyQtJhXv+eUSjr5Y6iiEEEItxVt8IYQQaikKVAghhFqKAhVCCKGWokCFEEKopShQIYQQaikKVOgYkizpmsr1iyX9fD/t+w5J39gf+xrhfubnlctXN4xPyo9vSWXsekmLRthfu3KvlDQlb2+VND5vHydpi6TpkuZKumy0s4TOEQUqdJLdwOkDT451IWnMPkxfDHzX9qwmt+0ELszfixp1eXHWVuYdA4yx/XLD+GdJK2CcYXsD8AhppZBx+z1s6EhRoEIn2UP6Euf3Gm9oPJKQ9O98OVPSs5JWSPqjpKWSeiT1SuqT9MnKbk6R9Ns8b27+/TGSrpK0TtIfJH2rst/Vku4G+prkWZD3v0nSlXnsUuBE4GZJVzV5fLtILRYWNt4g6byc4XlJ9zcUgWa5D5Z0e86wQdKsPL5I0n2SHgaekDRB0hpJG3PWLzfJ1cPQFQ0+AzwAnG27F8DpS5fPAHOb7COEIaJAhU5zA9Aj6aP78DufI/WgmkpazaPL9vGklh9LKvMmASeT2oHcnBvnLSatsN0NdAPnSZqc5x8P/MT2oL5eko4ErgRmA9OAbkmn2b4ceA7osf2DYbIuBb7f5Khsle1u2wN9oxaPkPt8ANtTgQXAMr3bCPBLwELbs4EzgcdtT8v/ThsZagbQuMDvg8AFtn/XMP4c0KzIhTBEFKjQUfIK8HeSmty1al3ux7Ub+DMw0M6hj/TkPmCF7X7bfyItPXQ08BXgnLw81VrgMODTeX6v7S1N7q8beCYvgLoHWE7q/dTK49sC9JIKR9Wx+Sipj3REc8wIuU8E7sr7fBF4BejK85+0PdCHaB1wbj6XNzX3Kms0gXR0V/UU8M0mhXQnaVXyEEYUBSp0omtJRxDV/kl7yP/f86Kj1fM4uyvb/ZXr/Qxer7JxXTADApbYnpZ/Jlf6Fb0xTD61+kCG8QvgRwz++72DdMQyFbgMqLZFHy73cN7JnZvmnQS8Btwl6Zwm8//TcH8AF+TLGxvGD87zQxhRFKjQcfKr/xUMfptrK3Bc3p4HHPQedj1f0gfyeakpwEvA48B3cqsUJHW10FhwLXCypPH5CGMB8GyrIfIRz2YGn8v5MLAj5+hpIfeagXmSuoCP5/FBJH2C1KPrFtJq+81afrwAfKphrD8/rqMkXV4Z7wI2tfI4Q4gCFTrVNUD103y3kIpCL/AFhj+62ZuXSIXkMeDbtv9LOk+1GVgvaRPwK0boEpA7oV4CrAaeB9bb3te2CVcAEyvXf0oqfE8CL7aQ+0ZgTH5L8DfAovwWZ6OZwEZJG4CvA9c1mfNInjdI3t880if3zs/Ds/L8EEYUq5mHEN4XSYeQiu0M22/vZd4RwN2257QtXPi/FgUqhPC+Sfoqqdnmtr3M6Qbest3sk4AhDBEFKoQQQi3FOagQQgi1FAUqhBBCLUWBCiGEUEtRoEIIIdRSFKgQQgi19D8Dy4OweRCO4QAAAABJRU5ErkJggg==\n"
                    },
                    "metadata": {
                        "needs_background": "light"
                    }
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "print( \"The best accuracy was with\", mean_acc.max(), \"with k=\",mean_acc.argmax()+1) ",
            "execution_count": 54,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "The best accuracy was with 0.34 with k= 5\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "### k in KNN, is the number of nearest neighbors to examine for better multilabel Accurancy Classification Score . "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "",
            "execution_count": null,
            "outputs": []
        }
    ],
    "metadata": {
        "kernelspec": {
            "name": "python3",
            "display_name": "Python 3.6",
            "language": "python"
        },
        "language_info": {
            "name": "python",
            "version": "3.6.9",
            "mimetype": "text/x-python",
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "pygments_lexer": "ipython3",
            "nbconvert_exporter": "python",
            "file_extension": ".py"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 1
}