Assignment1_Polynomial_Regression_1-1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nz_9QnWBj9_6"
   },
   "source": [
    "# Polynomial Regression Assignment (Please do not remove the output cells)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oNtRAlw1uQYi"
   },
   "source": [
    "## The objective is to apply polynomial regression on the provided data using 3 regularization techniques"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0Sh_Oqa2j9_7"
   },
   "source": [
    "Tasks are defined below:\n",
    "1-  Read the dataset, and get acquainted with its features and labels. Check the link below for more details about the data.  \n",
    "https://archive.ics.uci.edu/ml/datasets/Air+Quality  \n",
    "\n",
    "2-  This dataset has 5 different output labels. For this assignment we only need the NO2 output. Please drop other outputs.  \n",
    "\n",
    "3-  Handle missing data similar to what was covered in tutorial 3.  \n",
    "\n",
    "4-  Replace the date feature with 3 separate features (Day, Month and Year).  \n",
    "\n",
    "5-  Replace the time feature with 3 separate features (Hours, Minutes and Seconds).  \n",
    "\n",
    "6-  Apply feature scaling.  \n",
    "\n",
    "7-  Evaluate different degrees of lasso CV polynomial regression.  \n",
    "\n",
    "8-  Choose the best degree and print the errors on the test data, model coefficients and the lasso parameters.  \n",
    "\n",
    "9-  Repeat steps 4 and 5 using Ridge CV polynomial regression.  \n",
    "\n",
    "10- Repeat steps 4 and 5 using ElasticNet CV polynomial regression.  \n",
    "\n",
    "11- Compare the errors from the 3 regularization techniques, and save the best model.  \n",
    "\n",
    "12- Load the best model and test it on a sample that you manually created it.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sIwzxhGgxdrN"
   },
   "source": [
    "## Enter your IDs and Names below"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aLnMVhFQxdrO"
   },
   "source": [
    "1-  Salma El-Shafey 46-17002\n",
    "  \n",
    "2- Reem Abdelrazek 46-13896"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bsZFdAtSj9_8"
   },
   "source": [
    "\n",
    "### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "id": "nuN2-J1bj9_9"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "#import sklearn as sk "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hszifOuTj9__"
   },
   "source": [
    "### 1- Read the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 0
    },
    "id": "ZzX469qmUTu2",
    "outputId": "22450de6-53bf-4a87-9939-3695333e89ac"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Time</th>\n",
       "      <th>CO(GT)</th>\n",
       "      <th>PT08.S1(CO)</th>\n",
       "      <th>NMHC(GT)</th>\n",
       "      <th>C6H6(GT)</th>\n",
       "      <th>PT08.S2(NMHC)</th>\n",
       "      <th>NOx(GT)</th>\n",
       "      <th>PT08.S3(NOx)</th>\n",
       "      <th>NO2(GT)</th>\n",
       "      <th>PT08.S4(NO2)</th>\n",
       "      <th>PT08.S5(O3)</th>\n",
       "      <th>T</th>\n",
       "      <th>RH</th>\n",
       "      <th>AH</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>18:00:00</td>\n",
       "      <td>2.6</td>\n",
       "      <td>1360.0</td>\n",
       "      <td>150.0</td>\n",
       "      <td>11.9</td>\n",
       "      <td>1046.0</td>\n",
       "      <td>166.0</td>\n",
       "      <td>1056.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>1692.0</td>\n",
       "      <td>1268.0</td>\n",
       "      <td>13.6</td>\n",
       "      <td>48.9</td>\n",
       "      <td>0.7578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>19:00:00</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1292.0</td>\n",
       "      <td>112.0</td>\n",
       "      <td>9.4</td>\n",
       "      <td>955.0</td>\n",
       "      <td>103.0</td>\n",
       "      <td>1174.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>1559.0</td>\n",
       "      <td>972.0</td>\n",
       "      <td>13.3</td>\n",
       "      <td>47.7</td>\n",
       "      <td>0.7255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>20:00:00</td>\n",
       "      <td>2.2</td>\n",
       "      <td>1402.0</td>\n",
       "      <td>88.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>939.0</td>\n",
       "      <td>131.0</td>\n",
       "      <td>1140.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>1555.0</td>\n",
       "      <td>1074.0</td>\n",
       "      <td>11.9</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.7502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>21:00:00</td>\n",
       "      <td>2.2</td>\n",
       "      <td>1376.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>9.2</td>\n",
       "      <td>948.0</td>\n",
       "      <td>172.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>122.0</td>\n",
       "      <td>1584.0</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.7867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>22:00:00</td>\n",
       "      <td>1.6</td>\n",
       "      <td>1272.0</td>\n",
       "      <td>51.0</td>\n",
       "      <td>6.5</td>\n",
       "      <td>836.0</td>\n",
       "      <td>131.0</td>\n",
       "      <td>1205.0</td>\n",
       "      <td>116.0</td>\n",
       "      <td>1490.0</td>\n",
       "      <td>1110.0</td>\n",
       "      <td>11.2</td>\n",
       "      <td>59.6</td>\n",
       "      <td>0.7888</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Date      Time  CO(GT)  PT08.S1(CO)  NMHC(GT)  C6H6(GT)  \\\n",
       "0  3/10/2004  18:00:00     2.6       1360.0     150.0      11.9   \n",
       "1  3/10/2004  19:00:00     2.0       1292.0     112.0       9.4   \n",
       "2  3/10/2004  20:00:00     2.2       1402.0      88.0       9.0   \n",
       "3  3/10/2004  21:00:00     2.2       1376.0      80.0       9.2   \n",
       "4  3/10/2004  22:00:00     1.6       1272.0      51.0       6.5   \n",
       "\n",
       "   PT08.S2(NMHC)  NOx(GT)  PT08.S3(NOx)  NO2(GT)  PT08.S4(NO2)  PT08.S5(O3)  \\\n",
       "0         1046.0    166.0        1056.0    113.0        1692.0       1268.0   \n",
       "1          955.0    103.0        1174.0     92.0        1559.0        972.0   \n",
       "2          939.0    131.0        1140.0    114.0        1555.0       1074.0   \n",
       "3          948.0    172.0        1092.0    122.0        1584.0       1203.0   \n",
       "4          836.0    131.0        1205.0    116.0        1490.0       1110.0   \n",
       "\n",
       "      T    RH      AH  \n",
       "0  13.6  48.9  0.7578  \n",
       "1  13.3  47.7  0.7255  \n",
       "2  11.9  54.0  0.7502  \n",
       "3  11.0  60.0  0.7867  \n",
       "4  11.2  59.6  0.7888  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('AirQualityUCI.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TMHFhOfXuQYl"
   },
   "source": [
    "### 2- Drop unwanted labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 0
    },
    "id": "L7395CTWuQYm",
    "outputId": "ef08b3d5-56fb-47c0-c851-e471c0d03b38"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Time</th>\n",
       "      <th>PT08.S1(CO)</th>\n",
       "      <th>PT08.S2(NMHC)</th>\n",
       "      <th>PT08.S3(NOx)</th>\n",
       "      <th>NO2(GT)</th>\n",
       "      <th>PT08.S4(NO2)</th>\n",
       "      <th>PT08.S5(O3)</th>\n",
       "      <th>T</th>\n",
       "      <th>RH</th>\n",
       "      <th>AH</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>18:00:00</td>\n",
       "      <td>1360.0</td>\n",
       "      <td>1046.0</td>\n",
       "      <td>1056.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>1692.0</td>\n",
       "      <td>1268.0</td>\n",
       "      <td>13.6</td>\n",
       "      <td>48.9</td>\n",
       "      <td>0.7578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>19:00:00</td>\n",
       "      <td>1292.0</td>\n",
       "      <td>955.0</td>\n",
       "      <td>1174.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>1559.0</td>\n",
       "      <td>972.0</td>\n",
       "      <td>13.3</td>\n",
       "      <td>47.7</td>\n",
       "      <td>0.7255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>20:00:00</td>\n",
       "      <td>1402.0</td>\n",
       "      <td>939.0</td>\n",
       "      <td>1140.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>1555.0</td>\n",
       "      <td>1074.0</td>\n",
       "      <td>11.9</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.7502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>21:00:00</td>\n",
       "      <td>1376.0</td>\n",
       "      <td>948.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>122.0</td>\n",
       "      <td>1584.0</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.7867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>22:00:00</td>\n",
       "      <td>1272.0</td>\n",
       "      <td>836.0</td>\n",
       "      <td>1205.0</td>\n",
       "      <td>116.0</td>\n",
       "      <td>1490.0</td>\n",
       "      <td>1110.0</td>\n",
       "      <td>11.2</td>\n",
       "      <td>59.6</td>\n",
       "      <td>0.7888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9352</th>\n",
       "      <td>4/4/2005</td>\n",
       "      <td>10:00:00</td>\n",
       "      <td>1314.0</td>\n",
       "      <td>1101.0</td>\n",
       "      <td>539.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>1374.0</td>\n",
       "      <td>1729.0</td>\n",
       "      <td>21.9</td>\n",
       "      <td>29.3</td>\n",
       "      <td>0.7568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9353</th>\n",
       "      <td>4/4/2005</td>\n",
       "      <td>11:00:00</td>\n",
       "      <td>1163.0</td>\n",
       "      <td>1027.0</td>\n",
       "      <td>604.0</td>\n",
       "      <td>179.0</td>\n",
       "      <td>1264.0</td>\n",
       "      <td>1269.0</td>\n",
       "      <td>24.3</td>\n",
       "      <td>23.7</td>\n",
       "      <td>0.7119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9354</th>\n",
       "      <td>4/4/2005</td>\n",
       "      <td>12:00:00</td>\n",
       "      <td>1142.0</td>\n",
       "      <td>1063.0</td>\n",
       "      <td>603.0</td>\n",
       "      <td>175.0</td>\n",
       "      <td>1241.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>26.9</td>\n",
       "      <td>18.3</td>\n",
       "      <td>0.6406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9355</th>\n",
       "      <td>4/4/2005</td>\n",
       "      <td>13:00:00</td>\n",
       "      <td>1003.0</td>\n",
       "      <td>961.0</td>\n",
       "      <td>702.0</td>\n",
       "      <td>156.0</td>\n",
       "      <td>1041.0</td>\n",
       "      <td>770.0</td>\n",
       "      <td>28.3</td>\n",
       "      <td>13.5</td>\n",
       "      <td>0.5139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9356</th>\n",
       "      <td>4/4/2005</td>\n",
       "      <td>14:00:00</td>\n",
       "      <td>1071.0</td>\n",
       "      <td>1047.0</td>\n",
       "      <td>654.0</td>\n",
       "      <td>168.0</td>\n",
       "      <td>1129.0</td>\n",
       "      <td>816.0</td>\n",
       "      <td>28.5</td>\n",
       "      <td>13.1</td>\n",
       "      <td>0.5028</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>9357 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           Date      Time  PT08.S1(CO)  PT08.S2(NMHC)  PT08.S3(NOx)  NO2(GT)  \\\n",
       "0     3/10/2004  18:00:00       1360.0         1046.0        1056.0    113.0   \n",
       "1     3/10/2004  19:00:00       1292.0          955.0        1174.0     92.0   \n",
       "2     3/10/2004  20:00:00       1402.0          939.0        1140.0    114.0   \n",
       "3     3/10/2004  21:00:00       1376.0          948.0        1092.0    122.0   \n",
       "4     3/10/2004  22:00:00       1272.0          836.0        1205.0    116.0   \n",
       "...         ...       ...          ...            ...           ...      ...   \n",
       "9352   4/4/2005  10:00:00       1314.0         1101.0         539.0    190.0   \n",
       "9353   4/4/2005  11:00:00       1163.0         1027.0         604.0    179.0   \n",
       "9354   4/4/2005  12:00:00       1142.0         1063.0         603.0    175.0   \n",
       "9355   4/4/2005  13:00:00       1003.0          961.0         702.0    156.0   \n",
       "9356   4/4/2005  14:00:00       1071.0         1047.0         654.0    168.0   \n",
       "\n",
       "      PT08.S4(NO2)  PT08.S5(O3)     T    RH      AH  \n",
       "0           1692.0       1268.0  13.6  48.9  0.7578  \n",
       "1           1559.0        972.0  13.3  47.7  0.7255  \n",
       "2           1555.0       1074.0  11.9  54.0  0.7502  \n",
       "3           1584.0       1203.0  11.0  60.0  0.7867  \n",
       "4           1490.0       1110.0  11.2  59.6  0.7888  \n",
       "...            ...          ...   ...   ...     ...  \n",
       "9352        1374.0       1729.0  21.9  29.3  0.7568  \n",
       "9353        1264.0       1269.0  24.3  23.7  0.7119  \n",
       "9354        1241.0       1092.0  26.9  18.3  0.6406  \n",
       "9355        1041.0        770.0  28.3  13.5  0.5139  \n",
       "9356        1129.0        816.0  28.5  13.1  0.5028  \n",
       "\n",
       "[9357 rows x 11 columns]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.drop(df.columns[[2,4,5,7]], axis =1,inplace = True)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rMSYDzn1uQYm"
   },
   "source": [
    "### 3- Handle missing data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lKZf0YPcVknc",
    "outputId": "95d28c82-0597-4289-8e28-b34cd09d663d"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Date                0\n",
       "Time                0\n",
       "PT08.S1(CO)       366\n",
       "PT08.S2(NMHC)     366\n",
       "PT08.S3(NOx)      366\n",
       "NO2(GT)          1642\n",
       "PT08.S4(NO2)      366\n",
       "PT08.S5(O3)       366\n",
       "T                 366\n",
       "RH                366\n",
       "AH                366\n",
       "dtype: int64"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 424
    },
    "id": "0gV9yZfmxdra",
    "outputId": "bdf0622c-499c-4b40-98b7-4af675eeb03c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Time</th>\n",
       "      <th>PT08.S1(CO)</th>\n",
       "      <th>PT08.S2(NMHC)</th>\n",
       "      <th>PT08.S3(NOx)</th>\n",
       "      <th>NO2(GT)</th>\n",
       "      <th>PT08.S4(NO2)</th>\n",
       "      <th>PT08.S5(O3)</th>\n",
       "      <th>T</th>\n",
       "      <th>RH</th>\n",
       "      <th>AH</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>18:00:00</td>\n",
       "      <td>1360.0</td>\n",
       "      <td>1046.0</td>\n",
       "      <td>1056.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>1692.0</td>\n",
       "      <td>1268.0</td>\n",
       "      <td>13.6</td>\n",
       "      <td>48.9</td>\n",
       "      <td>0.7578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>19:00:00</td>\n",
       "      <td>1292.0</td>\n",
       "      <td>955.0</td>\n",
       "      <td>1174.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>1559.0</td>\n",
       "      <td>972.0</td>\n",
       "      <td>13.3</td>\n",
       "      <td>47.7</td>\n",
       "      <td>0.7255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>20:00:00</td>\n",
       "      <td>1402.0</td>\n",
       "      <td>939.0</td>\n",
       "      <td>1140.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>1555.0</td>\n",
       "      <td>1074.0</td>\n",
       "      <td>11.9</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.7502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>21:00:00</td>\n",
       "      <td>1376.0</td>\n",
       "      <td>948.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>122.0</td>\n",
       "      <td>1584.0</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.7867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3/10/2004</td>\n",
       "      <td>22:00:00</td>\n",
       "      <td>1272.0</td>\n",
       "      <td>836.0</td>\n",
       "      <td>1205.0</td>\n",
       "      <td>116.0</td>\n",
       "      <td>1490.0</td>\n",
       "      <td>1110.0</td>\n",
       "      <td>11.2</td>\n",
       "      <td>59.6</td>\n",
       "      <td>0.7888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9352</th>\n",
       "      <td>4/4/2005</td>\n",
       "      <td>10:00:00</td>\n",
       "      <td>1314.0</td>\n",
       "      <td>1101.0</td>\n",
       "      <td>539.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>1374.0</td>\n",
       "      <td>1729.0</td>\n",
       "      <td>21.9</td>\n",
       "      <td>29.3</td>\n",
       "      <td>0.7568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9353</th>\n",
       "      <td>4/4/2005</td>\n",
       "      <td>11:00:00</td>\n",
       "      <td>1163.0</td>\n",
       "      <td>1027.0</td>\n",
       "      <td>604.0</td>\n",
       "      <td>179.0</td>\n",
       "      <td>1264.0</td>\n",
       "      <td>1269.0</td>\n",
       "      <td>24.3</td>\n",
       "      <td>23.7</td>\n",
       "      <td>0.7119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9354</th>\n",
       "      <td>4/4/2005</td>\n",
       "      <td>12:00:00</td>\n",
       "      <td>1142.0</td>\n",
       "      <td>1063.0</td>\n",
       "      <td>603.0</td>\n",
       "      <td>175.0</td>\n",
       "      <td>1241.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>26.9</td>\n",
       "      <td>18.3</td>\n",
       "      <td>0.6406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9355</th>\n",
       "      <td>4/4/2005</td>\n",
       "      <td>13:00:00</td>\n",
       "      <td>1003.0</td>\n",
       "      <td>961.0</td>\n",
       "      <td>702.0</td>\n",
       "      <td>156.0</td>\n",
       "      <td>1041.0</td>\n",
       "      <td>770.0</td>\n",
       "      <td>28.3</td>\n",
       "      <td>13.5</td>\n",
       "      <td>0.5139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9356</th>\n",
       "      <td>4/4/2005</td>\n",
       "      <td>14:00:00</td>\n",
       "      <td>1071.0</td>\n",
       "      <td>1047.0</td>\n",
       "      <td>654.0</td>\n",
       "      <td>168.0</td>\n",
       "      <td>1129.0</td>\n",
       "      <td>816.0</td>\n",
       "      <td>28.5</td>\n",
       "      <td>13.1</td>\n",
       "      <td>0.5028</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8991 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           Date      Time  PT08.S1(CO)  PT08.S2(NMHC)  PT08.S3(NOx)  NO2(GT)  \\\n",
       "0     3/10/2004  18:00:00       1360.0         1046.0        1056.0    113.0   \n",
       "1     3/10/2004  19:00:00       1292.0          955.0        1174.0     92.0   \n",
       "2     3/10/2004  20:00:00       1402.0          939.0        1140.0    114.0   \n",
       "3     3/10/2004  21:00:00       1376.0          948.0        1092.0    122.0   \n",
       "4     3/10/2004  22:00:00       1272.0          836.0        1205.0    116.0   \n",
       "...         ...       ...          ...            ...           ...      ...   \n",
       "9352   4/4/2005  10:00:00       1314.0         1101.0         539.0    190.0   \n",
       "9353   4/4/2005  11:00:00       1163.0         1027.0         604.0    179.0   \n",
       "9354   4/4/2005  12:00:00       1142.0         1063.0         603.0    175.0   \n",
       "9355   4/4/2005  13:00:00       1003.0          961.0         702.0    156.0   \n",
       "9356   4/4/2005  14:00:00       1071.0         1047.0         654.0    168.0   \n",
       "\n",
       "      PT08.S4(NO2)  PT08.S5(O3)     T    RH      AH  \n",
       "0           1692.0       1268.0  13.6  48.9  0.7578  \n",
       "1           1559.0        972.0  13.3  47.7  0.7255  \n",
       "2           1555.0       1074.0  11.9  54.0  0.7502  \n",
       "3           1584.0       1203.0  11.0  60.0  0.7867  \n",
       "4           1490.0       1110.0  11.2  59.6  0.7888  \n",
       "...            ...          ...   ...   ...     ...  \n",
       "9352        1374.0       1729.0  21.9  29.3  0.7568  \n",
       "9353        1264.0       1269.0  24.3  23.7  0.7119  \n",
       "9354        1241.0       1092.0  26.9  18.3  0.6406  \n",
       "9355        1041.0        770.0  28.3  13.5  0.5139  \n",
       "9356        1129.0        816.0  28.5  13.1  0.5028  \n",
       "\n",
       "[8991 rows x 11 columns]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# numbers of null values in 8 columns is 366\n",
    "# what if they're all in the same rows?\n",
    "# just to check, drop rows with more than 7 values missing\n",
    "\n",
    "df_dropped_rows = df[df.isnull().sum(axis=1) < 8]\n",
    "df_dropped_rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "oeCbFWf_X_6J",
    "outputId": "4a4f1e11-6ece-44b1-c5f2-dc6993e5db90"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Date                0\n",
       "Time                0\n",
       "PT08.S1(CO)         0\n",
       "PT08.S2(NMHC)       0\n",
       "PT08.S3(NOx)        0\n",
       "NO2(GT)          1598\n",
       "PT08.S4(NO2)        0\n",
       "PT08.S5(O3)         0\n",
       "T                   0\n",
       "RH                  0\n",
       "AH                  0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_dropped_rows.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "id": "wSa3aLN1fIFS"
   },
   "outputs": [],
   "source": [
    "from sklearn.experimental import enable_iterative_imputer\n",
    "from sklearn.impute import IterativeImputer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "i5i93sVXfBe-"
   },
   "outputs": [],
   "source": [
    "#Imputation will be done after splitting the date and time columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v7vinFJmuQYn"
   },
   "source": [
    "### 4- Replace date feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 614
    },
    "id": "e1X8-GXvuQYn",
    "outputId": "17297b47-3640-42cf-842f-b63b49c7559b"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/n9/r3zlbmnd6csc4y4lg65njb6r0000gn/T/ipykernel_32496/3761602337.py:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_dropped_rows[[\"Day\", \"Month\", \"Year\"]] = df_dropped_rows[\"Date\"].str.split(\"/\", expand = True)\n",
      "/var/folders/n9/r3zlbmnd6csc4y4lg65njb6r0000gn/T/ipykernel_32496/3761602337.py:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_dropped_rows[[\"Day\", \"Month\", \"Year\"]] = df_dropped_rows[\"Date\"].str.split(\"/\", expand = True)\n",
      "/var/folders/n9/r3zlbmnd6csc4y4lg65njb6r0000gn/T/ipykernel_32496/3761602337.py:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_dropped_rows[[\"Day\", \"Month\", \"Year\"]] = df_dropped_rows[\"Date\"].str.split(\"/\", expand = True)\n",
      "/var/folders/n9/r3zlbmnd6csc4y4lg65njb6r0000gn/T/ipykernel_32496/3761602337.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_dropped_rows.drop('Date',axis = 1,inplace = True)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>PT08.S1(CO)</th>\n",
       "      <th>PT08.S2(NMHC)</th>\n",
       "      <th>PT08.S3(NOx)</th>\n",
       "      <th>NO2(GT)</th>\n",
       "      <th>PT08.S4(NO2)</th>\n",
       "      <th>PT08.S5(O3)</th>\n",
       "      <th>T</th>\n",
       "      <th>RH</th>\n",
       "      <th>AH</th>\n",
       "      <th>Day</th>\n",
       "      <th>Month</th>\n",
       "      <th>Year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>18:00:00</td>\n",
       "      <td>1360.0</td>\n",
       "      <td>1046.0</td>\n",
       "      <td>1056.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>1692.0</td>\n",
       "      <td>1268.0</td>\n",
       "      <td>13.6</td>\n",
       "      <td>48.9</td>\n",
       "      <td>0.7578</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>2004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>19:00:00</td>\n",
       "      <td>1292.0</td>\n",
       "      <td>955.0</td>\n",
       "      <td>1174.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>1559.0</td>\n",
       "      <td>972.0</td>\n",
       "      <td>13.3</td>\n",
       "      <td>47.7</td>\n",
       "      <td>0.7255</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>2004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>20:00:00</td>\n",
       "      <td>1402.0</td>\n",
       "      <td>939.0</td>\n",
       "      <td>1140.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>1555.0</td>\n",
       "      <td>1074.0</td>\n",
       "      <td>11.9</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.7502</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>2004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>21:00:00</td>\n",
       "      <td>1376.0</td>\n",
       "      <td>948.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>122.0</td>\n",
       "      <td>1584.0</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.7867</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>2004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>22:00:00</td>\n",
       "      <td>1272.0</td>\n",
       "      <td>836.0</td>\n",
       "      <td>1205.0</td>\n",
       "      <td>116.0</td>\n",
       "      <td>1490.0</td>\n",
       "      <td>1110.0</td>\n",
       "      <td>11.2</td>\n",
       "      <td>59.6</td>\n",
       "      <td>0.7888</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>2004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9352</th>\n",
       "      <td>10:00:00</td>\n",
       "      <td>1314.0</td>\n",
       "      <td>1101.0</td>\n",
       "      <td>539.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>1374.0</td>\n",
       "      <td>1729.0</td>\n",
       "      <td>21.9</td>\n",
       "      <td>29.3</td>\n",
       "      <td>0.7568</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>2005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9353</th>\n",
       "      <td>11:00:00</td>\n",
       "      <td>1163.0</td>\n",
       "      <td>1027.0</td>\n",
       "      <td>604.0</td>\n",
       "      <td>179.0</td>\n",
       "      <td>1264.0</td>\n",
       "      <td>1269.0</td>\n",
       "      <td>24.3</td>\n",
       "      <td>23.7</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>2005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9354</th>\n",
       "      <td>12:00:00</td>\n",
       "      <td>1142.0</td>\n",
       "      <td>1063.0</td>\n",
       "      <td>603.0</td>\n",
       "      <td>175.0</td>\n",
       "      <td>1241.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>26.9</td>\n",
       "      <td>18.3</td>\n",
       "      <td>0.6406</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>2005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9355</th>\n",
       "      <td>13:00:00</td>\n",
       "      <td>1003.0</td>\n",
       "      <td>961.0</td>\n",
       "      <td>702.0</td>\n",
       "      <td>156.0</td>\n",
       "      <td>1041.0</td>\n",
       "      <td>770.0</td>\n",
       "      <td>28.3</td>\n",
       "      <td>13.5</td>\n",
       "      <td>0.5139</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>2005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9356</th>\n",
       "      <td>14:00:00</td>\n",
       "      <td>1071.0</td>\n",
       "      <td>1047.0</td>\n",
       "      <td>654.0</td>\n",
       "      <td>168.0</td>\n",
       "      <td>1129.0</td>\n",
       "      <td>816.0</td>\n",
       "      <td>28.5</td>\n",
       "      <td>13.1</td>\n",
       "      <td>0.5028</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>2005</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8991 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          Time  PT08.S1(CO)  PT08.S2(NMHC)  PT08.S3(NOx)  NO2(GT)  \\\n",
       "0     18:00:00       1360.0         1046.0        1056.0    113.0   \n",
       "1     19:00:00       1292.0          955.0        1174.0     92.0   \n",
       "2     20:00:00       1402.0          939.0        1140.0    114.0   \n",
       "3     21:00:00       1376.0          948.0        1092.0    122.0   \n",
       "4     22:00:00       1272.0          836.0        1205.0    116.0   \n",
       "...        ...          ...            ...           ...      ...   \n",
       "9352  10:00:00       1314.0         1101.0         539.0    190.0   \n",
       "9353  11:00:00       1163.0         1027.0         604.0    179.0   \n",
       "9354  12:00:00       1142.0         1063.0         603.0    175.0   \n",
       "9355  13:00:00       1003.0          961.0         702.0    156.0   \n",
       "9356  14:00:00       1071.0         1047.0         654.0    168.0   \n",
       "\n",
       "      PT08.S4(NO2)  PT08.S5(O3)     T    RH      AH Day Month  Year  \n",
       "0           1692.0       1268.0  13.6  48.9  0.7578   3    10  2004  \n",
       "1           1559.0        972.0  13.3  47.7  0.7255   3    10  2004  \n",
       "2           1555.0       1074.0  11.9  54.0  0.7502   3    10  2004  \n",
       "3           1584.0       1203.0  11.0  60.0  0.7867   3    10  2004  \n",
       "4           1490.0       1110.0  11.2  59.6  0.7888   3    10  2004  \n",
       "...            ...          ...   ...   ...     ...  ..   ...   ...  \n",
       "9352        1374.0       1729.0  21.9  29.3  0.7568   4     4  2005  \n",
       "9353        1264.0       1269.0  24.3  23.7  0.7119   4     4  2005  \n",
       "9354        1241.0       1092.0  26.9  18.3  0.6406   4     4  2005  \n",
       "9355        1041.0        770.0  28.3  13.5  0.5139   4     4  2005  \n",
       "9356        1129.0        816.0  28.5  13.1  0.5028   4     4  2005  \n",
       "\n",
       "[8991 rows x 13 columns]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_dropped_rows[[\"Day\", \"Month\", \"Year\"]] = df_dropped_rows[\"Date\"].str.split(\"/\", expand = True)\n",
    "df_dropped_rows.drop('Date',axis = 1,inplace = True)\n",
    "df_dropped_rows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zHvSGF9fuQYo"
   },
   "source": [
    "### 5- Replace time feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 658
    },
    "id": "vWqvEVdsuQYo",
    "outputId": "1d57f61b-59d8-48ad-b7a9-237098eed773"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/n9/r3zlbmnd6csc4y4lg65njb6r0000gn/T/ipykernel_32496/854840442.py:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_dropped_rows[[\"Hours\", \"Minutes\", \"Seconds\"]] = df_dropped_rows[\"Time\"].str.split(\":\", expand = True)\n",
      "/var/folders/n9/r3zlbmnd6csc4y4lg65njb6r0000gn/T/ipykernel_32496/854840442.py:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_dropped_rows[[\"Hours\", \"Minutes\", \"Seconds\"]] = df_dropped_rows[\"Time\"].str.split(\":\", expand = True)\n",
      "/var/folders/n9/r3zlbmnd6csc4y4lg65njb6r0000gn/T/ipykernel_32496/854840442.py:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_dropped_rows[[\"Hours\", \"Minutes\", \"Seconds\"]] = df_dropped_rows[\"Time\"].str.split(\":\", expand = True)\n",
      "/var/folders/n9/r3zlbmnd6csc4y4lg65njb6r0000gn/T/ipykernel_32496/854840442.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df_dropped_rows.drop('Time',axis = 1,inplace = True)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PT08.S1(CO)</th>\n",
       "      <th>PT08.S2(NMHC)</th>\n",
       "      <th>PT08.S3(NOx)</th>\n",
       "      <th>NO2(GT)</th>\n",
       "      <th>PT08.S4(NO2)</th>\n",
       "      <th>PT08.S5(O3)</th>\n",
       "      <th>T</th>\n",
       "      <th>RH</th>\n",
       "      <th>AH</th>\n",
       "      <th>Day</th>\n",
       "      <th>Month</th>\n",
       "      <th>Year</th>\n",
       "      <th>Hours</th>\n",
       "      <th>Minutes</th>\n",
       "      <th>Seconds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1360.0</td>\n",
       "      <td>1046.0</td>\n",
       "      <td>1056.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>1692.0</td>\n",
       "      <td>1268.0</td>\n",
       "      <td>13.6</td>\n",
       "      <td>48.9</td>\n",
       "      <td>0.7578</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>2004</td>\n",
       "      <td>18</td>\n",
       "      <td>00</td>\n",
       "      <td>00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1292.0</td>\n",
       "      <td>955.0</td>\n",
       "      <td>1174.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>1559.0</td>\n",
       "      <td>972.0</td>\n",
       "      <td>13.3</td>\n",
       "      <td>47.7</td>\n",
       "      <td>0.7255</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>2004</td>\n",
       "      <td>19</td>\n",
       "      <td>00</td>\n",
       "      <td>00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1402.0</td>\n",
       "      <td>939.0</td>\n",
       "      <td>1140.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>1555.0</td>\n",
       "      <td>1074.0</td>\n",
       "      <td>11.9</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.7502</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>2004</td>\n",
       "      <td>20</td>\n",
       "      <td>00</td>\n",
       "      <td>00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1376.0</td>\n",
       "      <td>948.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>122.0</td>\n",
       "      <td>1584.0</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.7867</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>2004</td>\n",
       "      <td>21</td>\n",
       "      <td>00</td>\n",
       "      <td>00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1272.0</td>\n",
       "      <td>836.0</td>\n",
       "      <td>1205.0</td>\n",
       "      <td>116.0</td>\n",
       "      <td>1490.0</td>\n",
       "      <td>1110.0</td>\n",
       "      <td>11.2</td>\n",
       "      <td>59.6</td>\n",
       "      <td>0.7888</td>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>2004</td>\n",
       "      <td>22</td>\n",
       "      <td>00</td>\n",
       "      <td>00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9352</th>\n",
       "      <td>1314.0</td>\n",
       "      <td>1101.0</td>\n",
       "      <td>539.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>1374.0</td>\n",
       "      <td>1729.0</td>\n",
       "      <td>21.9</td>\n",
       "      <td>29.3</td>\n",
       "      <td>0.7568</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>2005</td>\n",
       "      <td>10</td>\n",
       "      <td>00</td>\n",
       "      <td>00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9353</th>\n",
       "      <td>1163.0</td>\n",
       "      <td>1027.0</td>\n",
       "      <td>604.0</td>\n",
       "      <td>179.0</td>\n",
       "      <td>1264.0</td>\n",
       "      <td>1269.0</td>\n",
       "      <td>24.3</td>\n",
       "      <td>23.7</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>2005</td>\n",
       "      <td>11</td>\n",
       "      <td>00</td>\n",
       "      <td>00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9354</th>\n",
       "      <td>1142.0</td>\n",
       "      <td>1063.0</td>\n",
       "      <td>603.0</td>\n",
       "      <td>175.0</td>\n",
       "      <td>1241.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>26.9</td>\n",
       "      <td>18.3</td>\n",
       "      <td>0.6406</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>2005</td>\n",
       "      <td>12</td>\n",
       "      <td>00</td>\n",
       "      <td>00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9355</th>\n",
       "      <td>1003.0</td>\n",
       "      <td>961.0</td>\n",
       "      <td>702.0</td>\n",
       "      <td>156.0</td>\n",
       "      <td>1041.0</td>\n",
       "      <td>770.0</td>\n",
       "      <td>28.3</td>\n",
       "      <td>13.5</td>\n",
       "      <td>0.5139</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>2005</td>\n",
       "      <td>13</td>\n",
       "      <td>00</td>\n",
       "      <td>00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9356</th>\n",
       "      <td>1071.0</td>\n",
       "      <td>1047.0</td>\n",
       "      <td>654.0</td>\n",
       "      <td>168.0</td>\n",
       "      <td>1129.0</td>\n",
       "      <td>816.0</td>\n",
       "      <td>28.5</td>\n",
       "      <td>13.1</td>\n",
       "      <td>0.5028</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>2005</td>\n",
       "      <td>14</td>\n",
       "      <td>00</td>\n",
       "      <td>00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8991 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      PT08.S1(CO)  PT08.S2(NMHC)  PT08.S3(NOx)  NO2(GT)  PT08.S4(NO2)  \\\n",
       "0          1360.0         1046.0        1056.0    113.0        1692.0   \n",
       "1          1292.0          955.0        1174.0     92.0        1559.0   \n",
       "2          1402.0          939.0        1140.0    114.0        1555.0   \n",
       "3          1376.0          948.0        1092.0    122.0        1584.0   \n",
       "4          1272.0          836.0        1205.0    116.0        1490.0   \n",
       "...           ...            ...           ...      ...           ...   \n",
       "9352       1314.0         1101.0         539.0    190.0        1374.0   \n",
       "9353       1163.0         1027.0         604.0    179.0        1264.0   \n",
       "9354       1142.0         1063.0         603.0    175.0        1241.0   \n",
       "9355       1003.0          961.0         702.0    156.0        1041.0   \n",
       "9356       1071.0         1047.0         654.0    168.0        1129.0   \n",
       "\n",
       "      PT08.S5(O3)     T    RH      AH Day Month  Year Hours Minutes Seconds  \n",
       "0          1268.0  13.6  48.9  0.7578   3    10  2004    18      00      00  \n",
       "1           972.0  13.3  47.7  0.7255   3    10  2004    19      00      00  \n",
       "2          1074.0  11.9  54.0  0.7502   3    10  2004    20      00      00  \n",
       "3          1203.0  11.0  60.0  0.7867   3    10  2004    21      00      00  \n",
       "4          1110.0  11.2  59.6  0.7888   3    10  2004    22      00      00  \n",
       "...           ...   ...   ...     ...  ..   ...   ...   ...     ...     ...  \n",
       "9352       1729.0  21.9  29.3  0.7568   4     4  2005    10      00      00  \n",
       "9353       1269.0  24.3  23.7  0.7119   4     4  2005    11      00      00  \n",
       "9354       1092.0  26.9  18.3  0.6406   4     4  2005    12      00      00  \n",
       "9355        770.0  28.3  13.5  0.5139   4     4  2005    13      00      00  \n",
       "9356        816.0  28.5  13.1  0.5028   4     4  2005    14      00      00  \n",
       "\n",
       "[8991 rows x 15 columns]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_dropped_rows[[\"Hours\", \"Minutes\", \"Seconds\"]] = df_dropped_rows[\"Time\"].str.split(\":\", expand = True)\n",
    "df_dropped_rows.drop('Time',axis = 1,inplace = True)\n",
    "df_dropped_rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 468
    },
    "id": "qlwTeJisF2LK",
    "outputId": "dfb30724-7791-4ccd-db2c-5ef10d848254"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PT08.S1(CO)</th>\n",
       "      <th>PT08.S2(NMHC)</th>\n",
       "      <th>PT08.S3(NOx)</th>\n",
       "      <th>NO2(GT)</th>\n",
       "      <th>PT08.S4(NO2)</th>\n",
       "      <th>PT08.S5(O3)</th>\n",
       "      <th>T</th>\n",
       "      <th>RH</th>\n",
       "      <th>AH</th>\n",
       "      <th>Day</th>\n",
       "      <th>Month</th>\n",
       "      <th>Year</th>\n",
       "      <th>Hours</th>\n",
       "      <th>Minutes</th>\n",
       "      <th>Seconds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1360.0</td>\n",
       "      <td>1046.0</td>\n",
       "      <td>1056.0</td>\n",
       "      <td>113.0</td>\n",
       "      <td>1692.0</td>\n",
       "      <td>1268.0</td>\n",
       "      <td>13.6</td>\n",
       "      <td>48.9</td>\n",
       "      <td>0.7578</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2004.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1292.0</td>\n",
       "      <td>955.0</td>\n",
       "      <td>1174.0</td>\n",
       "      <td>92.0</td>\n",
       "      <td>1559.0</td>\n",
       "      <td>972.0</td>\n",
       "      <td>13.3</td>\n",
       "      <td>47.7</td>\n",
       "      <td>0.7255</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2004.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1402.0</td>\n",
       "      <td>939.0</td>\n",
       "      <td>1140.0</td>\n",
       "      <td>114.0</td>\n",
       "      <td>1555.0</td>\n",
       "      <td>1074.0</td>\n",
       "      <td>11.9</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.7502</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2004.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1376.0</td>\n",
       "      <td>948.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>122.0</td>\n",
       "      <td>1584.0</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.7867</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2004.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1272.0</td>\n",
       "      <td>836.0</td>\n",
       "      <td>1205.0</td>\n",
       "      <td>116.0</td>\n",
       "      <td>1490.0</td>\n",
       "      <td>1110.0</td>\n",
       "      <td>11.2</td>\n",
       "      <td>59.6</td>\n",
       "      <td>0.7888</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2004.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8986</th>\n",
       "      <td>1314.0</td>\n",
       "      <td>1101.0</td>\n",
       "      <td>539.0</td>\n",
       "      <td>190.0</td>\n",
       "      <td>1374.0</td>\n",
       "      <td>1729.0</td>\n",
       "      <td>21.9</td>\n",
       "      <td>29.3</td>\n",
       "      <td>0.7568</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8987</th>\n",
       "      <td>1163.0</td>\n",
       "      <td>1027.0</td>\n",
       "      <td>604.0</td>\n",
       "      <td>179.0</td>\n",
       "      <td>1264.0</td>\n",
       "      <td>1269.0</td>\n",
       "      <td>24.3</td>\n",
       "      <td>23.7</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8988</th>\n",
       "      <td>1142.0</td>\n",
       "      <td>1063.0</td>\n",
       "      <td>603.0</td>\n",
       "      <td>175.0</td>\n",
       "      <td>1241.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>26.9</td>\n",
       "      <td>18.3</td>\n",
       "      <td>0.6406</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8989</th>\n",
       "      <td>1003.0</td>\n",
       "      <td>961.0</td>\n",
       "      <td>702.0</td>\n",
       "      <td>156.0</td>\n",
       "      <td>1041.0</td>\n",
       "      <td>770.0</td>\n",
       "      <td>28.3</td>\n",
       "      <td>13.5</td>\n",
       "      <td>0.5139</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8990</th>\n",
       "      <td>1071.0</td>\n",
       "      <td>1047.0</td>\n",
       "      <td>654.0</td>\n",
       "      <td>168.0</td>\n",
       "      <td>1129.0</td>\n",
       "      <td>816.0</td>\n",
       "      <td>28.5</td>\n",
       "      <td>13.1</td>\n",
       "      <td>0.5028</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8991 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      PT08.S1(CO)  PT08.S2(NMHC)  PT08.S3(NOx)  NO2(GT)  PT08.S4(NO2)  \\\n",
       "0          1360.0         1046.0        1056.0    113.0        1692.0   \n",
       "1          1292.0          955.0        1174.0     92.0        1559.0   \n",
       "2          1402.0          939.0        1140.0    114.0        1555.0   \n",
       "3          1376.0          948.0        1092.0    122.0        1584.0   \n",
       "4          1272.0          836.0        1205.0    116.0        1490.0   \n",
       "...           ...            ...           ...      ...           ...   \n",
       "8986       1314.0         1101.0         539.0    190.0        1374.0   \n",
       "8987       1163.0         1027.0         604.0    179.0        1264.0   \n",
       "8988       1142.0         1063.0         603.0    175.0        1241.0   \n",
       "8989       1003.0          961.0         702.0    156.0        1041.0   \n",
       "8990       1071.0         1047.0         654.0    168.0        1129.0   \n",
       "\n",
       "      PT08.S5(O3)     T    RH      AH  Day  Month    Year  Hours  Minutes  \\\n",
       "0          1268.0  13.6  48.9  0.7578  3.0   10.0  2004.0   18.0      0.0   \n",
       "1           972.0  13.3  47.7  0.7255  3.0   10.0  2004.0   19.0      0.0   \n",
       "2          1074.0  11.9  54.0  0.7502  3.0   10.0  2004.0   20.0      0.0   \n",
       "3          1203.0  11.0  60.0  0.7867  3.0   10.0  2004.0   21.0      0.0   \n",
       "4          1110.0  11.2  59.6  0.7888  3.0   10.0  2004.0   22.0      0.0   \n",
       "...           ...   ...   ...     ...  ...    ...     ...    ...      ...   \n",
       "8986       1729.0  21.9  29.3  0.7568  4.0    4.0  2005.0   10.0      0.0   \n",
       "8987       1269.0  24.3  23.7  0.7119  4.0    4.0  2005.0   11.0      0.0   \n",
       "8988       1092.0  26.9  18.3  0.6406  4.0    4.0  2005.0   12.0      0.0   \n",
       "8989        770.0  28.3  13.5  0.5139  4.0    4.0  2005.0   13.0      0.0   \n",
       "8990        816.0  28.5  13.1  0.5028  4.0    4.0  2005.0   14.0      0.0   \n",
       "\n",
       "      Seconds  \n",
       "0         0.0  \n",
       "1         0.0  \n",
       "2         0.0  \n",
       "3         0.0  \n",
       "4         0.0  \n",
       "...       ...  \n",
       "8986      0.0  \n",
       "8987      0.0  \n",
       "8988      0.0  \n",
       "8989      0.0  \n",
       "8990      0.0  \n",
       "\n",
       "[8991 rows x 15 columns]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Imputation will be done here\n",
    "\n",
    "imp = IterativeImputer(max_iter=10, random_state=0)\n",
    "imputed_df = imp.fit_transform(df_dropped_rows)\n",
    "imputed_df = pd.DataFrame(imputed_df, columns=df_dropped_rows.columns)\n",
    "imputed_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YUur1uWtiLvZ",
    "outputId": "49d1f0c7-60dd-41af-b5c2-606ee9193834"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PT08.S1(CO)      0\n",
       "PT08.S2(NMHC)    0\n",
       "PT08.S3(NOx)     0\n",
       "NO2(GT)          0\n",
       "PT08.S4(NO2)     0\n",
       "PT08.S5(O3)      0\n",
       "T                0\n",
       "RH               0\n",
       "AH               0\n",
       "Day              0\n",
       "Month            0\n",
       "Year             0\n",
       "Hours            0\n",
       "Minutes          0\n",
       "Seconds          0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imputed_df.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rV0bYDXzuQYo"
   },
   "source": [
    "### 6- Apply feature scaling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "id": "qIgBH_d2uQYo"
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler, PolynomialFeatures\n",
    "from sklearn.linear_model import RidgeCV, LassoCV, ElasticNetCV\n",
    "from sklearn.metrics import mean_absolute_error, mean_squared_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "id": "pmPqeyiuFU_z"
   },
   "outputs": [],
   "source": [
    "X = imputed_df.drop('NO2(GT)',axis = 1)\n",
    "y = imputed_df['NO2(GT)']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 424
    },
    "id": "BUXzGFRcGrv-",
    "outputId": "2f94fa0f-cb23-468b-ceb6-d4c32864d021"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PT08.S1(CO)</th>\n",
       "      <th>PT08.S2(NMHC)</th>\n",
       "      <th>PT08.S3(NOx)</th>\n",
       "      <th>PT08.S4(NO2)</th>\n",
       "      <th>PT08.S5(O3)</th>\n",
       "      <th>T</th>\n",
       "      <th>RH</th>\n",
       "      <th>AH</th>\n",
       "      <th>Day</th>\n",
       "      <th>Month</th>\n",
       "      <th>Year</th>\n",
       "      <th>Hours</th>\n",
       "      <th>Minutes</th>\n",
       "      <th>Seconds</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1360.0</td>\n",
       "      <td>1046.0</td>\n",
       "      <td>1056.0</td>\n",
       "      <td>1692.0</td>\n",
       "      <td>1268.0</td>\n",
       "      <td>13.6</td>\n",
       "      <td>48.9</td>\n",
       "      <td>0.7578</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2004.0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1292.0</td>\n",
       "      <td>955.0</td>\n",
       "      <td>1174.0</td>\n",
       "      <td>1559.0</td>\n",
       "      <td>972.0</td>\n",
       "      <td>13.3</td>\n",
       "      <td>47.7</td>\n",
       "      <td>0.7255</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2004.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1402.0</td>\n",
       "      <td>939.0</td>\n",
       "      <td>1140.0</td>\n",
       "      <td>1555.0</td>\n",
       "      <td>1074.0</td>\n",
       "      <td>11.9</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0.7502</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2004.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1376.0</td>\n",
       "      <td>948.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>1584.0</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.7867</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2004.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1272.0</td>\n",
       "      <td>836.0</td>\n",
       "      <td>1205.0</td>\n",
       "      <td>1490.0</td>\n",
       "      <td>1110.0</td>\n",
       "      <td>11.2</td>\n",
       "      <td>59.6</td>\n",
       "      <td>0.7888</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2004.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8986</th>\n",
       "      <td>1314.0</td>\n",
       "      <td>1101.0</td>\n",
       "      <td>539.0</td>\n",
       "      <td>1374.0</td>\n",
       "      <td>1729.0</td>\n",
       "      <td>21.9</td>\n",
       "      <td>29.3</td>\n",
       "      <td>0.7568</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8987</th>\n",
       "      <td>1163.0</td>\n",
       "      <td>1027.0</td>\n",
       "      <td>604.0</td>\n",
       "      <td>1264.0</td>\n",
       "      <td>1269.0</td>\n",
       "      <td>24.3</td>\n",
       "      <td>23.7</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8988</th>\n",
       "      <td>1142.0</td>\n",
       "      <td>1063.0</td>\n",
       "      <td>603.0</td>\n",
       "      <td>1241.0</td>\n",
       "      <td>1092.0</td>\n",
       "      <td>26.9</td>\n",
       "      <td>18.3</td>\n",
       "      <td>0.6406</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8989</th>\n",
       "      <td>1003.0</td>\n",
       "      <td>961.0</td>\n",
       "      <td>702.0</td>\n",
       "      <td>1041.0</td>\n",
       "      <td>770.0</td>\n",
       "      <td>28.3</td>\n",
       "      <td>13.5</td>\n",
       "      <td>0.5139</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8990</th>\n",
       "      <td>1071.0</td>\n",
       "      <td>1047.0</td>\n",
       "      <td>654.0</td>\n",
       "      <td>1129.0</td>\n",
       "      <td>816.0</td>\n",
       "      <td>28.5</td>\n",
       "      <td>13.1</td>\n",
       "      <td>0.5028</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2005.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8991 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      PT08.S1(CO)  PT08.S2(NMHC)  PT08.S3(NOx)  PT08.S4(NO2)  PT08.S5(O3)  \\\n",
       "0          1360.0         1046.0        1056.0        1692.0       1268.0   \n",
       "1          1292.0          955.0        1174.0        1559.0        972.0   \n",
       "2          1402.0          939.0        1140.0        1555.0       1074.0   \n",
       "3          1376.0          948.0        1092.0        1584.0       1203.0   \n",
       "4          1272.0          836.0        1205.0        1490.0       1110.0   \n",
       "...           ...            ...           ...           ...          ...   \n",
       "8986       1314.0         1101.0         539.0        1374.0       1729.0   \n",
       "8987       1163.0         1027.0         604.0        1264.0       1269.0   \n",
       "8988       1142.0         1063.0         603.0        1241.0       1092.0   \n",
       "8989       1003.0          961.0         702.0        1041.0        770.0   \n",
       "8990       1071.0         1047.0         654.0        1129.0        816.0   \n",
       "\n",
       "         T    RH      AH  Day  Month    Year  Hours  Minutes  Seconds  \n",
       "0     13.6  48.9  0.7578  3.0   10.0  2004.0   18.0      0.0      0.0  \n",
       "1     13.3  47.7  0.7255  3.0   10.0  2004.0   19.0      0.0      0.0  \n",
       "2     11.9  54.0  0.7502  3.0   10.0  2004.0   20.0      0.0      0.0  \n",
       "3     11.0  60.0  0.7867  3.0   10.0  2004.0   21.0      0.0      0.0  \n",
       "4     11.2  59.6  0.7888  3.0   10.0  2004.0   22.0      0.0      0.0  \n",
       "...    ...   ...     ...  ...    ...     ...    ...      ...      ...  \n",
       "8986  21.9  29.3  0.7568  4.0    4.0  2005.0   10.0      0.0      0.0  \n",
       "8987  24.3  23.7  0.7119  4.0    4.0  2005.0   11.0      0.0      0.0  \n",
       "8988  26.9  18.3  0.6406  4.0    4.0  2005.0   12.0      0.0      0.0  \n",
       "8989  28.3  13.5  0.5139  4.0    4.0  2005.0   13.0      0.0      0.0  \n",
       "8990  28.5  13.1  0.5028  4.0    4.0  2005.0   14.0      0.0      0.0  \n",
       "\n",
       "[8991 rows x 14 columns]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "y8erAknpGwgM",
    "outputId": "63610df5-63ce-4469-d552-cdbb5164f50b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       113.0\n",
       "1        92.0\n",
       "2       114.0\n",
       "3       122.0\n",
       "4       116.0\n",
       "        ...  \n",
       "8986    190.0\n",
       "8987    179.0\n",
       "8988    175.0\n",
       "8989    156.0\n",
       "8990    168.0\n",
       "Name: NO2(GT), Length: 8991, dtype: float64"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "id": "2iwFOKnJJKGZ"
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=101)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "id": "nPrawqeqFU_z"
   },
   "outputs": [],
   "source": [
    "scaler = StandardScaler()\n",
    "scaled_X_train = scaler.fit_transform(X_train)\n",
    "scaled_X_test = scaler.fit_transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-Bh7ou_4uQYp"
   },
   "source": [
    "### 7- Evaluate different degrees of lasso CV polynomial regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tLBRmfjuuQYp",
    "outputId": "824708a9-2d49-4651-db20-a1bd1ce58256"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "currently in iteration:  1\n",
      "currently in iteration:  2\n",
      "currently in iteration:  3\n",
      "currently in iteration:  4\n",
      "currently in iteration:  5\n"
     ]
    }
   ],
   "source": [
    "train_rmse_errors = []\n",
    "test_rmse_errors = []\n",
    "for d in range(1, 6):\n",
    "    print(\"currently in iteration: \", d)\n",
    "    polynomial_converter = PolynomialFeatures(degree=d,include_bias=False)\n",
    "    poly_features_train = polynomial_converter.fit_transform(scaled_X_train)\n",
    "    poly_features_test = polynomial_converter.fit_transform(scaled_X_test)\n",
    "    model =  LassoCV(eps=0.1,n_alphas=100,cv=5)\n",
    "    model.fit(poly_features_train,y_train)\n",
    "    train_pred = model.predict(poly_features_train)\n",
    "    test_pred = model.predict(poly_features_test)\n",
    "    train_RMSE = np.sqrt(mean_squared_error(y_train,train_pred))\n",
    "    test_RMSE = np.sqrt(mean_squared_error(y_test,test_pred))  \n",
    "    train_rmse_errors.append(train_RMSE)\n",
    "    test_rmse_errors.append(test_RMSE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pDn9gby-uQYp"
   },
   "source": [
    "### 8- Choose the best degree and print the errors, model coefficients and the lasso parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "id": "5TWFb_1pFU_0"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 296
    },
    "id": "xLdmHaPBuQYq",
    "outputId": "b26abc75-0bc6-4cd2-db2d-b3b42daac104"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x12ab59610>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3WUlEQVR4nO3dd3hUVfrA8e9LOiSUhFBDb0oHQxNlEewFG9KUjojdn+Kuuva1r11RV+nSqyCCCoirriIkEAhNmvQWQkkj/fz+uDdkEhMSIDM3ybyf58njzJwz9765kvfeOXPue8QYg1JKKe9RwekAlFJKeZYmfqWU8jKa+JVSysto4ldKKS+jiV8ppbyMr9MBFEf16tVNw4YNnQ5DKaXKlOjo6OPGmPD8r5eJxN+wYUOioqKcDkMppcoUEdlb0Os61KOUUl5GE79SSnkZTfxKKeVlysQYf0EyMjI4cOAAqampTofimMDAQCIiIvDz83M6FKVUGVJmE/+BAwcICQmhYcOGiIjT4XicMYb4+HgOHDhAo0aNnA5HKVWGlNmhntTUVMLCwrwy6QOICGFhYV79iUcpdWHKbOIHvDbp5/D2318pdWHKdOJXSqnyKiktkxcXbyYhNaPEt62J/wLFx8fTvn172rdvT61atahbt+7Z5yJC+/btad26NbfccgunTp3K89727dszYMCAPK8NGzaMefPmAdCzZ08iIyPPtkVFRdGzZ093/0pKqVIiLjGNgZ+v5svVe4nac6LEt6+J/wKFhYURExNDTEwMY8aM4f/+7//OPq9UqRIxMTFs2rSJ0NBQxo0bd/Z9W7duJSsri59//pnk5ORCt3/s2DGWLVvmiV9FKVWK7I1Ppu9nv7LjWCJfDLmMXpfULPF9aOJ3s27dunHw4MGzz2fOnMngwYO59tprWbRoUaHve/LJJ3n11Vc9EaJSqpTYeOAUd3zyKwlnMph5b1e3JH0ow9M5Xb309Wa2HEoo0W22rFOZF25pdVHbyMrKYuXKlYwcOfLsa7Nnz2b58uVs27aNjz76iEGDBhX43m7durFw4UJWrVpFSEjIRcWhlCr9/rs9jvunRRNayZ8pIzrTJDzYbfvSK343OHPmzNmx/6NHj3LNNdcA1lh99erVqV+/Pr1792b9+vWcOFH4+N2zzz7LK6+84qmwlVIOWbDuACMnr6VBWCUW3H+5W5M+lJMr/ou9Mi9pQUFBxMTEkJKSwnXXXce4ceN45JFHmDlzJtu2bSOnxHRCQgLz58/n3nvvLXA7vXr14tlnn2X16tUejF4p5SnGGP7z027eWLaNy5uE8Z/BlxES6P478fWK340qVqzIhx9+yDvvvEN6ejpz5swhNjaWPXv2sGfPHhYtWsTMmTPPuY1nn32Wt956y0MRK6U8JTvb8NLXW3hj2TZuaVeHScM75U36xsC2byA7u8T3rYnfzTp06EDbtm15/fXXqVu3LnXq1Dnb1qNHD7Zs2cLhw4cLff+NN95IePhf1lFQSpVhaZlZPDxrPZN/3cPIKxrxQf/2BPj65HbIyoRvHodZg2DzghLfvxhjSnyjJS0yMtLkX4hl69atXHrppQ5FVHrocVCqbElIzWD01ChW7z7BP2+8lHt7NM7bIT0Z5o2A7d9C98eg9wtQ4cKu0UUk2hgTmf/1cjHGr5RSZcHRhFSGTlzDrrgk3u/fnts61M3bIekYzOgPh2Pgxrehc8Hf/10sTfxKKeUBO48lMXTiGk6lpDNxWCeubJZvCPf4Tph+JyQehf7T4JKb3BaLJn6llHKz6L0nGTllLb4VKjD7vm60rlslb4d9v8PMASACw5ZAxF9GZ0qUJn6llHKjFVuO8tDMddSqHMjUEV2oH1Yxb4etX8P8UVC5Dtw9D8KauD0mndWjlFJuMmvNPkZ/GUWLmiHMu//yvyb93/8DswdDzdYwcrlHkj7oFb9SSpU4YwwfrtzJeyu207NFOOMGdaRSgEu6zc6G5c/Bbx9Di5vgzvHgX7HwDZYwTfwXKD4+nt69ewNw5MgRfHx8zs6337BhA+3atTvbd8CAATz11FMsWbKE5557juzsbDIyMnj00Uc5fvw4c+fOBSA2NpY2bdoAMGLECB555BEP/1ZKqYuVmZXNc4s2M3PNPu7sGMEbd7bBz8dlcCUjFb4aA5sXQqd74YY3oYJP4Rt0A53HXwJefPFFgoODGTt2LADBwcEkJSXl6ZORkUGDBg1Ys2YNERERpKWlsWfPHlq0aHG2T0HvK0ppOg5KebvUjCwenrme5VuO8uBVTRh7bYu8K+WlnIBZd8O+X+Gal+HyR6wvdN3EsXn8IuIDRAEHjTE3i0gjYBYQBkQDg40x6e6Ow2mJiYlkZmYSFhYGQEBAQJ6kr5Qq206lpDNyShTr9p3kpT6tGHp5w3wd9sG0vnDyT7hzArTp60ic4JmhnkeBrUBl+/mbwHvGmFki8hkwEvj0ovaw7Ck4EntRm/iLWm3ghjcu6K051TlzPP300/Tv358+ffrQoEEDevfuzc0338zAgQOpcIF35CmlSo+Dp84wdOIa9sWnMG5QR25sUztvh0MxMKMfZKbC4IXQ8ApH4szh1sQvIhHATcCrwONifebpBeQUoZ8CvMjFJv5SJqc6Z37jx48nNjaWFStW8Pbbb7N8+XImT57s8fiUUiVn25EEhk5cQ0p6FlNHdqZr47C8HXasgDlDIKgajFgENZwfmnX3Ff/7wN+BnJVEwoBTxphM+/kBoG4B70NERgOjAerXr3/uvVzglbkT2rRpQ5s2bRg8eDCNGjXSxK9UGbZ6dzz3To2ior8Pc8d045JalfN2WPclfP0o1GgJd8+FyrUL3pCHuW2cQURuBo4ZY6Iv5P3GmM+NMZHGmMjyUJ0yKSmJH3/88ezzmJgYGjRo4FxASqmLsjT2MEMmrKFm5UAWPNA9b9I3Bla9DosfgkY9YPjSUpP0wb1X/N2BPiJyIxCINcb/AVBVRHztq/4I4OA5tlEm5R/jv/766/nnP//JW2+9xX333UdQUBCVKlXSq32lyqgpv+7hxa8307F+NSYMjaRqRf/cxqwM+PoxiJkG7e+GWz4AH/cvrnI+3Jb4jTFPA08DiEhPYKwx5m4RmQv0xZrZMxQofMXxMuLFF1/M8zwrK6vAfkuXLj3nds53KqdSyrOMMfz7uz/45MddXNOyJh8N7ECgn8sc/LREmDMUdq2Evz0FPZ9y63TNC+XEDVz/AGaJyCvAemCCAzEopdR5ycjK5qn5scxfd4BBXerzcp9W+LremJVwGGbcBUe3QJ+PoOMQ54ItgkcSvzHmR+BH+/FuoLMn9quUUiUhOS2TB6av47/b43j8muY83Ktp3huzjm2D6X2tG7QGzYZm1zgXbDGU6ZINxpi8B9/LlIW7rpUq644npTFi8lo2HTzNG3e0YUDnfLMM9/xiLZHoE2B9iVunvSNxno8ym/gDAwOJj48nLCzMK5O/MYb4+HgCAwOdDkWpcmtffApDJv7OkYRUPh8cydUta+btEDsPvrofqjW0SipXKxsz9cps4o+IiODAgQPExcU5HYpjAgMDiYiIcDoMpcqlTQdPM2zSGjKzDdNHdeWyBtVyG42BXz+yKmzWvxwGTIeKoc4Fe57KbOL38/OjUaNGToehlCqHft4Rx5gvo6la0Z9ZIzrTtEZwbmN2Fnz7FKz5HFrdDrd9Bn5l65N3mU38SinlDl+tP8jYuRtoWiOYKSM6U7OyS1JPT4EF98K2JdDtIbjmX1AG621p4ldKKazvzb74eTevLd1G18ahfD4kksqBLjdeJcfDzP5wIAqufxO6jnEu2IukiV8p5fWysw2vLt3KhF/+5Ka2tXm3XzsCfF1uzDqx2yqpnHAQ+k2Fln2cC7YEaOJXSnm1tMwsxs7dyNcbDjG8e0Oeu6klFSq4zBQ8EG2VVDZZMGQx1O/iXLAlRBO/UsprJaZmcN+X0fy6K56nb7iE0T0a550evm0pzBsBwTXgnvlQvZlzwZYgTfxKKa90LCGVoZPWsuNoIu/2a8cdHfNNjV47HpY+CbXbwaA5VvIvJzTxK6W8zq64JIZMWMPJlHQmDutEj+Yupd+zs+GHl+GX96D59dB3IvhXci5YN9DEr5TyKuv3nWTE5LVUEGHW6K60jaia25iZBosehNi5cNlwuPFt8Cl/abL8/UZKKVWIlVuP8uCMddSsHMiU4Z1pWN3lSv7MKZh9D+z5GXo/D1c8XipLKpcETfxKKa8wZ+1+nl4YS8valZk0vBPVgwNyG08fsKZrxu+E2z+Hdv2dC9QDNPErpco1Ywwf/7CTd5Zvp0fzcD69uyOVAlxS35FNVknl9GS4Zx407ulYrJ6iiV8pVW5lZRueX7SJ6b/v444OdXmzb1v8XBdP2bUKZg+GgBAY8S3UbOVcsB6kiV8pVS6lZmTx6Kz1fLf5KGP+1oR/XN8i7xz9mJnWYujVW8Ddc6FKXeeC9TBN/EqpcudUSjqjpkQRve8kL9zSkuHdXSr5GgM/vw0/vAKNekD/aRBYxblgHaCJXylVrhw6dYahE9ewNz6FjwZ24Oa2dXIbszLhm8dh3RRo2x/6fAy+/s4F6xBN/EqpcuOPI4kMnbiG5LRMpozoTLcmYbmNaUkwbzjs+B6ufAJ6PVdup2sWRRO/Uqpc+H13PPdOjSLQz4c5Y7pxae3KuY1Jx2D6XXBkI9z8HkSOcC7QUkATv1KqzPt202EemRVDRLUgpo7oTES1irmNx3fAtDshOQ4GzIQW1zsXaCmhiV8pVaZ9+dsenl+8mQ71qjJhaCeqVXIZs9+3GmYOgAq+MGwJ1L3MuUBLEU38SqkyyRjD29//wbhVu7j60hp8NLAjQf4ui6dsWQTz74UqEVZJ5VBdozuHJn6lVJmTkZXNMwtimRt9gIGd6/GvW1vj63pj1m+fwHfPQEQnGDgLKoUVvjEvpIlfKVWmpKRn8uD0daz6I45Hezfjsaub5d6YlZ0N3/8TVn8Cl94Cd3wBfkHOBlwKaeJXSpUZ8UlpjJgSReyBU7x2exsGdamf25iRCgtHW0M8XcbAda9BBZ/CN+bFNPErpcqE/SdSGDJxDYdOneGzey7j2la1chtTTsCsQbDvN7j2Vej2oNfO0S8OTfxKqVJv08HTDJu0loysbKaP6kJkw9DcxpN7rJLKp/ZC30nQ+g7H4iwrNPErpUq1X3YcZ8y0aCoH+jJrdDea1gjJbTy0Hqb3g6x0GLIIGlzuXKBliCZ+pVSptSjmIGPnbqBJeDCTh3emVpXA3Mbt38PcYVAxzJqjH97CsTjLGk38SqlSafzPu3nlm610aRTK50MiqRLkl9sYPRmWPA61WsOgORBSq9DtqL/SxK+UKlWysw2vLd3K+F/+5MY2tXi3X3sC/ezZOcbAqtfgp7eg6dVw1xQICHY24DJIE79SqtRIz8xm7NwNLN5wiKHdGvD8La3wqWDPzslMh68fhQ0zoMNgq9iaj9+5N6gKpIlfKVUqJKZmcP+0dfyy8zh/v74F9/+tSe6NWakJMGcw7P4Rej4Df/u7Tte8CJr4lVKOO5aYyrCJa/njaCJv39WOvpdF5DYmHLJKKsdtg1s/gQ53OxdoOaGJXynlqN1xSQydtIbjiemMHxrJVS1q5DYe22rN0U89ZX2J27S3Y3GWJ25L/CISCPwEBNj7mWeMeUFEJgN/A07bXYcZY2LcFYdSqvRav+8kI6dEIcCs0V1pV69qbuOfP8Osu61aO8OXQe22ToVZ7rjzij8N6GWMSRIRP+AXEVlmtz1pjJnnxn0rpUq5VduO8cD0dVQP8WfqiC40ql4ptzF2Hnx1P4Q2hrvnQdV6zgVaDrkt8RtjDJBkP/Wzf4y79qeUKjvmRu3nqQWxXFo7hEnDOhMeEmA1GAP/ex9WvAgNroAB0yCompOhlksViu5y4UTER0RigGPAcmPM73bTqyKyUUTeE5GAQt47WkSiRCQqLi7OnWEqpTzEGMO4VTt5ct5GLm8SxqzR3XKTfnYWLB1rJf3Wd8LgBZr03cStid8Yk2WMaQ9EAJ1FpDXwNHAJ0AkIBf5RyHs/N8ZEGmMiw8PD3RmmUsoDsrINLyzezL+/+4Pb2tdhwtBOBAfYgw7pKTB7MKwdD90fhTvGg2+B14SqBLg18ecwxpwCVgHXG2MOG0saMAno7IkYlFLOSUnPZMy0aKb+tpf7ejTm3X7t8fe100/ycZhyC2xfBje+Dde8DBU8kpq8ljtn9YQDGcaYUyISBFwDvCkitY0xh8W6M+M2YJO7YlBKOe9YYiqjpkSx6eBpXrylJcO6u6x9G78Lpt0JiUeg/zS45CbnAvUi7pzVUxuYIiI+WJ8s5hhjlojID/ZJQYAYYIwbY1BKOWjH0USGTVrLieR0/jM4kmta1sxt3L8WZva3Hg/9Gup1ciZIL+TOWT0bgQ4FvN7LXftUSpUev+48zn3Togn082HOfd1oE1Elt3HbNzBvpFVV8575ENbEuUC9kN65q5QqcfOjD/DUgo00ql6JicM6EVGtYm7jmi9g2d+hTgfrbtxK1Z0L1Etp4ldKlRhjDO+v2MEHK3dweZMwPr3nstw6+tnZsPJF+N8H0OJGuHMC+Fc85/aUe2jiV0qViPTMbJ6av5EF6w/S97IIXru9Te7MnYxU607czQug0yi44S2o4ONswF5ME79S6qKdTslgzLRoftsdzxPXNOehXk1zSyonx8OsQbB/tTVV8/JHtKSywzTxK6Uuyv4TKQybtIZ9J1J4r387bu/gUlI5fhdM7wunD8Jdk6HV7Y7FqXJp4ldKXbCY/acYNWUtGVmGL0d2oWvjsNzGfath5kDr6n7o11C/i3OBqjw08SulLsh3m4/w6Kz1hIcEMGtYZ5rWcFn7dtMCWDgGqkTAPfOsKpuq1NDEr5Q6bxN++ZNXvtlCu4iqjB8aSfXgAqpr1u8GA2ZAxVAnQ1UF0MSvlCq2rGzDv5ZsYfKve7iuVU3e79+BIH97dk5WplVdM3qSVV3z1k/AL9DZgFWBNPErpYolJT2TR2auZ8XWY4y6ohFP33gpPhXs2TlpiTB3GOxcAVc+AVc9q4XWSjFN/EqpIh1LTGXk5Cg2HzrNy7e2Yki3hrmNpw/CjP5wbAvc8iFcNtSxOFXxaOJXSp3T9qOJDLcLrX0xJJLel7oUWjsSC9P7WVf8d8/VxdDLCE38SqlC/W/nccZMiybIz4e5Y7rRuq5LobUdK2DuUAisAiO+hVqtnQtUnRdN/EqpAs2N2s/TC2JpEh7MxOGdqFs1KLcxahJ88wTUbAmD5kLl2s4Fqs6bJn6lVB7GGN5bvp0Pf9jJFU2r88k9Hakc6Fpo7SVrymaza6HvRAgIcTRedf408SulzkrLzOKp+bEsXH+QfpERvHp7G/x8XAutjYHNCyFyBNzwb/DRFFIWnXO+lYj0cnncKF/bHe4KSinleadTMhgyYQ0L1x9k7LXNefPOtrlJPzkepvaxkv41/4Kb3tWkX4YVNdH2bZfH8/O1PVvCsSilHLIvPoXbP/0f6/ed4oMB7XmoV7Pc6prxu2DC1XB4A9w1Bbprdc2yrqhTthTyuKDnSqkyaP2+k4yaEkVmtuHLkZ3pcq5Ca/U6OxeoKjFFJX5TyOOCniulyphvNx3m0Vkx1KwcyKThnWgS7lpobT4svB+q1rPm6GuhtXKjqMTfWEQWY13d5zzGft6o8LcppUozYwwTfvmTV5dupX29qowfEkmYFlrzGkUl/ltdHr+dry3/c6VUGZCZlc3LS7Yw9be93NC6Fu/1b0+gn2uhtScgejK07gu3jtNCa+XQORO/Mea/rs9FxA9oDRw0xhxzZ2BKqZKXnGYVWlu57Rj39WjMP66/hAo5hdZSE6xCa7tWwpVj4ap/aqG1cuqciV9EPgM+MsZsFpEqwG9AFhAqImONMTM9EaRS6uIdTUhlxOS1bD2cwL9ua83grg1yG08fhBn94NhW6PMRdBziXKDK7Yoa6rnSGDPGfjwc2G6MuU1EagHLAE38SpUB244kMGLSWk6dyWDC0E5cdUmN3MbDG62kn5akhda8RFGJP93l8TXAXABjzBHRebxKlQm/7DjO/dOiCfL3Yc59+QutLbeGdwKrwsjvoGYrp8JUHlRU4j8lIjcDB4HuwEgAEfEFgs71RqWU8+as3c8zC2NpWiOYicM6UafAQmutYNAcLbTmRYpK/PcBHwK1gMeMMUfs13sD37gzMKXUhTPG8M732/l41U6ubFadT+7uSEieQmsvwv8+gGbX2YXWgs+5PVW+FDWrZztwfQGvfwd8566glFIXLi0zi7/P28iimEMM6FSPf93WOm+htYX3wZavIHIk3PCW1tzxQkXN6vnwXO3GmEdKNhyl1MU4lZLO6KnRrNlzgieva8EDPZvk1txJjodZA2H/Grj2Fej2kNbc8VJFnerHAJuAOcAhtD6PUqXW3vhkhk9ey4ETZ/hwYAf6tKuT2xi/C6b3hYRD0G8KtLy18A2pcq+oxF8buAvoD2QCs4F5xphTbo5LKXUe1tmF1rKNYfq9XejU0KXEwt7fYNYgu9DaEqjXyblAValwztvyjDHxxpjPjDFXYc3jrwpsEZHBnghOKVW0ZbGHGfj5akICfVlw/+V5k/6m+VYd/YqhMGqFJn0FFHMFLhHpCAzEmsu/DIh2Z1BKqaIZY/ji5928vmwbHepV5Yv8hdZ+ec9aJrH+5TBguhZaU2cV9eXuy8BNwFZgFvC0MSbTE4EppQqXmZXNi19vZtrqfdzUpjbv9GvnUmgtw5qfv24KtLnLKrTmG+BswKpUKeqK/1ngT6Cd/fOaPUNAAGOMaeve8JRS+SWnZfLQjHWs+iOO+/7WmH9cV0ihtR5PWoXWdOaOyqeoxH/BNfdFJBD4CQiw9zPPGPOCvXbvLCAMa8hosDEmvfAtKaVy5BRa23YkkVdvb83dXQootBa3Dfp8DB31qzhVsKJu4Npb0OsiUgFrzL/Adlsa0MsYk2SXc/5FRJYBjwPvGWNm2dU/RwKfXlD0SnmRbUcSGD5pLQlnMhg/NJKrWhRQaC092Sq01qSXc4GqUu+cs3pEpLKIPC0iH4vItWJ5GNgN9DvXe40lyX7qZ/8YoBcwz359CnDbxfwCSnmDn7bH0ffT38g2hjljuuVN+juWw6QbQHxgxLea9FWRihrq+RI4iVWHfxTwDNb4/m3GmJiiNi4iPljDOU2BccAu4JTLF8QHgLqFvHc0MBqgfv36Re1KqXJr1pp9/POrTTSrEcyk4Z2oXcW10NpE+GYs1GoNA2droTVVLEWuuWuMaQMgIuOBw0B9Y0xqcTZujMkC2otIVWAhcElxAzPGfA58DhAZGakLuyuvk51teGf5H4xbtYsezcMZN6iDFlpTJaKoxJ+R88AYkyUiB4qb9F0ZY06JyCqgG1BVRHztq/4IrJLPSikXqRlZPDlvI19vOMTAzvV5+dZWLoXWzsDCMVahtU6j4Po3tdCaOi9F/WtpJyIJ9mMBguznOdM5Kxf2RhEJBzLspB+EdfPXm8AqoC/WzJ6hwKKL/B2UKldOJqcz+sso1u45yVM3XMJ9PRq7FFo7DjMHwoG1cO2r0O1Bna6pzltRs3p8LmLbtYEp9jh/BWCOMWaJiGwBZonIK8B6YMJF7EOpcmXPcavQ2sFTZ/hoYAducS20dnynVWgt8bAWWlMXxW2fD40xG4EOBby+G+jsrv0qVVZF7z3BvVOjMcYwY1QXIv9SaG2gNXNHC62pi6QDg0qVAt9sPMz/zYmhTpVAJg3vTKPqlXIbY+fBV/dD1QbWHP3QC76vUilAE79SjjLG8J+fdvPGsm1ENqjG50MiCa3kn9OYW2itQXfoP00LrakSoYlfKYdkZmXzwuLNTP99Hze3rc3bd+UvtPY4rJuqhdZUidPEr5QDkuxCaz/+Ecf9PZvw5LUt8hVaGwq7ftBCa8otNPEr5WFHTqcyfPJath9N5PU72jCws8ud6acPwPR+cPwPLbSm3EYTv1IetOVQAiMmryUpLZOJwzrxt+bhuY15Cq3NgyZXOReoKtc08SvlIT/+cYwHp68jJNCPOfd1o2Udl/sft38P84ZDYFUY8R3UbOlYnKr808SvlAfM+H0fzy3aRIuaIUwc1olaVQJzG9dOgKVjoVYbGDQHQmo5F6jyCpr4lXKj7GzDW9/9wWf/3UXPFuF8PKgjwQG+OY2w4gX49UMttKY8ShO/Um6SmpHFE3M38M3Gw9zdpT4v9WmFb55Ca/fBlkXQ6V64/g0ttKY8Rv+lKeUGJ5LTGT01iqi9J3n6hksYXVihteteg64P6HRN5VGa+JUqYX8eT2b4pDUcOp3KuEEduamty+IoeQqtTYWWfZwLVHktTfxKlaCoPSe4d2oUIsLMe7twWQPXQmu/wqxBVqG1Yd9ARKRzgSqvpolfqRLy9YZDPDF3A3WrBjF5eCcahGmhNVU6aeJX6iIZY/jsv7t589ttdGpYjc8HR1ItT6G1d2Hly9DgCuj/pRZaU47TxK/URcjIyub5RZuYuWY/t7Srw7/7ts1baG3J/8H6L6FNP7j1Yy20pkoFTfxKXaDE1AwenLGen7bH8eBVTXjimnyF1uYMgd2roMff4apndOaOKjU08St1ATYfOs2D09ex/+QZ3rijDQMKK7R26zjocI9zgSpVAE38Sp0HYwwz1uzjpa+3EFrRn1mju9LJdYnEwxuspJ+RAvfMh8Y9HYtVqcJo4leqmJLSMnlmQSyLNxyiR/Nw3uvXjrBglzH77d/D3GHWl7eDtdCaKr008StVDNuOJPDAtHXsiU9m7LXNeaBn09zxfIC142Hpk1poTZUJmviVOgdjDHOi9vP8os1UDvJj+qiudGsSltshOxtWPA+/fgTNr4c7J2ihNVXqaeJXqhAp6Zk8u3ATC9YfpHvTMN7v34HwEJehHddCa51HW4XWKvg4F7BSxaSJX6kCbD+ayAPT17ErLonHrm7Gw72a4eM6tHN8p7Uu7tHNcN3r0PV+na6pygxN/ErlMy/6AM99tYlKAT5MG9mF7k2r5+2waT4sfgR8/K0lEptd7UygSl0gTfxK2c6kZ/HC4k3MiTpA18ahfDigAzUqu6yUlZEK3z0DUROgXldr4ZQqdZ0LWKkLpIlfKWDnsSQenL6O7ccSebhXUx7t3Sx30RSAE7utqZqHN0D3R6HXc+Dj51i8Sl0MTfzK6y2KOcjTC2IJ9PNhyvDO9GgenrfDlsWw6EGQCjBwNrS43plAlSohmviV10rNyOKlr7cwc80+OjcM5cOBHfIugp6ZDsufh98/hbqRcNckqFq/8A0qVUZo4lde6c/jyTwwfR1bDydwf88mPHFN87xDOyf3wrzhcDDaWhrx6pfA19+5gJUqQZr4lddZsvEQT82PxddHmDSsE1ddUiNvh21L4asxYID+0+DSWxyJUyl30cSvvEZqRhavfrOVL1fvpWP9qnw8qCN1qgbldsjKgJUvWXfh1m4Hd03RlbJUuaSJX3mFvfHJPDhjHZsOJjC6R2OevK4Ffq5DO6cPwLwRsP936DQKrn0V/AIL36BSZZgmflXuLYs9zN/nbaRCBWH8kEiublkzb4cdy2HBaOuKv+9EaH2nM4Eq5SGa+FW5lZ6ZzWtLtzL51z20q1eVjwd2oF5oxdwOWZmw6lVrTdyara2hnepNnQtYKQ/RxK/Kpf0nUnhoxjo2HDjNiO6NeOqGS/D3dRnaSTgM80fC3v9Bx6Fww5vgF1T4BpUqRzTxq3Ln+81HGDt3Awb47J6OXN+6dt4Ou1bB/FHWKlm3fw7t+jsSp1JOqVB0lwsjIvVEZJWIbBGRzSLyqP36iyJyUERi7J8b3RWD8i4ZWdm8smQLo7+MpkFYJb55+Mq8ST87C1a9Dl/eDpXCYfSPmvSVV3LnFX8m8IQxZp2IhADRIrLcbnvPGPO2G/etvMzBU2d4aMY61u87xZBuDfjnTZcS4OtSGz/pmHWV/+d/of3dcOO/wb+ScwEr5SC3JX5jzGHgsP04UUS2AlrKUJW4H7Yd5fE5G8jMMnw8qAM3t62Tt8OfP1vj+akJcOs46HCPM4EqVUq4bajHlYg0BDoAv9svPSQiG0VkoohUK+Q9o0UkSkSi4uLiPBGmKmMysrJ5Y9k2RkyOok6VIL5++Iq8ST87G376N0ztAwGV4d4fNOkrBYgxxr07EAkG/gu8aoxZICI1geNYN8T/C6htjBlxrm1ERkaaqKgot8apypYjp1N5eOY61u45yaAu9Xn+5pYE+rkM7SQft+bm71oJbe6Cm9/XtXCV1xGRaGNMZP7X3TqrR0T8gPnAdGPMAgBjzFGX9i+AJe6MQZU//90ex//NjiE1I4sPBrTn1vb5RhD3/mbdhZsSbyX8y4bpsohKuXBb4hcRASYAW40x77q8Xtse/we4HdjkrhhU+ZKZlc37K3bw8aqdXFIrhHF3d6RJuMtVfHY2/PohrHwZqjWAUSugdlvnAlaqlHLnFX93YDAQKyIx9mvPAANFpD3WUM8e4D43xqDKiWMJqTw8cz2//3mC/pH1eLFPK4L8XYZ2Uk7AwjGw4ztoeRv0+QgCKzsWr1KlmTtn9fwCFPT5eqm79qnKp//tPM6js9aTnJbFO3e1487LIvJ22L/Wqp2fdBRufNsqsqZDO0oVSu/cVaVWVrbhw5U7+PCHHTQND2bmvR1pVjMkt4MxsPoTa5WsynVhxHdQt6NzAStVRmjiV6VSXGIaj81ez/92xnNHx7q8cltrKvq7/HM9c8paB3fbErjkZmt+flBVp8JVqkzRxK9Knd92xfPIrPUknMngrb5t6RdZL2+Hg+tg7jBIOAjXvQ5d79ehHaXOgyZ+VWpkZxvGrdrJeyu207B6Jb4c2ZlLarl8QWsMrB0P3z0DlWrA8G+hXifnAlaqjNLEr0qF+KQ0Hpsdw887jnNr+zq8dnsbKgW4/PNMTYDFD8OWr6DZdXD7Z1Ax1LF4lSrLNPErx6358wQPz1zHyZQMXru9DQM710Nch24Ob4S5Q+HkXrj6Jbj8EajgkWojSpVLmviVY7KzDZ/9tIt3vt9OvWpBTHygE63qVMntYAxET4Zl/7Cu7od9Aw26ORavUuWFJn7liJPJ6Tw+J4ZVf8RxU9vavHFHG0IC/XI7pCXBkscgdi406QV3fAGVqjsWr1LliSZ+5XHRe0/w0Iz1xCel869bW3FP1wZ5h3aObrGGduJ3wlXPwpVP6NCOUiVIE7/yGGMM43/+kze/3UadqkHMv/9y2kRUydtp/TT4ZqxVbmHIImjUw5lglSrHNPErjzidksETczewYutRrm9Vi7fuaktl16Gd9BRYOhZipkPDK+HOCRBS07mAlSrHNPErt4vZf4oHp6/jWGIqL9zSkmGXN8w7tBP3B8wZCnHb4G//sH4q+BS+QaXURdHEr9zGGMOk/+3h9WVbqRESyNwxl9O+XtW8nTbMhiX/B35BMHiB9UWuUsqtNPErtzh9JoN/zNvIt5uPcPWlNXnnrnZUqegytJNxxpqmuW4K1L8c+k6EyrWdC1gpL6KJX5W42AOneWBGNIdPpfLsTZcy8opGeYd2ju+0au0cjYUrHoer/gk++k9RKU/RvzZVYowxfLl6L68s2Ur1YH9m39eNyxpUy9tp0wJY/IiV6AfNhebXOhOsUl5ME78qEYmpGTw1P5ZvYg9zVYtw3u3XnmqV/HM7ZKZZxdXWjoeIznDXJKgSUfgGlVJuo4lfXbTNh07z4PR17D95hqduuITRVzamQgWXoZ0Tf1o3ZB3eAN0egqtfBB+/QrenlHIvTfzqghljmLFmHy99vYXQiv7MGt2VTg3zVczc+jV89aC1COeAmXDJjY7EqpTKpYlfXZCktEyeWRDL4g2H6NE8nPf6tSMsOCC3Q2a6tSTi759CnY5w12So1sCxeJVSuTTxq/O27UgCD0xbx574ZMZe25wHejbNO7Rzap81a+dgNHQZA9f8C3z9C92eUsqzynXi//THXXy/5QjVgwOoHhxAeLA/1UMCzj6vbj8PCfDNO91QFcgYw5yo/Ty/aDOVg/yYPqor3ZqE5e30x7ew8D4w2dBvKrS81ZlglVKFKteJPzjQl4r+PuyLT2Hd3pOcSEnHmL/28/etQHjOiSDnpBCS+zgs2N9uD6BqRT+vPEmkpGfy7MJNLFh/kO5Nw3i/fwfCQ1yGdrIyYOXL8OuHUKsN3DUFwpo4F7BSqlDlOvEP7tqAwV1zx5Uzs7I5kZLO8cR0jielufykczwxjbikNA6dTmXjwdOcSE4nK/uvZwnfCkKY6wnCPkmE53tePTiAahX98alQ9k8S248m8sD0deyKS+Kxq5vxcK9meX+v0wdh3gjYvxoiR1gLoPsFOhewUuqcynXiz8/XpwI1QgKpEVJ0UsrONpw6k2GdGOyTwvGk9LPPc04Y248mcjwpjYysv54kKgiEVrI+SYSHuAwvBQcQ5vI4PCSA0Er++PmUvprz86IP8NxXm6gU4MO0kV3o3jTfYig7VsDC0dY8/TsnQJu+zgSqlCo2r0r856NCBSG0kj+hlfxpXjPknH2NMSScybRPDmkuJ4f03E8USWn8eTyZ40lppGZkF7idahX9XD41uJwYChh6CvB1b/XKM+lZvLB4E3OiDtC1cSgfDuhAjcouJ8ysTPjxdfj5bajRCvpNgerN3BqTUqpkaOIvASJClYp+VKnoR9Mawefsa4whOT3L5VNDGnH2UJPr0NPGA6c4nphGcnpWgdupHOh79ovqnO8nwvJ9aZ0z/BTkf34niZ3Hknhw+jq2H0vk4V5NebR3M3xdP40kHoF5I2HvL9BhMNzwFvhXPK99KKWco4nfw0SE4ABfggN8aVi9UpH9z6Rn2SeHNOILGGqKS0pj65EEjiemkZCaWeA2Kvn7uMxm8s/zqSI83/OVW4/y9IJYAv18mDK8Mz2ah+fd2O4fYf4oSE+G2z6D9gNL4KgopTypfCf+bUvhyEYICoWKoRBUzf6v/dw/GEr5DJ0gfx/qhVakXmjRV9RpmVm5J4ekNI4nprsMP1mfKnbHJbN2z0lOJKcXup3ODUP5cGAHalVxGdrJzoKf/g0/vgHVm8PQJVDjkpL4FZVSHla+E//uVbDm88LbffxdTgqhULFavudheU8UQaEQVLXUrg4V4OtDnapB1KkaVGTfzKxsTiSn535pbX+KqBTgy4BO9fIO7SQdgwX3Wlf7bQfAze+Cf9GfVpRSpZOYgia2lzKRkZEmKirqwt6clQlnTsKZE5ByIt9/4+3HJ//all3wsAkIBFaxTgQVw4o4cbj0KavTG/f8Yo3np56CG/9tjemX8k9JSimLiEQbYyLzv16+r/jBqvseHG79FJcxkJbocnIo7MRxAhIPw7Et1uOM5MK36Vex+J8qcvoEVnEuyWZnwy/vwqpXIbQx3DMfarV2JhalVIkq/4n/QohAYGXrp1rD4r8vM63gk0NK/F8/VRyJzf20QSGfusSn8JNCoSeOahdf8jg53pqbv3MFtL4TbvkAAs49pVUpVXZo4i9JvgHWurHns3Zsdhakni74hJH/vyf3wKF11uOstMK3GVA594vsvwxH5f+S2z5x+FW0Tnj7Vlt34SbHwU3vWnfi6tCOUuWKJn6nVbCv6iuGFt03hzGQkVL0p4qc/8bvtIar0k4Xvk2fACuGpGNQtR6MXA512l/0r6eUKn008ZdFItasGv9KVpIurqyMwk8OOd9nBFaFv/3d+n5BKVUuuS3xi0g9YCpQE2sQ+3NjzAciEgrMBhoCe4B+xpiT7opDufDxg+Aa1o9Symu5sypYJvCEMaYl0BV4UERaAk8BK40xzYCV9nOllFIe4rbEb4w5bIxZZz9OBLYCdYFbgSl2tynAbe6KQSml1F95pA6wiDQEOgC/AzWNMYftpiNYQ0EFvWe0iESJSFRcXJwnwlRKKa/g9sQvIsHAfOAxY0yCa5uxbhsucBK7MeZzY0ykMSYyPPw8br5SSil1Tm5N/CLih5X0pxtjFtgvHxWR2nZ7beCYO2NQSimVl9sSv1gL004Athpj3nVpWgwMtR8PBRa5KwallFJ/5c55/N2BwUCsiMTYrz0DvAHMEZGRwF6gnxtjUEoplY/bEr8x5hegsHv9e7trv0oppc6tTJRlFpE4rE8HF6I6cLwEwykpGtf50bjOj8Z1fkprXHBxsTUwxvxldkyZSPwXQ0SiCqpH7TSN6/xoXOdH4zo/pTUucE9sHpnHr5RSqvTQxK+UUl7GGxL/ORbddZTGdX40rvOjcZ2f0hoXuCG2cj/Gr5RSKi9vuOJXSinlQhO/Ukp5mXKR+EVkoogcE5FNhbSLiHwoIjtFZKOIdCwlcfUUkdMiEmP/PO+huOqJyCoR2SIim0Xk0QL6ePyYFTMujx8zEQkUkTUissGO66UC+gSIyGz7eP1uV6QtDXENE5E4l+M1yt1xuezbR0TWi8iSAto8fryKGZcjx0tE9ohIrL3PqALaS/bv0RhT5n+AHkBHYFMh7TcCy7DuJO4K/F5K4uoJLHHgeNUGOtqPQ4DtQEunj1kx4/L4MbOPQbD92A+rvHjXfH0eAD6zHw8AZpeSuIYBH3v635i978eBGQX9/3LieBUzLkeOF9ZqhNXP0V6if4/l4orfGPMTcOIcXW4FphrLaqBqToVQh+NyhCl8kRxXHj9mxYzL4+xjkGQ/9bN/8s+KcF1gaB7Q2y5U6HRcjhCRCOAmYHwhXTx+vIoZV2lVon+P5SLxF0NdYL/L8wOUgoRi62Z/VF8mIq08vXPJu0iOK0eP2TniAgeOmT08EINVRny5MabQ42WMyQROA2GlIC6AO+3hgXlirYXtCe8DfweyC2l35HgVIy5w5ngZ4HsRiRaR0QW0l+jfo7ck/tJqHVYtjXbAR8BXnty5nGORHCcVEZcjx8wYk2WMaQ9EAJ1FpLUn9luUYsT1NdDQGNMWWE7uVbbbiMjNwDFjTLS793U+ihmXx4+X7QpjTEfgBqz1yXu4c2fekvgPAq5n7gj7NUcZYxJyPqobY5YCfiJS3RP7loIXyXHlyDErKi4nj5m9z1PAKuD6fE1nj5eI+AJVgHin4zLGxBtj0uyn44HLPBBOd6CPiOwBZgG9RGRavj5OHK8i43LoeGGMOWj/9xiwEOicr0uJ/j16S+JfDAyxvxnvCpw2uev+OkZEauWMa4pIZ6z/H25PFvY+C1okx5XHj1lx4nLimIlIuIhUtR8HAdcA2/J1c11gqC/wg7G/lXMyrnzjwH2wvjdxK2PM08aYCGNMQ6wvbn8wxtyTr5vHj1dx4nLieIlIJREJyXkMXAvknwlYon+P7lyIxWNEZCbWbI/qInIAeAHriy6MMZ8BS7G+Fd8JpADDS0lcfYH7RSQTOAMMcPc/flthi+TUd4nNiWNWnLicOGa1gSki4oN1opljjFkiIi8DUcaYxVgnrC9FZCfWF/oD3BxTceN6RET6AJl2XMM8EFeBSsHxKk5cThyvmsBC+3rGF5hhjPlWRMaAe/4etWSDUkp5GW8Z6lFKKWXTxK+UUl5GE79SSnkZTfxKKeVlNPErpZSX0cSv3E5Esuyqg5tEZK6IVDxH32Ei8rEn43PZ98sicnURfSaLSN9C2saKyDb7d10rIkPcE+nZ/e25kJvXRKSOiMyzH7cXkRtLPjpVmmniV55wxhjT3hjTGkgHxjgdUEGMMc8bY1ZcyHvtOdfXAJ3tEgq9sSopljrGmEPGmJyTV3us+eHKi2jiV572M9BUREJF5Cu7GNZqEWnr2klEQkTkT7uEAyJSOee5iPwoIm+KVYt+u4hcafcJFJFJYtU1Xy8iV9mvD7P3tdy+Sn5IRB63+6wWkVC739mreRF53r5q3yQin+fcLXwOzwD359QWsktLTLG31dveV6xYazQE2K/vEZHX7U8IUSLSUUS+E5FdOTfviLX+wE8i8o2I/CEin4nIX/5uReQe+3jEiMh/xCre1sk+voH23aGbRaS1iDS0fy9/4GWgv/2+/iKyQ0TC7W1WEKv+e/gF/Z9WpZYmfuUxYtVkuQGIBV4C1tvFsJ4Bprr2tcsy/4hVQhesOzsXGGMy7Oe+xpjOwGNYd0QDPGi91bQBBmLd1Rpot7UG7gA6Aa8CKcaYDsBvQEFDMh8bYzrZn1KCgJvP8XtVBkKMMbsLaAsEJgP97bh8gftduuyzPyH8bPfri1Vv3XVRlc7Aw0BLoIn9e7ju41KgP9Dd3lYWcLcxZi3Wrf6vAG8B04wxZ0sBGGPSgeexauG3N8bMBqYBd9tdrgY2GGPiCvvdVdmkiV95QpBdgiEK2Id1u/4VwJcAxpgfgDA7gboaT+6t6cOBSS5tOQXcooGG9uMrsBIXxphtwF6gud22yhiTaCex01hVGME6CeW839VVYq0MFQv0Ai60/HML4E9jzHb7+RSsBXpyLHaJ43eXGNPErsMDrDHG7DbGZAEz7d/TVW+sYmJr7ePcG2hst72MNQQViZX8izKR3BPhCPIec1VOlItaParUO2NfiZ5V9MgJGGP+Zw9L9AR8XK9WgZwKilkU799xmsvjbJfn2fnfb1+lfwJEGmP2i8iLQCCFMMYkiEiSiDQu6Kq/mHG5xpQ/rvx1VfI/F2CKMebpArYfBgRj1YgKBJLPFYz9+x4VkV5YnzTuPld/VTbpFb9yys/YScVO7McLWRNgKtYyecW58nTdZnOs4m5/XEBsOUn+uFhrAxQ4iyef14FxOZ9aRCTYntXzB9BQRJra/QYD/z3PeDqLSCN7bL8/8Eu+9pVAXxGpYe87VEQa2G3/AZ4DpgNvFrDtRKxlLl2Nx/rkNNf+lKHKGU38yikvApeJyEbgDXJL9OY3HaiGNcRRlE+ACvbwzGxgmEtt9WKza9t/gVUa9ztgbTHe9ilWPfy1IrIJ6ySUbYxJxRqmmmvHlQ18dp4hrQU+xioR/CdWvXbXeLcAz2Kt4LQRawGR2vaJJ8MYMwPrGHeyr+RdrQJa5ny5a7+2GOtTgg7zlFNanVOVavYsm1uNMYOdjsUJ9qehscaYQr9cdsM+I4H3jDFXemqfyrN0jF+VWiLyEdYsIJ1n7iEi8hTWrCMd2y/H9IpfKaW8jI7xK6WUl9HEr5RSXkYTv1JKeRlN/Eop5WU08SullJf5f4v8maAulvglAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1,6),train_rmse_errors,label='TRAIN')\n",
    "plt.plot(range(1,6),test_rmse_errors,label='TEST')\n",
    "plt.xlabel(\"Polynomial Complexity\")\n",
    "plt.ylabel(\"RMSE\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "8PrGP3n0FU_0"
   },
   "outputs": [],
   "source": [
    "#after the plot, observe the best degree , choose the degree and train a new model accordingly "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "11VS-W9nFU_1"
   },
   "outputs": [],
   "source": [
    "polynomial_converter_lasso = PolynomialFeatures(degree=2,include_bias=False)\n",
    "poly_features_train = polynomial_converter.fit_transform(scaled_X_train)\n",
    "poly_features_test = polynomial_converter.fit_transform(scaled_X_test)\n",
    "lasso_cv_model =  LassoCV(eps=0.1,n_alphas=100,cv=5)\n",
    "lasso_cv_model.fit(poly_features_train,y_train)\n",
    "#train_pred = lasso_cv_model.predict(X_train)\n",
    "test_pred = lasso_cv_model.predict(poly_features_test)\n",
    "#train_RMSE = np.sqrt(mean_squared_error(y_train,train_pred))\n",
    "MAE = mean_absolute_error(y_test,test_pred)\n",
    "MSE = mean_squared_error(y_test,test_pred)\n",
    "RMSE_lasso = np.sqrt(mean_squared_error(y_test,test_pred))  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "J4oogbpwFU_1",
    "outputId": "b2635f38-329b-4879-818a-c9d6af473320"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "14.88330586144493"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MAE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PJP7WZY2FU_1",
    "outputId": "4b2e098d-1d9b-42a9-902e-7cfbc22fe24a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "420.3593307154854"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "J6QJnYZIFU_2",
    "outputId": "edfdd106-e685-4654-a058-779424f9f921"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20.502666429405846"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RMSE_lasso"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "O6puZ_fzRKbm",
    "outputId": "40daee86-3e33-4768-f0d7-d50b3319f1e9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "107.87544081225829"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imputed_df['NO2(GT)'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "XGgIBBgDFU_2",
    "outputId": "e1e8bb91-83b1-41b9-88a2-af0958e8d4df"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.4212243161164815"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lasso_cv_model.alpha_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MIf_ey0TFU_2",
    "outputId": "59b85e2f-59b0-490f-cf85-cad163057574"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  0.        ,  12.84042338,  -6.86276759,  -0.        ,\n",
       "        13.77933184,  -0.        ,  -0.        , -12.9556961 ,\n",
       "         0.        ,   0.        ,   0.        ,   4.22316175,\n",
       "         0.        ,   0.        ,  -0.        ,  -0.        ,\n",
       "         0.        ,  -0.        ,  -0.        ,  -0.        ,\n",
       "        -0.        ,  -0.        ,  -0.        ,   0.        ,\n",
       "         0.        ,  -0.        ,   0.        ,   0.        ,\n",
       "        -0.        ,   0.        ,  -0.        ,  -0.        ,\n",
       "         0.        ,  -0.        ,  -0.        ,  -0.        ,\n",
       "         0.        ,   0.        ,  -0.        ,   0.        ,\n",
       "         0.        ,  -0.        ,   0.76348194,   0.        ,\n",
       "         0.        ,   0.        ,   0.12438233,   0.        ,\n",
       "        -0.        ,  -0.06085445,   0.        ,   0.        ,\n",
       "         0.        ,  -0.        ,  -0.        ,   0.        ,\n",
       "        -0.        ,   0.        ,  -0.        ,   0.        ,\n",
       "        -0.        ,  -0.        ,   0.        ,   0.        ,\n",
       "         0.        ,  -0.        ,  -0.        ,  -0.        ,\n",
       "        -0.        ,   0.        ,   0.        ,   0.        ,\n",
       "         0.        ,   0.        ,   2.71609823,  -0.        ,\n",
       "         0.        ,   0.        ,  -0.        ,  -0.        ,\n",
       "        -0.        ,   0.        ,   0.        ,   0.        ,\n",
       "        -0.        ,  -0.        ,   0.        ,   0.        ,\n",
       "         0.        ,   0.        ,   0.        ,   0.        ,\n",
       "        -0.        ,   0.        ,  -0.        ,  -0.        ,\n",
       "         0.        ,   0.        ,   0.        ,   0.        ,\n",
       "        -0.        ,  -0.        ,   0.        ,   0.        ,\n",
       "        -0.        ,   0.        ,   0.        ,   0.        ,\n",
       "         0.        ,   6.70043591,   0.        ,   0.        ,\n",
       "         0.        ,  -0.        ,   0.        ,   0.        ,\n",
       "         0.        ,   0.        ,   0.        ])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lasso_cv_model.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M8SOg6KEuQYq"
   },
   "source": [
    "### 9a- Evaluate different degrees of Ridge CV polynomial regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "IqWaySbcuQYq",
    "outputId": "f0aacdf2-b68e-435a-d129-1412428eff1d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "currently in iteration:  1\n",
      "currently in iteration:  2\n",
      "currently in iteration:  3\n",
      "currently in iteration:  4\n",
      "currently in iteration:  5\n"
     ]
    }
   ],
   "source": [
    "train_rmse_errors = []\n",
    "test_rmse_errors = []\n",
    "for d in range(1,6):\n",
    "    print(\"currently in iteration: \", d)\n",
    "    polynomial_converter = PolynomialFeatures(degree=d,include_bias=False)\n",
    "    poly_features_train = polynomial_converter.fit_transform(scaled_X_train)\n",
    "    poly_features_test = polynomial_converter.fit_transform(scaled_X_test)\n",
    "    model =  RidgeCV(alphas=(0.1, 1.0, 10.0),scoring='neg_mean_absolute_error')    \n",
    "    model.fit(poly_features_train,y_train)\n",
    "    train_pred = model.predict(poly_features_train)\n",
    "    test_pred = model.predict(poly_features_test)\n",
    "    train_RMSE = np.sqrt(mean_squared_error(y_train,train_pred))\n",
    "    test_RMSE = np.sqrt(mean_squared_error(y_test,test_pred))\n",
    "    train_rmse_errors.append(train_RMSE)\n",
    "    test_rmse_errors.append(test_RMSE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8Bj_3j9RuQYr"
   },
   "source": [
    "### 9b- Choose the best degree and print the errors, model coefficients and the Ridge parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 296
    },
    "id": "4q5E5NnbuQYr",
    "outputId": "4e60dc08-f158-40a7-f4d0-6a7834330770"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x12aa2ce50>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3XElEQVR4nO3deXwU9f3H8dcnBwQIZ8KZEMIlghwhRi5FEBTvCy9UELRWbevRu57V2lpbW9ufV6vWqiiIN0rxREBFOUMIckuABBJCCOEI4Uyyn98fM4EQNySQ7M4m+TwfDx7Z3ZnZ+exo9p3vd+b7HVFVjDHGmIrCvC7AGGNMaLKAMMYY45cFhDHGGL8sIIwxxvhlAWGMMcavCK8LqE2xsbGamJjodRnGGFNnLF26dIeqtvW3rF4FRGJiIqmpqV6XYYwxdYaIZFW2zLqYjDHG+GUBYYwxxi8LCGOMMX7Vq3MQ/hQXF5Odnc3Bgwe9LsUTUVFRxMfHExkZ6XUpxpg6pt4HRHZ2Ns2bNycxMRER8bqcoFJVCgoKyM7OpmvXrl6XY4ypY+p9F9PBgweJiYlpcOEAICLExMQ02NaTMaZm6n1AAA0yHMo05M9ujKmZBhEQxhhTb22YA4tegNLiWn9rC4gAKigoICkpiaSkJDp06EBcXNyR5yJCUlISffv25dJLL2X37t3HbJuUlMS4ceOOeW3SpEm8++67AIwcOZKUlJQjy1JTUxk5cmSgP5IxJpSUFsMnv3MCIgD39rGACKCYmBjS09NJT0/njjvu4Be/+MWR582aNSM9PZ2VK1fSpk0bnnvuuSPbrVmzhtLSUubNm8e+ffsqff/t27fzySefBOOjGGNC0dJXYcf3MOaPENGo1t/eAiIEDB06lJycnCPPp02bxoQJExgzZgwffvhhpdv95je/4bHHHgtGicaYUHNgF8z9MyQOh14XBWQX9f4y1/L+8L9VrN5aWKvv2adTCx6+9LST3r60tJTZs2fzox/96Mhrb731FrNmzWLt2rU888wz3HDDDX63HTp0KNOnT2fu3Lk0b978pGswxtRBX//dCYnzH4MAXYxiLQiPHDhw4Mi5iby8PM477zzAOZcQGxtLQkICo0ePZtmyZezcubPS93nwwQf505/+FKyyjTGhoGCDc94h6UboOCBgu2lQLYia/KVf25o0aUJ6ejr79+/n/PPP57nnnuPuu+9m2rRprF27lrJpywsLC3nvvff48Y9/7Pd9Ro0axYMPPsjChQuDWL0xxlNfPAzhjWD0QwHdjbUgPNa0aVOefvppnnzySQ4fPszbb7/NihUryMzMJDMzkw8//JBp06Yd9z0efPBBnnjiiSBVbIzxVOY3sOZ/cNYvoHmHgO7KAiIEDBw4kP79+/P4448TFxdHp06djiw7++yzWb16Nbm5uZVuf9FFF9G2rd/7fRhj6hOfDz67H1rEwdCfBXx3ogG4dtYrKSkpWvGGQWvWrKF3794eVRQa7BgYU0+kvwEf/ATG/gf6X1srbykiS1U1xd8ya0EYY0xdcHgfzH4UOiVD36uDsssGdZLaGGPqrG+fhr25cM2rEBacv+2tBWGMMaFuTw58+xScdiUkDAnabi0gjDEm1M35I2gpnPtIUHcbsIAQkc4iMldEVovIKhG5x339ERHJEZF095/fMeIicoGIrBORDBG5N1B1GmNMSMtJg+XTYMhPoXViUHcdyHMQJcCvVDVNRJoDS0Vklrvsn6r698o2FJFw4DngPCAbWCIiM1R1dQDrNcaY0KLqXNbaNBaG/yrouw9YQKhqLpDrPt4rImuAuGpuPgjIUNWNACLyJnA5UKcCoqCggNGjRwOwbds2wsPDj4xXWL58OQMGHB0iP27cOO69915mzpzJQw89hM/no7i4mHvuuYcdO3bwzjvvALBixQr69esHwC233MLdd98d5E9ljAmaNTNg8wK45J8Q1SLouw/KVUwikggMBBYBZwJ3ishNQCpOK2NXhU3igC3lnmcDgyt579uA2wASEhJqt/AaKpvuG+CRRx4hOjqaX//61wBER0cfWVamuLiY2267jcWLFxMfH8+hQ4fIzMykV69ePPDAA5VuZ4yph0oOwazfQ9veMPAmT0oI+ElqEYkG3gN+rqqFwL+B7kASTgvjyZq8v6q+qKopqppS10cT7927l5KSEmJiYgBo3LgxvXr18rgqY4wnFr0AuzKd2VrDvRmRENC9ikgkTjhMVdX3AVQ1r9zy/wAz/WyaA3Qu9zzefa1mPrkXtq2o8dsco0M/uPAvJ7xZ2WyuZe677z6uu+46LrvsMrp06cLo0aO55JJLuP766wkL0jXPxpgQsW8HfP036DkGeoz2rIyABYSICPBfYI2q/qPc6x3d8xMAVwIr/Wy+BOgpIl1xgmEc4P+mCHVU2WyuFb300kusWLGCL774gr///e/MmjWLV199Nej1GWM89OXjzsjpMd5O5R/IFsSZwARghYiku6/dD1wvIkmAApnA7QAi0gl4SVUvUtUSEbkT+AwIB15W1VU1rugk/tL3Qr9+/ejXrx8TJkyga9euFhDGNCTb10LqK5ByC7T1tos5kFcxfQP4u83Rx5WsvxW4qNzzjytbt74qKioiNTWVkSNHApCenk6XLl28LcoYE1yfPwCNomHkfV5XYnMxeaXiOYgLLriABx54gCeeeILbb7+dJk2a0KxZM2s9GNOQrP8CMr5wupaaxXhdjQVEsDzyyCPHPC8tLfW73scfH7/RVFRUVFslGWNCSWmJ03po3RUG3eZ1NYAFhDHGhIa0yZC/Fq59HSIae10NYJP1GWOM9w7ugbl/hi5nQu9Lva7miAYREPXprnknqiF/dmPqjHlPwv4CZ1Cc+Lu2xxv1PiCioqIoKChokF+UqkpBQQFRUVFel2KMqczOTbDw3zDgeug00OtqjlHvz0HEx8eTnZ1Nfn6+16V4Iioqivj4eK/LMMZU5ouHISwCRj/kdSU/UO8DIjIykq5du3pdhjHG/FDWAlj9oTPmoUUnr6v5gXrfxWSMMSHJ54PP7oPmnWDYXV5X41e9b0EYY0xIWvEObF0GVzwPjZp5XY1f1oIwxphgO7wfZv8BOiZB/+u8rqZS1oIwxphgW/AsFObAVS9BCE/nH7qVGWNMfVSYC9/8E3pfBl2GeV3NcVlAGGNMMM35I/hK4Lw/eF1JlSwgjDEmWLamQ/obMPh2aNPN62qqZAFhjDHBoAqfPQBN28DwX3tdTbVYQBhjTDCs/QiyvnEGxTVp5XU11WIBYYwxgVZyGGY9BLG94PSbva6m2uwyV2OMCbQl/4GdG+HGdyG87nztBqwFISKdRWSuiKwWkVUico/7+t9EZK2IfCci00WkVSXbZ4rIChFJF5HUQNVpjDEBtX8nfPVX6D4aep7ndTUnJJBdTCXAr1S1DzAE+JmI9AFmAX1VtT/wPXC8O3Ofo6pJqpoSwDqNMSZwvnwcDu117vVQxwQsIFQ1V1XT3Md7gTVAnKp+rqol7moLAZuL2hhTP+V/D0v+C6dPgna9va7mhAXlJLWIJAIDgUUVFt0CfFLJZgp8LiJLRaTSO3iLyG0ikioiqQ31ng/GmBD1+YPORHwj7/e6kpMS8IAQkWjgPeDnqlpY7vUHcLqhplay6VmqmgxciNM9dba/lVT1RVVNUdWUtm3b1nL1xhhzkjbMgfWfwfBfQXTd/G4KaECISCROOExV1ffLvT4JuAS4USu5F6iq5rg/twPTgUGBrNUYY2qNrxQ+exBadYEhP/G6mpMWyKuYBPgvsEZV/1Hu9QuA3wKXqer+SrZtJiLNyx4DY4CVgarVGGNq1bLXYfsqOO9RiGjsdTUnLZAtiDOBCcAo91LVdBG5CHgWaA7Mcl97HkBEOonIx+627YFvRGQ5sBj4SFU/DWCtxhhTOw4Wwpw/QcJQ6HO519XUSMBGbKjqN4D4WfSxn9dQ1a3ARe7jjcCAQNVmjDEB880/YF8+3PAWiL+vwLrDptowxpjasisLFvzLuUtc3OleV1NjFhDGGFNbvngEJAxG/97rSmqFBYQxxtSGLYth1fsw7C5oWT/G/1pAGGNMTfl88Ol9EN0BzrzH62pqTd2ZVtAYY0LVqvchJxUu/xc0jva6mlpjLQhjjKmJ4gMw62Ho0B8GXO91NbXKWhDGGFMTC56Fwmy48nkIq19/c9evT2OMMcG0Nw/m/RNOvQS6Dve6mlpnAWGMMSdrzh+h9LAzpUY9ZAFhjDEnY9sKWDYFBt8OMd29riYgLCCMMeZEqcJn90OT1nD2r72uJmAsIIwx5kSt+wQ2fQ0j73NCop6ygDDGmBNRcti5U1zsKZBys9fVBJRd5mqMMSci9b+wcwPc8DaER3pdTUBZC8IYY6pr/0748i/QbST0HON1NQFnAWGMMdX11RNwqBDO/3Odv9dDdVhAGGNMdezIgCX/geSboP1pXlcTFBYQwJ4DxV6XYIwJdbMegogmcM4DXlcSNA0+IA6VlHLRU/O45dUlpG/Z7XU5xphQtPErWPcxDP8lRLfzupqgCVhAiEhnEZkrIqtFZJWI3OO+3kZEZonIeven34uIRWSiu856EZkYqDpV4YbBCaRt3sUVz33LxJcXk7Z5V6B2Z4ypa3yl8NkD0DIBhvzU62qCSlQ1MG8s0hHoqKppItIcWApcAUwCdqrqX0TkXqC1qv6uwrZtgFQgBVB329NV9bjf3CkpKZqamnpS9RYdKuH1BVn8Z95Gdu47zPCesdwzuicpiW1O6v2MMfVE2msw4y64+mXoe5XX1dQ6EVmqqin+lgWsBaGquaqa5j7eC6wB4oDLgcnuapNxQqOi84FZqrrTDYVZwAWBqhUgunEEPxnZnXm/PYf7LjyV1VsLufr5Bdz40kIWb9oZyF0bY0LVob0w508QPwhOG+t1NUEXlHMQIpIIDAQWAe1VNdddtA1o72eTOGBLuefZ7mv+3vs2EUkVkdT8/Pwa19qscQS3j+jOvN+dw4MX92bdtiKufWEB415cwIINBTV+f2NMHfLN/0FRHlzweIO4rLWigAeEiEQD7wE/V9XC8svU6d+qUR+Xqr6oqimqmtK2bduavNUxmjaK4Nbh3Zj323P4/SV92Ji/j+v/s5BrX1jA/IwdBKprzhgTInZvcW4G1O8aiPfbA1PvBTQgRCQSJxymqur77st57vmJsvMU2/1smgN0Lvc83n0t6Jo0CueWs7ry9W/P4ZFL+5BVsI8bXlrENc8vYN76fAsKY+qr2X9wfo5+2Ns6PBTIq5gE+C+wRlX/UW7RDKDsqqSJwId+Nv8MGCMird2rnMa4r3kmKjKcSWd25avfnMMfLz+NnN0HmPDfxVz17/l8uW67BYUx9Ul2Kqx4B4beCa06V71+PRXIq5jOAuYBKwCf+/L9OOch3gYSgCzgWlXdKSIpwB2qequ7/S3u+gCPqeorVe2zJlcxnahDJaW8k5rNv7/cQM7uAyR1bsU9o3sysldbpAH2VRpTb6jCf8fA7iy4ayk0bu51RQF1vKuYAhYQXjjpgCg+AJFNTmqfh0t8vJeWzXNzM8jedYD+8S25e1RPRvduZ0FhTF208j149xa47BlnWo16zgLieHyl8HQSxPZy/mfodeFJTeFbXOpjeloOz87NYPPO/ZzWqQV3j+7JmD7tLSiMqSuKD8KzZ0BUS7j9KwgL97qigPNkHESdUXII+o+DvFXw9gT4R2/4/CHYsf6E3iYyPIxrz+jM7F+N4G9X92ffoRJuf30pFz39DZ+uzMXnqz9BbEy9tfBfsGcznP9YgwiHqlgLooyvFDJmQ9pk+P5T8JVAwjCnVdHncmjU9ITerqTUx4zlW3l2TgYbd+zj1A7NuWtUTy7s24GwMGtRGBNyirbD08nQdThcP83raoLGuphO1N48WD7NGWK/cwM0bgn9r3HCouOAE3qrUp8y87utPD17PRvy99GzXTR3je7Jxf06Em5BYUzo+N89sGwK/HQRxPbwupqgsYA4WaqQNd8JitUfQMlBJyCSb4K+V0OTVtV+q1Kf8tGKXJ6ZvZ7124vo3rYZd43qyaUDOllQGOO1vFXw/Fkw6Ha48C9eVxNUFhC14cBu57rotMmwbYUzL/xpVzhhkTC02sPwfT7lk5XbeHr2etbl7aVbbDPuHNWDywZ0IiLcTgkZE3Sq8PoVsDUd7l4GTRvWBJ0WELVta7oTFCvedW4/GNPDCYoB11d7rnifT/ls1Taemr2etdv2khjTlJ+d04MrB8ZZUBgTTN9/Bm9cCxf8BYb8xOtqgs4CIlAO74PVHzpdUJsXQFiEc5ls8kToPqpaV0H4fMqsNXk8PXs9q7YWktCmKXee04Mrk+OItKAwJrBKi+Hfw0B98NOFJ3WJe11nAREM+d/DstcgfRrs3wEt4mDgeOdfq4QqN1dVZq/ZzlOz17MiZw/xrZvws3N6cFVyPI0iLCiMCYhFL8Inv4Fx0+DUi7yuxhMWEMFUchi+/8RpVWTMdl7rfo47CO9iiGh03M1VlbnrtvPUF+tZnr2HuFZN+MnI7lyTEk/jCLsu25hac2CXc1lrh75w04wGOZ031CAgRGSUqs5xH3dV1U3llo0tN0NrSAiJgChv9xZInwppr0NhNjSNcc5TJN8EbXsdd1NV5avv83lq9nqWbd5Nx5ZR/HRkd65J6UxUpAWFMTX22QOw4Dm4/Wvo2N/rajxTk4BIU9Xkio/9PQ8FIRcQZXylsHGu06pY+5EzCK/zECcoTrsCGjWrdFNV5ZuMHTz1xXpSs3bRoUUUd4zoxrhBCRYUxpysgg3w3GAYMA4uf9brajxVk4BYpqoDKz729zwUhGxAlFeUf3QQXsF6aNQc+l3thEWngZU2c1WV+RsKeOqL9SzO3Em75o25Y0R3bhhsQWHMCXvzRtgw17mstbm/m1o2HNaCCEWqsHmhExSrpkPJAWjfzwmK/tdAk9aVbrpgQwFPzf6ehRt3EhvdmDtGdOOGwQk0bRQRxA9gTB21aR5MvgRGPQhn/8brajxXk4DYDXwNCDDcfYz7/CxVrfxbzAN1KiDKO7jHGVOR9hrkpkN4Y2f+p+SbIPGsSlsVizYW8PSc9XybUUBsdCN+PLwbE4Z2saAwpjI+H7w4AvbvhLtST3qa//qkJgEx4nhvrKpf1bC2WlVnA6K83OXOSe3v3oZDe6BNNxg4AZJugOYd/G6SmrmTp2avZ976HbRpdjQoohtbUBhzjGVT4cOfwtiXnJa6qb3LXN17TPcFclTV372kPVUvAqJM8QFYPcNpVWR9AxIOp1zgtCp6nAvhP/zyX5q1i6dnr+er7/Np1TSSHw/vxk1Du9A8quEN/jHmBw7vcy5rbRkPt37RYC9rragmLYjngWdUdZWItAQWAKVAG+DXqhpSc+LWq4Aob0cGLHsd0t+AfduheScYeKMzCK914g9WT9+ym6dnr2fO2u20bBLJj87qyqQzE2lhQWEasrl/hq/+Crd8DgmDva4mZNQkIFap6mnu458DI1X1ChHpAHxiVzEFWWmxM29M2muQMcuZHqDbSKdVceolENH4mNW/y3aC4os122kRFcEtZ3Xl5jO70rKJBYVpYPbkwDOnO1PhXFPl7e0blNq6zPUj4B1VfbXiskq2fRm4BNiuqn3d194CykaItQJ2q2qSn20zgb04rZWSyoqvqN4HRHl7sp0WRdrrzh2wmrR2BuENnADt+xyz6sqcPTw9ez2fr86jeeMIbj4zkVvO6kqrpscf1W1MvfH+7c7VgncugdZdvK4mpNQkIOYCTwI5wFzgVFXdJiIRwEpVPfU4254NFAGvlQVEheVPAntU9VE/yzKBFFXdcdxPVkGDCogyPh9s+tJpVayZCb5iiD/DHYQ3FhpHH1l19dZCnpmznk9WbiO6cQQTh3Xh1rO60bqZBYWpx3KWwn9GwVm/gHMf8bqakFOTgDgFeBroAPxfudbD+cAYVf1VFTtOBGZWDAgREWAzMEpVf3DzZwuIk7RvB3z3FiydDDvWQaNo6DvWmV027vQjJ+XWbivkmdkZfLwyl6aR4dw0LJFbz+pKTHTjKnZgTB2jCq9cCAUZcFcaRLXwuqKQ49lkfccJiLOBf1RalMgmYBegwAuq+uJx9nEbcBtAQkLC6VlZWbVUfR2mCtlLnHtWrHwfivdDuz7uILzrjtwQ5fu8vTwzJ4OZ322lSWQ4E4Z04cdndyPWgsLUF6s+gHcmwiX/Byk3e11NSKpJC+Lp472xqt5dxY4T8R8Q/wYyVPXJSraLU9UcEWkHzALuUtWv/a1bXoNvQfhzsBBWvud0QW1Ng/BG0PtSdxDe2RAWRsb2vTw7J4MZy7fSKCKM8YO7cNuIbrRrHuV19cacvOKD8NwgpyV9x7xq3Z+lIapJQBwGVgJvA1txRlAfoaqTq9hxIhUCwj1/kQOcrqrZ1Sj+EaBIVf9e1boWEFXYtsIdhPemM3q7daI7CO9GaNGRDflFPDcngw/Sc4gMD+PGwV24Y0Q32rWwoDB10LdPwazfw4QPnCn3jV81CYgY4BrgOqAEeAt4V1V3V3PHifwwIC4A7lNVv6O0RaQZEKaqe93Hs4BHVfXTqvZnAVFNxQecE9ppkyFzHkgY9DzfaVX0HEPmrkM8OzeD6ctyCA8TbhiUwB0jutOhpQWFqSOK8uGZZOd+8Te+7XU1Ia1WzkGISDwwDvgl8DtVfb2K9acBI4FYIA94WFX/KyKvAgtV9fly63YCXlLVi0SkGzDdXRQBvKGqj1WnRguIk1CwAZZNce5bUZQH0R2caT0GjieLDvxr7gbeS8smTIRxgzpzx4judGpl89eYEDfzl7D0Vec2om1P8bqakFbjgBCRZOB64DxgKfCkqq6u1SprgQVEDZSWwPrPnXMV6z9zBuElDofkiWzpMJp/fZPDu0u3IAjXpMRz+9ndSYhp6nXVxvzQ9jXOfabPuBUu+pvX1YS8mnQxPQpcDKwB3gQ+VdWSgFRZCywgaknhVncQ3muwOwuiWkH/68jrcS1Pr2rM26lbKPEpI05py42DuzDq1HaEh9m8NiZETLnKuYrv7vQjV+yZytUkIHzAJmC/+1LZygKoqobUffosIGqZz+eco0h7DdbMgNLD0CmZPb2vZ1phf15ZXkRe4SE6tYzi+kEJXHdGZzuhbby1/guYehWMeQyG3el1NXVCTQLiuGPSVTWkBh1YQATQ/p3OILy012C707uozTuxPfpUvt0Xx8wd7VlLV5JO6834wV0Y2j0GsdkyTTCVlsDzZzp/yPx0EUTYDAHVUesD5UQkDLheVafWtLjaZAERBKrOeIrNC2FrunP/ih3fU9a4zKcVK0oTyWlyCh1PHcKgYaNo0T7RplY2gbfkJfjoV3DdVOh9idfV1Bk1aUG0AH4GxAEzcC45vRP4FbBcVS+v/XJPngWERw4VQd5KyF1Oac4yijYtJXpvBuH4ACgKb4mvwwCad01BOg6ATknQqouFhqk9B3Y7l7W27Q2TZtr/WyfgeAFR1S3HXseZ8mIBcCtwP875hytUNb02izR1WONoSBgCCUMIB1oCHN7PptVL+G7JlxRnL+PULZs4NedbIih1tolqCR0HQMck52engdC6K4SFefc5TN0170mnG/T8xywcalFVAdFNVfsBiMhLQC6QoKoHA16ZqdsaNaVr0gi6Jo1g78FiPkjfyn0LMtDtqzmj0WYuapZHv6JMohY97/QZAzRuAR36u8HhtjRietgUCeb4dm6CRc8743c6JXldTb1SVUAUlz1Q1VIRybZwMCeqeVQkE4Z0YfzgBJZmDWTKwixuXLGNw6U+hia24LZTD3FWdA6Red855zRSX4aSA87Gkc2gQ7+jgdFxAMT28nvLVdNAffEwhEXCqIe8rqTeqeocRCmwr+wp0ATnkteyy1xDau5cOwdRdxQUHeLdpdlMXbSZzTv3E9OsEdee0ZkbBiXQuWUj58R37nLITXd/fgfF7v+KEVHQvu+xodG2t1210hBlzXem8z7nARjxW6+rqZM8m+472Cwg6h6fT5mXsYMpC7OYvSYPBUac0pbxg7twTvkBeL5SZ1qQY0JjORwqdJaHN3KmNC8LjI5JzvNIG5dRb/l88J9zYF8+3JkKjWxk/8mwgDB1Qu6eA0xbvIU3F29m+95DxLVqwvWDOnPtGZ39Tz3u88GuTUcDo+yy24O7neVhEdCud7mT4UnQ/jT7Iqkvlr8J02+HK1+EAdd5XU2dZQFh6pTiUh+z1+QxZeFmvsnYQUSYcP5pHbhxSAJDu1UxAE8Vdm+uEBrpsL/AWS5h0PbUY6+g6tDvmFuzmjrg8D54JgWat4db59jVbzVgAWHqrI35RbyxaDPvLM1mz4FiurVtxo2Du3B1cjwtm0ZW701UoTDnaLdUWUujaJu7gkBsz2NDo2N/51JcE5q+/Ct8+We4+VPoMtTrauo0CwhT5x0sLuWj73KZsiiLZZt3ExUZxqX9OzF+SBcGdG51cm+6d9uxgZGb7gRJmTbdygWG+88mf/Ne4VZ45nToeR5c+5rX1dR5FhCmXlm1dQ9TFm7mw/Qc9h8upV9cS24cnMBlSZ1o2qiGl78W5cO2CqGxe/PR5a0Syg3uS3IeN4ut2T7Nifngp7DiHfjZYmjT1etq6jwLCFMv7T1YzAfLcpiycDPr8vbSPCqCq5LjuXFwAj3bN6+9He3febR7quzcxs6NR5e3iKsQGgOgeYfa2785ams6vDgSzrwbznvU62rqBQsIU6+pKqlZu5iyMItP3AF4g7q2YfyQLlxwWgcaRQTgBObBPc7YjPKhsWM9R2bEj+5wbGC06+O0PmxU+MlThVcvhvx1cHeanSOqJRYQpsEoKDrEO0uzecMdgBcb3YhrUzpz/aAEOrcJ8OWth/bCtpXHhkb+WufufADhjZ3zGjHdnSlEYns6P2N6Ouc2bA6h41vzP3hrPFz8DzjjR15XU29YQJgGx+dTvl6fz9RFm48MwBt5SlvGD+nCyF5BvAPe4f3OTLf5a6EgA3ZkOD93bgRf8dH1olqVC43uTmjE9HAeR9o9wCk5BM8NdkbR3/GNTbVSizwJCBF5GbgE2K6qfd3XHgF+DOS7q92vqh/72fYC4CkgHHhJVf9SnX1aQBh/tu4+wJuLN/Pmki3VG4AXDKUlsGfz0cAoyICC9c5o8fJXUgG07OyGhfsv1v3ZsnPD6bKa/yx8/gCMfw96nOt1NfWKVwFxNlAEvFYhIIpU9e/H2S4c+B44D8gGluDcnGh1Vfu0gDDHU1zq44vVeUxZlMW3GQXOALy+HRg/uAtDurUJnTvgHSpyWhhlgbFj/dEQKZtaBI52WcWWC4+ylkezGO/qr237CuDpgdB5EIx/1+tq6p2a3A/ipKnq1yKSeBKbDgIyVHUjgIi8CVwOVBkQxhxPZHgYF/bryIX9OrLBHYD37tJsPvoul+7uALyrTo+nZZNqDsALlMbRzkC9jhVu+a7qzDtUkHFsaOSvg3WfHttl1aR1ucAod86jTbe612X11V/gcBGM+ZPXlTQ4AT0H4QbEzAotiElAIZAK/EpVd1XY5mrgAlW91X0+ARisqn7vQC4itwG3ASQkJJyelRVSt8k2Ie5gcSkzv8tlysIs0rc4A/AuG+AMwOsf38rr8qqvtAR2ZzktjoL15UJkA+zdWm5Fcbusupc7Se6e82gZH3pdVvnr4F9DIeVmuPhJr6uplzw7Se0nINoDO3CuBfwj0FFVb6mwzQkFRHnWxWRqYmXOHqYuyuKDZVs5UOwMwBs/JIHLBsTRpFGIfXGeiENFsHPD0cA4EiAZcHjv0fXCG7thUf4kudvy8GoE+dRrYPMi57JWG5AYECETENVZJiJDgUdU9Xz3+X0Aqvp4VfuzgDC1ofDIALwsvs8rOjIAb/yQBHq0q8UBeF5ThaLt5U6Ql7vKatcm8JUcXbdJ66OhUf6cRyC7rDJmw5SxcN4fnYFxJiBCJiBEpKOq5rqPf4HTMhhXYZsInJPUo4EcnJPUN6jqqqr2ZwFhapOqsiRzF1MXHR2AN9gdgHd+oAbghYrSYmeKkYrnOwoyYG9uuRXdLqvYHsee84jtCS3iT36W1dISeGE4FO93ptSIaFwrH8v8kFdXMU0DRgKxQB7wsPs8CaeLKRO4XVVzRaQTzuWsF7nbXgT8H85lri+r6mPV2acFhAmUHUWHeCc1mzcWZ7Fl54HgDsALNYf2ul1V5UKjLEQOFx1dLyLKHRhYrquq7HFVXVapr8DMnzuT8fW5PKAfp6GzgXLG1BKfT/lqfT5TF25mzlpnAN45vdoxfkgCI04J4gC8UKQKRXn+Wx27Mit0WbXxPzCwTTcoPQzPJDuv3fyxjTAPMAsIYwIgp9wAvHx3AN4NgxO4NqUzbZtbl8gxSothV5b/8x1H7ssBINCkFRzYBT+eC3HJXlXcYFhAGBNAxaU+Zq3OY8rCLOZvKCAy3LkD3vghXRjcNYQG4IWqg4XOVVZHBgWud+7yd9YvvK6sQbCAMCZINuQXMXXhZt5duoXCgyX0aBfNjYMTGJscAgPwjPHDAsKYIDtwuJSZ321lyqLNLN+ym0YRYZzXuz1jk+M4+5S2RIbX4yugTJ1iAWGMh1bm7OHdpdnMWL6VnfsOE9OsEZcO6MTY5Dj6xbW0LijjKQsIY0JAcamPr9blM31ZDrNW53G41EePdtGMTY7jiqQ4OrWqY3MkmXrBAsKYELNnfzEfrchl+rJslmTuQgSGdothbHI8F/TtQHRju9+BCQ4LCGNCWFbBPqYvy2H6shyyCvbTJDKc809rz9jkeM7sEduwx1aYgLOAMKYOUFXSNu/ivbQcZi7fSuHBEto1b8wVA+MYmxzHqR1aeF2iqYcsIIypYw4WlzJ37XbeS8vhy3XbKfEpvTu24KrkOC5L6uTdnfBMvWMBYUwdVlB0iJnf5fJ+WjbLs/cQJjC8Z1vGJscxpk+Huj0VufGcBYQx9UTG9iKmL8tmeloOW/ccJLpxBBf27cDY5HgGd21DmJ2vMCfIAsKYesbnUxZt2sn7adl8vCKXfYdLiWvVhCsGduLKgfH0aBftdYmmjrCAMKYeO3C4lM9Xb+P9tBzmrc/HpzCgcyvGDozj0gGdaNOskdclmhBmAWFMA7G98CAzlm/lvbQc1uQWEhEmnHNqO8YOjGNU73Y0jrDzFeZYFhDGNEBrcguPjK/I33uIlk0iuaR/R8Ymx5Gc0Nqm+DCABYQxDVpJqY9vNxQwPS2bT1dt42Cxjy4xTRk7MJ4rB8aRENPA7ohnjmEBYYwBoOhQCZ+syGX6shwWbCxAFc5IbM2VA+O5uH9Hm5K8AbKAMMb8wNbdB/ggPYf303LI2F5kU5I3UJ4EhIi8DFwCbFfVvu5rfwMuBQ4DG4CbVXW3n20zgb1AKVBSWfEVWUAYc+JUlRU5e3g/LcemJG+AvAqIs4Ei4LVyATEGmKOqJSLyVwBV/Z2fbTOBFFXdcSL7tIAwpmZsSvKG53gBEbA5hVX1axFJrPDa5+WeLgSuDtT+jTEnLjI8jHP7tOfcPu2PmZL8iU/X8bfP1tmU5A1MQM9BuAExs6wFUWHZ/4C3VHWKn2WbgF2AAi+o6ovH2cdtwG0ACQkJp2dlZdVS9caYMhWnJI+KDOOC0zrYlOT1gGcnqSsLCBF5AEgBxqqfAkQkTlVzRKQdMAu4S1W/rmp/1sVkTGDZlOT1T0gFhIhMAm4HRqvq/mq8xyNAkar+vap1LSCMCR6bkrx+CJmAEJELgH8AI1Q1v5JtmgFhqrrXfTwLeFRVP61qfxYQxnjDpiSvu7y6imkaMBKIBfKAh4H7gMZAgbvaQlW9Q0Q6AS+p6kUi0g2Y7i6PAN5Q1ceqs08LCGO8VzYl+QfLtpKz+4BNSR7ibKCcMSbobEryusECwhjjKZuSPHRZQBhjQoa/KclH9mrHVck2JbkXLCCMMSHJ35TkF/fvyDWnxzMwobXX5TUIFhDGmJDmb0ryAfEtmTgskYv7d7RWRQBZQBhj6oy9B4uZviyHyfMz2ZC/j9joRtwwKIEbh3ShfQsbW1HbLCCMMXWOqvJNxg5e/TaTOeu2Ey7Chf06MmlYF7sjXi3yZLI+Y4ypCRFheM+2DO/ZlqyCfby2IIu3U7fwv+Vb6RvXgolDE7l0QCeiIq37KVCsBWGMqTP2HSo50v20fnsRbZo14vpBnRk/pAsdW9pU5CfDupiMMfWKqjJ/QwGvzs/kizV5hIlwwWkdmDgskTMSrfvpRFgXkzGmXhERzuwRy5k9Ytmycz+vL8zizcWb+WhFLn06tmDSsEQuS7Lup5qyFoQxpl7Yf7iED5ZtZfL8TNbl7aV100jGDUpg/JAuxNmd8CplXUzGmAZDVVm4cSevzt/ErNV5AIzp04FJZyYyuGsb636qwLqYjDENhogwtHsMQ7vHkL3L6X56a8kWPl21jVM7NGfisESuSIqzKcirwVoQxph678DhUmYsz+GVbzNZu20vLZtEMu4M5+qnzm2ael2ep6yLyRhjcLqfFm/ayeQFmXy2Kg9V5dze7Zk0LJGh3WMaZPeTdTEZYwxO99PgbjEM7hbD1t0HmLIwi2mLN/P56jxOaR/NxGGJXDkwjqaN7KsRrAVhjGngDhaXMmO5c/XTqq2FtIiK4NqUztw0NJGEmPrf/WRdTMYYUwVVZWnWLl6Zn8mnK7fhU2X0qe2YOCyRs3rE1tvuJ+tiMsaYKogIKYltSElsw7Y9B5m6KIs3Fm3mizWL6dEumolDuzA2OZ5mjRvO12ZYIN9cRF4Wke0isrLca21EZJaIrHd/+r0riIhMdNdZLyITA1mnMcaU16FlFL8a04tv7x3Fk9cMoElkOA99uIohf57No/9bTeaOfV6XGBQB7WISkbOBIuA1Ve3rvvYEsFNV/yIi9wKtVfV3FbZrA6QCKYACS4HTVXXX8fZnXUzGmEBQVdI272by/Ew+XpFLqSojT2nLpDO7MrxHLGFhdbf7ydNzECKSCMwsFxDrgJGqmisiHYEvVbVXhW2ud9e53X3+grvetOPtywLCGBNoeYUHmbpoM28s2syOokN0i23GxGGJXHV6PNF1sPvpeAER0C6mSrRX1Vz38TagvZ914oAt5Z5nu6/9gIjcJiKpIpKan59fu5UaY0wF7VtE8cvzTuHbe8/hn9cNoHmTSB6e4XQ/PTJjFRvzi7wusdZ4GneqqiJSoyaMqr4IvAhOC6JWCjPGmCo0jgjnyoHxXDkwnvQtTvfT1EVZvDo/kxGntGXSsERGnNK2Tnc/edGCyHO7lnB/bvezTg7QudzzePc1Y4wJOUmdW/HP65L49t5R/OLcU1idW8jNry5h1JNf8vI3myg8WOx1iSfFi3MQfwMKyp2kbqOqv62wTRucE9PJ7ktpOCepdx5vX3YOwhgTCg6X+PhkZS6T52eStnk3zRqFMzY5nonDutCjXXOvyzuGZyepRWQaMBKIBfKAh4EPgLeBBCALuFZVd4pICnCHqt7qbnsLcL/7Vo+p6itV7c8CwhgTar7L3s2r8zOZuTyXw6U+hveMZdKwREb2akd4CHQ/2UhqY4zx2I6iQ0xbtJkpi7LIKzxEQpum3DS0C9ekdKZlk0jP6rKAMMaYEFFc6uPTlduYPD+T1KxdNIkMZ2xyHJOGJdKzffC7nywgjDEmBK3M2cPk+Zl8uHwrh0t8nNkjholDExndu33Qup8sIIwxJoQVFB3izSVbmLIwi9w9B4lv3YQJQ7pw3RmdadW0UUD3bQFhjDF1QEmpj89X5/Hq/EwWb9pJVGQYVw6MY+KwRE7t0CIg+7SAMMaYOmb11kImz8/kg/QcDpX4GNKtDZOGJXJu7/ZEhNfeEDYLCGOMqaN27Tt8pPspZ/cB4lo1YfyQLow7ozOtm9W8+8kCwhhj6riSUh9frNnOq/M3sXDjThpHhHFFktP91KfTyXc/WUAYY0w9snZbIZPnZzF9WTYHi30M6tqG1380iMYR4Sf8XnZHOWOMqUdO7dCCx8f243cX9OLt1C1szN93UuFQFQsIY4ypo1o1bcRtZ3cP2Pt7MZurMcaYOsACwhhjjF8WEMYYY/yygDDGGOOXBYQxxhi/LCCMMcb4ZQFhjDHGLwsIY4wxftWrqTZEJB/nPtcnIxbYUYvl1Bar68RYXSfG6jox9bGuLqra1t+CehUQNSEiqZXNR+Ilq+vEWF0nxuo6MQ2tLutiMsYY45cFhDHGGL8sII560esCKmF1nRir68RYXSemQdVl5yCMMcb4ZS0IY4wxfllAGGOM8atBBYSIvCwi20VkZSXLRUSeFpEMEflORJJDpK6RIrJHRNLdf78PUl2dRWSuiKwWkVUico+fdYJ+zKpZV9CPmYhEichiEVnu1vUHP+s0FpG33OO1SEQSQ6SuSSKSX+543RrousrtO1xElonITD/Lgn68qlmXJ8dLRDJFZIW7zx/cX7nWfx9VtcH8A84GkoGVlSy/CPgEEGAIsChE6hoJzPTgeHUEkt3HzYHvgT5eH7Nq1hX0Y+Yeg2j3cSSwCBhSYZ2fAs+7j8cBb4VIXZOAZ4P9/5i7718Cb/j77+XF8apmXZ4cLyATiD3O8lr9fWxQLQhV/RrYeZxVLgdeU8dCoJWIdAyBujyhqrmqmuY+3gusAeIqrBb0Y1bNuoLOPQZF7tNI91/Fq0AuBya7j98FRouIhEBdnhCReOBi4KVKVgn68apmXaGqVn8fG1RAVEMcsKXc82xC4IvHNdTtIvhERE4L9s7dpv1AnL8+y/P0mB2nLvDgmLndEunAdmCWqlZ6vFS1BNgDxIRAXQBXud0S74pI50DX5Po/4LeAr5LlnhyvatQF3hwvBT4XkaUicpuf5bX6+2gBUTek4cyXMgB4BvggmDsXkWjgPeDnqloYzH0fTxV1eXLMVLVUVZOAeGCQiPQNxn6rUo26/gckqmp/YBZH/2oPGBG5BNiuqksDva8TUc26gn68XGepajJwIfAzETk7kDuzgDhWDlD+L4F49zVPqWphWReBqn4MRIpIbDD2LSKROF/CU1X1fT+reHLMqqrLy2Pm7nM3MBe4oMKiI8dLRCKAlkCB13WpaoGqHnKfvgScHoRyzgQuE5FM4E1glIhMqbCOF8eryro8Ol6oao77czswHRhUYZVa/X20gDjWDOAm90qAIcAeVc31uigR6VDW7yoig3D+uwX8S8Xd53+BNar6j0pWC/oxq05dXhwzEWkrIq3cx02A84C1FVabAUx0H18NzFH37KKXdVXop74M57xOQKnqfaoar6qJOCeg56jq+AqrBf14VacuL46XiDQTkeZlj4ExQMUrH2v19zHipKutg0RkGs7VLbEikg08jHPCDlV9HvgY5yqADGA/cHOI1HU18BMRKQEOAOMC/UviOhOYAKxw+68B7gcSytXmxTGrTl1eHLOOwGQRCccJpLdVdaaIPAqkquoMnGB7XUQycC5MGBfgmqpb190ichlQ4tY1KQh1+RUCx6s6dXlxvNoD092/eyKAN1T1UxG5AwLz+2hTbRhjjPHLupiMMcb4ZQFhjDHGLwsIY4wxfllAGGOM8csCwhhjjF8WECYkiEipO0PlShF5R0SaHmfdSSLybDDrK7fvR0Xk3CrWeVVErq5k2a9FZK37WZeIyE2BqfTI/jJPZoCgiHQSkXfdx0kiclHtV2dCnQWECRUHVDVJVfsCh4E7vC7IH1X9vap+cTLbuternwcMcqe9GI0z62bIUdWtqloWckk419abBsYCwoSieUAPEWkjIh+4E6ItFJH+5VcSkeYissmddgMRaVH2XES+FJG/inMfhO9FZLi7TpSIvCLOnPrLROQc9/VJ7r5muX913ykiv3TXWSgibdz1jrQOROT3bitgpYi8WDZy+zjuB35SNm+UOx3IZPe9Rrv7WiHO/UEau69nisjjbosjVUSSReQzEdlQNkBKnHtffC0iH4nIOhF5XkR+8LstIuPd45EuIi+IM4HfGe7xjXJH6q4Skb4ikuh+rkbAo8B17nbXich6EWnrvmeYOPceaHtS/6VNSLOAMCFFnPl2LgRWAH8AlrkTot0PvFZ+XXeq7y9xpmUGZ5Tt+6pa7D6PUNVBwM9xRqcD/MzZVPsB1+OMMI5yl/UFxgJnAI8B+1V1ILAA8NcV9KyqnuG2epoAlxznc7UAmqvqRj/LooBXgevcuiKAn5RbZbPb4pjnrnc1zlz/5W/8Mwi4C+gDdHc/R/l99AauA85036sUuFFVl+BMz/An4Algiqoemb5BVQ8Dv8e5D0OSqr4FTAFudFc5F1iuqvmVfXZTd1lAmFDRxJ02IxXYjDPFwlnA6wCqOgeIcb9oy3uJo9MJ3Ay8Um5Z2SR+S4FE9/FZOF9wqOpaIAs4xV02V1X3ul92e3Bm7AQnrMq2L+8cce5ytgIYBZzslOK9gE2q+r37fDLOTaTKzChXx6JyNR4Sd44lYLGqblTVUmCa+znLG40zodwS9ziPBrq5yx7F6fpKwQmJqrzM0cC8hWOPualHGtRcTCakHXD/sj2i6h4bUNVv3e6QkUB4+b9+gbLZNkup3v/rh8o99pV77qu4vftX/7+AFFXdIiKPAFFUQlULRaRIRLr5a0VUs67yNVWsq+KcORWfCzBZVe/z8/4xQDTO/F9RwL7jFeN+3jwRGYXTcrnxeOubustaECaUzcP98nEDYEcl96N4DefWkNX5S7b8e56CM8HfupOorSwMdohzXwq/Vy1V8DjwXFkrSESi3auY1gGJItLDXW8C8NUJ1jNIRLq65x6uA76psHw2cLWItHP33UZEurjLXgAeAqYCf/Xz3ntxbu1a3ks4LbF33FaLqYcsIEwoewQ4XUS+A/7C0WmfK5oKtMbpWqnKv4Awt1voLWBSuXn9q829r8J/cKZb/gxYUo3N/o1zL4YlIrISJ6x8qnoQp3vsHbcuH/D8CZa0BHgWZ9rpTTj3Cihf72rgQZy7kX2Hc5Objm5AFavqGzjH+Ay3ZVDeXKBP2Ulq97UZOK0O616qx2w2V1PnuVcVXa6qE7yuxQtu6+rXqlrpSfIA7DMF+KeqDg/WPk3w2TkIU6eJyDM4Vz3ZdfpBIiL34lxlZece6jlrQRhjjPHLzkEYY4zxywLCGGOMXxYQxhhj/LKAMMYY45cFhDHGGL/+HxVA7h4f/iF7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1,6),train_rmse_errors,label='TRAIN')\n",
    "plt.plot(range(1,6),test_rmse_errors,label='TEST')\n",
    "plt.xlabel(\"Polynomial Complexity\")\n",
    "plt.ylabel(\"RMSE\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "id": "YJ6f2sjIFU_3"
   },
   "outputs": [],
   "source": [
    "#after the plot, observe the best degree , choose the degree and train a new model accordingly "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "id": "lGKBxaNLFU_3"
   },
   "outputs": [],
   "source": [
    "polynomial_converter_ridge = PolynomialFeatures(degree=2,include_bias=False)\n",
    "poly_features_train = polynomial_converter.fit_transform(scaled_X_train)\n",
    "poly_features_test = polynomial_converter.fit_transform(scaled_X_test)\n",
    "ridge_cv_model = RidgeCV(alphas=(0.1, 1.0, 10.0),scoring='neg_mean_absolute_error') \n",
    "ridge_cv_model.fit(scaled_X_train,y_train)\n",
    "#train_pred = ridge_cv_model.predict(X_train)\n",
    "test_pred = ridge_cv_model.predict(scaled_X_test)\n",
    "#train_RMSE = np.sqrt(mean_squared_error(y_train,train_pred))\n",
    "MAE = mean_absolute_error(y_test,test_pred)\n",
    "MSE = mean_squared_error(y_test,test_pred)\n",
    "RMSE_ridge = np.sqrt(MSE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lBQLdjRpFU_3",
    "outputId": "a59b6090-7c4e-444a-c852-f1b9b1e2adc9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13.309288639785585"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MAE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "BaaDfDACFU_3",
    "outputId": "3498873e-e639-4b04-cca6-2f404042f323"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "377.5771223031417"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "-xWt0h-XFU_4",
    "outputId": "61ee82c6-9376-4fce-be12-20278bdab68e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "19.4313438110477"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RMSE_ridge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "NtSn-febFU_4",
    "outputId": "025ab315-89cd-413b-d169-a848aef1f4e8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.1"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge_cv_model.alpha_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PC5WrOYJFU_4",
    "outputId": "b35fd948-82d3-46c2-ee9e-2667bd0b8934"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ -0.85928238,  28.58179009,  -6.46179935, -18.7492228 ,\n",
       "        15.01342692,   6.83824093,   1.27839856, -11.95057662,\n",
       "        -2.74433938,   0.30964067,   6.67982473,   5.15269654,\n",
       "         0.        ,   0.        ])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge_cv_model.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MN4RwweLuQYr"
   },
   "source": [
    "### 10a- Evaluate different degrees of ElasticNet CV polynomial regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "qK9SePrtuQYr",
    "outputId": "39d3dd0d-a8e5-47f0-cc44-bda8de535cc9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "currently in iteration:  1\n",
      "currently in iteration:  2\n",
      "currently in iteration:  3\n",
      "currently in iteration:  4\n",
      "currently in iteration:  5\n",
      "currently in iteration:  6\n",
      "currently in iteration:  7\n",
      "currently in iteration:  8\n",
      "currently in iteration:  9\n"
     ]
    }
   ],
   "source": [
    "train_rmse_errors = []\n",
    "test_rmse_errors = []\n",
    "for d in range(1,10):\n",
    "    # CREATE POLY DATA SET FOR DEGREE \"d\"\n",
    "    print(\"currently in iteration: \", d)\n",
    "    polynomial_converter = PolynomialFeatures(degree=d,include_bias=False)\n",
    "    poly_features_train = polynomial_converter.fit_transform(scaled_X_train)\n",
    "    poly_features_test = polynomial_converter.fit_transform(scaled_X_test)\n",
    "    model = ElasticNetCV(l1_ratio=[.1, .5, .7,.9, .95, .99, 1],tol=1e-4)\n",
    "    model.fit(scaled_X_train,y_train)\n",
    "    train_pred = model.predict(scaled_X_train)\n",
    "    test_pred = model.predict(scaled_X_test)\n",
    "    train_RMSE = np.sqrt(mean_squared_error(y_train,train_pred))\n",
    "    test_RMSE = np.sqrt(mean_squared_error(y_test,test_pred))\n",
    "    train_rmse_errors.append(train_RMSE)\n",
    "    test_rmse_errors.append(test_RMSE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JSt5s73XuQYr"
   },
   "source": [
    "### 10b- Choose the best degree and print the errors, model coefficients and ElasticNet parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 296
    },
    "id": "UkbcQLlLuQYr",
    "outputId": "1c28e8ea-11a0-4dc4-e9ad-7e319f271a6c"
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'plt' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Input \u001b[0;32mIn [1]\u001b[0m, in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mplt\u001b[49m\u001b[38;5;241m.\u001b[39mplot(\u001b[38;5;28mrange\u001b[39m(\u001b[38;5;241m1\u001b[39m,\u001b[38;5;241m8\u001b[39m),train_rmse_errors,label\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mTRAIN\u001b[39m\u001b[38;5;124m'\u001b[39m)\n\u001b[1;32m      2\u001b[0m plt\u001b[38;5;241m.\u001b[39mplot(\u001b[38;5;28mrange\u001b[39m(\u001b[38;5;241m1\u001b[39m,\u001b[38;5;241m8\u001b[39m),test_rmse_errors,label\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mTEST\u001b[39m\u001b[38;5;124m'\u001b[39m)\n\u001b[1;32m      3\u001b[0m plt\u001b[38;5;241m.\u001b[39mxlabel(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mPolynomial Complexity\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
      "\u001b[0;31mNameError\u001b[0m: name 'plt' is not defined"
     ]
    }
   ],
   "source": [
    "plt.plot(range(1,8),train_rmse_errors,label='TRAIN')\n",
    "plt.plot(range(1,8),test_rmse_errors,label='TEST')\n",
    "plt.xlabel(\"Polynomial Complexity\")\n",
    "plt.ylabel(\"RMSE\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "id": "MNc5dLI7FU_5"
   },
   "outputs": [],
   "source": [
    "#after the plot, observe the best degree , choose the degree and train a new model accordingly "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "f5bV54CdFU_5"
   },
   "outputs": [],
   "source": [
    "#d = ?\n",
    "polynomial_converter_elastic = PolynomialFeatures(degree=d,include_bias=False)\n",
    "poly_features_train = polynomial_converter.fit_transform(scaled_X_train)\n",
    "poly_features_test = polynomial_converter.fit_transform(scaled_X_test)\n",
    "elastic_model = ElasticNetCV(l1_ratio=[.1, .5, .7,.9, .95, .99, 1],tol=0.01)\n",
    "elastic_model.fit(scaled_X_train,y_train)\n",
    "#train_pred = elastic_model.predict(X_train)\n",
    "test_pred = elastic_model.predict(scaled_X_test)\n",
    "#train_RMSE = np.sqrt(mean_squared_error(y_train,train_pred))\n",
    "MAE = mean_absolute_error(y_test,test_pred)\n",
    "MSE = mean_squared_error(y_test,test_pred)\n",
    "RMSE_elastic = np.sqrt(MSE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Kl7iGrKfFU_5"
   },
   "outputs": [],
   "source": [
    "MAE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "AfQuRjNxFU_5"
   },
   "outputs": [],
   "source": [
    "MSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "HeIdwZFhFU_5"
   },
   "outputs": [],
   "source": [
    "RMSE_elastic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SzDjGjm7FU_5"
   },
   "outputs": [],
   "source": [
    "elastic_model.l1_ratio_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "0PsOvb4PFU_6"
   },
   "outputs": [],
   "source": [
    "elastic_model.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "O_cuFINquQYs"
   },
   "source": [
    "### 11- Compare the errors from the 3 regularization techniques, and save the best model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "qn58GoGpuQYs"
   },
   "outputs": [],
   "source": [
    "least_error = min(min(RMSE_lasso,RMSE_ridge),RMSE_elastic)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ewWbm36-FU_6"
   },
   "outputs": [],
   "source": [
    "if (least_error == RMSE_lasso)\n",
    "    final_model = lasso_cv_model\n",
    "    final_poly_converter = polynomial_converter_lasso\n",
    "else if (least_error == RMSE_rigde)\n",
    "    final_model = ridge_cv_model\n",
    "    final_poly_converter = polynomial_converter_ridge\n",
    "else \n",
    "    final_model = elastic_model\n",
    "    final_poly_converter = polynomial_converter_elastic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1jcOsOBsFU_6"
   },
   "outputs": [],
   "source": [
    "from joblib import dump, load"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ta3nLXrkFU_6"
   },
   "outputs": [],
   "source": [
    "dump(final_model, 'sales_poly_model.joblib') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_XaU-tHVFU_6"
   },
   "outputs": [],
   "source": [
    "dump(final_poly_converter,'poly_converter.joblib')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IUGAAZY4uQYs"
   },
   "source": [
    "### 12- Load the best model and test it on a sample that you manually created it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "SIw1mo3kuQYs"
   },
   "outputs": [],
   "source": [
    "loaded_poly = load('poly_converter.joblib')\n",
    "loaded_model = load('sales_poly_model.joblib')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "G3dUEoJjj-AK"
   },
   "source": [
    "## Great work!\n",
    "----"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "collapsed_sections": [],
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}