kernels/kernel.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "e6a8376b4f432d323d023d8a938ed56fecf352f2"
   },
   "source": [
    "# Predicting Housing Prices in Ames, Iowa"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "22080ccd36043ff0dd260cfcde044a7ee474aadb"
   },
   "source": [
    "The goal of this exercise to build a housing price predictor for the Ames, Iowa dataset. The Ames Housing dataset was compiled by Dean De Cock for use in data science education. It's a wonderful alternative for data scientists looking for a modernized and expanded version of the often cited Boston Housing dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "48ac98ee10626fcfc4f1edc83c627a3e370c981d"
   },
   "source": [
    "## Part 1: Preliminary checks on file and loading the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "4622d34d5253f625721574d3856446b9610fd2a5"
   },
   "source": [
    "It's always good to do a few preliminary checks such as file size, line numbers, and printing the first few lines to not load corrupt data or exceed one's memory constraints."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "a5e604e7-3c20-409b-ad29-5c2a4fe40738",
    "_uuid": "44b399828f0b07fe63abbdcdf74bbf3b22bb8067"
   },
   "outputs": [],
   "source": [
    "from os import path\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "iowa_file = '../input/train.csv'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_uuid": "f50092271c3111ba40a041c3805fe2df37f9e7ee"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File size in MB:\n",
      "0.4393348693847656\n"
     ]
    }
   ],
   "source": [
    "print('File size in MB:')\n",
    "print(path.getsize(iowa_file) / (1<<20))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "_uuid": "b5c1319fc0083b44297b293ca59bad418d694324"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of rows in file:\n",
      "1461\n"
     ]
    }
   ],
   "source": [
    "print('Number of rows in file:')\n",
    "with open(iowa_file) as file:\n",
    "    print(sum(1 for line in file))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "33b2c9c6398c50b82a2ba88e16cc7fe542e9abdb"
   },
   "source": [
    "First few rows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "_uuid": "44c4c5b29051ba666f2abf38451deba29475dc01"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Id,MSSubClass,MSZoning,LotFrontage,LotArea,Street,Alley,LotShape,LandContour,Utilities,LotConfig,LandSlope,Neighborhood,Condition1,Condition2,BldgType,HouseStyle,OverallQual,OverallCond,YearBuilt,YearRemodAdd,RoofStyle,RoofMatl,Exterior1st,Exterior2nd,MasVnrType,MasVnrArea,ExterQual,ExterCond,Foundation,BsmtQual,BsmtCond,BsmtExposure,BsmtFinType1,BsmtFinSF1,BsmtFinType2,BsmtFinSF2,BsmtUnfSF,TotalBsmtSF,Heating,HeatingQC,CentralAir,Electrical,1stFlrSF,2ndFlrSF,LowQualFinSF,GrLivArea,BsmtFullBath,BsmtHalfBath,FullBath,HalfBath,BedroomAbvGr,KitchenAbvGr,KitchenQual,TotRmsAbvGrd,Functional,Fireplaces,FireplaceQu,GarageType,GarageYrBlt,GarageFinish,GarageCars,GarageArea,GarageQual,GarageCond,PavedDrive,WoodDeckSF,OpenPorchSF,EnclosedPorch,3SsnPorch,ScreenPorch,PoolArea,PoolQC,Fence,MiscFeature,MiscVal,MoSold,YrSold,SaleType,SaleCondition,SalePrice\n",
      "\n",
      "1,60,RL,65,8450,Pave,NA,Reg,Lvl,AllPub,Inside,Gtl,CollgCr,Norm,Norm,1Fam,2Story,7,5,2003,2003,Gable,CompShg,VinylSd,VinylSd,BrkFace,196,Gd,TA,PConc,Gd,TA,No,GLQ,706,Unf,0,150,856,GasA,Ex,Y,SBrkr,856,854,0,1710,1,0,2,1,3,1,Gd,8,Typ,0,NA,Attchd,2003,RFn,2,548,TA,TA,Y,0,61,0,0,0,0,NA,NA,NA,0,2,2008,WD,Normal,208500\n",
      "\n",
      "2,20,RL,80,9600,Pave,NA,Reg,Lvl,AllPub,FR2,Gtl,Veenker,Feedr,Norm,1Fam,1Story,6,8,1976,1976,Gable,CompShg,MetalSd,MetalSd,None,0,TA,TA,CBlock,Gd,TA,Gd,ALQ,978,Unf,0,284,1262,GasA,Ex,Y,SBrkr,1262,0,0,1262,0,1,2,0,3,1,TA,6,Typ,1,TA,Attchd,1976,RFn,2,460,TA,TA,Y,298,0,0,0,0,0,NA,NA,NA,0,5,2007,WD,Normal,181500\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open(iowa_file) as file:\n",
    "    for lnum, line in enumerate(file):\n",
    "        print(line)\n",
    "        if lnum > 1:\n",
    "            break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "9e56f1936cd6fcd5146acd6546bbcd03aeb96b0d"
   },
   "source": [
    "Looks good! Note that the data comes split into a train and a test portion. In practice, you'd need to run these tests on each portion but since this is a demonstration we proceed immediately with loading both files into memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "_uuid": "a91bb59aab4b7e6ffd531499194775388a92c881"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "iowa_train_data = pd.read_csv('../input/train.csv')\n",
    "iowa_test_data = pd.read_csv('../input/test.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "_uuid": "7621475c4bed1de283a7128857531a251f632b6b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of train data portion: 1460\n",
      "Length of test data portion: 1459\n"
     ]
    }
   ],
   "source": [
    "print('Length of train data portion: {}\\nLength of test data portion: {}'.format(len(iowa_train_data),\n",
    "len(iowa_test_data)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "_uuid": "c5431f53c62fc94a323cd451f8bb907100462774"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>MSZoning</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>LotShape</th>\n",
       "      <th>LandContour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>LotConfig</th>\n",
       "      <th>LandSlope</th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Condition1</th>\n",
       "      <th>Condition2</th>\n",
       "      <th>BldgType</th>\n",
       "      <th>HouseStyle</th>\n",
       "      <th>OverallQual</th>\n",
       "      <th>OverallCond</th>\n",
       "      <th>YearBuilt</th>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <th>RoofStyle</th>\n",
       "      <th>RoofMatl</th>\n",
       "      <th>Exterior1st</th>\n",
       "      <th>Exterior2nd</th>\n",
       "      <th>MasVnrType</th>\n",
       "      <th>MasVnrArea</th>\n",
       "      <th>ExterQual</th>\n",
       "      <th>ExterCond</th>\n",
       "      <th>Foundation</th>\n",
       "      <th>BsmtQual</th>\n",
       "      <th>BsmtCond</th>\n",
       "      <th>BsmtExposure</th>\n",
       "      <th>BsmtFinType1</th>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <th>BsmtFinType2</th>\n",
       "      <th>BsmtFinSF2</th>\n",
       "      <th>BsmtUnfSF</th>\n",
       "      <th>TotalBsmtSF</th>\n",
       "      <th>Heating</th>\n",
       "      <th>...</th>\n",
       "      <th>CentralAir</th>\n",
       "      <th>Electrical</th>\n",
       "      <th>1stFlrSF</th>\n",
       "      <th>2ndFlrSF</th>\n",
       "      <th>LowQualFinSF</th>\n",
       "      <th>GrLivArea</th>\n",
       "      <th>BsmtFullBath</th>\n",
       "      <th>BsmtHalfBath</th>\n",
       "      <th>FullBath</th>\n",
       "      <th>HalfBath</th>\n",
       "      <th>BedroomAbvGr</th>\n",
       "      <th>KitchenAbvGr</th>\n",
       "      <th>KitchenQual</th>\n",
       "      <th>TotRmsAbvGrd</th>\n",
       "      <th>Functional</th>\n",
       "      <th>Fireplaces</th>\n",
       "      <th>FireplaceQu</th>\n",
       "      <th>GarageType</th>\n",
       "      <th>GarageYrBlt</th>\n",
       "      <th>GarageFinish</th>\n",
       "      <th>GarageCars</th>\n",
       "      <th>GarageArea</th>\n",
       "      <th>GarageQual</th>\n",
       "      <th>GarageCond</th>\n",
       "      <th>PavedDrive</th>\n",
       "      <th>WoodDeckSF</th>\n",
       "      <th>OpenPorchSF</th>\n",
       "      <th>EnclosedPorch</th>\n",
       "      <th>3SsnPorch</th>\n",
       "      <th>ScreenPorch</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>PoolQC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>MiscFeature</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "      <th>SaleType</th>\n",
       "      <th>SaleCondition</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>65.0</td>\n",
       "      <td>8450</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>CollgCr</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>2Story</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2003</td>\n",
       "      <td>2003</td>\n",
       "      <td>Gable</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>196.0</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>PConc</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>No</td>\n",
       "      <td>GLQ</td>\n",
       "      <td>706</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0</td>\n",
       "      <td>150</td>\n",
       "      <td>856</td>\n",
       "      <td>GasA</td>\n",
       "      <td>...</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>856</td>\n",
       "      <td>854</td>\n",
       "      <td>0</td>\n",
       "      <td>1710</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>Gd</td>\n",
       "      <td>8</td>\n",
       "      <td>Typ</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>2003.0</td>\n",
       "      <td>RFn</td>\n",
       "      <td>2</td>\n",
       "      <td>548</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>0</td>\n",
       "      <td>61</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>208500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>80.0</td>\n",
       "      <td>9600</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>Veenker</td>\n",
       "      <td>Feedr</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>1Story</td>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "      <td>1976</td>\n",
       "      <td>1976</td>\n",
       "      <td>Gable</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>MetalSd</td>\n",
       "      <td>MetalSd</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>CBlock</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>Gd</td>\n",
       "      <td>ALQ</td>\n",
       "      <td>978</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0</td>\n",
       "      <td>284</td>\n",
       "      <td>1262</td>\n",
       "      <td>GasA</td>\n",
       "      <td>...</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>1262</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1262</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>6</td>\n",
       "      <td>Typ</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>1976.0</td>\n",
       "      <td>RFn</td>\n",
       "      <td>2</td>\n",
       "      <td>460</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>298</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>181500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>68.0</td>\n",
       "      <td>11250</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>CollgCr</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>2Story</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2001</td>\n",
       "      <td>2002</td>\n",
       "      <td>Gable</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>162.0</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>PConc</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>Mn</td>\n",
       "      <td>GLQ</td>\n",
       "      <td>486</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0</td>\n",
       "      <td>434</td>\n",
       "      <td>920</td>\n",
       "      <td>GasA</td>\n",
       "      <td>...</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>920</td>\n",
       "      <td>866</td>\n",
       "      <td>0</td>\n",
       "      <td>1786</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>Gd</td>\n",
       "      <td>6</td>\n",
       "      <td>Typ</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>2001.0</td>\n",
       "      <td>RFn</td>\n",
       "      <td>2</td>\n",
       "      <td>608</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>223500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>70</td>\n",
       "      <td>RL</td>\n",
       "      <td>60.0</td>\n",
       "      <td>9550</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>Crawfor</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>2Story</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>1915</td>\n",
       "      <td>1970</td>\n",
       "      <td>Gable</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>Wd Shng</td>\n",
       "      <td>None</td>\n",
       "      <td>0.0</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>BrkTil</td>\n",
       "      <td>TA</td>\n",
       "      <td>Gd</td>\n",
       "      <td>No</td>\n",
       "      <td>ALQ</td>\n",
       "      <td>216</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0</td>\n",
       "      <td>540</td>\n",
       "      <td>756</td>\n",
       "      <td>GasA</td>\n",
       "      <td>...</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>961</td>\n",
       "      <td>756</td>\n",
       "      <td>0</td>\n",
       "      <td>1717</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>Gd</td>\n",
       "      <td>7</td>\n",
       "      <td>Typ</td>\n",
       "      <td>1</td>\n",
       "      <td>Gd</td>\n",
       "      <td>Detchd</td>\n",
       "      <td>1998.0</td>\n",
       "      <td>Unf</td>\n",
       "      <td>3</td>\n",
       "      <td>642</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>272</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2006</td>\n",
       "      <td>WD</td>\n",
       "      <td>Abnorml</td>\n",
       "      <td>140000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>84.0</td>\n",
       "      <td>14260</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NoRidge</td>\n",
       "      <td>Norm</td>\n",
       "      <td>Norm</td>\n",
       "      <td>1Fam</td>\n",
       "      <td>2Story</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>2000</td>\n",
       "      <td>2000</td>\n",
       "      <td>Gable</td>\n",
       "      <td>CompShg</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>BrkFace</td>\n",
       "      <td>350.0</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>PConc</td>\n",
       "      <td>Gd</td>\n",
       "      <td>TA</td>\n",
       "      <td>Av</td>\n",
       "      <td>GLQ</td>\n",
       "      <td>655</td>\n",
       "      <td>Unf</td>\n",
       "      <td>0</td>\n",
       "      <td>490</td>\n",
       "      <td>1145</td>\n",
       "      <td>GasA</td>\n",
       "      <td>...</td>\n",
       "      <td>Y</td>\n",
       "      <td>SBrkr</td>\n",
       "      <td>1145</td>\n",
       "      <td>1053</td>\n",
       "      <td>0</td>\n",
       "      <td>2198</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>Gd</td>\n",
       "      <td>9</td>\n",
       "      <td>Typ</td>\n",
       "      <td>1</td>\n",
       "      <td>TA</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>2000.0</td>\n",
       "      <td>RFn</td>\n",
       "      <td>3</td>\n",
       "      <td>836</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>192</td>\n",
       "      <td>84</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "      <td>250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id  MSSubClass MSZoning    ...     SaleType  SaleCondition SalePrice\n",
       "0   1          60       RL    ...           WD         Normal    208500\n",
       "1   2          20       RL    ...           WD         Normal    181500\n",
       "2   3          60       RL    ...           WD         Normal    223500\n",
       "3   4          70       RL    ...           WD        Abnorml    140000\n",
       "4   5          60       RL    ...           WD         Normal    250000\n",
       "\n",
       "[5 rows x 81 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iowa_train_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "0bf6c343dd46681dcec16dae4cf3f57e5fb37ecf"
   },
   "source": [
    "## Part 2: Building a Random Forest model and handling missing values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ba33c1a01e204297242c2246572d52e59632c2a3"
   },
   "source": [
    "First, we define the prediction features and target. For simplicity, we'll ignore categorical (non-numerical) data for now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "_uuid": "e146fd6ef49b654dc33611bfd4dd34f5d882605f"
   },
   "outputs": [],
   "source": [
    "iowa_target = iowa_train_data['SalePrice']\n",
    "iowa_predictors = iowa_train_data.drop(['SalePrice'], axis=1)\n",
    "\n",
    "# For the sake of keeping the example simple, we'll use only numeric predictors. \n",
    "iowa_numeric_predictors = iowa_predictors.select_dtypes(exclude=['object'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "88d1c1c9-91ba-4bef-aee5-f4a19a68e61c",
    "_uuid": "de7bdb4f005022ea45742b5d25f47cba7a6d698d"
   },
   "source": [
    "Let's also create a function to measure the quality of an approach. This function reports the out-of-sample mean absolute error (MAE) score from a Random Forest. We begin by splitting our training data into a a train and a test portion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "_cell_guid": "986dbfe4-9386-4a03-b2a4-9e99bf1b08f5",
    "_kg_hide-input": true,
    "_uuid": "6088bfdac20ece9c040e83beb28ff169d17f0666"
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.metrics import mean_absolute_error\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(iowa_numeric_predictors, \n",
    "                                                    iowa_target,\n",
    "                                                    train_size=0.7, \n",
    "                                                    test_size=0.3, \n",
    "                                                    random_state=0)\n",
    "\n",
    "def score_dataset(X_train, X_test, y_train, y_test):\n",
    "    model = RandomForestRegressor(random_state=0)\n",
    "    model.fit(X_train, y_train)\n",
    "    preds = model.predict(X_test)\n",
    "    return mean_absolute_error(y_test, preds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "7184fc99-e266-4bd7-af0d-0da9c96887f0",
    "_uuid": "df0103056e52ffb3c500f5fc1437bd175a41adad"
   },
   "source": [
    "We first try to handle missing values by dropping the entire column containing them and measure the resulting MAE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "_cell_guid": "64ae7a0a-95aa-47a7-aa29-a1589ebcbd18",
    "_uuid": "2957c4a7c4e6ed990b1406d5c88a9aa4c738b28f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error from dropping columns with missing values:\n",
      "19458\n"
     ]
    }
   ],
   "source": [
    "cols_with_missing = [col for col in X_train.columns if X_train[col].isnull().any()]\n",
    "reduced_X_train = X_train.drop(cols_with_missing, axis = 1)\n",
    "reduced_X_test  = X_test.drop(cols_with_missing, axis = 1)\n",
    "print(\"Mean Absolute Error from dropping columns with missing values:\")\n",
    "print(int(score_dataset(reduced_X_train, reduced_X_test, y_train, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "b446d3e1-0a5c-4552-8718-e07ab5f4496a",
    "_uuid": "e78f8751db60278737a388a4b85b72bed4d3f45b"
   },
   "source": [
    "We next try to replace missing values through standard imputation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "_cell_guid": "8cb756dc-9623-43b7-92c7-dfd87c70f450",
    "_uuid": "7f1030b598d08cd586a56d4ab33d1f99f6535784"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error from imputation:\n",
      "19202\n"
     ]
    }
   ],
   "source": [
    "from sklearn.impute import SimpleImputer\n",
    "\n",
    "my_imputer = SimpleImputer()\n",
    "imputed_X_train = my_imputer.fit_transform(X_train)\n",
    "imputed_X_test = my_imputer.transform(X_test)\n",
    "print(\"Mean Absolute Error from imputation:\")\n",
    "print(int(score_dataset(imputed_X_train, imputed_X_test, y_train, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "74d86db6e782a18cfe286f6d55da95e88ac38d23"
   },
   "source": [
    "Our error seems smaller but the improvement is so little that a different `random_state` could easily make the error bigger. Why haven't we improved? Let's look at how many missing values there actually are."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "_uuid": "56bcb82486d45a9a60975cacea3a560d75e93694"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LotFrontage    189\n",
       "MasVnrArea       5\n",
       "GarageYrBlt     54\n",
       "dtype: int64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "missing = X_train.isnull().sum()\n",
    "missing[missing != 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "dfe3f8be93897749d36595b96977ffdaab11b64b"
   },
   "source": [
    "There are only few missing values (remember, there are 1460 rows) and they are concentrated in only 3 out of 80 features/columns. Replacing these with their mean values (which is what `SimpleImputer` does) is thus no improvement over simply dropping the three columns. Still, let's see if keeping track of which columns have missing values that are replaced improves the situation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "_cell_guid": "39ba8166-9b03-41cb-9403-d728b342d5e7",
    "_uuid": "914b9e57b99d7964013f007537c300fe57e0bf91"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error from imputation while tracking what was imputed:\n",
      "19525\n"
     ]
    }
   ],
   "source": [
    "imputed_X_train_plus = X_train.copy()\n",
    "imputed_X_test_plus = X_test.copy()\n",
    "\n",
    "cols_with_missing = (col for col in X_train.columns if X_train[col].isnull().any())\n",
    "for col in cols_with_missing:\n",
    "    imputed_X_train_plus[col + '_was_missing'] = imputed_X_train_plus[col].isnull()\n",
    "    imputed_X_test_plus[col + '_was_missing'] = imputed_X_test_plus[col].isnull()\n",
    "\n",
    "# Imputation\n",
    "my_imputer = SimpleImputer()\n",
    "imputed_X_train_plus = my_imputer.fit_transform(imputed_X_train_plus)\n",
    "imputed_X_test_plus = my_imputer.transform(imputed_X_test_plus)\n",
    "\n",
    "print(\"Mean Absolute Error from imputation while tracking what was imputed:\")\n",
    "print(int(score_dataset(imputed_X_train_plus, imputed_X_test_plus, y_train, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "35888f7d87fa506a07afbb0cb57c5b489bd5f621"
   },
   "source": [
    "It does not. Again, the `random_state` variance is greater than the error difference."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "1dc9e6072bad39ead6ba047f7909fbd0eddfe999"
   },
   "source": [
    "## Part 3: Converting categorical data to dummy variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "49669f889f61347fd132dd983c1dbb2e14fcf740"
   },
   "source": [
    "Let's see if we can improve our model by one-hot encoding our categoricals, i.e. turning them into dummy variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "_uuid": "873e1270d2b461d39d3816f1c6fe9fa5c2cc8a88"
   },
   "outputs": [],
   "source": [
    "one_hot_encoded_iowa_predictors = pd.get_dummies(iowa_predictors)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "_uuid": "d68f071944b758f0ffda5b32de7448d3878d1f5f"
   },
   "outputs": [],
   "source": [
    "X_train_1hot, X_test_1hot, y_train, y_test = train_test_split(one_hot_encoded_iowa_predictors, \n",
    "                                                    iowa_target,\n",
    "                                                    train_size=0.7, \n",
    "                                                    test_size=0.3, \n",
    "                                                    random_state=0)\n",
    "\n",
    "cols_with_missing = [col for col in X_train_1hot.columns if X_train_1hot[col].isnull().any()]\n",
    "reduced_X_train_1hot = X_train_1hot.drop(cols_with_missing, axis = 1)\n",
    "reduced_X_test_1hot  = X_test_1hot.drop(cols_with_missing, axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "_uuid": "e6406aace1eb4020d5e2552126c4afc757f44634"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error from including categorical data:\n",
      "18613\n"
     ]
    }
   ],
   "source": [
    "print(\"Mean Absolute Error from including categorical data:\")\n",
    "print(int(score_dataset(reduced_X_train_1hot, reduced_X_test_1hot, y_train, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "752e00ad53d4e773cf21e3f8b07a7f5bba40262d"
   },
   "source": [
    "We have reduced the error by about $1,000. This should be more than the `random_state` variance and a first small piece of progress."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "96e7cda13c65241facf4ad673ca8001edd966754"
   },
   "source": [
    "## Part 4: XGBoost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ddd653836a22b5d06e0b42038c3654a9050b144d"
   },
   "source": [
    "Let's use the powerful XGBoost regressor and run it on our data set with missing values dropped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "_uuid": "8e9246c2fd9e486de4cb587a1607647b30d106b6"
   },
   "outputs": [],
   "source": [
    "from xgboost import XGBRegressor\n",
    "\n",
    "xgb = XGBRegressor()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "_uuid": "363de4bf8ad2f2022c492738585240f486e959a9"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "XGBRegressor(base_score=0.5, booster='gbtree', colsample_bylevel=1,\n",
       "       colsample_bytree=1, gamma=0, learning_rate=0.1, max_delta_step=0,\n",
       "       max_depth=3, min_child_weight=1, missing=None, n_estimators=100,\n",
       "       n_jobs=1, nthread=None, objective='reg:linear', random_state=0,\n",
       "       reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=None,\n",
       "       silent=True, subsample=1)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb.fit(reduced_X_train_1hot, y_train, verbose=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "_uuid": "de91420d4dc687e7210cbd6616afef7ae1430425",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error: 16380\n"
     ]
    }
   ],
   "source": [
    "xgb_preds = xgb.predict(reduced_X_test_1hot)\n",
    "print(\"Mean Absolute Error: \" + str(int(mean_absolute_error(y_test, xgb_preds))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "729d3186f7f9c11c046918cf216b2fb6833c9c66"
   },
   "source": [
    "Great! We've reduced our error by another $2300. To further improve, let's find the optimal number of estimators to use in our XGBoost."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "_uuid": "f92134279d906f09def5872fb2fdded4e2ab1542"
   },
   "outputs": [],
   "source": [
    "def get_xgb_mae(n_estimators, learning_rate=0.1):\n",
    "    xgb = XGBRegressor(n_estimators=n_estimators, learning_rate=learning_rate)\n",
    "    xgb.fit(reduced_X_train_1hot, y_train, eval_set=[(reduced_X_test_1hot, y_test)], verbose=False)\n",
    "    xgb_preds = xgb.predict(reduced_X_test_1hot)\n",
    "    mae = mean_absolute_error(y_test, xgb_preds)\n",
    "    print(\"Mean Absolute Error from using {} estimators at a learning rate of {}: {}\"\\\n",
    "          .format(str(n_estimators), learning_rate, str(int(mae))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "_uuid": "f8173d2426eda140279960537521813c518c1ea9",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error from using 100 estimators at a learning rate of 0.1: 16380\n",
      "Mean Absolute Error from using 200 estimators at a learning rate of 0.1: 16028\n",
      "Mean Absolute Error from using 300 estimators at a learning rate of 0.1: 15967\n",
      "Mean Absolute Error from using 400 estimators at a learning rate of 0.1: 16014\n",
      "Mean Absolute Error from using 500 estimators at a learning rate of 0.1: 16080\n",
      "Mean Absolute Error from using 600 estimators at a learning rate of 0.1: 16082\n",
      "Mean Absolute Error from using 700 estimators at a learning rate of 0.1: 16061\n",
      "Mean Absolute Error from using 800 estimators at a learning rate of 0.1: 16100\n",
      "Mean Absolute Error from using 900 estimators at a learning rate of 0.1: 16159\n",
      "Mean Absolute Error from using 1000 estimators at a learning rate of 0.1: 16176\n"
     ]
    }
   ],
   "source": [
    "for i in range(100, 1100, 100):\n",
    "    get_xgb_mae(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "09c14b4813e55901f5c0d39c62041f95d102ddf6"
   },
   "source": [
    "We see that the best number of estimators is 300. By using a slower learning rate we may be able to further improve our results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "_uuid": "7fc3cc5be1676e269f99a873794a3a656f863e91"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error from using 300 estimators at a learning rate of 0.05: 15935\n"
     ]
    }
   ],
   "source": [
    "get_xgb_mae(300, 0.05)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ec152d8ee5e035fbd1b15063ad647ae88fde495a"
   },
   "source": [
    "## Part 5: Training our model on the whole data set and submitting our work"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "67e48833a4feb0431c1f9a5447b2dbaed6cac326"
   },
   "source": [
    "Now that we have determined the best model and parameters, we train our model on the entire data set and submit our work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "_uuid": "de12bfb39a99c550ca3fde23016a4aa41eebc0fd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[121293.23 162708.9  175002.23 ... 154983.5  122153.27 235333.69]\n"
     ]
    }
   ],
   "source": [
    "# Starting data\n",
    "iowa_target = iowa_train_data['SalePrice']\n",
    "y_train = iowa_target\n",
    "iowa_predictors = iowa_train_data.drop(['SalePrice'], axis=1)\n",
    "\n",
    "# Drop missing values from training and test data\n",
    "cols_with_missing = [col for col in iowa_predictors.columns if iowa_predictors[col].isnull().any()]\n",
    "reduced_iowa_predictors = iowa_predictors.drop(cols_with_missing, axis = 1)\n",
    "reduced_iowa_test_data = iowa_test_data.drop(cols_with_missing, axis = 1)\n",
    "\n",
    "# Convert categoricals to dummies in training and test data and aligning the columns\n",
    "one_hot_encoded_reduced_iowa_predictors = pd.get_dummies(reduced_iowa_predictors)\n",
    "one_hot_encoded_reduced_iowa_test_data = pd.get_dummies(reduced_iowa_test_data)\n",
    "X_train, X_test = one_hot_encoded_reduced_iowa_predictors.align(one_hot_encoded_reduced_iowa_test_data,\n",
    "                                                                    join='left', \n",
    "                                                                    axis=1)\n",
    "\n",
    "# Build best currently known model and use it to make predictions\n",
    "xgb = XGBRegressor(n_estimators=300, learning_rate=0.05)\n",
    "xgb.fit(X_train, y_train, verbose=False)\n",
    "predicted_prices = xgb.predict(X_test)\n",
    "\n",
    "# Print result\n",
    "print(predicted_prices)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "_uuid": "845384b1ffebe278b3378c696a9a42d3554dbc3c"
   },
   "outputs": [],
   "source": [
    "# Prepare data for submission\n",
    "my_submission = pd.DataFrame({'Id': X_test.Id, 'SalePrice': predicted_prices})\n",
    "my_submission.to_csv('submission.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "7964dbbf6e826fb2dccca64a63bd1b9c3ec7e739"
   },
   "source": [
    "## Part 6: Partial dependence plots"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "8c9c47e3bc218a32f68579887e837b0874c6d97c"
   },
   "source": [
    "Let's see how the predicted price depends on individual features. The plot visualizing this dependence is known as a partial dependence plot. The obvious features to chose from are the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "_uuid": "ba84966713348fad755e5aa623c9a12de0dd4324"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>OverallQual</th>\n",
       "      <th>OverallCond</th>\n",
       "      <th>YearBuilt</th>\n",
       "      <th>YearRemodAdd</th>\n",
       "      <th>BsmtFinSF1</th>\n",
       "      <th>BsmtFinSF2</th>\n",
       "      <th>BsmtUnfSF</th>\n",
       "      <th>TotalBsmtSF</th>\n",
       "      <th>1stFlrSF</th>\n",
       "      <th>2ndFlrSF</th>\n",
       "      <th>LowQualFinSF</th>\n",
       "      <th>GrLivArea</th>\n",
       "      <th>BsmtFullBath</th>\n",
       "      <th>BsmtHalfBath</th>\n",
       "      <th>FullBath</th>\n",
       "      <th>HalfBath</th>\n",
       "      <th>BedroomAbvGr</th>\n",
       "      <th>KitchenAbvGr</th>\n",
       "      <th>TotRmsAbvGrd</th>\n",
       "      <th>Fireplaces</th>\n",
       "      <th>GarageCars</th>\n",
       "      <th>GarageArea</th>\n",
       "      <th>WoodDeckSF</th>\n",
       "      <th>OpenPorchSF</th>\n",
       "      <th>EnclosedPorch</th>\n",
       "      <th>3SsnPorch</th>\n",
       "      <th>ScreenPorch</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>60</td>\n",
       "      <td>8450</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2003</td>\n",
       "      <td>2003</td>\n",
       "      <td>706</td>\n",
       "      <td>0</td>\n",
       "      <td>150</td>\n",
       "      <td>856</td>\n",
       "      <td>856</td>\n",
       "      <td>854</td>\n",
       "      <td>0</td>\n",
       "      <td>1710</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>548</td>\n",
       "      <td>0</td>\n",
       "      <td>61</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>20</td>\n",
       "      <td>9600</td>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "      <td>1976</td>\n",
       "      <td>1976</td>\n",
       "      <td>978</td>\n",
       "      <td>0</td>\n",
       "      <td>284</td>\n",
       "      <td>1262</td>\n",
       "      <td>1262</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1262</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>460</td>\n",
       "      <td>298</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>60</td>\n",
       "      <td>11250</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>2001</td>\n",
       "      <td>2002</td>\n",
       "      <td>486</td>\n",
       "      <td>0</td>\n",
       "      <td>434</td>\n",
       "      <td>920</td>\n",
       "      <td>920</td>\n",
       "      <td>866</td>\n",
       "      <td>0</td>\n",
       "      <td>1786</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>608</td>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>70</td>\n",
       "      <td>9550</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>1915</td>\n",
       "      <td>1970</td>\n",
       "      <td>216</td>\n",
       "      <td>0</td>\n",
       "      <td>540</td>\n",
       "      <td>756</td>\n",
       "      <td>961</td>\n",
       "      <td>756</td>\n",
       "      <td>0</td>\n",
       "      <td>1717</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>642</td>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>272</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>60</td>\n",
       "      <td>14260</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>2000</td>\n",
       "      <td>2000</td>\n",
       "      <td>655</td>\n",
       "      <td>0</td>\n",
       "      <td>490</td>\n",
       "      <td>1145</td>\n",
       "      <td>1145</td>\n",
       "      <td>1053</td>\n",
       "      <td>0</td>\n",
       "      <td>2198</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>836</td>\n",
       "      <td>192</td>\n",
       "      <td>84</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>2008</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id  MSSubClass  LotArea   ...    MiscVal  MoSold  YrSold\n",
       "0   1          60     8450   ...          0       2    2008\n",
       "1   2          20     9600   ...          0       5    2007\n",
       "2   3          60    11250   ...          0       9    2008\n",
       "3   4          70     9550   ...          0       2    2006\n",
       "4   5          60    14260   ...          0      12    2008\n",
       "\n",
       "[5 rows x 34 columns]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reduced_iowa_predictors.select_dtypes(exclude=['object']).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "5c23cf71e3834adccd7923e567a89ad0635de900"
   },
   "source": [
    "Let's see how lot area, above ground living area, and the size of the garage affect the price. We hypothesize that prices will rise with increasing number of square footage for these values. We also look at the year the house was built. Again, we expect the price to be higher for more recently built houses."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ebc00603185c6974149ee71dc70cd1dde40e70fb"
   },
   "source": [
    " *Note:* The plots are relative to the average price which is centered at 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "_uuid": "89528b09fd24e252377dc910eb0f314677ef401d"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAb0AAADPCAYAAABhsw86AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl4VNX5wPHvS9hBICwisoZNQRCFgOCCCIpBrCAVK1pBi/tG1bbiVtD+RK1WW6XuIGhZFDcQlFVwqSCbYAIIhq2AbLILJJDk/f1xT2SIWYZkZu5M5v08zzy5c+bOPe9M5p0z9557zxFVxRhjjIkHZfwOwBhjjIkUa/SMMcbEDWv0jDHGxA1r9IwxxsQNa/SMMcbEDWv0jDHGxA1r9IwxxsQNa/SMMcbEDWv0jDHGxI2yfgcQbWrXrq1NmjTxOwxTii1ZsuQnVa3jdxyRZHllSmr1T6s5nHWYtnXbkiAJv3o82LyyRi+PJk2asHjxYr/DMKWYiGz0O4ZIs7wyJTFy4Uju/vRuxvQZw6CzBuW7TrB5ZYc3jTHGRK0NezcwdPZQUpqnMLDdwBJvzxo9Y4wxUUlVufnjmxERXr38VUSkxNu0w5vGGGOi0pvL3mT2utm8dNlLNKreKCTbtD09Y4wxUefHAz9y34z76Nq4K7cm3xqy7VqjZ4wxJqqoKrdPu50j2UcYdcUoykjomio7vGmMMSaqvLPiHaasnsKzlzxL85rNQ7pt29MzxhgTNXYe3Mndn95Np/qd+GPnP4Z8+9boGWOMiRr3TL+HfRn7GH3FaBLK/Poi9JLyrdETkYoislBElovIChF5zJUnicg3IpIuIu+ISHlXXsHdT3ePNwnY1oOufLWIXBpQnuLK0kVkaKRfozF+sNwysWry95OZmDaRR7s+yhknnxGWOvzc08sEuqtqO+AsIEVEOgNPA8+ranNgDzDYrT8Y2OPKn3frISKtgWuAM4AU4CURSRCRBODfQC+gNTDArWtMaWe5ZWLO3oy93D7tdtrVbcfQ88P3O8q3Rk89P7u75dxNge7Ae658LNDXLfdx93GP9xDvSsU+wERVzVTV9UA60Mnd0lV1naoeASa6dY0p1Sy3TCy6f8b97Di4g9F9RlMuoVzY6vG1T8/9alwG7ABmAWuBvaqa5VbZDNR3y/WBTQDu8X1ArcDyPM8pqNyYUs9yy8SSWWtnMXrZaP587p9pX699WOvytdFT1WxVPQtogPfr8XQ/4hCRW0RksYgs3rlzpx8hGBNS0ZBbllcmGD8f+ZmbP76Z02qdxrBuw8JeX1Scvamqe4G5QBeghojkXj/YANjilrcADQHc49WBXYHleZ5TUHl+9b+mqsmqmlynTlzN+GJKOT9zy/LKBOPB2Q/yv33/Y3Sf0VQsWzHs9fl59mYdEanhlisBlwCr8BL0KrfaIGCyW57i7uMe/0xV1ZVf485ASwJaAAuBRUALd8ZaebwO+Snhf2XG+Mtyy8SKLzd+ychFI7nnnHs4t+G5EanTzxFZ6gFj3ZlgZYB3VXWqiKwEJorI/wHfAqPc+qOAt0UkHdiNl2io6goReRdYCWQBd6pqNoCI3AXMABKA0aq6InIvzxjfWG6ZqHf46GEGTxlMUo0knuj+RMTqFe8HncmVnJysNtmlCScRWaKqyX7HEUmWVyavB2Y9wN+//juzr59Nj6Y9Sry9YPMqKvr0jDHGxI9FWxbx7Pxnubn9zSFp8E6ENXrGGGMi5kj2Ef4w5Q/Uq1qPZy55JuL12ywLxhhjImbElyNI25HGxwM+pnrF6hGv3/b0jDHGRETq9lSe+PIJrmt7HZe3vNyXGKzRM8YYE3YLtyzkug+uI7FiIv9M+advcdjhTWOMMWGhqszdMJcRX45gzvo51KhYg3H9xlG7cm3fYrJGzxhjTEjlaA5TVk/hya+eZOGWhZxS9RSeueQZbu1wKydVOMnX2KzRM8YYExJHs48yMW0iT/33KVbuXElSjSRe6f0Kg84aFJEhxoJhjZ4xxpgSOXz0MG8ue5Nnvn6GDXs30ObkNozrN46rz7iasmWiq5mJrmiMMcbEjP2Z+3l50cs8v+B5th/cTpcGXXgh5QV6t+xNGYnO8ySt0TPGGHNCdh7cyb+++RcjF45kX+Y+ejbryUPnP0TXxl3x5h+OXtboGWOMCcrejL0MmzuM15e+TkZWBv1a9ePB8x+kw6kd/A4taNboGWOMCco9n97D+NTxXN/ueh447wFOr+3LvN8lEtRBVxFpLCIXu+VKIuLvOafGlAKWVyaWpG5P5T/f/Yf7u9zPm33ejMkGD4Jo9ETkZuA94FVX1AD4KJxBGVPaWV6ZWPPI3EeoVqEaD5z/gN+hlEgwe3p3AucB+wFU9Qfg5HAGZUwcsLwyMWP+pvlMWT2Fv5z3F2pWqul3OCUSTKOXqapHcu+ISFnAZp41pmQsr0xMUFUenPMgdavUZcg5Q/wOp8SCafQ+F5GHgEoicgkwCfg4vGEZU+pZXpmYMHPtTD7f+DmPdn2UKuWr+B1OiQXT6A0FdgKpwK3AJ8Aj4QzKmDhgeWWiXo7m8NBnD5FUI4mbO9zsdzghEcwlC5WA0ar6OoCIJLiyQ+EMzJhSzvLKRL33Vr7H0q1LeavvW5RPKO93OCERzJ7eHLxkzFUJmB2ecIyJG5ZXJqpl5WTx6NxHaXNyG65te63f4YRMMHt6FVX159w7qvqziFQOY0zGxAPLKxPVxiwbw5pda5h8zWQSyiT4HU7IBLOnd1BE2ufeEZEOwOGSViwiDUVkroisFJEVIjLEldcUkVki8oP7m+jKRUReEJF0EfkuT0yD3Po/iMigwFhFJNU95wWJ9kHhTDwJS165bVlumRI5fPQww+cNp0uDLvym5W/8Diekgmn0/ghMEpEvReQr4B3grhDUnQXcr6qtgc7AnSLSGq+Df46qtsA7BDTUrd8LaOFutwAvg5fIwDDgHKATMCw3md06Nwc8LyUEcRsTCuHKK7DcMiX00qKX2HJgCyN6jIj6AaRPVJGHN1V1kYicDpzmilar6tGSVqyqW4GtbvmAiKwC6gN9gG5utbHAPOABV/6WqiqwQERqiEg9t+4sVd0NICKzgBQRmQdUU9UFrvwtoC/waUljN6akwpVXbtuWW6bY9mXs48mvnuTSZpfSrUk3v8MJuWAHnO4INHHrtxcRVPWtUAUhIk2As4FvgLouaQG2AXXdcn1gU8DTNruywso351NuTLQIa16B5ZY5cf+Y/w92Hd7FiB4j/A4lLIps9ETkbaAZsAzIdsUKhCQ5RaQq8D7wR1XdH7grraoqImEfpUJEbsE7rEOjRo3CXZ0xYc8rV4evuWV5FXt2HNzBc/Ofo3/r/rSv177oJ8SgYPb0koHW7tBHSIlIObykHKeqH7ji7SJST1W3ukMsO1z5FqBhwNMbuLItHDtkk1s+z5U3yGf9X1HV14DXAJKTk20oKBMJYcsriI7csryKPSO+HEFGVgZ/u+hvfocSNsGcyJIGnBLqit3ZXqOAVar6XMBDU4Dcs8QGAZMDyge6M806A/vcoZoZQE8RSXSd7D2BGe6x/SLS2dU1MGBbxvgtLHkFllumeDbu3cjLi1/mxrNu5LTapxX9hBgVzJ5ebWCliCwEMnMLVfWKEtZ9HnA9kCoiy1zZQ8BTwLsiMhjYCFztHvsEuAxIxxu14kYXx24R+RuwyK33eG7HO3AHMAbvwt9PsY52Ez3ClVdguWWKYfjnwxGEYd2G+R1KWElRR1dE5ML8ylX187BE5LPk5GRdvHix32GYUkxElgD35/eY5ZXxw8qdK2n7clvu7Xwvz/Z81u9wikVElqhqclHrBXPJwuci0hhooaqzxRs1ovRcnm+MDyyvTDR55LNHqFKuCkPPH1r0yjGuODOn18dmeDamRCyvTLRYuGUhH37/IX8690/Urlzb73DCzmZON8YfllcmKjw05yHqVK7DvZ3v9TuUiAjmRJZMVT2Se42P2AzPxoSC5ZXx3ex1s5mzfg7/vPSfnFThJL/DiQibOd0Yf1heGV+pKg/NeYhG1RtxW/JtfocTMTZzujH+sLwyvvrw+w9Z9OMiHuv2GBXKVvA7nIgJ5uzNHOB1dzPGhIDllfFTVk4WD3/2MK1qt+L6M6/3O5yIKrDRE5FUCuljUNUzwxKRMaVY27ZtAVqLyHf5PW55ZSLh7eVv8/1P3/P+1e+Xqglig1HYnt7l7u+d7u/b7u/vsQ53Y4pl6tSpNGnSJB2Y7oosr0xEZWRlMPzz4XQ8tSNXnn6l3+FEXIGNnqpuBBCRS1T17ICHHhCRpRybgNIYE6TGjRsDHAEsr4wvXln8Cv/b9z9GXzG61E0QG4xgTmQRETkv4M65QT7PGFMwyysTcVk5WTz11VP0SOpBj6Y9/A7HF8FcpzcYGC0i1QEB9gB/CGtUxpR+llcm4uasm8P2g9t5pdMrfofim2DO3lwCtHPJiaruC3tUxpRyllfGDxPSJlC9QnV6Ne/ldyi+CWbm9ArAb4EmQNncY8Cq+nhYIzOmFLO8MpF2+OhhPlj1Af1b94+r6/LyCubw5mRgH7CEgHm/jDElYnllIuqTHz7hwJEDXNv2Wr9D8VUwjV4DVU0JeyTGxBfLKxNR49PGc0rVU+jWpJvfofgqmLPFvhaRtmGPxJj4YnllImZfxj6mrZnG7874XdxdjJ5XMHt65wM3iMh6vMMwAqiNHGFMiVhemYj58PsPyczOZECbAX6H4rtgGr34Pc3HmPCxvDIRMyFtAk0Tm9Kpfie/Q/FdkYc33cgsDYHubvlQMM8zxhTM8spEyvaftzN73WwGtBkQlyOw5FVkkonIMOAB4EFXVA74TziDMqa0s7wykTJp5SRyNCfuz9rMFcwvyyuBK4CDAKr6IxAfU+waEz6WVyYixqeO58y6Z9K6Tmu/Q4kKwTR6R1RVcSPAi0iVUFUuIqNFZIeIpAWU1RSRWSLyg/ub6MpFRF4QkXQR+U5E2gc8Z5Bb/wcRGRRQ3kFEUt1zXhDbtzfRw/LKhN36PeuZv3k+17axvbxcwTR674rIq0ANEbkZmE3oJr4cA+S9VmkoMEdVWwBzODbqfC+ghbvdArwMXjIDw4BzgE7AsNyEduvcHPA8uy7KRAvLKxN2E9MmAnBNm2t8jiR6BHMiy7PAe8D7QEvgr6r6YigqV9UvgN15ivsAY93yWKBvQPlb6lmA92VRD7gUmKWqu1V1DzALSHGPVVPVBe4X9VsB2zLGV5ZXJhLGp43nvIbn0bhGY79DiRrBXLIAkApUwjsUkxq+cACoq6pb3fI2oK5brg9sClhvsysrrHxzPuXGRAvLKxM2qdtTSduRxsheI/0OJaoEc/bmTcBCoB9wFbBARCIyBUpgn0c4icgtIrJYRBbv3Lkz3NUZY3llwm5C2gQSJIH+Z/T3O5SoEkyf3p+Bs1X1BlUdBHTAO9U6XLa7Qyi4vztc+Ra865pyNXBlhZU3yKf8V1T1NVVNVtXkOnXqhORFGFMEyysTNqrKhLQJXNz0Yk6ucrLf4USVYA5v7gIOBNw/4MrCZQowCHjK/Z0cUH6XiEzE61zfp6pbRWQGMCKgk70n8KCq7haR/SLSGfgGGAiEpM/ERL/snGz2ZOwhOyebbM3+5W+O5vyqLDvHlQeUCUJSYhL1qtYL1wW9llcmbBZsXsCGvRt4rNtjfocSdYJp9NKBb0RkMt4hkT7AdyJyH4CqPlfcykVkAtANqC0im/HOFnsK78y2wcBG4Gq3+ifAZS6eQ8CNrv7dIvI3YJFb73FVze3EvwPvTLZKwKfuZkqxDXs3MPrb0Yz+djRbDuS7A3JCqleozum1T6dVnVa0qu1udVqRVCOppAP3Wl6ZsJmQNoGKZSvS93Q7xygv8Q7vF7KCN3JEgVS1VP2USE5O1sWLF/sdhjkBR7KPMPn7ybzx7RvMWjsLgJTmKVza7FLKJ5SnjJQhoUwCCZJAQpkE775bTpCEfB/PzskmfXc63//0Pat+WsWqn1ax7edtv9RZIaECLWu1/FVj2LJWSyqWrVhovCKyBPi4sHUsr0xxZeVkUf+5+nRt3JVJ/Sf5HU7EiMgSVU0uar0i9/Ryk09EKqvqoVAEZ0worNq5ilHfjmLs8rH8dOgnGlVvxLALh3Hj2TfSqHqjkNe35/CeY43gTq8hXPzjYiatmIS680LKSBmSaiTRq3kvXrys4KN+llcmXOaun8uOgztsRoUCFNnoiUgXYBRQFWgkIu2AW1X1jnAHZ0xeh44eYtKKSby+9HX+u+m/lC1Tlj6n9eGm9jdxSdNLwjpXWGKlRLo07EKXhl2OKz989DBrdq05bq8wiL09yysTFuPTxlOtQjUua3GZ36FEpWD69P6Jd6HqFABVXS4iXcMalTF5LN26lDeWvsG41HHsz9xPy1ot+fvFf2dgu4HUrVq36A2EUaVylWh3SjvandLuRJ5meWVCLiMrgw9WfUC/Vv2K/OEVr4K6OF1VN+U5gy07POEYc8y+jH2MTx3P60tf59tt31KxbEX6t+7PTe1v4oJGF8T8NCmWVybUPvnhE/Zn7rexNgsRTKO3SUTOBVREygFDgFXhDcvEu7QdaZw/+nz2Ze6jXd12jOw1kmvbXktipcSinxwbLK9MyE1Im8DJVU7moqSL/A4lagXT6N0G/AtvqKEtwEzgznAGZeLbz0d+pv+k/lQsW5GZ18+k46kdY36vLh+WVyak9mfu5+PVH3NLh1soWybYESbjTzBnb/4EXBeBWIxBVblj2h2s/mk1swfOplP9Tn6HFBaWVybUPvr+IzKzM+2szSIU2OiJyIsUMj6fqt4TlohMXBuzbAxvf/c2wy8cTvek7n6HE3J33303QEMReSG/xy2vTHGNTx1PkxpN6Nygs9+hRLXCxt5cDCwBKgLtgR/c7SygfPhDM/EmbUcad35yJ92TuvNI10f8DicskpOTwRv5xPLKhMyOgzuYvW42A9oMKI1dASFV4J6eqo4FEJHbgfNVNcvdfwX4MjLhmXiR249XrUI1xvUbF9br7fw0aNAgbrjhhl3AmVhemRCZtGIS2ZrNtW3trM2iBDPLQiJQLeB+VVdmTEgE9uON/+14Tql6it8hRYLllQmZCWkTaHNyG9qc3MbvUKJeMKf4PAV8KyJzAQG6AsPDGZSJL6W9H68AllcmJDbu3ch/N/2XEd1H+B1KTAjm7M03ReRTvGlHAB5Q1W2FPceYYMVDP15+LK9MqExMmwjANW2u8TmS2BDsiCzbODb/ljEhES/9eAWxvDKhMD5tPF0adCEpMcnvUGJCMH16xoRcnPbjGRNSK3as4Lvt39m1eSfAGj3ji9x+vGEXDounfjxjQmpC2gTKSBmuPuPqolc2QOEXp9cs7IkBsygbc0LitR8PYPfu3QAJBeWX5ZUJlqoyIW0CPZJ6+D7TSCwprE9vCd6ILPld6ahA07BEZEq1eO/H69ChA0BrvPzKy/LKBG3hloWs27OOR7s+6ncoMaWwi9OtV9SEVG4/3ppda5h1/ay47Mdbv349IpKqqsl+x2Ji24S0CVRIqMCVp1/pdygxJaizN0UkEWiBN3QSAKr6RbiCMqVTbj/eY90es348LK9M8WXnZDMxbSK9W/amesXqfocTU4ps9ETkJry5vhoAy4DOwHzAvrVM0HL78Xok9eDhCx72OxzfWV6Zkpi7YS7bD263szaLIZizN4cAHYGNqnoRcDawN6xRmVIltx+vesXqcdmPVwDLK1NsE1IncFL5k+jdorffocScYBq9DFXNABCRCqr6PXBaeMMKHRFJEZHVIpIuIkP9jifeBPbjje833s4yOyam8wost/ySmZXJ+6ve58pWV1KpXCW/w4k5wfTpbRaRGsBHwCwR2QNsDG9YoSEiCcC/gUuAzcAiEZmiqiv9jSx+BPbjXZR0kd/hRJOYzSuw3PLLml1rGD5vOPsy93FtG5tRoTiCGXsz99Sg4W5w3OrA9LBGFTqdgHRVXQcgIhOBPoAlZhjty9jHuj3rWLFzhfXjFSDG8wostyJq6dalPPnVk7y/8n0qlK3AfZ3v4+KmF/sdVkwq7OL0aqq6P89FtKnub1UgFi6irQ9sCri/mWMD/Jpiys7JZvP+zazbs461e9aybs+645Z3Hz720WhYraH14wXYv38/8KvBH2Itr8ByK+xUlS82fsGTXz3JjLUzqFahGkPPH8qQc4ZYN0EJFLanNx64nOMvUg/8W2ouohWRW4BbABo1auRzNNFj+8/b+WbLN6zZtea4hm3j3o0czTn6y3ply5SlSY0mNE1sSsdTO9I0sSlNE5vSLLEZLWu1tH6HANde+8shKcsrky9VZeqaqTz51ZPM3zyfk6uczIjuI7ij4x12eUIIFHZx+uXubyxfpL4FaBhwv4ErO46qvga8BpCcnKyRCS26HMk+wvJty5m/eT4LNi9g/ub5bNi74ZfHEysm0jSxKe3rteeqVld5jVrNZjRNbEqDag0oWyaoSz7j3tSpUxGRWM8rCCK3LK9OTFZOFu+ueJenvnqK1B2pNK7emJG9RvKHs/9gPxxDKJjr9Oaoao+iyqLUIqCFiCThJeQ1gPX+Alv2b/mlcVuweQFLti4hIysDgFNPOpUuDbpwV8e7OKfBOZxR5wwSK9mk3qEU43kFllshk5GVwZhlY3jm62dYt2cdrWq3YmzfsQxoM4ByCeX8Dq/UKaxPryJQGajtRo7IHYOzGt7x/KinqlkichcwA0gARqvqCp/DirjMrEyWbl16XCO3ab/XHVM+oTwd6nXg9uTb6dKgC50bdKZh9YZFbNEUV0ZGBnifxZjNK7DcCoUDmQd4ZfErPLfgObb9vI1O9Tvxj57/4IrTrqCM2AQ44VLYnt6twB+BU/H6H3KTcz8wMsxxhYyqfgJ84nccftj28zYe/exR3vruLY5kHwGgcfXGnNvw3F8auLNOOYsKZSv4HGn8ePXVV8EbcBpiOK8gvnOrpP654J889vlj7M3YS4+kHozrN46LmlyESH7j+5tQEtWCD7W7a3EeUtW/RS4kfyUnJ+vixYv9DqNEDh89zPMLnufJr54kMyuTwWcPpmeznpzT4BxOPelUv8OLeyKyBJhseRWfXvzmRe6Zfg8pzVN4vNvjdKzf0e+QSgURWRLMQO6F9umparaI9APiJjljWe78WkNnD2XT/k1cefqVPH3x07So1cLv0MyvWV7FoUkrJjFk+hD6nt6X9/q/Z5fy+CCYA8dzROS3YvvdUe3rTV/TZVQXrvvgOmpXrs3cQXP54HcfWIMXvSyv4sy8DfP4/Ye/59yG5zK+33hr8HwSzHnmtwL3AVkikoG7nkhVq4U1MhOUDXs3MHT2UN5Z8Q71qtbjzT5vMrDdQOsIj36WV3EkdXsqfSf2pVliM6YMmGKXIPgomGHITopEIObE7M/cz5NfPsnzC56njJRh2IXD+PO5f6ZK+Sp+h2aCYHkVP/6373+kjEuhavmqTP/9dGpWqln0k0zY2CSyMSYrJ4tRS0fx6NxH2XloJwPbDeSJ7k/QoFoDv0MzJ8jyqvTbfXg3Kf9J4eCRg3x545c0qm4j0/jNJpGNITPXzuT+mfeTtiONCxpdwCeXfkLyqUWerGSikOVV6Xf46GF+M+E3rN2zlpm/n0nbum39Dslgk8jGhFU7V9F7fG8u/c+lHDp6iPf6v8fnN3xuDV5ss7wqxbJyshjw/gDmb5rPuH7juLDJhX6HZJxgDm9mqGqGiPwy2aWIxNRkl7FKVXl1yasMmT6ESmUr8ewlz3JXp7vsYvLSwfKqlFJV7vrkLiavnswLKS9wVeur/A7JBCjVk8jGsgOZB7h16q1MSJtASvMUxvYdy8lVTvY7LBM6llel1P998X+8uuRVhp43lLvPudvvcEwepX0S2ZiUuj2V/pP688PuH3ii+xMMPX+oXYJQylhelU5vLH2Dv877KwPbDWREjxF+h2PyUdSA07cBzfEmuRylqp9HKrB4NWbZGO6YdgfVKlRj9vWzuSjpIr9DMiHkBpw+WURGYnlVqkxdM5Xbpt5GSvMU3vjNGzaOZpQqbPdhLJCMl5i9gH9EJKI4dejoIQZPHsyNk2/knAbnsOy2ZdbglUKDBg0CqILlVamyYPMCrp50NWfXO5tJ/SfZlEBRrLDDm61VtS2AiIwCFkYmpPiz+qfV9J/Un7QdaTxywSMM7zbchigqpVauXAmwXlVftbwqHVb/tJrLx19O/Wr1mXbtNKqWr+p3SKYQhTV6R3MX3NxZEQgn/ryT9g43fXwTFRIq8Ml1n5DSPMXvkEwYlSt3bA/A8ir2/XjgRy79z6UklElg+nXT7WSzGFBYo9dORPa7ZQEqufs2RmAIZGZlct+M+3hp8Uuc2/BcJv52ok3eGgeWL18OcHZALllexah9Gfu4bNxl/HToJz6/4XOa1Wzmd0gmCAU2eqpqx9fCZP2e9fSf1J8lW5dwf5f7ebLHk9YHECeys7MRkW+DmffLRK/MrEz6vduPFTtXMO3aaXQ4tYPfIZkgBTX2pgmdyd9PZtBHgxARPvrdR/Q5vY/fIRljTsCho4fo904/Plv/GW/1fYuezXr6HZI5AXbxV4QczT7K/TPup+87fWlRqwVLb1lqDZ4xMeZA5gF6j+/NzLUzGXXFKK5vd73fIZkTZHt6IaSq7Mvcx65Du9h1eBe7D+/+ZXli2kTmb57PnR3v5B89/2FDiRkTY/Zm7OWycZexcMtCxvUbx4C2A/wOyRSDNXpBmrpmKlv2b2HX4V3sOrSL3RnHGrTcv3sO7yFbs/N9fvUK1Zn424n8rs3vIhy5Maakdh3aRc//9CR1eyrv9n+Xfq36+R2SKSZr9IL0p5l/YvWu1QBULleZWpVqUbNSTWpVrsWZdc887n6tSrWoVdndd8uJFRPt2jtjYtC2n7dxyduXkL47nY+u+YjLWlzmd0imBHxp9ESkPzAcaAV0UtXFAY89CAwGsoF7VHWGK08B/gUkAG+o6lOuPAmYCNQClgDXq+oREakAvAV0AHYBv1PVDcWNedq106hYtiK1KteiYtmKRT/BGB/EYm5Fs837N9PjrR5s3r+ZaddOo3uSTXcY6/w6kSUN6AccN0u0iLQGrgHOAFKAl0QkQUQSgH/jDdvUGhjg1gWO9ngyAAAQcUlEQVR4GnheVZsDe/CSGvd3jyt/3q1XbM1qNqN+tfrW4JloF3O5Fa3W71lP1ze7svXAVmb+fqY1eKWEL42eqq5S1dX5PNQHmKiqmaq6HkgHOrlbuqquU9UjeL8++4g3nEV34D33/LFA34BtjXXL7wE9xIa/MKWc5VZorNm1hq5jurI3Yy9zBs7hvEbn+R2SCZFou2ShPrAp4P5mV1ZQeS1gr6pm5Sk/blvu8X1ufWPikeVWkNJ2pNH1za5kZmUy74Z5dKzf0e+QTAiFrU9PRGYDp+Tz0MOqOjlc9RaHiNwC3ALQqFEjn6MxpnCxkluxmFdLty6l59s9qVC2AvMGzuP02qf7HZIJsbA1eqp6cTGetgUIHICygSujgPJdQA0RKet+cQaun7utzSJSFm+Szl0FxPoa8BpAcnKyFiNuYyImVnIr1vJq/qb59BrXi+oVq/PZwM9sLM1SKtoOb04BrhGRCu7MsRZ4U68sAlqISJKIlMfrkJ+iqgrMBa5yzx8ETA7Y1iC3fBXwmVvfmHhkuVWIeRvmccnbl1CnSh2+vPFLa/BKMV8aPRG5UkQ2A12AaSIyA0BVVwDvAiuB6cCdqprtfmneBcwAVgHvunUBHgDuE5F0vH6FUa58FFDLld8HDI3MqzPGP5ZbJ25G+gx6jetF4xqN+eKGL2hUPTYOxZrikRj/gRZyycnJunjx4qJXNKaYRGRJvM2yEK15Nfn7yVz93tW0qt2KWdfPok6VOn6HZIop2LyKtsObxhgTdqrK2GVjuWrSVZx1ylnMHTTXGrw4YcOQGWPiyraft3HHtDv48PsPubDxhUwZMIVqFWzu3nhhjZ4xJi6oKuNSxzFk+hAOHjnI0xc/zX1d7qNsGfsajCf23zbGlHo/HviRW6feytQ1U+nSoAuj+4y2a/DilDV6xphSS1UZs2wM9864lyPZR3iu53Pcc849NuNJHLNGzxhTKm3at4lbpt7C9PTpXNDoAkZdMYoWtVr4HZbxmTV6xphSRVV5Y+kb3D/zfnI0hxd7vcgdHe+gjNjJ6sYaPWNMKbJh7wZu/vhmZq+bzUVNLmLUFaNISkzyOywTRazRi6Bu3boBMG/evIhur7D1wvHYicZ3ouueiHBt14TXif7fcjSH0647jXXN1lG5cmVe7v0yt3S4pdC9u3B9PkORlyVZNxTPi7Y6QskaPWNMTFu7ey03fXwT6aelk7g7kWV/WmZDiZkCWaNnjIlJew7vYcyyMTwy9xHKlSlHy1UtOWXbKdbgmUJZo2eMiQk5msPSrUuZnj6dT9M/ZcHmBeRoDr1b9ObVy1/luiuu8ztEEwOs0TPGRK2dB3cyc+1Mpq+dzoz0Gew8tBNBSD41mYcveJhezXvRuUFnRMTvUE2MsEbPGBM1snKyWLhlIeuT1rOn5h7qPlsXRalTuQ6XNr+UlGYp9GzW0waHNsVmjZ4xxldbD2xlevp0pq+dzqy1s9iTsQcaQ7X91Xj8osdJaZ5C+3rt7To7ExLW6BljfDN48mBGLxsNQL2q9eh7el9SmqfwryH/olxWOR557hGfIzSljU0im4eI7AQ2BhTVBn7yIRQ/6rXXGpl6q6hqXB2fyyevihKJ/0+kPgNWT2TqaBxMXlmjVwQRWezHLNd+1GuvtfTWG2si8T5F6n9h9URXHXaQ3BhjTNywRs8YY0zcsEavaK/FUb32WktvvbEmEu9TpP4XVk8U1WF9esYYY+KG7ekZY4yJG9boOSKSIiKrRSRdRIbm8/htIpIqIstE5CsRaR3uOt06V4vIShFZISLjS1pnMPWKSGMRmSMi34nIPBFpEII6R4vIDhFJK+Dx61x9qSLytYi0i0Cd3URkn/ufLhORv5a0ziDrrS4iH4vIcvd/vTEU9cYSERkiImnu9f/RldUUkVki8oP7m+jKRURecJ/X70SkfSHb/dV7LyLPiMj37rkfikiNgMcedNtdLSKXBpQXlSP51TNcRLYEfJ4uK0k9BdRxlogscNtfLCKdinqPRGSQe09/EJFB+byWhiIyN+B7Zogr7+/u54hIcp7nFOf15FtPwOP3i4iKSO2SvqZCqWrc34AEYC3QFCgPLAda51mnWsDyFcD0CNTZAvgWSHT3T47Qa50EDHLL3YG3Q1BvV6A9kFbA4+cGvM5ewDcRqLMbMDUMn6ei6n0IeNot1wF2A+VDHUe03oA2QBpQGW+AjNlAc+DvwFC3ztCA9+gy4FNAgM6FfTbye++BnkBZt/x0wHZbu89/BSDJ5UVCkDmSXz3DgT/lE1Ox6imgjplAr4D3ZV5h7xFQE1jn/ia65cQ88dUD2rvlk4A1LuZWwGnAPCA5BK8n33rc/YbADLxrOWuX9DUVdrM9PU8nIF1V16nqEWAi0CdwBVXdH3C3ClDSztAi6wRuBv6tqntcDDtKWGew9bYGPnPLc/N5/ISp6hd4X+4FPf517usEFgAl3rssqs5wCaJeBU4SEQGqunWzIhFblGiF9wV2SFWzgM+Bfnifs7FunbFAX7fcB3hLPQuAGiJSL78N5/feq+pMVw8c/9nqA0xU1UxVXQ+k4+VHMN8HJ/LZKlY9BdShQDW3XB34MaCO/N6jS4FZqrrb5dcsICXPa9mqqkvd8gFgFVBfVVep6uoQvp5863EPPw/8heO/V4v9mgpjjZ6nPrAp4P5mjv0zfiEid4rIWrxfpPdEoM6WQEsR+a87pBH0P7aE9S7H+xICuBLvC7pWCOoO1mC8X3iR0MUdZvxURM6IUJ0j8b74fwRSgSGqmhOhuqNBGnCBiNQSkcp4v+gbAnVVdatbZxtQ1y0HlZ9B+gPHPlsFbbck9d3lDsWNzj08G+J6/gg8IyKbgGeBB0NZh4g0Ac4GvikkhhLXFViPiPQBtqjq8lDXkx9r9E6Aqv5bVZsBDwCRGBSwLN4hzm7AAOD1wP6IMPoTcKGIfAtcCGwBsiNQLyJyEV6j90AEqluKN3RRO+BF4KMI1AneL9VlwKnAWcBIEalW+FNKD1VdhXeYcSYwHe+9yM6zjlLyoynHEZGH8faox4VyuwFeBprh/U+3Av8IQx23A/eqakPgXmBUqDYsIlWB94E/5jmyFVKB9eD9Px4CQtKfHgxr9Dxb8H5p5mrgygoykWOHXsJZ52ZgiqoedYcR1uA1gmGtV1V/VNV+qno28LAr21vCeoskImcCbwB9VHVXuOtT1f2q+rNb/gQol9uJHmY3Ah+4wzbpwHrg9AjUGzVUdZSqdlDVrsAevM/29tzDlu5v7uH8E83PXxGRG4DLgetcg1rYdotVn6puV9Vst9f+Ot7hvlDXMwj4wC1PClUdIlIOryEap6of5H08j2LXlU89zfD6BZeLyAb3nKUickpJX1OBgu38K803vD2qde7Nz+2APSPPOi0Cln8DLI5AnSnAWLdcG2+XvlYE6q0NlHHLTwCPh+h9bkLBJ3c0wusbODfE/9vC6jyFY9eqdgL+l3s/zPW+DAx3y3VdwtYO5euO9hvupCz3f/8eqAE8w/EnsvzdLffm+BMaFp7Ie+/yaCVQJ896Z3D8CRnr8E7GKDJHCqinXsDyvXj9XiWqJ586VgHd3HIPYElh7xHeyR7r8U74SHTLNfPUIcBbwD8LeD/ncfyJLMV6PUXV49bZwLETWYr9mgr9fPj94Y+WG16/whq8s48edmWPA1e45X8BK/AOxczNLwnCUKcAz7mETQWuidBrvQr4wa3zBlAhBHVOwDvkcxRvD3YwcBtwm3v8Dbxf/MvcrUQ/KoKs8y73P12Od4JDSBrcIOo9Fe/QXipe/9bv/f78R/oGfOk+18uBHq6sFjDHffZm536RuTz4t/u8phLwBRzke5+O94Mx97P1SsD6D7vtrsadFVlQjgRRz9suvu+AKRzfCJ5wPQXUcT6wxL1v3wAdinqP8Pox093txnxey/l4h5K/C3iPLsPrz98MZALbgRklfD351pNnnQ0ca/SK/ZoKu9mILMYYY+KG9ekZY4yJG9boGWOMiRvW6BljjIkb1ugZY4yJG9boGROk/AYADuI5v3WD6CYXvbYxJtys0YtTIvLzCazbV/LMKiEiZUVkp4g8FfrootYYTmCMPxE5CRhC4UM6mRglInVFZLyIrBORJSIyX0SuzGe9Jvn9UBKRx0Xk4iDqOcv9cArFMIRxzxo9E4y+eINQB7oE73qc/m7g5F8RkYRwBxZJms8AwCLSTESmuy+9L0UkcGSVv+ENt5URyThN+LnP/EfAF6raVFU7ANeQZ6B0ESlb0DZU9a+qOjuI6gYAX7m/+cYiIvZdHiR7o8wv3C/Sz9yAuXNEpJGInIs3ldIz4s3h1cytPgDvgv3/AV0CtrFBRJ4WkaV4DWK+jYKI/EZEvhGRb0VktojUJTa9BtztvvT+BLwE4Ob+aqiq0/wMzoRNd+CIqr6SW6CqG1X1RRG5QUSmiMhneBfb50tExojIVeLNQTcpoLybiEx1ywL0B24ALhGRiq68iXjz1r2FN8BBQxHp6fY2l4rIJDfGJSLyVxFZJN78ha8V9CM1XlijZwK9iDfs2Zl4g/K+oKpf440u8WdVPUtV17rEuxj4GG/UiLy/QHepantVnUgBjQLeL9fO6o3vORFvWpGY4r5UzgUmicgy4FWgnvvV/Rxwv5/xmbA6A2/A8oK0B65S1QuD2NZs4BwRqeLu/w4vJ8D7fK1X1bV4w4H1DnheC+AlVT0DOIg3CP7FqtoeWAzc59YbqaodVbUNUAlvDNK4ZY2eCdQFyJ2d/W28YYPyczkwV1UP4w0e2zfPocx3oOBGwa3TAJghIqnAn/G+RGJNGWCv+zGQe2uFN0FmG2CeG0S3MzDFTmYpvUTk3+JNUbXIFc1S1aDm21Nvrr/pwG/c4dDewGT38ACONYATOf4H5kb15pkD7zPWGvivy7VBQGP32EXuqEoq3h5qLOZayBR4vNmYQgwAzndf6OCNmdgdbzJH8H51QkCjkM82XgSeU9UpItINb9bpmKKq+0VkvYj0V9VJ7rDRmerNC/bLbA0iMg9vRu3FfsVqQm4F8NvcO6p6p5uhI/d/fDDfZxVsIt5YsLvxxp094H5I/hboI960SALUcidI5a1D8Bra4466uKMyL+GNW7lJRIYDFU8wtlLF9vRMoK/xOuMBrsMbFBjgAN7eC+LN+3YB0EhVm6hqE+BO8ulkV29OrvUi0t89V0SknXu4OsemAxkU+pcSeiIyAZgPnCYim0VkMN77NFhEluN9EZZ4lnkTEz4DKorI7QFllUuwvc/xDonezLE9ux7Ad6ra0OVaY7wjK786QxRvwPTzRKQ5gIhUEZGWHGvgfnJHXq4qQYylgu3pxa/KIrI54P5zwN3AmyLyZ2An3rxv4CXh6yJyD94Za5+pambAcycDfxeRCvnUcx3wsog8ApRz21qOt2c3SUT24H2BJIXslYVJ3l/RAQo9lVxVu4U+GuMnVVUR6Qs8LyJ/wcuXg3iTH1fK5ymn5cm3e/NsL9udvHIDx34EDgA+zLOd9/Emkv0iz/N3ijdn4ISAPHxEVdeIyOt4J7tsAxYR52yWBWOMMXHDDm8aY4yJG9boGWOMiRvW6BljjIkb1ugZY4yJG9boGWOMiRvW6BljjIkb1ugZY4yJG9boGWOMiRv/D0THNgLTbGasAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbcAAADPCAYAAAB2kZ/zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8VFX6+PHPQy+CdAQRAQVdywoYiqIuggK6KCJFUNEVFUVsuK7o6k/ZRYVFFiILiKisICJVioogKBZQWpBepH8JIigIUXqS5/fHPZExmzJJZuZOZp736zWvuXNuOc+d5MnJbeeIqmKMMcbEkiJ+B2CMMcaEmjVuxhhjYo41bsYYY2KONW7GGGNijjVuxhhjYo41bsYYY2KONW7GGGNijjVuxhhjYo41bsYYY2JOMb8D8EuVKlW0Tp06fodhCqmjp46y6adNVChVgXoV62W5TFJS0k+qWjXCofnK8sqEW7B5FbeNW506dVixYoXfYZhC6OCxg1w+5nJqpdciqVcSVctmnWcisivCofnO8sqEW7B5FbeNmzH5ka7p9JjRgz0pe/jqnq+ybdiMMf6yxs2YPHjxyxeZs2UOo24cRbNazfwOxxiTDbuhxJggzd06l/6f9+fOP97JgwkP+h2OMSYH1rgZE4Sdh3Zyx/t3cEm1S3i9/euIiN8hGWNyYI2bMbk4nnqczlM6k5qeyvSu0ylTvIzfIRljcmHX3IzJxWMfP0bS3iRm3jaT+pXr+x2OMSYIduRmTA7eXvU2Y1aO4ekWT9Phwg5+h2OMCZI1bsZkY9UPq+j9UW9a1W3FgFYD/A7HGJMH1rgZk4Wfj/3MrZNvpXLpyrzX6T2KFbEz+MYUJpaxxmSS8aB2ckoyX97zJdXKVvM7JGNMHlnjZkwmL3/1Mh9t+YgRN4ygea3mfodjjMkHOy1pTIBPtn3C8wuf545L7+ChJg/5HY4xJp+scTPG2XVoF7dPv52Lq11sD2obU8hZ42YMcCL1BF2mduFU+immd51O2RJl/Q7JGFMAds3NGODxuY+z/PvlvN/1fRpUbuB3OMaYArIjNxP3xq8ez+ik0Tx15VN0/ENHv8MxxoSA742biOwUkbUiskpEVriySiIyX0S2uPeKrlxEZLiIbBWRNSLSOGA7d7vlt4jI3X7tjylcVv+wmgc+fIBr61zLS61f8juckLG8MvHO98bNuVZVG6pqgvv8NPCpqtYHPnWfAW4A6rtXL+A18JIWeAFoBjQFXshIXGOyc/j4YTpN6USl0pVi9UFtyysTt6KlccusAzDOTY8DbgkoH6+eJUAFEakBtAXmq+pBVf0ZmA+0i3TQpnB55etX2P7zdqZ2mUr1M6r7HU4kWF6ZuBENjZsCn4hIkoj0cmXVVXWvm/4ByPjLczawO2DdZFeWXbkxWUo5kcKIZSPo+IeOXHnOlX6HEw6WVyauRcN5mKtUdY+IVAPmi8imwJmqqiKioajIJXkvgNq1a4dik6aQGr1iNIdPHOaZq57xO5Rwsbwycc33IzdV3ePe9wMz8M7t73OnRXDv+93ie4BzAlav5cqyK89c1xhVTVDVhKpVq4Z6V0whcTz1OMOWDOO6eteRUDMh9xUKIcsrE+98bdxEpKyIlMuYBtoA64DZQMadWXcDs9z0bOAud3dXc+CwO80yD2gjIhXdBe82rsyY//H2qrf54dcfYvaozfLKGP9PS1YHZrhujooBE1V1rogsB6aIyL3ALqCrW34OcCOwFTgK3AOgqgdFZACw3C33T1U9GLndMIVFanoqgxcPpunZTbm2zrV+hxMullcm7vnauKnqduCyLMoPAK2zKFegTzbbGguMDXWMJrZMWT+FHYd2MLTt0JjtO9LyypgouOZmTKSoKoMWDeKiqhdx8wU3+x2OMSaM/D4taUzEzNkyh7X71zLulnEUEfu/zphYZhlu4sbARQM598xz6X5Jd79DMcaEmTVuJi58tesrFu9ezJNXPknxosX9DscYE2bWuJm4MHDRQKqWqUrPRj39DsUYEwFBNW4icq6IXOemS2c8Q2NMYbDqh1V8vPVjHmv2GGWKl/E7nN9YXhkTPrk2biJyPzANeN0V1QJmhjMoY0Jp0KJBlCtRjj5Ns7zb3ReWV8aEVzBHbn2AFkAKgKpuAaqFMyhjQmXrwa1M3TCV3gm9qVCqgt/hBLK8MiaMgmncTqjqyYwPIlIMr8dxY6Le4MWDKV6kOH2v6Ot3KJlZXhkTRsE0bl+IyN+B0iJyPTAV+CC8YRlTcN//8j3jVo/jnob3cNYZZ/kdTmaWV8aEUTCN29PAj8Ba4AG8fuieC2dQxoTC0G+Gkpqeyt9a/M3vULJieWVMGAXTQ0lpYKyqvgEgIkVd2dFwBmZMQRw8dpDRK0bT7ZJu1KtYz+9wsmJ5ZUwYBXPk9ile0mUoDSwITzjGhMbIZSM5cuoIT7d42u9QsmN5ZUwYBdO4lVLVXzM+uOnoeVjImEyOnDzCq0tfpX2D9lxa/VK/w8mO5ZUxYRRM43ZERBpnfBCRy4Fj4QvJmIJ5c+WbHDh2INoHI7W8MiaMgrnm9jgwVUS+BwQ4C7gtrFEZk08n004y5JshXHPuNVx5zpV+h5MTyytjsrDhxw0Iwh+q/qFA28m1cVPV5SJyIXCBK9qsqqcKVKsxYfLumndJTknmjZve8DuUHFleGfO/jp46SpepXTieepzND2+mWJH8j8oW7JpNgDpu+cYigqqOz3etxoRBWnoa/1r8Lxqe1ZC257X1O5xgWF4ZE6Dv3L5s+HED8+6cV6CGDYLrW/IdYAhwFV4yNgESClSrt91zRGShiGwQkfUi8pgr7y8ie0RklXvdGLDOMyKyVUQ2i0jbgPJ2rmyriETt7XEmvGZumsnmA5t5usXTiIjf4eQoXHnltm25ZQqdqeunMmblGJ668inanNemwNsLpmlMAC5S1VB3DZQK/FVVV7re0JNEZL6bN0xVhwQuLCIXAd2Ai4GawAIRaeBmjwSuB5KB5SIyW1U3hDheE8VUlYGLBnJ+pfPpfFFnv8MJRrjyCiy3TCGz89BO7v/gfpqd3YwXW70Ykm0G07itw7vYvTckNTqqujdjm6r6i4hsBM7OYZUOwCRVPQHsEJGtQFM3b6uqbgcQkUluWUvAOLJg+wKS9iYxpv0YihYp6nc4wQhLXoHllilcTqWdovv07ijKe53eC9lgwsE8ClAF2CAi80RkdsYrJLU7IlIHaAQsdUUPi8gaERkrIhVd2dnA7oDVkl1ZduUmjgxcNJCa5Wpy12V3+R1KsMKeV2C5ZaLfC5+/wJLkJYxpP4a6FeuGbLvBHLn1D1ltWRCRM4DpwOOqmiIirwED8HpIHwD8GwjJ8Mki0gvoBVC7du1QbNJEgaXJS1m4cyFDrh9CyWIl/Q4nWP3DXUGkcsvyyuTXgu0LGLRoEPc1uo/bLgntkzC5Hrmp6hfATqC4m14OrAxF5SJSHC/53lXV9119+1Q1TVXTgTc4fXpkD3BOwOq1XFl25VntyxhVTVDVhKpVq4ZiF0wUGLhoIBVLVaTX5b38DiVo4cwriGxuWV6Z/Nh/ZD89ZvTgwioX8uoNr4Z8+/kZiftsQjBisHi3s70FbFTVoQHlNQIW64h3bQJgNtBNREqKSF2gPrAM749CfRGpKyIl8C6Mh/z0jolOG37cwKzNs3ik6SOUK1nO73CCFq68ctu23DJRLV3TuXvm3fx87Gcmd55MmeKh73kumNOSffD+w1sK3ojBIhKKEYNbAD2AtSKyypX9HeguIg3xTp3sxBsOBFVdLyJT8C5mpwJ9VDUNQEQeBuYBRfF6Wl8fgvhMIfCvxf+iTPEyPNrsUb9Dyatw5RVYbpkoN+ybYczdOpeRN44MW/+vwTRuJ1T1ZMZzQ6EaMVhVF+F1O5TZnBzWeQl4KYvyOTmtZ2LTnpQ9TFw7kYcSHqJymcp+h5NXYckrsNwy0W35nuU8/enTdLywI70TeoetHhuJ2xRaI5ePJC09jceaP+Z3KPlheWXiTsqJFLpN70aNM2rw5s1vhrWzBRuJ2xRKR04e4fWk1+n4h47ROhhpbiyvTFxRVXp/1Judh3YysdNEKpWuFNb6guk4OePOqujuidbElfGrx3Pw2EH6Nu/rdyj5Ynll4s241eOYuHYiA64dwFW1rwp7fdk2biKylhyuAajqH8MSkTG5SNd0Epcm0qRmE1qc08LvcPLE8srEo00/baLPnD60rNMyYuMs5nTk1t6993Hv77j3OwnRhW9j8uPjLR/z3YHvmHjrxKjvIDkLllcmrhxPPU63ad0oXaw0EzpOiFj3eNk2bqq6C0BErlfVRgGz+onISrxrBsZE3NAlQ6lVvlZh6SD5dyyvTLx5av5TrN63mg+6f8DZ5SPXe1swN5SIiLQI+HBlkOsZE3Krf1jNZzs+45Gmj4Ssg1WfWF6ZmDdr0yz+s+w/PN7scdo3aJ/7CiEUzHNu9wJjReRMvGdnfiZEfT0ak1fDlgyjbPGy3N/4fr9DKSjLKxPTklOS6Tm7J43OasSg6wZFvP5g7pZMAi5zSYiqHg57VMZkYe8ve5m4diIPXP4AFUtXzH2FKGZ5ZWJZWnoad7x/BydSTzCp8yRfOjTPtXETkZJAJ6AOUCzjAr6q/jOskRmTyajlo0hNTy2sD23/juWVCbcDRw+QciLFl7rfXPkmX+76kvG3jKdB5Qa5rxAGwZyWnAUcBpKAE+ENx5isHTt1jNdWvMbNF9zM+ZXO9zucULC8MiGnqizYvoCRy0fywXcfkK7pvsXS44896HFZD9/qD6Zxq6Wq7cIeiTE5eGfNOxw4dqDQPrSdBcsrEzIpJ1IYt2ocI5ePZPOBzVQpU4WnrnyKC6tc6Es8pYuX5uYLbval7gzBNG5fi8ilqro27NEYk4V0TSdxSSKNazTmmnOv8TucULG8MgW2fv96Ri4fyTtr3uHXk7/S7OxmjL9lPF0u7kKpYqX8Ds9XwTRuVwF/EZEdeKdPBFDrScFEyryt89j400be6fhOYXxoOzuWVyZfUtNTmb15NiOWjWDhzoWULFqSbpd0o0+TPjQ5u4nf4UWNYBq3G8IehTE5GLZkGDXL1aTrxV39DiWULK9Mnuw/sp83kt5gdNJoklOSqX1mbQa1HsS9je+lSpkqfocXdYJ5FGCXiFwF1FfV/4pIVeCM8IdmDKzdt5b52+fzcquXKVG0hN/hhIzllQmGqrJ0z1JGLBvB1A1TOZl2kuvqXceIG0bQvkH7iHVlVRgF8yjAC0ACcAHwX6A4MAFvtF9jwipxSSJlipfhgYQH/A4lpCyvTG5WfL+CBz98kKS9SZQrUY4HLn+Ah5o85NtNIoVNMKclOwKNgJUAqvq9iJQLa1TGAPt+3ceEtRO4t9G9YR/7yQeWVyZb+4/sp8OkDgjCyBtH0uOPPShX0n498iKYxu2kqqqIKICIlA1zTMYA8NqK1ziZdpLHmz/udyjhYHllspSWnkaPGT04cPQAS+5bQsOzGvodUqEUTEetU0TkdaCCiNwPLCAKB1gUkXYisllEtoqI9axeyB1PPc6o5aNo36C9bz0chJnllcnSy1+9zCfbPmH4DcOtYSuAYG4oGSIi1wMpQAPgeVWdH/bI8kBEigIjgeuBZGC5iMxW1Q3+Rmby69017/Lj0R95ovkTfocSFpZXJiuf7fiM/l/0545L74iFzsF9FcxpSYC1QGm8wRSj8aHTpsBWVd0OICKTgA6AJWEhpKoMWzKMy6pfRss6Lf0OJ5wsr8xv9v6yl9un306Dyg0Y3X50LD3T6YtcT0uKyH3AMuBWoDOwRESibWiOs4HdAZ+TXZkphOZvn8/6H9fzxBVPxGyCW16ZQKnpqdz+/u2knEhhapepnFHCngopqGCO3P4GNFLVAwAiUhn4GhgbzsDCQUR6Ab0Aateu7XM0JjtDvxnKWWecRbdLuvkdSjhZXpnf/OPzf/D5zs95u8PbXFLtEr/DiQnB3FByAPgl4PMvriya7AHOCfhcy5X9jqqOUdUEVU2oWrVqxIIzwdvw4wbmbZvHw00ejqmHtrNgeWUAr3u5l756iXsa3sPdDe/2O5yYEcyR21ZgqYjMwrs20AFYIyJPAKjq0DDGF6zlQH0RqYuXfN2A2/0NyeRH4pJEShUrFXMPbWfB8sqQnJLMnTPu5OJqFzPixhF+hxNTgmnctrlXhlnuPWqeKFTVVBF5GJgHFAXGqup6n8MyefTjkR8Zv3o8f2n4l3joK8/yKs6dSjtFt2ndOHbqGFO7TKVM8TJ+hxRTgnkU4B8AIlJGVY+GP6T8UdU5wBy/4zD5N3rFaE6knYjVh7Z/x/LKPPfZcyzevZh3b33XutQKg2DulrxCRDYAm9zny0RkVNgjM3HlROoJRi4fyY31b4yLRLe8im8ffvchg78ezIOXP8jtl9qZ3nAI5oaSRKAt7mK3qq4GYmbESBMd3lv3HvuO7IulkbZzY3kVp3Yd2sVdM+6i0VmNGNZumN/hxKxgGjdUdXemorQwxGLilKoy9JuhXFrtUlrXbe13OBFjeRV/TqadpOu0rqRpGlO6TIn70bLDKZgbSnaLyJWAikhx4DFgY3jDMvHksx2fsXb/WsbePDZmH9rOguVVHOo3vx/L9ixjWpdpnF/pfL/DiWnBHLk9CPTB65lgD9DQfTYmJIYuGUr1stXj7dqD5VWceX/j+yQuTeTRpo/S6aJOfocT84K5W/In4I4IxGJ8suvQLhqPaczPx34u0HaKSJF8vXan7OYfLf9ByWIlQ7RH0c/yKr5sO7iNnrN60qRmE15p84rf4cSFbBs3EfkP3sOlWVLVR8MSkYm44UuHk3IihWeueibfw9arKoqSrul5fpUsWpJHmj4S4r2KTpZX8ed46nG6TuuKiDCly5RY73knauR05LbCvbcALgImu89dsF7BY8YvJ37hzW/fpMtFXXip9Ut+hxMPLK/izF/n/ZWVe1cyq9ss6lSo43c4cSPbxk1VxwGISG/gKlVNdZ9HA19FJjwTbm+vepuUEylx8eB0NLC8KlzSNZ2NP24kNT01X+t/k/wNo1aM4skrnuTmC24OcXQmJ8HcLVkRKA8cdJ/PcGWmkEtLT+PVpa9yRa0raHp2U7/DiTeWV1Hup6M/cfv025m/vWBjyF55zpW83PrlEEVlghVM4zYI+FZEFgKC96Bp/3AGZSLjoy0fse3nbQxsPdDvUOKR5VUUW7ZnGZ2ndGbfkX0Mvm5wvm/bLyJFaF2vNcWLFg9xhCY3wdwt+V8R+Rho5or6qeoP4Q3LRELikkTOKX8OHf/Q0e9Q4o7lVXRSVV5Pep3H5j5GjTNqsLjnYhJqJvgdlsmHYI7ccEk3K9cFTaGx+ofVLNy5kMHXDaZYkaB+DUyIWV5Fl6OnjtL7o96MXz2edue3Y0LHCVQuU9nvsEw+2V+1OJW4NJGyxctyX+P7/A7FGN9tPbiVTlM6sXbfWl740ws8/6fnKSJB9U5oopQ1bnFo36/7mLh2Ivc3vp+Kpe0eBhPfZm+ezV0z7qKIFOGj2z/ihvo3+B2SCYGcHuKulNOKqnowp/kmeo1eMZqTaSd5tJk9LxxpllfRIzU9lecXPs/ARQO5vMblTOs6zZ5DiyE5Hbkl4fWkkFVPtgrUC0tEJqyOpx5n1IpR/Ln+n2lQuYHf4cQjy6sosP/IfrpP785nOz7j/sb3M/yG4dZDf4zJ6SHuupEMxETGpHWT2H9kfzyNmxZVLK/8tyR5CZ2ndObAsQOMvXks9zS6x++QTBgEdc1NRCoC9YHf/rVR1S/DFZQJD1UlcUkil1S7hFZ1W/kdTtyzvIosVWXU8lH0ndeXWuVr8XXPr2lUo5HfYZkwyfV2IBG5D/gSmAf8w733L0ilIvKKiGwSkTUiMkNEKrjyOiJyTERWudfogHUuF5G1IrJVRIaLG/hLRCqJyHwR2eLe7Q6JbHy+83NW71vN480ej6dx06JSOPLKbddyKwtHTh6hx4wePPzxw7Q5rw1JvZKsYYtxwdzr+hjQBNilqtcCjYBDBax3PnCJqv4R+A54JmDeNlVt6F4PBpS/BtyP959ufaCdK38a+FRV6wOfus8mC4lLE6lSpgp3/NFGWokC4cgrsNz6H98d+I7mbzVn4tqJDLh2ALO7z7a7hONAMI3bcVU9DiAiJVV1E3BBQSpV1U8yOowFlgC1clpeRGoA5VV1iaoqMB64xc3uAIxz0+MCyk2ArQe38sHmD+id0NsunEeHkOcVWG5l9uvJX2k1rhV7f9nL3Dvn8tw1z9nza3EimGtuye7Uxkxgvoj8DOwKYQw9OT3sB0BdEfkWSAGeU9Wv8EYrTg6MyZUBVFfVvW76B6B6CGOLGf9Z+h+KFSlG74TefodiPOHOK7DcYvDiwez5ZQ+Ley7mynOu9DscE0HB9C2Z0fFgf9fJ65nA3NzWE5EFwFlZzHpWVWe5ZZ4FUoF33by9QG1VPSAilwMzReTi3Hfjt1hVRLIdCFJEegG9AGrXrh3sZgu9w8cPM3bVWLpd0o0a5Wr4HY4h/3kF0Zdb0ZpXuw/vZsjXQ7jt4tusYYtDOT3EXV5VUzI9dLrWvZ/B6aE6sqSq1+U0X0T+ArQHWrvTIajqCeCEm04SkW1AA2APvz+9UsuVAewTkRqqutedYtmfQ0xjgDEACQkJ2TaCseatb9/i15O/2phtUaCgeQXRl1vRmld//+zvpGs6g64b5Hcoxgc5nXye6N6T8EYPzvyebyLSDngKuFlVjwaUVxWRom66Ht7F7e3u1EiKiDR3d3LdxekOZ2cDd7vpu7GOaH8nNT2V4UuHc82519C4RmO/wzFhzCuw3MqwfM9yJqyZQN/mfa3XkTiV00Pc7d17OB46HQGUxLvWALDE3b11DfBPETkFpAMPBnRH9BDwNlAa+Ni9wBsXa4qI3It3zaJrGOIttGZtmsWuw7sY1naY36EYwp5XYLmFqtJ3Xl+qla3GM1c/k/sKJibles1NRD5V1da5leWFqmY58p+qTgemZzNvBXBJFuUHgHzHEusSlyZSt0JdG+I+yoQjr8ByC2Dahmks3r2YMe3HUL5keb/DMT7J6ZpbKaAMUMU9vJnx1G95Tt9NZaLYiu9XsOj/FjGs7TCKFinqdzgGy6twO556nH4L+nFptUvp2ain3+EYH+V05PYA8DhQE+96QEYSpuCd+jBR7tWlr1KuRDlL8uhieRVGw5cOZ8ehHczvMd/+oYtzOV1ze1VERgB/V9UBEYzJhMD3v3zPpHWT6NOkj52aiSKWV+Gz/8h+XvzyRdo3aM919XK8odTEgRwf1VfVNODWCMViQmjU8lGkpafZmG1RyPIqPF5Y+ALHUo/xyvWv+B2KiQLB9EPzqYh0yuhM1US/Y6eOMXrFaDpc2IF6FW14sChleRVC6/avY8zKMfRO6M2FVS70OxwTBYJp3B4ApgInRCRFRH4RkZQwx2UKYMKaCRw4doDHm9lD21HM8iqEnvzkScqXLM8Lf3rB71BMlAim+61ykQjEhIaqkrg0kUZnNeKac6/xOxyTDcur0Pl4y8fM2zaPoW2GUrlMZb/DMVHCBiuNMQu2L2DDjxsYd8s4G7MtylleFdyptFP89ZO/cn6l8+nTtI/f4ZgoEsxD3PfhjT1VC1gFNAe+AWwo5yiUuDSR6mWrc9vFt/kdismB5VVovLHyDTb+tJEZt82gRNESfodjoohfg5WaMNj00ybmbJnDQ00eomSxkn6HY3JmeVVAh44f4vmFz9OyTks6XNDB73BMlPFlsFITHsOXDqdk0ZI8mPBg7gsbv1leFdCLX77IwWMHGdpmqJ2CN/8jGgYrNSFw8NhBxq0exx2X3kG1stX8DsfkzvKqALYd3MbwpcP5S8O/0KhGI7/DMVEobIOVmrxJOZFC7496s/vw7nytf+DYAY6eOmpjthUSllcF89SCpyhRtAQvtXrJ71BMlMqt4+QHgfPxBlN8S1W/iFRg8SQtPY3u07szb+s8rj73aoS8n2KpXrY6NzW4iUurXxqGCE2oWF4V3Je7vuT9je8z4NoBNrK8yVZOR27jgFPAV8ANwEV4F8FNiPVb0I85W+bw2p9fs+tlsc/yqgDSNZ0n5j1BrfK1eOKKJ/wOx0SxnBq3i1T1UgAReQtYFpmQ4stbK9/i39/8m0eaPmINW3ywvCqAd1a/Q9LeJCZ0nECZ4mX8DsdEsZzuljyVMaGqqRGIJe58uetLen/UmzbntWFo26F+h2Miw/Iqn46cPMLfP/s7TWo2oful3f0Ox0S5nI7cLgvo606A0u6zAKqqNo5KAWz/eTu3Tr6VehXrMbnzZIoVCaqzGFP4WV7l0ytfv8L3v3zPlM5TKCLBPMVk4llO47nZSH9hknIihZveu4l0TeeD7h9QoVQFv0MyEWJ5lT/JKckMXjyYrhd3pUXtFn6HYwoBX/79EZH+IrJHRFa5140B854Rka0isllE2gaUt3NlW0Xk6YDyuiKy1JVPFpGo7oMn487I7w58x/Su06lfub7fIZkYEqu59exnz5KmaQxqPcivEEwh4+e5sGGqOiSwQEQuAroBFwM1gQUi0sDNHglcDyQDy0VktqpuAP7ltjVJREYD9wKvRWon8uqp+U8xZ8scRv95NNfWvdbvcExsirrcuvP9O/n2h2/ztTOqysafNtKvRT/qVqybr22Y+BNtF3o6AJNU9QSwQ0S2Ak3dvK2quh1ARCYBHURkI15Hs7e7ZcYB/YnSxu2tlW8xdMlQHm36KA8kPOB3OCa++JpbdSrU4UTaiXwH37pua569+tl8r2/ij5+N28MichewAvirqv4MnA0sCVgm2ZUB7M5U3gyoDBwKuOsscPn/ISK9gF4AtWvXDsU+BO2LnV/Q+6PetD2vLf9u+++I1m3iTkRzK5i8erHVi/naEWPyK2zX3ERkgYisy+LVAe+/v/OAhsBeICJ/7VV1jKomqGpC1apVI1El4N0Z2WlKJ86rdB6TOk+yOyNNgURbbvmVV8bkJGx/ZVX1umCWE5E3gA/dxz3AOQGza7kysik/AFQQkWLuP8zA5aPC4eOHuem9m1DU7ow0IWG5ZUzu/LpbMrBDuI7AOjc9G+gmIiVFpC7eKMXLgOVAfXf3Vgm8C+OzVVWBhUBnt/7dwKzbs7CaAAAOW0lEQVRI7EMwAu+MnNZlGudXOt/vkEyMi5fcMiY3fp0fGywiDQEFdgIPAKjqehGZAmwAUoE+qpoGICIPA/OAosBYVV3vttUPmCQiLwLfAm9Fckdy8rf5f+PjrR/zevvX7c5IEylxkVvG5Ea8f9DiT0JCgq5YsSJs239z5Zvc/8H9PNbsMRLbJYatHhO9RCRJVRP8jiOSwp1XxgSbV9aHTRh8vvPz3+6MHNJmSO4rGGOMCSlr3EJs28FtdJrSifqV6lufkcYY4xNr3EIo485IgA+6f8CZpc70OSJjjIlPdlgRIifTTtJ5ame2HNzC/B7zOa/SeX6HZIwxccsatxBI13TumXUPC7YvYNwt42hZp6XfIRljTFyz05Ih0G9+PyauncjA1gO567K7/A7HGGPinjVuBZS4JJEh3wzh4SYP069FP7/DMcYYgzVuBTJ53WT6zutL54s6k9guERHxOyRjjDFY45ZvC3cs5K6Zd3HNudfwTsd3KFrEBlg2xphoYY1bPqz+YTW3TL6F+pXqM/O2mZQqVsrvkIwxxgSwxi2Pdh3axQ3v3kD5kuWZe+dcKpau6HdIxhhjMrFHAfLgwNEDtJ3QlmOpx1h0zyJqla/ld0jGGGOyYI1bkI6eOspN793EzkM7md9jPhdXu9jvkIwxxmTDGrcgpKan0n16d5YkL2Fa12lcfe7Vv5vfsmVLAD7//PPIB5dJQWPJy/p5rSsU31Oov+tgtpd5mWj6eZuCy8/vQKS2n9fyYOvLy3L5EQ05Yo1bLlSVPh/1Yfbm2Yy4YQS3/uFWv0MyxhiTC7uhJBcDvhzAmJVjeOaqZ+jTtI/f4RhjjAmCNW45eHPlm7zw+QvcfdndvNTqJb/DMcYYEyRr3LLx4Xcf8uCHD9Lu/Ha8cdMb1vuIMcYUIta4ZWFJ8hK6Tu1KoxqNmNplKsWLFvc7JGOMMXlgjVsmm3/aTPuJ7alZriYf3f4RZ5Q4w++QjDHG5JE1bgEOHD1Au3fbUUSKMPfOuVQrW83vkIwxxuSDqKrfMfhCRH4EduVxtSrAT2EIJ1rrjde6Q1XvuapaNQTbKTRyySs/f5eiJQa/64+FGILKq7ht3PJDRFaoakK81Buvdfu5z7EsGr5Xv2Pwu/54isFOSxpjjIk51rgZY4yJOda45c2YOKs3Xuv2c59jWTR8r37H4Hf9ECcx2DU3Y4wxMceO3IwxxsScuG7cRGSsiOwXkXUBZZeJyDcislZEPhCR8q78ehFJcuVJItIqYJ3LXflWERkuQfTVlZe6A+bXFpFfReTJgLJ2IrLZ1f10qOsVkT+6eevd/FKR2GcRKS4i41z5RhF5pgD7fI6ILBSRDW4/HnPllURkvohsce8VXbm4fdoqImtEpHHAtu52y28Rkbtzq9vk/edVgHpC9nMuYBxFReRbEfnQfa4rIktdPZNFpIQrL+k+b3Xz64So/goiMk1ENrncucKH76Cv+xmsE5H3RKRUpL8HVDVuX8A1QGNgXUDZcuBPbronMMBNNwJquulLgD0B6ywDmgMCfAzcEMq6A+ZPA6YCT7rPRYFtQD2gBLAauCiE+1wMWANc5j5XBopGYp+B24FJbroMsBOok899rgE0dtPlgO+Ai4DBwNOu/GngX276RrdP4vZxqSuvBGx37xXddEW/f4+j+ZWfn1cB6grJzzkEcTwBTAQ+dJ+nAN3c9Gigt5t+CBjtprsBk0NU/zjgPjddAqgQye8AOBvYAZQO2P+/RPx7iNQvebS+3B/MwD+2hzl9LfIcYEMW6whwECjpEmpTwLzuwOuhrhu4BXgF6M/pxu0KYF7AMs8Az4SqXveLPyGL9cO+z26bH+A1sJXx/lBVyu8+Z4phFnA9sBmoEbBPm93060D3gOU3u/m/28/My9kry++6wD+vAtSdr59zAeusBXwKtAI+dH8rfgKKZf4+gHnAFW66mFtOClj/mXgNi2Qqj+R3cDaw2+VrMfc9tI3k96Cq8X1aMhvrgQ5uugveH9zMOgErVfUE3g8yOWBesisLWd0icgbQD/hHpuUzfokKWnd2+9wAUBGZJyIrReSpgHrDus94R6lHgL3A/wFDVPUgBdxnd8qjEbAUqK6qe92sH4Dqbjq7OkL1fccTX76zAv6cCyIReApId58rA4dUNTWLOn6r380/7JYviLrAj8B/3anRN0WkLBH8DlR1DzAEL2/34u1XEpH9Hqxxy0JP4CERScI7tXEycKaIXAz8C3gggnX3B4ap6q9hqDOneosBVwF3uPeOItI6QnU3BdKAmngJ+1cRqVeQitw/CdOBx1U1JXCeev822q3DMcCvn7OItAf2q2pSOLYfpGJ4p/5fU9VGeP8g/u46Z7h/1931vA54eVsTKAu0C1d92SkW6QqjnapuAtoAiEgD4M8Z80SkFjADuEtVt7niPXinIjLUcmWhrLsZ0FlEBuOdP08XkeN4/w0FHlnmq+4c6k0GvlTVn9y8OXiJM4Hw7/PtwFxVPQXsF5HFQALef3h53mcRKY73B+9dVX3fFe8TkRqquldEagD7XfmebOrYA7TMVP55cHsat7L7LsMiRD/n/GoB3CwiNwKlgPLAq0AFESnmjkoC68ioP1lEiuGdUjxQgPrBy9lkVV3qPk/Da9wi9R0AXAfsUNUfAUTkfbzvJpLfgx25ZSYi1dx7EeA5vAufiEgF4CO8i7KLM5Z3h/opItJcRAS4C+9cf8jqVtWrVbWOqtbBO+3xsqqOwLsZo767C6kE3sXY2aGqF+9c+KUiUsb90v0J75pY2PcZ75RGKzevLN7F7k3kY59djG8BG1V1aMCs2UDGHY93B+zDbOAudydZc+Cw2+d5QBsRqej+O23jykz2QvI7GowQ/pzzRVWfUdVaLk+7AZ+p6h3AQqBzNvVnxNXZLV+gIypV/QHYLSIXuKLWwAYi9B04/wc0d383JCCGiH0PQHzfUAK8h3dO+BTefzz3Ao/h3bzwHTCI0zc7PId3iL8q4FXNzUsA1uHdFTaCIC6G5qXuTOv1x91Q4j7f6JbfBjwb6nqBO/Gui60DBgeUh3WfgTPw7gxdj5cYfyvAPl+FdxpmTcDP7ka88/qfAluABUAlt7wAI9321wIJAdvqCWx1r3v8/h0uDK+8/rwKUE/Ifs4hiKUlp++WrId3d/FW9ztd0pWXcp+3uvn1QlR3Q2CF+x5m4t3ZG9HvAO/+gE3ub8Q7eDffRfR7sB5KjDHGxBw7LWmMMSbmWONmjDEm5ljjZowxJuZY42aMMSbmWONmjDEm5ljjFmPc8yqLROSGgLIuIjI3BNueICI7RGSVeD2OPxfEOh1F5G9u+kURedxN9xSRswoakzGhJCLVRWSiiGwXb/SPb0Sko88xzRSRJX7GUBhZ4xZj1Hu240FgqHjDTJwBvAz0Kch23UPcAH1VtSFev333i0hWfW8GxjNDVV/JYlZPwBo3EzXcA8cz8Xrlqaeql+M9jF0r5zV/Wz/kPT65ziMuB87Mrvu5cNQbC6xxi0Gqug6vR/1+wPPAeFXdJt5YZMvckdco1ysIIjJGRFaIN/7S8xnbEZFkERkkIt8Cmf97LY33wOzRgGUruOnmIrLATd8nIomBK4rIbXgPmk52sZQIx/dgTB61Ak6qakYvOajqLlX9j4jUEZGvxOtAfKWIXAkgIi1d+Wy8zgYyjrSSXD71ytiWiNwrIt+5HHxDREa48qoiMl1ElrtXi4CYbsXL5Ul4DW3Gtt4WkdEishQYLCJlxRsvcZl4HSZ3cMtlGXdcCFdvAfby94XXWelmvF4HSuKNQTeT00NOjAFud9MZvRUUA77CjbeF14vIEwHbnIA3nMYqvN5a/hkwLxmo4KabAwvc9H1Aopt+Ea8zW4BFQEO/vyd72SvjBTyK10F5VvPKAKXcdH1ghZtu6XKhbsCyGflUGq+Hjsp4HQjvxBsGprjLsxFuuYnAVW66Nl73YRnbmg9cjTdCx9qA8rfxhpLJGF/xZeBON10Br0eYstnFHQ8vO5yNUap6REQmA7+q6gkRuQ5oAqzwzr5QmtNDXXQXkXvxGreaeAM8bnDzJmfadF9VnSki5YCFIvKhqi4L9/4YE2kiMhKvS6+TeJ0BjxCRhnijVTQIWHSZqu4I+PxowHW6c/AalbOAL9QbtgkRmRqwjeuAi+T0YPbl3eWEsm7dRaqqInJKRC5R78wMwFRVTXPTbfA6bX7SfS6F11B+n0PcMc0at9iWzulxpQQYq6r/L3ABEamP179jU1U9JCIT8BIjw5GsNqyqv4jIF3jJvwxI5fRp7lJZrWNMlFuPN1YjAKraR0Sq4PXT2BfYB1yG93t+PGC933JERFriNVZXqOpREfmc3POhCNBcVQO3iYjcg9cv5A7X8JXHGzD32cz14uV3J1XdnGkb/XOIO6bZNbf4sQDo6pIVEaksIrXxEuYXvF7+a+CNmJsr8YYWaYrX4Sp4p1wud9Odslonk1/wxm8zJlp8BpQSkd4BZWXc+5nAXlVNB3oARbPZxpnAz65huxDvFD14oyP8SbwRJYrx+xz5BHgk44M7ygKvIWunp0cEybjBJSvzgEfcTTGISKM8xh1zrHGLE6q6Fq+n7gUisgYvoaoDK/FOQW4CxgOLs92IZ5iIrMLrcTyJ08OX9AdGichyMg3wmo3/Am/aDSUmWqh3YeoWvEZoh4gsA8bh3Zg1CrhbRFYDF5LNGQ1gLlBMRDbijXKxxG17D951sWV4ObYTb8Rp8K71JYjIGhHZADwo3kji52as77axAzgsIs2yqHcA3rW8NSKy3n0mD3HHHBsVwBhjIkBEzlDVX92R2wy8ywQz/I4rVtmRmzHGREZ/d9ZjHd5dxzN9jiem2ZGbMcaYmGNHbsYYY2KONW7GGGNijjVuxhhjYo41bsYYY2KONW7GGGNijjVuxhhjYs7/B4w0SoZkAGvdAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# We need to load a gradient booster that is compatible with scikit-learn's partial dependence module (not XGBoost).\n",
    "from sklearn.ensemble import GradientBoostingRegressor\n",
    "from sklearn.ensemble.partial_dependence import partial_dependence, plot_partial_dependence\n",
    "\n",
    "my_model = GradientBoostingRegressor()\n",
    "my_model.fit(X_train, y_train)\n",
    "my_plots = plot_partial_dependence(my_model,       \n",
    "                                   features=[X_train.columns.get_loc(\"LotArea\"), X_train.columns.get_loc(\"GrLivArea\")], # column numbers of plots we want to show\n",
    "                                   X=X_train,            # raw predictors data.\n",
    "                                   feature_names=list(X_train.columns.values), # labels on graphs\n",
    "                                   grid_resolution=10) # number of values to plot on x axis\n",
    "my_plots = plot_partial_dependence(my_model,       \n",
    "                                   features=[X_train.columns.get_loc(\"YearBuilt\"), X_train.columns.get_loc(\"GarageArea\")], # column numbers of plots we want to show\n",
    "                                   X=X_train,            # raw predictors data.\n",
    "                                   feature_names=list(X_train.columns.values), # labels on graphs\n",
    "                                   grid_resolution=10) # number of values to plot on x axis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "945592a86e5af9b8da35a8d3ffd071a46469fe51"
   },
   "source": [
    "As expected, prices grow with larger available space (lot area, ground floor living area, garage) and more recent year built."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "_uuid": "f02de7dd07e0dd8059be3fb69caad8a11ea5dc44"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}