06 - Regression Examples.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regression Example: Predict the Price of a House\n",
    "Here we will use some housing data to predict the price of a house based on a number of data points.\n",
    "The data contains a lot of issues we will need to solve before we can get a result:\n",
    "\n",
    "* String values (all features need to be numbers so we can do math on them)\n",
    "* null values\n",
    "* NaN values\n",
    "* enumerable values (e.g. Sale Condition = [\"Normal\", \"Abnormal\", \"Partial\",...])\n",
    "* Some of the data points may not be relevant to the sale price\n",
    "* Some data points might be best combined into a single data point (added or multiplied)\n",
    "\n",
    "We won't have time to address all of the issues, but let's explore some of them by importing the data and using dataframes and matplotlib to visualize it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the common packages for exploring Machine Learning\n",
    "%matplotlib notebook\n",
    "import numpy as np  # <-- common convention for short names of packages...\n",
    "import pandas as pd\n",
    "import sklearn\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib\n",
    "from sklearn import datasets, linear_model\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_squared_error, r2_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load housing data into DataFrame (Pandas reads & writes CSVs and many other data formats)\n",
    "# data source: linked from https://ww2.amstat.org/publications/jse/v19n3/decock.pdf \n",
    "\n",
    "# Download this file to our Jupyter filesystem\n",
    "!wget http://www.amstat.org/publications/jse/v19n3/decock/AmesHousing.txt;\n",
    "# file is 'Tab Separated' with a generic extension, so tell Pandas which separator to use:  \\t\n",
    "df = pd.read_csv('AmesHousing.txt',sep='\\t');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DataFrames in Pandas are easy to sample or use head(n) or tail(n)\n",
    "\n",
    "# df.head(3)\n",
    "# df.tail(3)\n",
    "df.sample(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wow, 82 is lots of columns - let's sort them so we can find what we're looking for more easily\n",
    "df.columns.sort_values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Maybe we want to grab just a subset of data columns - it's easy with Pandas. \n",
    "# Don't forget the double [[]] syntax for multiple selections\n",
    "\n",
    "# let's start with the easy stuff and grab only the numeric columns\n",
    "df2 = df[['SalePrice','Lot Area','Bedroom AbvGr','Year Built','Yr Sold','1st Flr SF', '2nd Flr SF','BsmtFin SF 1','BsmtFin SF 2']]\n",
    "df2.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Maybe you want to use Pandas Dataframes to engineer a new aggregate feature column\n",
    "# It's easy to do opeations across columns (eg. add all the Square Footage columns into a new one 'Total SF')\n",
    "df3 = pd.DataFrame(df2['1st Flr SF']+df2['2nd Flr SF']+df2['BsmtFin SF 1']+df2['BsmtFin SF 2'], columns=['Total SF'])\n",
    "df3.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Combining DataFrame's is easy to\n",
    "# use 'axis=1' for adding columns (features); 'axis=0' for more rows (examples)\n",
    "df4 = pd.concat([df2,df3],axis=1) \n",
    "df4.sample(3) # <-- now we have a new 'Total SF' feature column appended"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's Try a Simple Linear Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's see if we can predict a Sale Price based on single feature 'Gross Living Area'\n",
    "# Create a new DataFrame with only the data we need\n",
    "data = df[['SalePrice','Gr Liv Area']]\n",
    "data.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sklearn complains if these are shape [100,] vs [100,1]\n",
    "# just one of the many gotchas you'll find :)\n",
    "X = data['Gr Liv Area'].values.reshape(-1,1) \n",
    "# Y is typically used for the Truth Labels\n",
    "Y = data['SalePrice'].values.reshape(-1,1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "# Let's plot Square Foot vs Sale Price to understand our data\n",
    "plt.xlabel('Gross Living Area in $ft^2$')\n",
    "plt.ylabel('Sale Price in $')\n",
    "plt.plot(X,Y,'rx');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's Split the Data so We Can Evaluate How We'll We Can Predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use SKLearns builtin method to split our data & shuffle it into test & train\n",
    "# Split the data into training/testing sets\n",
    "# By default, train_test_split will split the data into 75%/25% train/test\n",
    "housing_X_train, housing_X_test, housing_Y_train, housing_Y_test = train_test_split(\n",
    "    X,Y,\n",
    "    random_state=3\n",
    ")\n",
    "print('housing_X_train',len(housing_X_train),'examples')\n",
    "print('housing_X_test',len(housing_X_test),'examples')\n",
    "print('housing_Y_train',len(housing_Y_train),'examples')\n",
    "print('housing_Y_test',len(housing_Y_test),'examples')\n",
    "\n",
    "# Create linear regression object\n",
    "regr = linear_model.LinearRegression(normalize=True)\n",
    "# regr = linear_model.SGDRegressor(n_iter=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The fit function will train the model using the training set\n",
    "regr.fit(housing_X_train, housing_Y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make predictions using the testing set\n",
    "housing_Y_pred = regr.predict(housing_X_test)\n",
    "\n",
    "# The coefficients\n",
    "# print('Coefficients: \\n', regr.coef_)\n",
    "# TODO: this number is huge...\n",
    "print(\"Mean squared error: %.2f\" % mean_squared_error(housing_Y_test, housing_Y_pred))\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "print('Variance score: %.2f' % r2_score(housing_Y_test, housing_Y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "# Plot outputs\n",
    "plt.scatter(housing_X_train, housing_Y_train, alpha=.5, color='black', label='train')\n",
    "plt.scatter(housing_X_test, housing_Y_test, alpha=.5, color='red', label='test')\n",
    "plt.plot(housing_X_test, housing_Y_pred,color='blue', linewidth=3)\n",
    "\n",
    "plt.xticks()\n",
    "plt.yticks()\n",
    "plt.legend()\n",
    "plt.xlabel('Gross Living Area in $ft^2$')\n",
    "plt.ylabel('Sale Price in $')\n",
    "\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This looks OK, but how well did we really do?\n",
    "\n",
    "Let's create a histogram showing how off we were from the truth.\n",
    "\n",
    "If our model is good, we'll have a lot of hits in the middle and a nice tall bell curve. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "\n",
    "# Plot hist of predictions vs actual\n",
    "y_lr = np.reshape(housing_Y_test,housing_Y_test.shape[0])\n",
    "yhat_lr = np.reshape(housing_Y_pred,housing_Y_pred.shape[0])\n",
    "ydiff_lr = np.subtract(y_lr,yhat_lr)\n",
    "\n",
    "plt.ylim([0,40])\n",
    "plt.hist(ydiff_lr,bins=100,range=[-100000, 100000])\n",
    "plt.title('Linear Regression Single Variable')\n",
    "plt.xlabel('Difference: Predicted vs Sale Price')\n",
    "plt.ylabel('Number of Predictions')\n",
    "\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This doesn't look that great.\n",
    "\n",
    "We can do better if we consider multiple features of data and use a more complex model.\n",
    "\n",
    "## Adding a Neural Network\n",
    "\n",
    "> Before creating a neural network, pop over to [Tensorflow Playground](http://playground.tensorflow.org/) and we'll play with it visually to understand what a neural network is doing.\n",
    "\n",
    "![playground.tensorflow.org](images/playground.tensorflow.org.png)\n",
    "\n",
    "\n",
    "## Run a Basic Network\n",
    "For simplicity, let's start with just using all of the numerical columns in the data.\n",
    "We aren't going to worry about featurizing non-numeric fields yet since there is probably useful data already in the dataset that won't require a lot of work to setup.\n",
    "\n",
    "There are a lot of neural networks and tools to choose from. In this example, we are going to use an [MLPRegressor](http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html), built into scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sklearn.neural_network.MLPRegressor\n",
    "# sklearn.neural_network.MLPRegressor(hidden_layer_sizes=(100, ), activation=’relu’, solver=’adam’, alpha=0.0001, \n",
    "# batch_size=’auto’, learning_rate=’constant’, learning_rate_init=0.001, power_t=0.5, max_iter=200, shuffle=True, \n",
    "# random_state=None, tol=0.0001, verbose=False, warm_start=False, momentum=0.9, nesterovs_momentum=True, \n",
    "# early_stopping=False, validation_fraction=0.1, beta_1=0.9, beta_2=0.999, epsilon=1e-08)\n",
    "from sklearn.neural_network import MLPRegressor\n",
    "\n",
    "# Use all numerical columns to predict 'SalePrice'\n",
    "columns = list([\n",
    "    'Gr Liv Area', # this is our single linear regression point of reference\n",
    "    # can we do better by adding these other features?\n",
    "    '1st Flr SF', \n",
    "    '2nd Flr SF', \n",
    "    '3Ssn Porch', \n",
    "    'Bedroom AbvGr',\n",
    "    'Bsmt Full Bath',\n",
    "    'Bsmt Half Bath', \n",
    "    'Bsmt Unf SF', \n",
    "    'BsmtFin SF 1',\n",
    "    'BsmtFin SF 2',\n",
    "    'Enclosed Porch',\n",
    "    'Fireplaces',\n",
    "    'Full Bath',\n",
    "    'Garage Area', \n",
    "    'Garage Cars',\n",
    "    'Garage Yr Blt', \n",
    "    'Half Bath', \n",
    "    'Kitchen AbvGr',\n",
    "    'Lot Area',\n",
    "    'Lot Frontage', \n",
    "    'Low Qual Fin SF',\n",
    "    'Mas Vnr Area',\n",
    "    'Mo Sold', \n",
    "    'Open Porch SF',\n",
    "    'Pool Area',\n",
    "    'TotRms AbvGrd', \n",
    "    'Total Bsmt SF',\n",
    "    'Wood Deck SF', \n",
    "    'Year Built', \n",
    "    'Year Remod/Add', \n",
    "    'Yr Sold'\n",
    "])\n",
    "print(columns)\n",
    "# Create new dataframe with columns\n",
    "X_NN = df[columns]\n",
    "print(X_NN.shape)\n",
    "#sklearn complains if these are shape [100,] vs [100,1]\n",
    "Y_NN = df['SalePrice'].values.reshape(-1,1)\n",
    "print(Y_NN.shape)\n",
    "# remove NaN values & replace with 0's\n",
    "X_NN = X_NN.fillna(0)\n",
    "X_NN = X_NN.values # convert to plain NumPy array\n",
    "\n",
    "# TODO: scaling & centering data\n",
    "# scale & center our data\n",
    "from sklearn import preprocessing\n",
    "X_scaled = preprocessing.scale(X_NN);\n",
    "\n",
    "# use SKLearns builtin method to split our data & shuffle it into test & train\n",
    "# Split the data into training/testing sets\n",
    "housing_X_train_nn, housing_X_test_nn, housing_Y_train_nn, housing_Y_test_nn = train_test_split(\n",
    "    X_scaled,Y_NN,\n",
    "    random_state=2# what if we lock down the random seed number? (set to 1, 3, 10, or False)\n",
    ")\n",
    "print('housing_X_train',len(housing_X_train_nn),'examples')\n",
    "print('housing_X_test',len(housing_X_test_nn),'examples')\n",
    "\n",
    "score1 = r2_score(housing_Y_test, housing_Y_pred)\n",
    "print('\\nLinear Regression (Single variable) VARIANCE:',round(score1,2))\n",
    "\n",
    "\n",
    "# you can run this multiple times to check the variable starting points\n",
    "# each run will be different--and may be significantly difference since the initialization variables will change\n",
    "# and that will affect how the model converges\n",
    "# for i in range(2,8): # try setting one of the variables to i\n",
    "# Explore settings logarithmically (0.1, 0.01, 0.001, 0.00001)\n",
    "nn_regr = MLPRegressor(\n",
    "    # what if we change our layer sizes?\n",
    "    hidden_layer_sizes=(2,8,2), \n",
    "    # what if we change our learning rate?\n",
    "    learning_rate_init=0.01,\n",
    "    # what if we change our activation function? (relu, tanh, identity)\n",
    "    activation='relu',\n",
    "    max_iter=2000,\n",
    "    random_state=2, # if set to None, this is random, to an int, static seed\n",
    "    # set this to True to see how well we are learning over the iterations\n",
    "    verbose=False\n",
    ");\n",
    "\n",
    "# Train it\n",
    "nn_regr.fit(housing_X_train_nn,housing_Y_train_nn.reshape(housing_Y_train_nn.size))\n",
    "\n",
    "# Make predictions using the testing set\n",
    "housing_Y_pred_nn = nn_regr.predict(housing_X_test_nn)\n",
    "\n",
    "# Variance scores or Linear Regression vs NN\n",
    "score2 = r2_score(housing_Y_test_nn, housing_Y_pred_nn)\n",
    "\n",
    "#print(\"Mean squared error: %.2f\" % mean_squared_error(housing_Y_test_nn, housing_Y_pred_nn))\n",
    "# Explained variance score: 1 is perfect prediction\n",
    "answer = ((score2-score1)/score1)*100\n",
    "print(\n",
    "    'NN MLP Regression (Multi variable) VARIANCE: {} {:0.0f}% ({:0.2f}x) over 1 variable linear regression'.format(\n",
    "        round(score2,2), answer, score2/score1\n",
    "    )\n",
    ")\n",
    "\n",
    "\n",
    "pd.DataFrame(nn_regr.loss_curve_).plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## So How is That Bell Curve?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "\n",
    "# Plot histogram of difference between predicted & actual sale price outputs\n",
    "y_nn = np.reshape(housing_Y_test_nn,housing_Y_test_nn.shape[0])\n",
    "yhat_nn = np.reshape(housing_Y_pred_nn,housing_Y_pred_nn.shape[0])\n",
    "ydiff = np.subtract(y_nn,yhat_nn)\n",
    "\n",
    "plt.ylim([0,40])\n",
    "plt.hist(ydiff,bins=100,range=[-100000, 100000])\n",
    "\n",
    "plt.title('Neural Net MLP Muli Variable')\n",
    "plt.xlabel('Difference: Predicted vs Sale Price')\n",
    "plt.ylabel('Number of Predictions')\n",
    "\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's significantly better!\n",
    "\n",
    "But can we do even better?\n",
    "\n",
    "## Let's Visualize The Relationship Each Feature Has With Price\n",
    "\n",
    "We can plot a chart of each feature mapped to the sales price to easily see how a feature corresponds to the rising price.\n",
    "\n",
    "If a feature doesn't show a correlation with increased price, this is not likely a good feature for our model to consider--and we can omit it from the data. If your dataset is large enough, a good neural network will learn to ignore meaningless or redundant data (more data is usually better), but if your dataset isn't large enough or if extra features are [spurriously correlated](http://www.tylervigen.com/spurious-correlations), they can cause chaos within your network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Notes:\n",
    "# Data not strongly correllate\n",
    "# remove 'Bsmt Unf SF',\n",
    "# adjust 'Garage Yr Blt' - notice poor logic of replacing NA values with 0 - created outliers/data scaling issue\n",
    "# watch out for scaling - Lot Area\n",
    "%matplotlib notebook\n",
    "# How many columns do we have?\n",
    "print(len(columns))\n",
    "\n",
    "# for each feature, show how it relates to sales price\n",
    "for i in range(0,len(columns)):\n",
    "    # by specifying a figure, the plotter will create multiple figures\n",
    "    plt.figure(i)\n",
    "    plt.scatter(X_NN[:,i], Y, alpha=.2, color='blue', label='train samples')\n",
    "#     plt.scatter(housing_X_train[:,i], housing_Y_train, alpha=.3, color='black', label='train samples')\n",
    "#     plt.scatter(housing_X_test[:,i], housing_Y_test, alpha=.3, color='red', label='test samples')\n",
    "#     plt.scatter(housing_X_test[:,i], housing_Y_pred_nn,color='magenta', linewidth=1, alpha=.5, label='predictions')\n",
    "\n",
    "    plt.xticks()\n",
    "    plt.yticks()\n",
    "    plt.legend()\n",
    "    plt.xlabel(columns[i])\n",
    "    plt.ylabel('Sale Price in $')\n",
    "\n",
    "    plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Which Features Are Not Useful?\n",
    "Let's remove all of the features that seem less useful and run it again...\n",
    "\n",
    "Just go up to the cell that has our feature columns and comment out (command+/) any fields you want to remove.\n",
    "\n",
    "What's the best score you can get?\n",
    "\n",
    "## Box Plots\n",
    "\n",
    "Some of the scatter plots only have whole numbers on the x-axis. This makes it difficult to look at them and decide whether the features represented in those graphs are useful or not since the graphs are not continuous. The following box plots make it easier to look at these features and decide whether they are useful based on the amount of overlap between the box plots in each graph. If there is a lot of overlap between box plots, then the feature they are representing is not useful. If there is not a lot of overlap between box plots, then the feature they are representing is useful.\n",
    "\n",
    "For information on what box plots are and how to read them, see https://en.wikipedia.org/wiki/Box_plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "import numpy\n",
    "\n",
    "def createBoxplot(attribute, figNum):\n",
    "    for i in range(0, len(columns)):\n",
    "        a = []\n",
    "        if(columns[i] == attribute):\n",
    "            a = X_NN[:,i]\n",
    "            break\n",
    "    unique, counts = numpy.unique(a, return_counts = True)\n",
    "    d = dict(zip(unique, counts))\n",
    "    index = 0\n",
    "    if 0 in d:\n",
    "        index = d[0];\n",
    "    pltNum = 1\n",
    "    data_to_plot = list()\n",
    "    length = len(d)\n",
    "    while pltNum <= length:\n",
    "        temp = []\n",
    "        if pltNum in d:\n",
    "            endIndex = index + d[pltNum]\n",
    "            temp = []\n",
    "            while index < endIndex:\n",
    "                temp = numpy.append(temp,Y[index])\n",
    "                index = index + 1\n",
    "        data_to_plot.append(temp)\n",
    "        pltNum = pltNum + 1\n",
    "    fig = plt.figure(figNum, figsize=(9,6))\n",
    "    ax = fig.add_subplot(111)\n",
    "    bp = ax.boxplot(data_to_plot)\n",
    "    fig.savefig(\"fig\"+str(figNum)+\".png\", bbox_inches='tight')\n",
    "    plt.xlabel(attribute)\n",
    "    plt.ylabel(\"Sale Price in $\")\n",
    "        \n",
    "createBoxplot(\"Bedroom AbvGr\", 1)\n",
    "createBoxplot(\"Kitchen AbvGr\", 2)\n",
    "createBoxplot(\"Half Bath\", 3)\n",
    "createBoxplot(\"Garage Cars\", 4)\n",
    "createBoxplot(\"Full Bath\", 5)\n",
    "createBoxplot(\"Fireplaces\", 6)\n",
    "createBoxplot(\"Bsmt Half Bath\", 7)\n",
    "createBoxplot(\"Bsmt Full Bath\", 8)\n",
    "createBoxplot(\"TotRms AbvGrd\", 9)\n",
    "createBoxplot(\"Mo Sold\", 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Number of Zeros in Each Column \n",
    "Many of the scatter plots show features with 0s in their data. Most of the time, these 0s are just noise. For example, for the feature \"2nd Flr SF\", many houses had 0 in their data for this feature meaning that they did not have a second floor. It would be more meaningful for the graph to just display the data for houses that have a second floor. To do that, the houses that don't have a second floor would be excluded from the graph. It is still useful to know how many houses do not have a second floor though which is why the number of 0s for each feature should be counted even if they are excluded from the graphs.\n",
    "The box plots exclude 0s from their graphs as some houses have 0s in their feature vector. That makes the resulting visualizations more meaningful."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#calculate the number of 0s in the columns with at least one 0. These 0s are left out of the box chart data\n",
    "for i in range(0, len(columns)):\n",
    "    numZeros = 0\n",
    "    a = X_NN[:,i]\n",
    "    for j in range(0, len(a)):\n",
    "        if(a[j]) == 0:\n",
    "            numZeros = numZeros + 1\n",
    "    if(numZeros > 0):\n",
    "        print(columns[i] + \": \" + str(numZeros))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Aggregate Graphs\n",
    "The following graph shows the relationship between the total square footage in a house and the price of the house. It is an example of an aggregate graph because it adds up three different features provided in the data, \"1st Flr SF\", \"2nd Flr SF', and \"Total Bsmt SF\", to compute the total square footage of each house. \n",
    "Aggregate graphs can show potentially useful relationships that aren't immediately obvious from the data provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib\n",
    "\n",
    "#Show a plot of the total square footage in a house compared with the price\n",
    "firstFloorIndex = 0\n",
    "secondFloorIndex = 0\n",
    "basementIndex = 0\n",
    "for i in range(0, len(columns)):\n",
    "    if(columns[i] == '1st Flr SF'):\n",
    "        firstFloorIndex = i\n",
    "    elif(columns[i] == '2nd Flr SF'):\n",
    "        secondFloorIndex = i\n",
    "    elif(columns[i] == 'Total Bsmt SF'):\n",
    "        basementIndex = i\n",
    "sumSquareFootage = []\n",
    "sumSquareFootage = X_NN[:, firstFloorIndex]\n",
    "sumSquareFootage = X_NN[:, secondFloorIndex] + sumSquareFootage\n",
    "sumSquareFootage = X_NN[:, basementIndex] + sumSquareFootage\n",
    "\n",
    "plt.figure(1)\n",
    "plt.scatter(sumSquareFootage, Y, color = 'blue', label = 'train samples')\n",
    "plt.xticks()\n",
    "plt.yticks()\n",
    "plt.legend()\n",
    "plt.xlabel('Total Square Footage')\n",
    "plt.ylabel('Sale Price in $')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next Up\n",
    "[Where Do You Go From Here?](07%20-%20From%20Here.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}