Notebook 1 - Linear Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear Regression - Housing Prices\n",
    "\n",
    "For those of you unfamiliar with Jupyter notebooks, this is a concise explanation.\n",
    "Each \"cell\" of the notebook (a rectangular area) contains either explanatory text (\"Markdown,\" like this cell) or Python/Keras commands (\"Code\"). By clicking on this cell, you select it (and the contours are highlighted). If you press \"__Run__\" in the menu, Jupyter processes the contents of this cell and moves on to the next. Scroll to the next cell, read the command and press __Run__ again. The result of the command (if any) will become visible. Just proceed through the notebook in this fashion, and return to previous cells, whenever necessary (either to re-read an explanation or command, or to change parameters). Please note that if you want to restart the entire notebook, you have to start at the top.\n",
    "\n",
    "Let's start with importing the required plugins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt \n",
    "import pandas as pd  \n",
    "%matplotlib inline\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.metrics import mean_squared_error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Linear Regression is usually the first machine learning algorithm that every data scientist comes across. It is a simple model, but nontheless good to understand, as it lays the foundation for other machine learning algorithms. In statistics, Linear Regression is a linear approach to modelling the relationship between a dependent variable and one or more independent variables.\n",
    "\n",
    "Next, we will load the housing data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_boston\n",
    "boston_dataset = load_boston()\n",
    "print(\"Data loaded.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We print the values of the __boston_dataset__ to understand what it contains."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(boston_dataset.DESCR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The house prices, indicated by the variable __MEDV__, is our target variable. The remaining variables are the feature variables, based on which we will predict the value of a house.\n",
    "\n",
    "We will now load the data into a dataframe. To get an idea of the generated dataframe, we then print the first 5 rows of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "boston = pd.DataFrame(boston_dataset.data, columns=boston_dataset.feature_names)\n",
    "boston['MEDV'] = boston_dataset.target\n",
    "boston.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's a good practice to explore your data before starting with the linear regression. Next, we will use some visualizations to understand the relationship of the target variable (house prices) with other features in the data.\n",
    "\n",
    "Let’s first plot the distribution of the target variable __MEDV__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.hist(boston['MEDV'],30)\n",
    "plt.title('Distribution house prices')\n",
    "plt.xlabel('House price (*$1000)')\n",
    "plt.ylabel('Occurance')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can observe that the housing prices are normally distributed, with a few outliers. We suspect that the number of rooms in a house, represented by __RM__, has a strong correlation with the actual price of a house. Visualizing this using a scatterplot may confirm our suspicion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(20, 5))\n",
    "\n",
    "features = boston['RM']\n",
    "target = boston['MEDV']\n",
    "\n",
    "plt.scatter(features, target)\n",
    "plt.title('House prices VS Number of rooms')\n",
    "plt.ylabel('House price (*$1000)')\n",
    "plt.xlabel('Number of rooms')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks like we were right! The house prices seem to increase as the number of rooms increases. Now we can build our Linear Regression model. First, we load the relevant data (in our case the house prices __MEDV__ and the number of rooms __RM__) to a separate data frame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = pd.DataFrame(boston['RM'], columns = ['RM'])\n",
    "Y = boston['MEDV']\n",
    "print(\"Data loaded.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we can set up a Linear Regression model, and fit the data to this model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lin_model = LinearRegression()\n",
    "lin_model.fit(X, Y)\n",
    "print(\"Model loaded.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, we generate a prediction by the model and visualize the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Y_pred = lin_model.predict(X)\n",
    "plt.figure(figsize=(20, 5))\n",
    "plt.scatter(X, Y)\n",
    "plt.plot(X, Y_pred, color='red')\n",
    "plt.title('Linear Regression - Prediction')\n",
    "plt.ylabel('House price (*$1000)')\n",
    "plt.xlabel('Number of rooms')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The red line in the plot shows how our model predicts house prices will change, based on the number of rooms available in a house. A line for which the the error between the predicted values and the observed values (blue dots) is minimum is called the best fit line or the regression line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These errors are also called residuals. The residuals can be visualized as the 'distance' between the observed data values and the regression line.\n",
    "\n",
    "![](images/residuals.png)\n",
    ">_http://wiki.engageeducation.org.au/further-maths/data-analysis/residuals/_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}