Absenteeism Exercise - Logistic Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating a Logistic Regression to Predict Absenteeism"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import the Relevant Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_preprocessed = pd.read_csv('Absenteeism_preprocessed.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Reason 1</th>\n",
       "      <th>Reason 2</th>\n",
       "      <th>Reason 3</th>\n",
       "      <th>Reason 4</th>\n",
       "      <th>Month Value</th>\n",
       "      <th>Day of the Week</th>\n",
       "      <th>Transportation Expense</th>\n",
       "      <th>Distance to Work</th>\n",
       "      <th>Age</th>\n",
       "      <th>Daily Work Load Average</th>\n",
       "      <th>Body Mass Index</th>\n",
       "      <th>Education</th>\n",
       "      <th>Children</th>\n",
       "      <th>Pets</th>\n",
       "      <th>Absenteeism Time in Hours</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>289</td>\n",
       "      <td>36</td>\n",
       "      <td>33</td>\n",
       "      <td>239.554</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>118</td>\n",
       "      <td>13</td>\n",
       "      <td>50</td>\n",
       "      <td>239.554</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>179</td>\n",
       "      <td>51</td>\n",
       "      <td>38</td>\n",
       "      <td>239.554</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>279</td>\n",
       "      <td>5</td>\n",
       "      <td>39</td>\n",
       "      <td>239.554</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>289</td>\n",
       "      <td>36</td>\n",
       "      <td>33</td>\n",
       "      <td>239.554</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Reason 1  Reason 2  Reason 3  Reason 4  Month Value  Day of the Week  \\\n",
       "0         0         0         0         1            7                1   \n",
       "1         0         0         0         0            7                1   \n",
       "2         0         0         0         1            7                2   \n",
       "3         1         0         0         0            7                3   \n",
       "4         0         0         0         1            7                3   \n",
       "\n",
       "   Transportation Expense  Distance to Work  Age  Daily Work Load Average  \\\n",
       "0                     289                36   33                  239.554   \n",
       "1                     118                13   50                  239.554   \n",
       "2                     179                51   38                  239.554   \n",
       "3                     279                 5   39                  239.554   \n",
       "4                     289                36   33                  239.554   \n",
       "\n",
       "   Body Mass Index  Education  Children  Pets  Absenteeism Time in Hours  \n",
       "0               30          0         2     1                          4  \n",
       "1               31          0         1     0                          0  \n",
       "2               31          0         0     0                          2  \n",
       "3               24          0         2     0                          4  \n",
       "4               30          0         2     1                          2  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_preprocessed.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model itself will give us a fair indication of which variables are important for the analysis.\n",
    "\n",
    "Logistic Regression is a type of classification."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the Targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create two classes:\n",
    "   1. Moderately Absent\n",
    "   2. Excessively Absent\n",
    "  \n",
    "Will take the median value of the Absenteeism Time in Hours and use it as a cut-off line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.0"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_preprocessed['Absenteeism Time in Hours'].median()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Less than or Equal to 3.0 --> Moderately Absent \n",
    "2. Greater than 3.0 --> Excessively Absent\n",
    "\n",
    "In ML, 0s and 1s are TARGETS\n",
    "\n",
    "Using the median to classify essentially balances the data, thus half of the data fits into classification 1 and the other half into classification 2. Therefore, prevents our model from learning to output only 0s or only 1s."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0,\n",
       "       1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1,\n",
       "       0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1,\n",
       "       0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1,\n",
       "       0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0,\n",
       "       0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0,\n",
       "       1, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1,\n",
       "       0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1,\n",
       "       1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1,\n",
       "       0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 0,\n",
       "       0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0,\n",
       "       0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1,\n",
       "       1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0,\n",
       "       1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0,\n",
       "       1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1,\n",
       "       1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1,\n",
       "       1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1,\n",
       "       0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1,\n",
       "       1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1,\n",
       "       1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0,\n",
       "       1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1,\n",
       "       0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1,\n",
       "       0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0,\n",
       "       1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1,\n",
       "       1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# np.where(condition, value if True, value is False) --> checks if a condition has been satisfied and assigns a value accordingly \n",
    "targets = np.where(data_preprocessed['Absenteeism Time in Hours'] > \n",
    "                   data_preprocessed['Absenteeism Time in Hours'].median(), 1, 0)\n",
    "targets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Reason 1</th>\n",
       "      <th>Reason 2</th>\n",
       "      <th>Reason 3</th>\n",
       "      <th>Reason 4</th>\n",
       "      <th>Month Value</th>\n",
       "      <th>Day of the Week</th>\n",
       "      <th>Transportation Expense</th>\n",
       "      <th>Distance to Work</th>\n",
       "      <th>Age</th>\n",
       "      <th>Daily Work Load Average</th>\n",
       "      <th>Body Mass Index</th>\n",
       "      <th>Education</th>\n",
       "      <th>Children</th>\n",
       "      <th>Pets</th>\n",
       "      <th>Absenteeism Time in Hours</th>\n",
       "      <th>(Excessive Absenteeism, Daily Work Load Average, Distance to Work)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>289</td>\n",
       "      <td>36</td>\n",
       "      <td>33</td>\n",
       "      <td>239.554</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>118</td>\n",
       "      <td>13</td>\n",
       "      <td>50</td>\n",
       "      <td>239.554</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>179</td>\n",
       "      <td>51</td>\n",
       "      <td>38</td>\n",
       "      <td>239.554</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>279</td>\n",
       "      <td>5</td>\n",
       "      <td>39</td>\n",
       "      <td>239.554</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>289</td>\n",
       "      <td>36</td>\n",
       "      <td>33</td>\n",
       "      <td>239.554</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Reason 1  Reason 2  Reason 3  Reason 4  Month Value  Day of the Week  \\\n",
       "0         0         0         0         1            7                1   \n",
       "1         0         0         0         0            7                1   \n",
       "2         0         0         0         1            7                2   \n",
       "3         1         0         0         0            7                3   \n",
       "4         0         0         0         1            7                3   \n",
       "\n",
       "   Transportation Expense  Distance to Work  Age  Daily Work Load Average  \\\n",
       "0                     289                36   33                  239.554   \n",
       "1                     118                13   50                  239.554   \n",
       "2                     179                51   38                  239.554   \n",
       "3                     279                 5   39                  239.554   \n",
       "4                     289                36   33                  239.554   \n",
       "\n",
       "   Body Mass Index  Education  Children  Pets  Absenteeism Time in Hours  \\\n",
       "0               30          0         2     1                          4   \n",
       "1               31          0         1     0                          0   \n",
       "2               31          0         0     0                          2   \n",
       "3               24          0         2     0                          4   \n",
       "4               30          0         2     1                          2   \n",
       "\n",
       "   (Excessive Absenteeism, Daily Work Load Average, Distance to Work)  \n",
       "0                                                  1                   \n",
       "1                                                  0                   \n",
       "2                                                  0                   \n",
       "3                                                  1                   \n",
       "4                                                  0                   "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create new column in dataframe for Excessive Absenteeism\n",
    "data_preprocessed['Excessive Absenteeism', 'Daily Work Load Average', 'Distance to Work'] = targets\n",
    "data_preprocessed.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Comment on the Targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To prove our model is not learning to output only 0s and 1s..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.45571428571428574"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "targets.sum() / targets.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Around 46% of the targets are 1s. A 60-40 split will usually work for a logistic regression, but not true for other algorithms such as neural networks. \n",
    "\n",
    "45-55 is almost always sufficient."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Drop Absenteeism Time in Hours\n",
    "data_with_targets = data_preprocessed.drop(['Absenteeism Time in Hours'], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check that at this point, there is a checkpoint of the data\n",
    "# Using 'is' --> true = the 2 variables refer to the same object\n",
    "#                false = the 2 variable refer to different objects\n",
    "data_with_targets is data_preprocessed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Reason 1</th>\n",
       "      <th>Reason 2</th>\n",
       "      <th>Reason 3</th>\n",
       "      <th>Reason 4</th>\n",
       "      <th>Month Value</th>\n",
       "      <th>Day of the Week</th>\n",
       "      <th>Transportation Expense</th>\n",
       "      <th>Distance to Work</th>\n",
       "      <th>Age</th>\n",
       "      <th>Daily Work Load Average</th>\n",
       "      <th>Body Mass Index</th>\n",
       "      <th>Education</th>\n",
       "      <th>Children</th>\n",
       "      <th>Pets</th>\n",
       "      <th>(Excessive Absenteeism, Daily Work Load Average, Distance to Work)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>289</td>\n",
       "      <td>36</td>\n",
       "      <td>33</td>\n",
       "      <td>239.554</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>118</td>\n",
       "      <td>13</td>\n",
       "      <td>50</td>\n",
       "      <td>239.554</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>179</td>\n",
       "      <td>51</td>\n",
       "      <td>38</td>\n",
       "      <td>239.554</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>279</td>\n",
       "      <td>5</td>\n",
       "      <td>39</td>\n",
       "      <td>239.554</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>289</td>\n",
       "      <td>36</td>\n",
       "      <td>33</td>\n",
       "      <td>239.554</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Reason 1  Reason 2  Reason 3  Reason 4  Month Value  Day of the Week  \\\n",
       "0         0         0         0         1            7                1   \n",
       "1         0         0         0         0            7                1   \n",
       "2         0         0         0         1            7                2   \n",
       "3         1         0         0         0            7                3   \n",
       "4         0         0         0         1            7                3   \n",
       "\n",
       "   Transportation Expense  Distance to Work  Age  Daily Work Load Average  \\\n",
       "0                     289                36   33                  239.554   \n",
       "1                     118                13   50                  239.554   \n",
       "2                     179                51   38                  239.554   \n",
       "3                     279                 5   39                  239.554   \n",
       "4                     289                36   33                  239.554   \n",
       "\n",
       "   Body Mass Index  Education  Children  Pets  \\\n",
       "0               30          0         2     1   \n",
       "1               31          0         1     0   \n",
       "2               31          0         0     0   \n",
       "3               24          0         2     0   \n",
       "4               30          0         2     1   \n",
       "\n",
       "   (Excessive Absenteeism, Daily Work Load Average, Distance to Work)  \n",
       "0                                                  1                   \n",
       "1                                                  0                   \n",
       "2                                                  0                   \n",
       "3                                                  1                   \n",
       "4                                                  0                   "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_with_targets.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Select the Inputs for the Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(700, 15)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_with_targets.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Reason 1</th>\n",
       "      <th>Reason 2</th>\n",
       "      <th>Reason 3</th>\n",
       "      <th>Reason 4</th>\n",
       "      <th>Month Value</th>\n",
       "      <th>Day of the Week</th>\n",
       "      <th>Transportation Expense</th>\n",
       "      <th>Distance to Work</th>\n",
       "      <th>Age</th>\n",
       "      <th>Daily Work Load Average</th>\n",
       "      <th>Body Mass Index</th>\n",
       "      <th>Education</th>\n",
       "      <th>Children</th>\n",
       "      <th>Pets</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>289</td>\n",
       "      <td>36</td>\n",
       "      <td>33</td>\n",
       "      <td>239.554</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>118</td>\n",
       "      <td>13</td>\n",
       "      <td>50</td>\n",
       "      <td>239.554</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>179</td>\n",
       "      <td>51</td>\n",
       "      <td>38</td>\n",
       "      <td>239.554</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>279</td>\n",
       "      <td>5</td>\n",
       "      <td>39</td>\n",
       "      <td>239.554</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>289</td>\n",
       "      <td>36</td>\n",
       "      <td>33</td>\n",
       "      <td>239.554</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>695</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>179</td>\n",
       "      <td>22</td>\n",
       "      <td>40</td>\n",
       "      <td>237.656</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>696</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>225</td>\n",
       "      <td>26</td>\n",
       "      <td>28</td>\n",
       "      <td>237.656</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>697</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>330</td>\n",
       "      <td>16</td>\n",
       "      <td>28</td>\n",
       "      <td>237.656</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>698</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>235</td>\n",
       "      <td>16</td>\n",
       "      <td>32</td>\n",
       "      <td>237.656</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>699</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>291</td>\n",
       "      <td>31</td>\n",
       "      <td>40</td>\n",
       "      <td>237.656</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>700 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Reason 1  Reason 2  Reason 3  Reason 4  Month Value  Day of the Week  \\\n",
       "0           0         0         0         1            7                1   \n",
       "1           0         0         0         0            7                1   \n",
       "2           0         0         0         1            7                2   \n",
       "3           1         0         0         0            7                3   \n",
       "4           0         0         0         1            7                3   \n",
       "..        ...       ...       ...       ...          ...              ...   \n",
       "695         1         0         0         0            5                2   \n",
       "696         1         0         0         0            5                2   \n",
       "697         1         0         0         0            5                3   \n",
       "698         0         0         0         1            5                3   \n",
       "699         0         0         0         1            5                3   \n",
       "\n",
       "     Transportation Expense  Distance to Work  Age  Daily Work Load Average  \\\n",
       "0                       289                36   33                  239.554   \n",
       "1                       118                13   50                  239.554   \n",
       "2                       179                51   38                  239.554   \n",
       "3                       279                 5   39                  239.554   \n",
       "4                       289                36   33                  239.554   \n",
       "..                      ...               ...  ...                      ...   \n",
       "695                     179                22   40                  237.656   \n",
       "696                     225                26   28                  237.656   \n",
       "697                     330                16   28                  237.656   \n",
       "698                     235                16   32                  237.656   \n",
       "699                     291                31   40                  237.656   \n",
       "\n",
       "     Body Mass Index  Education  Children  Pets  \n",
       "0                 30          0         2     1  \n",
       "1                 31          0         1     0  \n",
       "2                 31          0         0     0  \n",
       "3                 24          0         2     0  \n",
       "4                 30          0         2     1  \n",
       "..               ...        ...       ...   ...  \n",
       "695               22          1         2     0  \n",
       "696               24          0         1     2  \n",
       "697               25          1         0     0  \n",
       "698               25          1         0     0  \n",
       "699               25          0         1     1  \n",
       "\n",
       "[700 rows x 14 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 'DataFrame.iloc[row indices, column indices]' --> selects (slices) data by position when given rows and columns wanted\n",
    "data_with_targets.iloc[:,0:14]  # all rows and columns 0 through 13; '.loc' is inclusive of the range whereas '.iloc' is exclusive \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Reason 1</th>\n",
       "      <th>Reason 2</th>\n",
       "      <th>Reason 3</th>\n",
       "      <th>Reason 4</th>\n",
       "      <th>Month Value</th>\n",
       "      <th>Day of the Week</th>\n",
       "      <th>Transportation Expense</th>\n",
       "      <th>Distance to Work</th>\n",
       "      <th>Age</th>\n",
       "      <th>Daily Work Load Average</th>\n",
       "      <th>Body Mass Index</th>\n",
       "      <th>Education</th>\n",
       "      <th>Children</th>\n",
       "      <th>Pets</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>289</td>\n",
       "      <td>36</td>\n",
       "      <td>33</td>\n",
       "      <td>239.554</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>118</td>\n",
       "      <td>13</td>\n",
       "      <td>50</td>\n",
       "      <td>239.554</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>179</td>\n",
       "      <td>51</td>\n",
       "      <td>38</td>\n",
       "      <td>239.554</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>279</td>\n",
       "      <td>5</td>\n",
       "      <td>39</td>\n",
       "      <td>239.554</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>289</td>\n",
       "      <td>36</td>\n",
       "      <td>33</td>\n",
       "      <td>239.554</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>695</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>179</td>\n",
       "      <td>22</td>\n",
       "      <td>40</td>\n",
       "      <td>237.656</td>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>696</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>225</td>\n",
       "      <td>26</td>\n",
       "      <td>28</td>\n",
       "      <td>237.656</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>697</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>330</td>\n",
       "      <td>16</td>\n",
       "      <td>28</td>\n",
       "      <td>237.656</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>698</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>235</td>\n",
       "      <td>16</td>\n",
       "      <td>32</td>\n",
       "      <td>237.656</td>\n",
       "      <td>25</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>699</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>291</td>\n",
       "      <td>31</td>\n",
       "      <td>40</td>\n",
       "      <td>237.656</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>700 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Reason 1  Reason 2  Reason 3  Reason 4  Month Value  Day of the Week  \\\n",
       "0           0         0         0         1            7                1   \n",
       "1           0         0         0         0            7                1   \n",
       "2           0         0         0         1            7                2   \n",
       "3           1         0         0         0            7                3   \n",
       "4           0         0         0         1            7                3   \n",
       "..        ...       ...       ...       ...          ...              ...   \n",
       "695         1         0         0         0            5                2   \n",
       "696         1         0         0         0            5                2   \n",
       "697         1         0         0         0            5                3   \n",
       "698         0         0         0         1            5                3   \n",
       "699         0         0         0         1            5                3   \n",
       "\n",
       "     Transportation Expense  Distance to Work  Age  Daily Work Load Average  \\\n",
       "0                       289                36   33                  239.554   \n",
       "1                       118                13   50                  239.554   \n",
       "2                       179                51   38                  239.554   \n",
       "3                       279                 5   39                  239.554   \n",
       "4                       289                36   33                  239.554   \n",
       "..                      ...               ...  ...                      ...   \n",
       "695                     179                22   40                  237.656   \n",
       "696                     225                26   28                  237.656   \n",
       "697                     330                16   28                  237.656   \n",
       "698                     235                16   32                  237.656   \n",
       "699                     291                31   40                  237.656   \n",
       "\n",
       "     Body Mass Index  Education  Children  Pets  \n",
       "0                 30          0         2     1  \n",
       "1                 31          0         1     0  \n",
       "2                 31          0         0     0  \n",
       "3                 24          0         2     0  \n",
       "4                 30          0         2     1  \n",
       "..               ...        ...       ...   ...  \n",
       "695               22          1         2     0  \n",
       "696               24          0         1     2  \n",
       "697               25          1         0     0  \n",
       "698               25          1         0     0  \n",
       "699               25          0         1     1  \n",
       "\n",
       "[700 rows x 14 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_with_targets.iloc[:,:-1]    #will give the same results without have to count the number of columns; just want all columns but the last"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "unscaled_inputs = data_with_targets.iloc[:,:-1] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standardize the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# THE FOLLOWING CODE (WHICH HAS BEEN COMMENTED OUT) IS BAD PRACTICE BECAUSE IT ALSO STANDARDIZES THE DUMMY VARIABLES\n",
    "#from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# declare standard scaler object\n",
    "#absenteeism_scaler = StandardScaler()       #EMPTY scaler object; no information in it yet\n",
    "                                           # will be use to subtract the mean and divide by the standard deviation variablewise (featurewise)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use this instead\n",
    "from sklearn.base import BaseEstimator, TransformerMixin\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "class CustomScaler(BaseEstimator,TransformerMixin):\n",
    "    \n",
    "    def __init__(self,columns,copy=True,with_mean=True,with_std=True):\n",
    "        self.scaler = StandardScaler(copy,with_mean,with_std)\n",
    "        self.columns = columns\n",
    "        self.mean_ = None\n",
    "        self.var_ = None\n",
    "    \n",
    "    def fit(self, X, y=None):\n",
    "        self.scaler.fit(X[self.columns], y)\n",
    "        self.mean_ = np.mean(X[self.columns])\n",
    "        self.var_ = np.var(X[self.columns])\n",
    "        return self\n",
    "    \n",
    "    def transform(self, X, y=None, copy=None):\n",
    "        init_col_order = X.columns\n",
    "        X_scaled = pd.DataFrame(self.scaler.transform(X[self.columns]), columns=self.columns)\n",
    "        X_not_scaled = X.loc[:,~X.columns.isin(self.columns)]\n",
    "        return pd.concat([X_not_scaled, X_scaled], axis=1)[init_col_order]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Reason 1', 'Reason 2', 'Reason 3', 'Reason 4', 'Month Value',\n",
       "       'Day of the Week', 'Transportation Expense', 'Distance to Work',\n",
       "       'Age', 'Daily Work Load Average', 'Body Mass Index', 'Education',\n",
       "       'Children', 'Pets'], dtype=object)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unscaled_inputs.columns.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# scale columns except for the (4) dummy variables\n",
    "#columns_to_scale = ['Month Value',\n",
    "#       'Day of the Week', 'Transportation Expense', 'Distance to Work',\n",
    "#       'Age', 'Daily Work Load Average', 'Body Mass Index', 'Education',\n",
    "#       'Children', 'Pets']\n",
    "\n",
    "columns_to_omit = ['Reason_1', 'Reason_2', 'Reason_3', 'Reason_4', 'Education']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List Comprehension -- a syntactic construct which allos us to create a list from existing lists based on loops, conditionals, etc.\n",
    "columns_to_scale = [x for x in unscaled_inputs.columns.values if x not in columns_to_omit]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "absenteeism_scaler = CustomScaler(columns_to_scale)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CustomScaler(columns=['Reason 1', 'Reason 2', 'Reason 3', 'Reason 4',\n",
       "                      'Month Value', 'Day of the Week',\n",
       "                      'Transportation Expense', 'Distance to Work', 'Age',\n",
       "                      'Daily Work Load Average', 'Body Mass Index', 'Children',\n",
       "                      'Pets'],\n",
       "             copy=None, with_mean=None, with_std=None)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "absenteeism_scaler.fit(unscaled_inputs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Reason 1</th>\n",
       "      <th>Reason 2</th>\n",
       "      <th>Reason 3</th>\n",
       "      <th>Reason 4</th>\n",
       "      <th>Month Value</th>\n",
       "      <th>Day of the Week</th>\n",
       "      <th>Transportation Expense</th>\n",
       "      <th>Distance to Work</th>\n",
       "      <th>Age</th>\n",
       "      <th>Daily Work Load Average</th>\n",
       "      <th>Body Mass Index</th>\n",
       "      <th>Education</th>\n",
       "      <th>Children</th>\n",
       "      <th>Pets</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>-0.577350</td>\n",
       "      <td>-0.092981</td>\n",
       "      <td>-0.314485</td>\n",
       "      <td>0.821365</td>\n",
       "      <td>0.030796</td>\n",
       "      <td>-0.800950</td>\n",
       "      <td>1.005844</td>\n",
       "      <td>0.412816</td>\n",
       "      <td>-0.536062</td>\n",
       "      <td>-0.806331</td>\n",
       "      <td>0.767431</td>\n",
       "      <td>0</td>\n",
       "      <td>0.880469</td>\n",
       "      <td>0.268487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>-0.577350</td>\n",
       "      <td>-0.092981</td>\n",
       "      <td>-0.314485</td>\n",
       "      <td>-1.217485</td>\n",
       "      <td>0.030796</td>\n",
       "      <td>-0.800950</td>\n",
       "      <td>-1.574681</td>\n",
       "      <td>-1.141882</td>\n",
       "      <td>2.130803</td>\n",
       "      <td>-0.806331</td>\n",
       "      <td>1.002633</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.019280</td>\n",
       "      <td>-0.589690</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>-0.577350</td>\n",
       "      <td>-0.092981</td>\n",
       "      <td>-0.314485</td>\n",
       "      <td>0.821365</td>\n",
       "      <td>0.030796</td>\n",
       "      <td>-0.232900</td>\n",
       "      <td>-0.654143</td>\n",
       "      <td>1.426749</td>\n",
       "      <td>0.248310</td>\n",
       "      <td>-0.806331</td>\n",
       "      <td>1.002633</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.919030</td>\n",
       "      <td>-0.589690</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1.732051</td>\n",
       "      <td>-0.092981</td>\n",
       "      <td>-0.314485</td>\n",
       "      <td>-1.217485</td>\n",
       "      <td>0.030796</td>\n",
       "      <td>0.335149</td>\n",
       "      <td>0.854936</td>\n",
       "      <td>-1.682647</td>\n",
       "      <td>0.405184</td>\n",
       "      <td>-0.806331</td>\n",
       "      <td>-0.643782</td>\n",
       "      <td>0</td>\n",
       "      <td>0.880469</td>\n",
       "      <td>-0.589690</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>-0.577350</td>\n",
       "      <td>-0.092981</td>\n",
       "      <td>-0.314485</td>\n",
       "      <td>0.821365</td>\n",
       "      <td>0.030796</td>\n",
       "      <td>0.335149</td>\n",
       "      <td>1.005844</td>\n",
       "      <td>0.412816</td>\n",
       "      <td>-0.536062</td>\n",
       "      <td>-0.806331</td>\n",
       "      <td>0.767431</td>\n",
       "      <td>0</td>\n",
       "      <td>0.880469</td>\n",
       "      <td>0.268487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>695</td>\n",
       "      <td>1.732051</td>\n",
       "      <td>-0.092981</td>\n",
       "      <td>-0.314485</td>\n",
       "      <td>-1.217485</td>\n",
       "      <td>-0.568019</td>\n",
       "      <td>-0.232900</td>\n",
       "      <td>-0.654143</td>\n",
       "      <td>-0.533522</td>\n",
       "      <td>0.562059</td>\n",
       "      <td>-0.853789</td>\n",
       "      <td>-1.114186</td>\n",
       "      <td>1</td>\n",
       "      <td>0.880469</td>\n",
       "      <td>-0.589690</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>696</td>\n",
       "      <td>1.732051</td>\n",
       "      <td>-0.092981</td>\n",
       "      <td>-0.314485</td>\n",
       "      <td>-1.217485</td>\n",
       "      <td>-0.568019</td>\n",
       "      <td>-0.232900</td>\n",
       "      <td>0.040034</td>\n",
       "      <td>-0.263140</td>\n",
       "      <td>-1.320435</td>\n",
       "      <td>-0.853789</td>\n",
       "      <td>-0.643782</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.019280</td>\n",
       "      <td>1.126663</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>697</td>\n",
       "      <td>1.732051</td>\n",
       "      <td>-0.092981</td>\n",
       "      <td>-0.314485</td>\n",
       "      <td>-1.217485</td>\n",
       "      <td>-0.568019</td>\n",
       "      <td>0.335149</td>\n",
       "      <td>1.624567</td>\n",
       "      <td>-0.939096</td>\n",
       "      <td>-1.320435</td>\n",
       "      <td>-0.853789</td>\n",
       "      <td>-0.408580</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.919030</td>\n",
       "      <td>-0.589690</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>698</td>\n",
       "      <td>-0.577350</td>\n",
       "      <td>-0.092981</td>\n",
       "      <td>-0.314485</td>\n",
       "      <td>0.821365</td>\n",
       "      <td>-0.568019</td>\n",
       "      <td>0.335149</td>\n",
       "      <td>0.190942</td>\n",
       "      <td>-0.939096</td>\n",
       "      <td>-0.692937</td>\n",
       "      <td>-0.853789</td>\n",
       "      <td>-0.408580</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.919030</td>\n",
       "      <td>-0.589690</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>699</td>\n",
       "      <td>-0.577350</td>\n",
       "      <td>-0.092981</td>\n",
       "      <td>-0.314485</td>\n",
       "      <td>0.821365</td>\n",
       "      <td>-0.568019</td>\n",
       "      <td>0.335149</td>\n",
       "      <td>1.036026</td>\n",
       "      <td>0.074838</td>\n",
       "      <td>0.562059</td>\n",
       "      <td>-0.853789</td>\n",
       "      <td>-0.408580</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.019280</td>\n",
       "      <td>0.268487</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>700 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Reason 1  Reason 2  Reason 3  Reason 4  Month Value  Day of the Week  \\\n",
       "0   -0.577350 -0.092981 -0.314485  0.821365     0.030796        -0.800950   \n",
       "1   -0.577350 -0.092981 -0.314485 -1.217485     0.030796        -0.800950   \n",
       "2   -0.577350 -0.092981 -0.314485  0.821365     0.030796        -0.232900   \n",
       "3    1.732051 -0.092981 -0.314485 -1.217485     0.030796         0.335149   \n",
       "4   -0.577350 -0.092981 -0.314485  0.821365     0.030796         0.335149   \n",
       "..        ...       ...       ...       ...          ...              ...   \n",
       "695  1.732051 -0.092981 -0.314485 -1.217485    -0.568019        -0.232900   \n",
       "696  1.732051 -0.092981 -0.314485 -1.217485    -0.568019        -0.232900   \n",
       "697  1.732051 -0.092981 -0.314485 -1.217485    -0.568019         0.335149   \n",
       "698 -0.577350 -0.092981 -0.314485  0.821365    -0.568019         0.335149   \n",
       "699 -0.577350 -0.092981 -0.314485  0.821365    -0.568019         0.335149   \n",
       "\n",
       "     Transportation Expense  Distance to Work       Age  \\\n",
       "0                  1.005844          0.412816 -0.536062   \n",
       "1                 -1.574681         -1.141882  2.130803   \n",
       "2                 -0.654143          1.426749  0.248310   \n",
       "3                  0.854936         -1.682647  0.405184   \n",
       "4                  1.005844          0.412816 -0.536062   \n",
       "..                      ...               ...       ...   \n",
       "695               -0.654143         -0.533522  0.562059   \n",
       "696                0.040034         -0.263140 -1.320435   \n",
       "697                1.624567         -0.939096 -1.320435   \n",
       "698                0.190942         -0.939096 -0.692937   \n",
       "699                1.036026          0.074838  0.562059   \n",
       "\n",
       "     Daily Work Load Average  Body Mass Index  Education  Children      Pets  \n",
       "0                  -0.806331         0.767431          0  0.880469  0.268487  \n",
       "1                  -0.806331         1.002633          0 -0.019280 -0.589690  \n",
       "2                  -0.806331         1.002633          0 -0.919030 -0.589690  \n",
       "3                  -0.806331        -0.643782          0  0.880469 -0.589690  \n",
       "4                  -0.806331         0.767431          0  0.880469  0.268487  \n",
       "..                       ...              ...        ...       ...       ...  \n",
       "695                -0.853789        -1.114186          1  0.880469 -0.589690  \n",
       "696                -0.853789        -0.643782          0 -0.019280  1.126663  \n",
       "697                -0.853789        -0.408580          1 -0.919030 -0.589690  \n",
       "698                -0.853789        -0.408580          1 -0.919030 -0.589690  \n",
       "699                -0.853789        -0.408580          0 -0.019280  0.268487  \n",
       "\n",
       "[700 rows x 14 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# scale the inputs\n",
    "scaled_inputs = absenteeism_scaler.transform(unscaled_inputs)\n",
    "scaled_inputs\n",
    "\n",
    "# new_data_raw = pd.read_csv('new_data.csv')\n",
    "# new_data_scaled = absenteeism_scaler.transform(new_data_raw)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(700, 14)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scaled_inputs.shape   # print out size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the Data into Train & Test, then Shuffle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import the Relevant Module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "#splits arrays or matrices into random train and test subsets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[     Reason 1  Reason 2  Reason 3  Reason 4  Month Value  Day of the Week  \\\n",
       " 523  1.732051 -0.092981 -0.314485 -1.217485     0.929019         0.335149   \n",
       " 113 -0.577350 -0.092981 -0.314485  0.821365    -0.268611        -0.232900   \n",
       " 457 -0.577350 -0.092981 -0.314485  0.821365    -0.268611         0.903199   \n",
       " 619 -0.577350 -0.092981 -0.314485  0.821365    -0.568019         0.335149   \n",
       " 461 -0.577350 -0.092981 -0.314485  0.821365    -0.268611        -0.232900   \n",
       " ..        ...       ...       ...       ...          ...              ...   \n",
       " 252 -0.577350 -0.092981  3.179797 -1.217485    -0.867426         0.903199   \n",
       " 544 -0.577350 -0.092981 -0.314485  0.821365     1.228426        -1.368999   \n",
       " 104 -0.577350 -0.092981  3.179797 -1.217485     0.330204        -0.232900   \n",
       " 82   1.732051 -0.092981 -0.314485 -1.217485    -0.568019        -1.368999   \n",
       " 121 -0.577350 -0.092981 -0.314485  0.821365     1.228426        -0.800950   \n",
       " \n",
       "      Transportation Expense  Distance to Work       Age  \\\n",
       " 523                2.213108         -0.871500 -0.849811   \n",
       " 113                0.040034         -0.263140 -1.320435   \n",
       " 457                0.356940         -0.330735  0.718933   \n",
       " 619                0.387122         -0.330735  1.660180   \n",
       " 461                0.040034         -0.263140 -1.320435   \n",
       " ..                      ...               ...       ...   \n",
       " 252                1.005844          0.412816 -0.536062   \n",
       " 544               -0.654143          1.426749  0.248310   \n",
       " 104               -1.574681         -1.344669  0.091435   \n",
       " 82                -0.654143          1.426749  0.248310   \n",
       " 121               -1.574681         -1.344669  0.091435   \n",
       " \n",
       "      Daily Work Load Average  Body Mass Index  Education  Children      Pets  \n",
       " 523                 0.326336        -0.408580          0  1.780219 -0.589690  \n",
       " 113                 0.919937        -0.643782          0 -0.019280  1.126663  \n",
       " 457                -0.446195        -0.878984          0 -0.919030 -0.589690  \n",
       " 619                -1.240355         1.237836          0  0.880469  0.268487  \n",
       " 461                -1.037971        -0.643782          0 -0.019280  1.126663  \n",
       " ..                       ...              ...        ...       ...       ...  \n",
       " 252                -0.154696         0.767431          0  0.880469  0.268487  \n",
       " 544                -0.082083         1.002633          0 -0.919030 -0.589690  \n",
       " 104                -0.262439         0.297027          0 -0.919030 -0.589690  \n",
       " 82                  0.863727         1.002633          0 -0.919030 -0.589690  \n",
       " 121                 0.919937         0.297027          0 -0.919030 -0.589690  \n",
       " \n",
       " [525 rows x 14 columns],\n",
       "      Reason 1  Reason 2  Reason 3  Reason 4  Month Value  Day of the Week  \\\n",
       " 148 -0.577350 -0.092981 -0.314485  0.821365    -1.466241        -0.800950   \n",
       " 270  1.732051 -0.092981 -0.314485 -1.217485    -0.568019        -1.368999   \n",
       " 352  1.732051 -0.092981 -0.314485 -1.217485     1.527833        -0.800950   \n",
       " 55  -0.577350 -0.092981 -0.314485 -1.217485     0.629611        -0.800950   \n",
       " 545 -0.577350 -0.092981 -0.314485  0.821365     1.228426        -1.368999   \n",
       " ..        ...       ...       ...       ...          ...              ...   \n",
       " 556 -0.577350 -0.092981 -0.314485  0.821365    -0.268611        -1.368999   \n",
       " 413 -0.577350 -0.092981 -0.314485  0.821365     0.030796        -0.800950   \n",
       " 653 -0.577350 -0.092981 -0.314485  0.821365    -1.166834        -0.232900   \n",
       " 169  1.732051 -0.092981 -0.314485 -1.217485    -1.166834         0.335149   \n",
       " 34  -0.577350 -0.092981 -0.314485  0.821365     0.330204        -1.368999   \n",
       " \n",
       "      Transportation Expense  Distance to Work       Age  \\\n",
       " 148               -0.654143         -0.263140 -1.006686   \n",
       " 270               -0.654143          1.426749  0.248310   \n",
       " 352                0.190942         -0.668713  1.032682   \n",
       " 55                -1.574681         -1.141882  2.130803   \n",
       " 545               -1.574681         -1.344669  0.091435   \n",
       " ..                      ...               ...       ...   \n",
       " 556                1.036026          0.074838  0.562059   \n",
       " 413                0.568211          1.359154 -0.065439   \n",
       " 653                2.213108         -0.871500 -0.849811   \n",
       " 169               -0.654143         -0.533522  0.562059   \n",
       " 34                -0.654143          1.426749  0.248310   \n",
       " \n",
       "      Daily Work Load Average  Body Mass Index  Education  Children      Pets  \n",
       " 148                 0.769711        -1.819793          1 -0.919030 -0.589690  \n",
       " 270                 0.560476         1.002633          0 -0.919030 -0.589690  \n",
       " 352                -0.879469         2.649049          0 -0.019280 -0.589690  \n",
       " 55                 -0.758273         1.002633          0 -0.019280 -0.589690  \n",
       " 545                -0.082083         0.297027          0 -0.919030 -0.589690  \n",
       " ..                       ...              ...        ...       ...       ...  \n",
       " 556                 0.218718        -0.408580          0 -0.019280  0.268487  \n",
       " 413                -0.809957        -0.878984          0  2.679969 -0.589690  \n",
       " 653                -1.240355        -0.408580          0  1.780219 -0.589690  \n",
       " 169                 1.786584        -1.114186          1  0.880469 -0.589690  \n",
       " 34                 -1.647399         1.002633          0 -0.919030 -0.589690  \n",
       " \n",
       " [175 rows x 14 columns],\n",
       " array([1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0,\n",
       "        0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0,\n",
       "        1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1,\n",
       "        0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1,\n",
       "        1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 1, 0,\n",
       "        1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1,\n",
       "        0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0,\n",
       "        0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1,\n",
       "        1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0,\n",
       "        1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1,\n",
       "        1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0,\n",
       "        1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1,\n",
       "        1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 1,\n",
       "        1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0,\n",
       "        0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0,\n",
       "        1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1,\n",
       "        1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0,\n",
       "        1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 0,\n",
       "        0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1,\n",
       "        0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0,\n",
       "        0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0,\n",
       "        1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1,\n",
       "        0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0,\n",
       "        1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0]),\n",
       " array([0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0,\n",
       "        0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0,\n",
       "        1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1,\n",
       "        1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0,\n",
       "        1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0,\n",
       "        1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1,\n",
       "        0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1,\n",
       "        1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0])]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_test_split(scaled_inputs, targets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# default split shows training size = 75% and test size = 25%\n",
    "# add parameters 'train _size = 0.8' to change training size = 80%\n",
    "# 'sklearn.mode_selection.train_test_split(inputs, targets, train_size, shuffle=True, random_state)'\n",
    "# splits arrays or matrices into random train and test subsets\n",
    "# rerunning our code, we get a different split every time due to shuffle by default \n",
    "\n",
    "x_train, x_test, y_train, y_test = train_test_split(scaled_inputs,targets, train_size = 0.8, random_state = 20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(560, 14) (560,)\n"
     ]
    }
   ],
   "source": [
    "print(x_train.shape, y_train.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(140, 14) (140,)\n"
     ]
    }
   ],
   "source": [
    "print(x_test.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression with Sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# declare Logistic Regression object\n",
    "reg = LogisticRegression(solver='liblinear')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
       "                   multi_class='warn', n_jobs=None, penalty='l2',\n",
       "                   random_state=None, solver='liblinear', tol=0.0001, verbose=0,\n",
       "                   warm_start=False)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7821428571428571"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg.score(x_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# can conclude the model has an accuracy of about 80%\n",
    "# also stated as 80% of the model outputs match the targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Manually Check the Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0,\n",
       "       0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1,\n",
       "       0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0,\n",
       "       0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0,\n",
       "       0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0,\n",
       "       0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0,\n",
       "       0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1,\n",
       "       1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0,\n",
       "       0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1,\n",
       "       1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1,\n",
       "       1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1,\n",
       "       0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0,\n",
       "       0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0,\n",
       "       1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1,\n",
       "       1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1,\n",
       "       0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0,\n",
       "       1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0,\n",
       "       0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0,\n",
       "       0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0,\n",
       "       1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0,\n",
       "       1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0,\n",
       "       0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1,\n",
       "       0, 1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1,\n",
       "       0, 1, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 0,\n",
       "       0, 1, 0, 1, 1, 1, 0, 0, 0, 0])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_outputs = reg.predict(x_train)\n",
    "model_outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0,\n",
       "       1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1,\n",
       "       1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0,\n",
       "       0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1,\n",
       "       1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0,\n",
       "       0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1,\n",
       "       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1,\n",
       "       0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0,\n",
       "       0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1,\n",
       "       1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0,\n",
       "       1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0,\n",
       "       0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0,\n",
       "       0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0,\n",
       "       1, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0,\n",
       "       0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1,\n",
       "       1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1,\n",
       "       0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0,\n",
       "       1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0,\n",
       "       0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1,\n",
       "       0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,\n",
       "       0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0,\n",
       "       1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0,\n",
       "       1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1,\n",
       "       0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0,\n",
       "       0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0,\n",
       "       0, 0, 0, 1, 1, 1, 1, 0, 1, 0])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ True,  True, False,  True,  True,  True,  True,  True,  True,\n",
       "        True, False,  True, False, False,  True,  True,  True,  True,\n",
       "       False,  True, False,  True, False, False,  True,  True,  True,\n",
       "       False,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "       False, False, False, False,  True, False,  True,  True, False,\n",
       "        True,  True,  True,  True,  True, False,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True, False,  True,  True,  True,\n",
       "        True,  True,  True,  True, False,  True, False,  True,  True,\n",
       "        True,  True,  True, False,  True,  True,  True,  True,  True,\n",
       "       False,  True, False,  True,  True, False, False, False,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True, False,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True, False,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True, False,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True, False,  True,  True,  True,\n",
       "        True, False,  True,  True,  True,  True,  True,  True, False,\n",
       "        True, False,  True, False,  True,  True,  True,  True, False,\n",
       "        True, False,  True,  True, False,  True,  True,  True,  True,\n",
       "        True, False,  True, False,  True, False,  True, False,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True, False,  True,  True,  True,\n",
       "        True, False,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True, False,  True, False,\n",
       "       False,  True,  True,  True,  True,  True,  True,  True, False,\n",
       "        True, False,  True, False,  True,  True,  True,  True, False,\n",
       "        True, False, False,  True,  True,  True,  True,  True, False,\n",
       "       False, False,  True, False,  True,  True,  True, False,  True,\n",
       "        True,  True,  True,  True,  True,  True, False,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True, False, False,  True,  True,  True,  True,  True,  True,\n",
       "       False,  True,  True,  True,  True,  True,  True,  True, False,\n",
       "       False,  True,  True,  True,  True, False,  True, False,  True,\n",
       "        True,  True,  True,  True,  True,  True, False,  True, False,\n",
       "       False,  True,  True,  True,  True,  True, False,  True,  True,\n",
       "        True,  True, False, False,  True, False,  True,  True,  True,\n",
       "        True,  True, False,  True,  True, False,  True,  True, False,\n",
       "        True, False,  True,  True,  True,  True,  True,  True, False,\n",
       "        True,  True,  True,  True, False,  True,  True,  True, False,\n",
       "        True, False,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "       False,  True,  True, False,  True, False,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True, False,  True,\n",
       "        True,  True,  True, False,  True,  True,  True, False,  True,\n",
       "        True,  True,  True, False,  True, False,  True, False,  True,\n",
       "        True,  True,  True,  True,  True, False,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True, False,  True, False,  True,  True,  True,\n",
       "        True,  True,  True, False,  True,  True, False,  True, False,\n",
       "        True,  True,  True,  True,  True, False, False,  True, False,\n",
       "        True,  True,  True,  True,  True,  True,  True, False,  True,\n",
       "       False,  True,  True,  True, False, False,  True,  True,  True,\n",
       "        True, False,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True, False,  True,  True, False, False,  True,  True,\n",
       "       False,  True,  True,  True,  True,  True,  True, False,  True,\n",
       "        True,  True, False,  True,  True,  True,  True, False,  True,\n",
       "        True,  True,  True,  True, False,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True, False,  True,  True,  True,  True,\n",
       "       False,  True,  True,  True,  True,  True, False,  True,  True,\n",
       "       False, False, False,  True,  True, False,  True,  True,  True,\n",
       "       False,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True, False,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True, False,  True,  True,  True,  True, False,  True,\n",
       "       False,  True])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# hard to compare with the naked eye\n",
    "# use code to compare the outputs to the targets\n",
    "model_outputs == y_train\n",
    "\n",
    "# True = matches\n",
    "# False = does not match"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "438"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Boolean: True = 1 and False = 0\n",
    "\n",
    "np.sum((model_outputs==y_train))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "560"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_outputs.shape[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7821428571428571"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum((model_outputs==y_train)) / model_outputs.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding the Intercept and Coefficients of Linear Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.14352457])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg.intercept_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2.06899633,  0.32883249,  1.56068887,  1.31090995,  0.02582743,\n",
       "        -0.08621396,  0.72169608, -0.05835588, -0.20543182, -0.0273594 ,\n",
       "         0.32990512, -0.39927087,  0.38392076, -0.32004597]])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we wnat to know what variables these coeffecients refer to"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Reason 1', 'Reason 2', 'Reason 3', 'Reason 4', 'Month Value',\n",
       "       'Day of the Week', 'Transportation Expense', 'Distance to Work',\n",
       "       'Age', 'Daily Work Load Average', 'Body Mass Index', 'Education',\n",
       "       'Children', 'Pets'], dtype=object)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unscaled_inputs.columns.values\n",
    "# 'scaled_inputs.columns.values' will receive an error due to employing sklearn, the results are arrays and not dataframes\n",
    "# thus must use unscaled_inputs and then create a dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_name = unscaled_inputs.columns.values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Feature Name</th>\n",
       "      <th>Coefficient</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>Reason 1</td>\n",
       "      <td>2.068996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>Reason 2</td>\n",
       "      <td>0.328832</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>Reason 3</td>\n",
       "      <td>1.560689</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>Reason 4</td>\n",
       "      <td>1.310910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>Month Value</td>\n",
       "      <td>0.025827</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>Day of the Week</td>\n",
       "      <td>-0.086214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>Transportation Expense</td>\n",
       "      <td>0.721696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>Distance to Work</td>\n",
       "      <td>-0.058356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>Age</td>\n",
       "      <td>-0.205432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>Daily Work Load Average</td>\n",
       "      <td>-0.027359</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>Body Mass Index</td>\n",
       "      <td>0.329905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>Education</td>\n",
       "      <td>-0.399271</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12</td>\n",
       "      <td>Children</td>\n",
       "      <td>0.383921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13</td>\n",
       "      <td>Pets</td>\n",
       "      <td>-0.320046</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Feature Name  Coefficient\n",
       "0                  Reason 1     2.068996\n",
       "1                  Reason 2     0.328832\n",
       "2                  Reason 3     1.560689\n",
       "3                  Reason 4     1.310910\n",
       "4               Month Value     0.025827\n",
       "5           Day of the Week    -0.086214\n",
       "6    Transportation Expense     0.721696\n",
       "7          Distance to Work    -0.058356\n",
       "8                       Age    -0.205432\n",
       "9   Daily Work Load Average    -0.027359\n",
       "10          Body Mass Index     0.329905\n",
       "11                Education    -0.399271\n",
       "12                 Children     0.383921\n",
       "13                     Pets    -0.320046"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "summary_table = pd.DataFrame(columns=['Feature Name'], data = feature_name)\n",
    "\n",
    "# must transpose the array, because by default, np.arrays are rows and we want columns\n",
    "summary_table['Coefficient'] = np.transpose(reg.coef_)\n",
    "\n",
    "summary_table     # prints summary tables of the variables and correlating coefficients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Feature Name</th>\n",
       "      <th>Coefficient</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>Intercept</td>\n",
       "      <td>-0.143525</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>Reason 1</td>\n",
       "      <td>2.068996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>Reason 2</td>\n",
       "      <td>0.328832</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>Reason 3</td>\n",
       "      <td>1.560689</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>Reason 4</td>\n",
       "      <td>1.310910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>Month Value</td>\n",
       "      <td>0.025827</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>Day of the Week</td>\n",
       "      <td>-0.086214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>Transportation Expense</td>\n",
       "      <td>0.721696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>Distance to Work</td>\n",
       "      <td>-0.058356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>Age</td>\n",
       "      <td>-0.205432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>Daily Work Load Average</td>\n",
       "      <td>-0.027359</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>Body Mass Index</td>\n",
       "      <td>0.329905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12</td>\n",
       "      <td>Education</td>\n",
       "      <td>-0.399271</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13</td>\n",
       "      <td>Children</td>\n",
       "      <td>0.383921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14</td>\n",
       "      <td>Pets</td>\n",
       "      <td>-0.320046</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Feature Name  Coefficient\n",
       "0                 Intercept    -0.143525\n",
       "1                  Reason 1     2.068996\n",
       "2                  Reason 2     0.328832\n",
       "3                  Reason 3     1.560689\n",
       "4                  Reason 4     1.310910\n",
       "5               Month Value     0.025827\n",
       "6           Day of the Week    -0.086214\n",
       "7    Transportation Expense     0.721696\n",
       "8          Distance to Work    -0.058356\n",
       "9                       Age    -0.205432\n",
       "10  Daily Work Load Average    -0.027359\n",
       "11          Body Mass Index     0.329905\n",
       "12                Education    -0.399271\n",
       "13                 Children     0.383921\n",
       "14                     Pets    -0.320046"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add one to all indices of dataframe 'summary_table'\n",
    "summary_table.index = summary_table.index + 1\n",
    "summary_table.loc[0] = ['Intercept', reg.intercept_[0]]\n",
    "summary_table = summary_table.sort_index()\n",
    "summary_table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Standardized Coefficients are the coefficients of a regression where all variables have been standardized"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interpreting the Coefficients"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "log(odds) = intercept + (b_1 * x_1) + (b_2 * x_2) +...+ (b_n * x_n)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Feature Name</th>\n",
       "      <th>Coefficient</th>\n",
       "      <th>Odds_ratio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>Intercept</td>\n",
       "      <td>-0.143525</td>\n",
       "      <td>0.866300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>Reason 1</td>\n",
       "      <td>2.068996</td>\n",
       "      <td>7.916873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>Reason 2</td>\n",
       "      <td>0.328832</td>\n",
       "      <td>1.389345</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>Reason 3</td>\n",
       "      <td>1.560689</td>\n",
       "      <td>4.762101</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>Reason 4</td>\n",
       "      <td>1.310910</td>\n",
       "      <td>3.709548</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Feature Name  Coefficient  Odds_ratio\n",
       "0    Intercept    -0.143525    0.866300\n",
       "1     Reason 1     2.068996    7.916873\n",
       "2     Reason 2     0.328832    1.389345\n",
       "3     Reason 3     1.560689    4.762101\n",
       "4     Reason 4     1.310910    3.709548"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "summary_table['Odds_ratio'] = np.exp(summary_table.Coefficient)\n",
    "summary_table.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Feature Name</th>\n",
       "      <th>Coefficient</th>\n",
       "      <th>Odds_ratio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>Reason 1</td>\n",
       "      <td>2.068996</td>\n",
       "      <td>7.916873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>Reason 3</td>\n",
       "      <td>1.560689</td>\n",
       "      <td>4.762101</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>Reason 4</td>\n",
       "      <td>1.310910</td>\n",
       "      <td>3.709548</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>Transportation Expense</td>\n",
       "      <td>0.721696</td>\n",
       "      <td>2.057921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13</td>\n",
       "      <td>Children</td>\n",
       "      <td>0.383921</td>\n",
       "      <td>1.468029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>Body Mass Index</td>\n",
       "      <td>0.329905</td>\n",
       "      <td>1.390836</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>Reason 2</td>\n",
       "      <td>0.328832</td>\n",
       "      <td>1.389345</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>Month Value</td>\n",
       "      <td>0.025827</td>\n",
       "      <td>1.026164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>Daily Work Load Average</td>\n",
       "      <td>-0.027359</td>\n",
       "      <td>0.973011</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>Distance to Work</td>\n",
       "      <td>-0.058356</td>\n",
       "      <td>0.943314</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>Day of the Week</td>\n",
       "      <td>-0.086214</td>\n",
       "      <td>0.917398</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>Intercept</td>\n",
       "      <td>-0.143525</td>\n",
       "      <td>0.866300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>Age</td>\n",
       "      <td>-0.205432</td>\n",
       "      <td>0.814296</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14</td>\n",
       "      <td>Pets</td>\n",
       "      <td>-0.320046</td>\n",
       "      <td>0.726116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12</td>\n",
       "      <td>Education</td>\n",
       "      <td>-0.399271</td>\n",
       "      <td>0.670809</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Feature Name  Coefficient  Odds_ratio\n",
       "1                  Reason 1     2.068996    7.916873\n",
       "3                  Reason 3     1.560689    4.762101\n",
       "4                  Reason 4     1.310910    3.709548\n",
       "7    Transportation Expense     0.721696    2.057921\n",
       "13                 Children     0.383921    1.468029\n",
       "11          Body Mass Index     0.329905    1.390836\n",
       "2                  Reason 2     0.328832    1.389345\n",
       "5               Month Value     0.025827    1.026164\n",
       "10  Daily Work Load Average    -0.027359    0.973011\n",
       "8          Distance to Work    -0.058356    0.943314\n",
       "6           Day of the Week    -0.086214    0.917398\n",
       "0                 Intercept    -0.143525    0.866300\n",
       "9                       Age    -0.205432    0.814296\n",
       "14                     Pets    -0.320046    0.726116\n",
       "12                Education    -0.399271    0.670809"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 'DataFrame.sort_values(Series)' --> sorts the values in a data frame with respect to a given column (series)\n",
    "summary_table.sort_values('Odds_ratio', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A Feature is not particulary important if..\n",
    " - coefficient is close to '0':\n",
    "         - implies that no matter the feature value, we will multiply is by 0 (in the modela)\n",
    "         \n",
    " - coefficient is close to '1':\n",
    "         - for a unit change in the standardized fature, the odds increase by a multiple equal to the odds ratio (1 = no change)\n",
    "         - i.e.   odds * odds ratio = new odds\n",
    "                  5:1  *      2     = 10:1\n",
    "                  5:1  *     0.2    = 1:1\n",
    "                  5:1  *      1     = 5:1 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interpreting the Summary Table:\n",
    " - the variables 'Daily Work Load Average', 'Distance to Work', and 'Day of the Week' may be dropped due to their coefficients being close to 0\n",
    " - When employees give Reasons 1 through 4, there seems to be a likelihood of future absenteeism"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BACKWARD ELIMINATION\n",
    " - The idea is that we can simplify our model by removing all features which have close to no contribution to the model\n",
    " - When we ahve the p-values, we get rid of all coeff with p-values > 0.05\n",
    " - if the weight is small enough, it won't make a difference; if these variables are removed, the rest of the model should not really change in terms of coefficient values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Testing the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7357142857142858"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg.score(x_test, y_test)   #reg.score(train, test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on data the model has NEVER seen before, in 73.6% of the cases, the model will predict (correctly) if a person is going to be excessively absent\n",
    "\n",
    "Often the test accuracy is 10-20% lower than the train accuract (due to overfitting)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.76424392, 0.23575608],\n",
       "       [0.61299367, 0.38700633],\n",
       "       [0.4064611 , 0.5935389 ],\n",
       "       [0.7852416 , 0.2147584 ],\n",
       "       [0.06091598, 0.93908402],\n",
       "       [0.26799665, 0.73200335],\n",
       "       [0.28704915, 0.71295085],\n",
       "       [0.06715365, 0.93284635],\n",
       "       [0.75947822, 0.24052178],\n",
       "       [0.76485305, 0.23514695],\n",
       "       [0.47335262, 0.52664738],\n",
       "       [0.13663258, 0.86336742],\n",
       "       [0.04179783, 0.95820217],\n",
       "       [0.68987638, 0.31012362],\n",
       "       [0.20845446, 0.79154554],\n",
       "       [0.47262008, 0.52737992],\n",
       "       [0.49323601, 0.50676399],\n",
       "       [0.53775049, 0.46224951],\n",
       "       [0.38057849, 0.61942151],\n",
       "       [0.0294033 , 0.9705967 ],\n",
       "       [0.73485646, 0.26514354],\n",
       "       [0.77686748, 0.22313252],\n",
       "       [0.42793913, 0.57206087],\n",
       "       [0.44441853, 0.55558147],\n",
       "       [0.15455462, 0.84544538],\n",
       "       [0.76056028, 0.23943972],\n",
       "       [0.45911251, 0.54088749],\n",
       "       [0.89668308, 0.10331692],\n",
       "       [0.16377217, 0.83622783],\n",
       "       [0.7682632 , 0.2317368 ],\n",
       "       [0.58763671, 0.41236329],\n",
       "       [0.259638  , 0.740362  ],\n",
       "       [0.29863569, 0.70136431],\n",
       "       [0.47065131, 0.52934869],\n",
       "       [0.77686748, 0.22313252],\n",
       "       [0.38342183, 0.61657817],\n",
       "       [0.74322116, 0.25677884],\n",
       "       [0.16947703, 0.83052297],\n",
       "       [0.52457476, 0.47542524],\n",
       "       [0.39314269, 0.60685731],\n",
       "       [0.75714385, 0.24285615],\n",
       "       [0.57877705, 0.42122295],\n",
       "       [0.76588815, 0.23411185],\n",
       "       [0.90151002, 0.09848998],\n",
       "       [0.17172251, 0.82827749],\n",
       "       [0.27037303, 0.72962697],\n",
       "       [0.23984565, 0.76015435],\n",
       "       [0.75817192, 0.24182808],\n",
       "       [0.7302298 , 0.2697702 ],\n",
       "       [0.7980944 , 0.2019056 ],\n",
       "       [0.49711478, 0.50288522],\n",
       "       [0.92842303, 0.07157697],\n",
       "       [0.29462596, 0.70537404],\n",
       "       [0.74007061, 0.25992939],\n",
       "       [0.13268147, 0.86731853],\n",
       "       [0.56010403, 0.43989597],\n",
       "       [0.04803194, 0.95196806],\n",
       "       [0.70897706, 0.29102294],\n",
       "       [0.92154238, 0.07845762],\n",
       "       [0.92306618, 0.07693382],\n",
       "       [0.25233829, 0.74766171],\n",
       "       [0.31986943, 0.68013057],\n",
       "       [0.71391821, 0.28608179],\n",
       "       [0.14304253, 0.85695747],\n",
       "       [0.76112337, 0.23887663],\n",
       "       [0.76229842, 0.23770158],\n",
       "       [0.98844088, 0.01155912],\n",
       "       [0.75588413, 0.24411587],\n",
       "       [0.15821939, 0.84178061],\n",
       "       [0.68250406, 0.31749594],\n",
       "       [0.75683439, 0.24316561],\n",
       "       [0.92234701, 0.07765299],\n",
       "       [0.07761553, 0.92238447],\n",
       "       [0.53223815, 0.46776185],\n",
       "       [0.34815673, 0.65184327],\n",
       "       [0.7682632 , 0.2317368 ],\n",
       "       [0.150593  , 0.849407  ],\n",
       "       [0.24205347, 0.75794653],\n",
       "       [0.242469  , 0.757531  ],\n",
       "       [0.33826026, 0.66173974],\n",
       "       [0.73586181, 0.26413819],\n",
       "       [0.9873703 , 0.0126297 ],\n",
       "       [0.72010814, 0.27989186],\n",
       "       [0.27129905, 0.72870095],\n",
       "       [0.52363006, 0.47636994],\n",
       "       [0.8978327 , 0.1021673 ],\n",
       "       [0.28375317, 0.71624683],\n",
       "       [0.4539096 , 0.5460904 ],\n",
       "       [0.95504001, 0.04495999],\n",
       "       [0.26916225, 0.73083775],\n",
       "       [0.84419723, 0.15580277],\n",
       "       [0.88739397, 0.11260603],\n",
       "       [0.73938252, 0.26061748],\n",
       "       [0.77077697, 0.22922303],\n",
       "       [0.7820104 , 0.2179896 ],\n",
       "       [0.09046086, 0.90953914],\n",
       "       [0.77499868, 0.22500132],\n",
       "       [0.21908004, 0.78091996],\n",
       "       [0.76498174, 0.23501826],\n",
       "       [0.81556585, 0.18443415],\n",
       "       [0.35637467, 0.64362533],\n",
       "       [0.25033532, 0.74966468],\n",
       "       [0.27139118, 0.72860882],\n",
       "       [0.31030061, 0.68969939],\n",
       "       [0.54314558, 0.45685442],\n",
       "       [0.55132673, 0.44867327],\n",
       "       [0.78102356, 0.21897644],\n",
       "       [0.10329858, 0.89670142],\n",
       "       [0.23009999, 0.76990001],\n",
       "       [0.86211279, 0.13788721],\n",
       "       [0.98754576, 0.01245424],\n",
       "       [0.06828707, 0.93171293],\n",
       "       [0.2478512 , 0.7521488 ],\n",
       "       [0.58373315, 0.41626685],\n",
       "       [0.39339725, 0.60660275],\n",
       "       [0.37094988, 0.62905012],\n",
       "       [0.1618807 , 0.8381193 ],\n",
       "       [0.1438226 , 0.8561774 ],\n",
       "       [0.42717286, 0.57282714],\n",
       "       [0.71061606, 0.28938394],\n",
       "       [0.77672404, 0.22327596],\n",
       "       [0.88733527, 0.11266473],\n",
       "       [0.1335998 , 0.8664002 ],\n",
       "       [0.51177559, 0.48822441],\n",
       "       [0.75373351, 0.24626649],\n",
       "       [0.7162    , 0.2838    ],\n",
       "       [0.78313081, 0.21686919],\n",
       "       [0.86412402, 0.13587598],\n",
       "       [0.29854157, 0.70145843],\n",
       "       [0.72001847, 0.27998153],\n",
       "       [0.34734232, 0.65265768],\n",
       "       [0.77321023, 0.22678977],\n",
       "       [0.7758966 , 0.2241034 ],\n",
       "       [0.70507004, 0.29492996],\n",
       "       [0.71849301, 0.28150699],\n",
       "       [0.39296857, 0.60703143],\n",
       "       [0.5279669 , 0.4720331 ],\n",
       "       [0.70712899, 0.29287101],\n",
       "       [0.73754885, 0.26245115],\n",
       "       [0.49852796, 0.50147204]])"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 'sklearn.linear_model.LogisticRegression.predict_proba(x)' -- returns the probability estimates for all possible outputs (classes)\n",
    "predicted_proba = reg.predict_proba(x_test)\n",
    "predicted_proba  # first column: probability of being 0, second column: prob of being 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(140, 2)"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predicted_proba.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.23575608, 0.38700633, 0.5935389 , 0.2147584 , 0.93908402,\n",
       "       0.73200335, 0.71295085, 0.93284635, 0.24052178, 0.23514695,\n",
       "       0.52664738, 0.86336742, 0.95820217, 0.31012362, 0.79154554,\n",
       "       0.52737992, 0.50676399, 0.46224951, 0.61942151, 0.9705967 ,\n",
       "       0.26514354, 0.22313252, 0.57206087, 0.55558147, 0.84544538,\n",
       "       0.23943972, 0.54088749, 0.10331692, 0.83622783, 0.2317368 ,\n",
       "       0.41236329, 0.740362  , 0.70136431, 0.52934869, 0.22313252,\n",
       "       0.61657817, 0.25677884, 0.83052297, 0.47542524, 0.60685731,\n",
       "       0.24285615, 0.42122295, 0.23411185, 0.09848998, 0.82827749,\n",
       "       0.72962697, 0.76015435, 0.24182808, 0.2697702 , 0.2019056 ,\n",
       "       0.50288522, 0.07157697, 0.70537404, 0.25992939, 0.86731853,\n",
       "       0.43989597, 0.95196806, 0.29102294, 0.07845762, 0.07693382,\n",
       "       0.74766171, 0.68013057, 0.28608179, 0.85695747, 0.23887663,\n",
       "       0.23770158, 0.01155912, 0.24411587, 0.84178061, 0.31749594,\n",
       "       0.24316561, 0.07765299, 0.92238447, 0.46776185, 0.65184327,\n",
       "       0.2317368 , 0.849407  , 0.75794653, 0.757531  , 0.66173974,\n",
       "       0.26413819, 0.0126297 , 0.27989186, 0.72870095, 0.47636994,\n",
       "       0.1021673 , 0.71624683, 0.5460904 , 0.04495999, 0.73083775,\n",
       "       0.15580277, 0.11260603, 0.26061748, 0.22922303, 0.2179896 ,\n",
       "       0.90953914, 0.22500132, 0.78091996, 0.23501826, 0.18443415,\n",
       "       0.64362533, 0.74966468, 0.72860882, 0.68969939, 0.45685442,\n",
       "       0.44867327, 0.21897644, 0.89670142, 0.76990001, 0.13788721,\n",
       "       0.01245424, 0.93171293, 0.7521488 , 0.41626685, 0.60660275,\n",
       "       0.62905012, 0.8381193 , 0.8561774 , 0.57282714, 0.28938394,\n",
       "       0.22327596, 0.11266473, 0.8664002 , 0.48822441, 0.24626649,\n",
       "       0.2838    , 0.21686919, 0.13587598, 0.70145843, 0.27998153,\n",
       "       0.65265768, 0.22678977, 0.2241034 , 0.29492996, 0.28150699,\n",
       "       0.60703143, 0.4720331 , 0.29287101, 0.26245115, 0.50147204])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predicted_proba[:,1]\n",
    "\n",
    "# if the probability is below 0.5, it places a 0 and vice versa, a 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Save the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "'pickle[module]' -- is a Python module used to convert a Python object into a character stream"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- file name: model\n",
    "- write bytes: wb\n",
    "- dump = save\n",
    "- reg = object to be dumped"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('model', 'wb') as file:\n",
    "    pickle.dump(reg, file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Note on Pickling\n",
    " \n",
    "\n",
    "There are several popular ways to save (and finalize) a model. To name some, you can use Joblib (a part of the SciPy ecosystem), and JSON. Certainly, each of those choices has its pros and cons. Pickle is probably the most intuitive and definitely our preferred choice.\n",
    "\n",
    "Once again, ‘pickle’ is the standard Python tool for serialization and deserialization. In simple words, pickling means: converting a Python object (no matter what) into a string of characters. Logically, unpickling is about converting a string of characters (that has been pickled) into a Python object.\n",
    "\n",
    "\n",
    "\n",
    "There are some potential issues you should be aware of, though!\n",
    "\n",
    "Pickle and Python version.\n",
    "\n",
    "Pickling is strictly related to Python version. It is not recommended to (de)serialize objects across different Python versions. Logically, if you’re working on your own this will never be an issue (unless you upgrade/downgrade your Python version). \n",
    "\n",
    "\n",
    "\n",
    "Pickle is slow.\n",
    "\n",
    "Well, you will barely notice that but for complex structures it may take loads of time to pickle and unpickle.\n",
    "\n",
    "\n",
    "\n",
    "Pickle is not secure.\n",
    "\n",
    "This is evident from the documentation of pickle, quote: “Never unpickle data received from an untrusted or unauthenticated source.” The reason is that just about anything can be pickled, so you can easily unpickle malicious code.\n",
    "\n",
    "\n",
    "\n",
    "Now, if you are unpickling your own code, you are more or less safe.\n",
    "\n",
    "\n",
    "\n",
    "If, however, you receive pickled objects from someone you don’t fully trust, you should be very cautious. That’s how viruses affect your operating system.\n",
    "\n",
    "\n",
    "\n",
    "Finally, even your own file may be changed by an attacker. Thus, the next time you unpickle, you can unpickle just about anything (that this unethical person put there)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}