S10_Final_Exercises.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": true,
    "toc-nb-collapsed": true
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Execute before using this notebook if using google colab\n",
    "kernel = str(get_ipython())\n",
    "if 'google.colab' in kernel:    \n",
    "    !wget https://raw.githubusercontent.com/fredzett/rmqa/master/utils.py -P local_modules -nc \n",
    "    !npx degit fredzett/rmqa/data data\n",
    "    import sys\n",
    "    sys.path.append('local_modules')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import scipy.stats as stats\n",
    "import statsmodels.api as sm\n",
    "from patsy import dmatrices\n",
    "from sklearn.cluster import KMeans\n",
    "\n",
    "plt.rcParams['font.size'] = 12\n",
    "plt.rcParams['figure.figsize'] = [9,7]\n",
    "plt.rcParams['figure.dpi'] = 80\n",
    "plt.rcParams[\"axes.spines.right\"] = False\n",
    "plt.rcParams[\"axes.spines.top\"] = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simulation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use simulation to answer the following two exercises."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A game involving a toin coss has the following rules: \n",
    "\n",
    "The player tosses a coin repeatedly until a tail appears or tosses it a maximum of 1000 times if no tail appears. The initial stake starts at 2 dollars and is doubled every time heads appears. The first time tails appears, the game ends and the player wins whatever is in the pot. Thus the player wins 2 dollars if tails appears on the first toss, 4 dollars if heads appears on the first toss and tails on the second, 8 dollars if heads appears on the first two tosses and tails on the third, and so on. \n",
    "\n",
    "_(Note: the coin is a fair coin with two sides (head and tail))_\n",
    "\n",
    "__Question:__ What is the probability of profit if it costs __30__ dollars to participate in the game?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A box contains 100 coins, where 99 are fair but one is double-handed, i.e. \n",
    "\n",
    "- 99 of 100 coins have an equal probability of 50% to land either head or tail\n",
    "- 1 of 100 coins will always land head\n",
    "\n",
    "A coin is chosen at random from the box. The chosen coin is then flipped $7$ times, and it lands heads all 7 times. \n",
    "\n",
    "__Question:__ Given this information, what is the probability that the chosen coin is a fair coin."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Resampling method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are using a statistical model (e.g. linear regression) to predict a response variable $Y$ for a particular value of $X$. Describe how we might estimate the standard deviation of our prediction using a __bootstrap__ approach. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Answer the following questions regarding __cross-validation__\n",
    "\n",
    "1. why cross validation is useful to evaluate the predictive performance of a statistical model\n",
    "\n",
    "2. Explain how k-fold cross-validation is implemented\n",
    "\n",
    "3. Explain the advantates and disadvantages of k-fold cross-validation relative to\n",
    "\n",
    "    1. validation set approach\n",
    "    \n",
    "    2. LOOCV approach"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Statistical models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 5: Regression model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A friend of yours argues that he earns money by predicting today's stock return from looking at the previous day's returns (so called \"lags\"). Specifically, he argues that today's returns can be explained by the five previous days.\n",
    "\n",
    "Example: \n",
    "\n",
    "- today's return is +3%\n",
    "- returns in the five previous days where +1%, -2%, +2%, +0.4% and -2% (called lag1 to lag5)\n",
    "\n",
    "Then he argues that the +3% can be explained by the previous five day returns. \n",
    "\n",
    "__Exercise:__ \n",
    "\n",
    "Below you are given a dataset of approximately 5 years of daily returns of the German Stock Index (DAX 30). Use linear regression analysis to analyse the validity of your friend's statments. Specifically do the following:\n",
    "\n",
    "1. build a linear regression model specifying y and X such that the above statmenent can be scrutinized in a reasonable way\n",
    "\n",
    "2. evaluate and interpret the model's goodness of fit (i.e. how good is the overall model in explaining stock returns)\n",
    "\n",
    "3. evaluate and interpret the regression coefficients (i.e. how do individual variables contribute in explaining stock returns). Also include the intercept (i.e. $\\beta_0$) in your discussion\n",
    "\n",
    "5. What is your conclusion of your analysis. What would tell your friend regarding his statement?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "         Date       volume     today      lag1      lag2      lag3      lag4  \\\n",
       "0  2016-01-04  119844300.0 -0.042778 -0.010785  0.019357 -0.006873  0.000000   \n",
       "1  2016-01-05   84894800.0  0.002592 -0.042778 -0.010785  0.019357 -0.006873   \n",
       "2  2016-01-06   90465700.0 -0.009319  0.002592 -0.042778 -0.010785  0.019357   \n",
       "3  2016-01-07  128029000.0 -0.022926 -0.009319  0.002592 -0.042778 -0.010785   \n",
       "4  2016-01-08   98631100.0 -0.013077 -0.022926 -0.009319  0.002592 -0.042778   \n",
       "\n",
       "       lag5  \n",
       "0  0.022776  \n",
       "1  0.000000  \n",
       "2 -0.006873  \n",
       "3  0.019357  \n",
       "4 -0.010785  "
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Date</th>\n      <th>volume</th>\n      <th>today</th>\n      <th>lag1</th>\n      <th>lag2</th>\n      <th>lag3</th>\n      <th>lag4</th>\n      <th>lag5</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>2016-01-04</td>\n      <td>119844300.0</td>\n      <td>-0.042778</td>\n      <td>-0.010785</td>\n      <td>0.019357</td>\n      <td>-0.006873</td>\n      <td>0.000000</td>\n      <td>0.022776</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2016-01-05</td>\n      <td>84894800.0</td>\n      <td>0.002592</td>\n      <td>-0.042778</td>\n      <td>-0.010785</td>\n      <td>0.019357</td>\n      <td>-0.006873</td>\n      <td>0.000000</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>2016-01-06</td>\n      <td>90465700.0</td>\n      <td>-0.009319</td>\n      <td>0.002592</td>\n      <td>-0.042778</td>\n      <td>-0.010785</td>\n      <td>0.019357</td>\n      <td>-0.006873</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>2016-01-07</td>\n      <td>128029000.0</td>\n      <td>-0.022926</td>\n      <td>-0.009319</td>\n      <td>0.002592</td>\n      <td>-0.042778</td>\n      <td>-0.010785</td>\n      <td>0.019357</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>2016-01-08</td>\n      <td>98631100.0</td>\n      <td>-0.013077</td>\n      <td>-0.022926</td>\n      <td>-0.009319</td>\n      <td>0.002592</td>\n      <td>-0.042778</td>\n      <td>-0.010785</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 12
    }
   ],
   "source": [
    "df = pd.read_csv(\"./data/dax_lags.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "            Date       volume     today      lag1      lag2      lag3  \\\n",
       "1253  2020-12-15   67265000.0  0.010566  0.008301 -0.013646 -0.003338   \n",
       "1254  2020-12-16   77798600.0  0.015200  0.010566  0.008301 -0.013646   \n",
       "1255  2020-12-17   77206700.0  0.007465  0.015200  0.010566  0.008301   \n",
       "1256  2020-12-18  156772100.0 -0.002688  0.007465  0.015200  0.010566   \n",
       "1257  2020-12-21   98290900.0 -0.028187 -0.002688  0.007465  0.015200   \n",
       "\n",
       "          lag4      lag5  \n",
       "1253  0.004652  0.000564  \n",
       "1254 -0.003338  0.004652  \n",
       "1255 -0.013646 -0.003338  \n",
       "1256  0.008301 -0.013646  \n",
       "1257  0.010566  0.008301  "
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Date</th>\n      <th>volume</th>\n      <th>today</th>\n      <th>lag1</th>\n      <th>lag2</th>\n      <th>lag3</th>\n      <th>lag4</th>\n      <th>lag5</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>1253</th>\n      <td>2020-12-15</td>\n      <td>67265000.0</td>\n      <td>0.010566</td>\n      <td>0.008301</td>\n      <td>-0.013646</td>\n      <td>-0.003338</td>\n      <td>0.004652</td>\n      <td>0.000564</td>\n    </tr>\n    <tr>\n      <th>1254</th>\n      <td>2020-12-16</td>\n      <td>77798600.0</td>\n      <td>0.015200</td>\n      <td>0.010566</td>\n      <td>0.008301</td>\n      <td>-0.013646</td>\n      <td>-0.003338</td>\n      <td>0.004652</td>\n    </tr>\n    <tr>\n      <th>1255</th>\n      <td>2020-12-17</td>\n      <td>77206700.0</td>\n      <td>0.007465</td>\n      <td>0.015200</td>\n      <td>0.010566</td>\n      <td>0.008301</td>\n      <td>-0.013646</td>\n      <td>-0.003338</td>\n    </tr>\n    <tr>\n      <th>1256</th>\n      <td>2020-12-18</td>\n      <td>156772100.0</td>\n      <td>-0.002688</td>\n      <td>0.007465</td>\n      <td>0.015200</td>\n      <td>0.010566</td>\n      <td>0.008301</td>\n      <td>-0.013646</td>\n    </tr>\n    <tr>\n      <th>1257</th>\n      <td>2020-12-21</td>\n      <td>98290900.0</td>\n      <td>-0.028187</td>\n      <td>-0.002688</td>\n      <td>0.007465</td>\n      <td>0.015200</td>\n      <td>0.010566</td>\n      <td>0.008301</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 13
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 6: Logistic regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extend your stock return analysis by using a _logistic regression model_. \n",
    "\n",
    "__Exercise:__\n",
    "\n",
    "Use the same data data set as in Exercise 5. Build a logistic regression model to analyse your friend's statement that money can be earned from predicting stock returns. \n",
    "\n",
    "Specifically do the following:\n",
    "\n",
    "1. build a logistic regression model using y and X. Keep in mind that logistic regression requires a binary response variable $y$. For this create a variable \"direction\" that takes a value of 1 if today's stock return was positive and a value of 0 if today's stock return was negative (hint: use `np.where`). For X in addition to lag1 to lag5 also consider trading volume (variable \"volume\") in addition to the variables lag1 to lag5. \n",
    "\n",
    "2. evaluate and interpret the model's goodness of fit (i.e. how good is the overall model in explaining stock return direction)\n",
    "\n",
    "3. evaluate and interpret the regression coefficients (i.e. how do individual variables contribute in explaining stock return direction). Also include the intercept (i.e. $\\beta_0$) in your discussion\n",
    "\n",
    "5. What is your conclusion of your analysis. What would tell your friend regarding his statement?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "         Date       volume     today      lag1      lag2      lag3      lag4  \\\n",
       "0  2016-01-04  119844300.0 -0.042778 -0.010785  0.019357 -0.006873  0.000000   \n",
       "1  2016-01-05   84894800.0  0.002592 -0.042778 -0.010785  0.019357 -0.006873   \n",
       "2  2016-01-06   90465700.0 -0.009319  0.002592 -0.042778 -0.010785  0.019357   \n",
       "3  2016-01-07  128029000.0 -0.022926 -0.009319  0.002592 -0.042778 -0.010785   \n",
       "4  2016-01-08   98631100.0 -0.013077 -0.022926 -0.009319  0.002592 -0.042778   \n",
       "\n",
       "       lag5  \n",
       "0  0.022776  \n",
       "1  0.000000  \n",
       "2 -0.006873  \n",
       "3  0.019357  \n",
       "4 -0.010785  "
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Date</th>\n      <th>volume</th>\n      <th>today</th>\n      <th>lag1</th>\n      <th>lag2</th>\n      <th>lag3</th>\n      <th>lag4</th>\n      <th>lag5</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>2016-01-04</td>\n      <td>119844300.0</td>\n      <td>-0.042778</td>\n      <td>-0.010785</td>\n      <td>0.019357</td>\n      <td>-0.006873</td>\n      <td>0.000000</td>\n      <td>0.022776</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2016-01-05</td>\n      <td>84894800.0</td>\n      <td>0.002592</td>\n      <td>-0.042778</td>\n      <td>-0.010785</td>\n      <td>0.019357</td>\n      <td>-0.006873</td>\n      <td>0.000000</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>2016-01-06</td>\n      <td>90465700.0</td>\n      <td>-0.009319</td>\n      <td>0.002592</td>\n      <td>-0.042778</td>\n      <td>-0.010785</td>\n      <td>0.019357</td>\n      <td>-0.006873</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>2016-01-07</td>\n      <td>128029000.0</td>\n      <td>-0.022926</td>\n      <td>-0.009319</td>\n      <td>0.002592</td>\n      <td>-0.042778</td>\n      <td>-0.010785</td>\n      <td>0.019357</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>2016-01-08</td>\n      <td>98631100.0</td>\n      <td>-0.013077</td>\n      <td>-0.022926</td>\n      <td>-0.009319</td>\n      <td>0.002592</td>\n      <td>-0.042778</td>\n      <td>-0.010785</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 14
    }
   ],
   "source": [
    "df = pd.read_csv(\"./data/dax_lags.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Clustering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 7 - Clustering analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generate a simulated data set with 150 observation and 50 variables. The dataset should contain three classes such that:\n",
    "\n",
    "- class 1: has 50 observations and 50 variables, where all observations should be sampled from a normal distribution with $\\mu = 0$ and $\\sigma = 1$\n",
    "- class 2: has 50 observations and 50 variables, where all observations should be sampled from a normal distribution with $\\mu = 5$ and $\\sigma = 1$\n",
    "- class 3: has 50 observations and 50 variables, where all observations should be sampled from a normal distribution with $\\mu = 10$ and $\\sigma = 1$\n",
    "\n",
    "Perform a __K-Means__ clustering analysis with the above observations with $K=3$ and $K=5$. \n",
    "\n",
    "- Explain how KMeans clustering works in your own words\n",
    "\n",
    "- compare the results from the clustering analysis to the true clusters\n",
    "\n",
    "- explain your findings? \n",
    "\n",
    "Hint: \n",
    "- create each class separateley using `scipy.stats.norm` (remember that `.rvs` can be used to create random data from a defined distribution)\n",
    "- combine all three classes into $X$ using `np.vstack((class1, class2, class3))`\n",
    "- use `KMeans` from `sklearn.cluster` to conduct clustering analysis. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3-final"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}