notebooks/supporting/what-prices-to-predict.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Supporting Notebook 2: (Predictable and unpredicable prices)\n",
    "\n",
    "## Modeling: What to Predict\n",
    "It is not straightforward to choose how to model stock behavior.\n",
    "\n",
    "Do we predict prices at 1 minute ahead, 10 minutes, or daily.\n",
    "\n",
    "Or should we predict all?\n",
    "\n",
    "What kind of price do we want to predict within an interval:\n",
    "\n",
    "- EndPrice\n",
    "- Mean(EndPrice)\n",
    "- WeightedMean(EndPrice)\n",
    "- MedianPrice\n",
    "- or the complete distribution of prices\n",
    "- if the complete distribution is to be predicted, do we predict the mean, variance and the skew. Do we discretize the distribution or we treat the available prices as samples from the distribution that we predict\n",
    "\n",
    "Another decision is whether to predict:\n",
    "- Percent of Change of Price\n",
    "- The sign of the price movement\n",
    "- The log of the actual price\n",
    "\n",
    "From a practical perspective, even in the presence of perfect predictions of prices in the future, one may not be able to make any money. This depends on\n",
    "- fees (the fee to enter and exit a position) should be lower than the expected change in the price\n",
    "- on seeing a quoted price how much above (when buying) and how far below (when selling) will one execute\n",
    "- volumes in the market. If one trades with very large volumes one will disrupt the supply and demand mechanism, therefore one will not execute at the desired price.\n",
    "- how the broker executes trades matters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Insights: what is predictable\n",
    "\n",
    "In the current notebook we try to get an understanding what is easier and what is difficult to predict.\n",
    "For example, we find out that when we organize the data into intervals of say 10, 15, or 30 minutes,\n",
    "the mean price (`Mean(EndPrice)`) is much easier to predict than the end price `EndPrice`.\n",
    "\n",
    "We also find that when normalizing for `PctChange:X` of a feature `X`, we should use features that are normalized in the same way. One way to normalize the features is to divide them by `X[t - 1]`. \n",
    "\n",
    "One way to avoid this difficulty would be the following:\n",
    "- generate different linear combination of prices (averages are also linear combinations, and so are absolute returns)\n",
    "- compute the logs of all types of prices (or averages) of prices\n",
    "- we may also attempt to predict linear combination of prices, by taking logs\n",
    "- notice that is essentially a non-linear model, because we apply logs on sums of raw input features\n",
    "- when using logs care must be taken to avoid numbers close to zero as well as negative numbers.\n",
    "- however, log returns are approximately equal to percent change, so logs can be avoided\n",
    "\n",
    "We explore different ways to normalize the prices. One way would be to choose a more stable price as an \"anchor\", and replace all prices as linear functions such as:\n",
    "```\n",
    "AnchorPrice = mean(Price[t - k, t - 1])\n",
    "NormPrice:X[i] = (Price:X[i] - AnchorPrice)/AnchorPrice, for i = t, t - 1, ...\n",
    "```\n",
    "\n",
    "\n",
    "Regarding the prediction of an `EndPrice` we hypothesis that one would do better by predicting:\n",
    "- predict two prices, one 10 minutes ahead, one 5 minutes ahead\n",
    "- find the differences between the predictions\n",
    "\n",
    "\n",
    "In the future we will also explore the possiblity to predict entire distribution of prices, instead of just a single end-price, or averaged prices.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "mpl.rcParams['figure.figsize'] = (5, 3) # use bigger graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usual we first load the data we prepared in notebook 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_file = '/data/processed/cooked_v3.pkl'\n",
    "df = pd.read_pickle(input_file)\n",
    "df['CalcDateTime'] = df.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we prepare a dataset consting of a single stock and compute derivative features\n",
    "(percent change) of a number of price features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "price_features = ['MaxPrice', 'MinPrice', 'LastEndPrice', 'FirstStartPrice', 'MeanEndPrice', 'MedianEndPrice', \n",
    "                  'MeanStartEndPrice', 'MeanMaxMinPrice', 'MeanAvg4Price',\n",
    "                  'Direction1', 'Direction2', 'StdEndPrice', 'VolumeWeightedEndPrice']\n",
    "\n",
    "log_ret_features = ['MaxPrice', 'MinPrice', 'LastEndPrice', 'FirstStartPrice', 'MeanEndPrice', 'MedianEndPrice', \n",
    "                  'MeanStartEndPrice', 'MeanMaxMinPrice', 'MeanAvg4Price']\n",
    "\n",
    "indicator_features = [\n",
    "    'F1', 'F2', 'F3', 'F4', 'F5', 'F6', 'F7', 'F8',\n",
    "    'G1', 'G2', 'G3', 'G4'\n",
    "]\n",
    "\n",
    "def pct_change_of(feature):\n",
    "        return 'PctChange:' + feature\n",
    "    \n",
    "def log_return_f(feature):\n",
    "    return 'LogReturn:' + feature\n",
    "\n",
    "def adj_log_return_f(feature):\n",
    "    return 'AdjLogReturn:' + feature\n",
    "\n",
    "def rev_pct_change_of_at_t(feature, t):\n",
    "        return 'RevPctChange[t - {}]:{}'.format(str(t), feature)\n",
    "    \n",
    "def shifted(feature):\n",
    "    return feature + '[t - 1]'\n",
    "\n",
    "def closer_to(pnt, a, b):\n",
    "    return (np.absolute(pnt - a) - np.absolute(pnt - b))/pnt\n",
    "\n",
    "def closer_to_with_normalization(pnt, a, b, norm):\n",
    "    return (np.absolute(pnt - a) - np.absolute(pnt - b))/norm\n",
    "\n",
    "def closer_to_or(pnt1, a, pnt2, b):\n",
    "    return 2.0*(np.absolute(pnt1 - a) - np.absolute(pnt2 - b))/(pnt1 + pnt2)\n",
    "\n",
    "def rev_pct_change(a, t):\n",
    "    one_step_in_past = a.shift(1)\n",
    "    t_steps_in_past = a.shift(1 + t)\n",
    "    return (one_step_in_past - t_steps_in_past)/one_step_in_past\n",
    "    \n",
    "def log_return(a):\n",
    "    return np.log(a) - np.log(a.shift(1))\n",
    "\n",
    "def adj_log_return(a, norm):\n",
    "    return np.log(a) - np.log(norm)\n",
    "\n",
    "def norm_feature(feature_family, t, norm_feature):\n",
    "    return \"{}@WithNorm({})[t - {}]\".format(feature_family, norm_feature, t)    \n",
    "    \n",
    "def weighted_mean(prices, volumes, interval):\n",
    "    prices_times_volumes = prices * volumes\n",
    "    num = prices_times_volumes.resample(interval).sum()\n",
    "    denom = volumes.resample(interval).sum()\n",
    "    return num/denom\n",
    "    \n",
    "# use this to experiment with custom weights    \n",
    "def custom_linear_comb(prices):\n",
    "    prices = prices.values[:,]\n",
    "    prices = prices[~np.isnan(prices)]\n",
    "    if prices.shape[0] == 0:\n",
    "        return np.nan\n",
    "    weights = np.zeros_like(prices)\n",
    "    if prices.shape[0] < 3:\n",
    "        return prices[-1]\n",
    "    weights[-1] = 1.0\n",
    "    weights[-2] = 0.5\n",
    "    return np.sum(np.multiply(prices, weights))/np.sum(weights)    \n",
    "\n",
    "def prepare_single_stock(mnemonic, interval):\n",
    "    # TODO: add traded volume to averaging of prices, also traded volume weighted differently (e.g. exponential weighting)\n",
    "    # TODO: one can weight max/min/start/end price differently depending on the move\n",
    "    # TODO: add exponential weighting of the prices within a window (also there are different ways to center)\n",
    "    single_stock = df[df.Mnemonic == mnemonic].copy()\n",
    "    single_stock['StartEndPrice'] = 0.5*(single_stock['StartPrice'] + single_stock['EndPrice'])\n",
    "    single_stock['MaxMinPrice'] = 0.5*(single_stock['MaxPrice'] + single_stock['MinPrice'])\n",
    "    single_stock['Avg4Price'] = 0.25*(single_stock['MaxPrice'] + single_stock['MinPrice'] + \n",
    "                                      single_stock['StartPrice'] + single_stock['EndPrice'])\n",
    "    \n",
    "    # TODO: add smoothed traded volume\n",
    "    single_stock['PctChange'] = single_stock['EndPrice'].pct_change()\n",
    "    single_stock['SmoothedTradedVolume'] = single_stock['TradedVolume'].ewm(com=2.5).mean()\n",
    "    \n",
    "    single_stock['Direction'] = \\\n",
    "        2.0*(single_stock['EndPrice'] - single_stock['StartPrice'])/ \\\n",
    "        (single_stock['EndPrice'] + single_stock['StartPrice'])\n",
    "        \n",
    "    single_stock['F1'] = - closer_to(single_stock['EndPrice'], single_stock['MaxPrice'], single_stock['MinPrice'])\n",
    "        \n",
    "\n",
    "    resampled = pd.DataFrame({\n",
    "        'MaxPrice': single_stock['MaxPrice'].resample(interval).max(),\n",
    "        'MinPrice': single_stock['MinPrice'].resample(interval).min(),\n",
    "        'MeanStartEndPrice': single_stock['StartEndPrice'].resample(interval).mean(),  \n",
    "        'MeanMaxMinPrice': single_stock['MaxMinPrice'].resample(interval).mean(), \n",
    "        'MeanAvg4Price': single_stock['Avg4Price'].resample(interval).mean(),  \n",
    "        'CustomLinearComb': single_stock['Avg4Price'].resample(interval).apply(custom_linear_comb),         \n",
    "        'FirstStartPrice': single_stock['StartPrice'].resample(interval).first(),        \n",
    "        'LastEndPrice': single_stock['EndPrice'].resample(interval).last(), \n",
    "        'MeanEndPrice': single_stock['EndPrice'].resample(interval).mean(), \n",
    "        'MedianEndPrice': single_stock['EndPrice'].resample(interval).median(),\n",
    "        'VolumeWeightedEndPrice': weighted_mean(single_stock['EndPrice'], single_stock['SmoothedTradedVolume'], interval),\n",
    "        'VolumeWeightedPctChange': weighted_mean(single_stock['PctChange'], single_stock['SmoothedTradedVolume'], interval),\n",
    "        \n",
    "        'StdEndPrice': single_stock['EndPrice'].resample(interval).std(),\n",
    "        'HasTrade': single_stock['HasTrade'].resample(interval).max(),\n",
    "        'G1': single_stock['Direction'].resample(interval).mean(),\n",
    "        'G2': np.sign(single_stock['Direction']).resample(interval).mean(),\n",
    "        'G3': single_stock['F1'].resample(interval).mean(),\n",
    "        'G4': np.sign(single_stock['F1']).resample(interval).mean()\n",
    "    })\n",
    "    resampled['AdjustedPctChange'] = (resampled['LastEndPrice'] - resampled['MeanEndPrice'])/resampled['MeanEndPrice']\n",
    "    resampled['AdjustedPctChange[t - 1]'] = resampled['AdjustedPctChange'].shift(1)\n",
    "    \n",
    "    resampled['MeanPctChangeV2'] = (resampled['MeanEndPrice'] - resampled['LastEndPrice'].shift(1))/resampled['LastEndPrice'].shift(1)\n",
    "    \n",
    "    anchor = resampled['MeanEndPrice'].shift(1)\n",
    "    resampled[adj_log_return_f('LastEndPrice')] = adj_log_return(resampled['LastEndPrice'], anchor)\n",
    "    resampled[shifted(adj_log_return_f('LastEndPrice'))] = \\\n",
    "        adj_log_return(resampled['LastEndPrice'].shift(1), anchor)\n",
    "    \n",
    "    resampled['Direction1'] = \\\n",
    "        2.0*(resampled['LastEndPrice'] - resampled['FirstStartPrice'])/ \\\n",
    "        (resampled['LastEndPrice'] + resampled['FirstStartPrice'])\n",
    "    \n",
    "    resampled['Direction2'] = \\\n",
    "        2.0*(resampled['LastEndPrice'] - resampled['FirstStartPrice'])/ \\\n",
    "        resampled['MeanEndPrice'].shift(1)\n",
    "        \n",
    "    resampled[shifted('Direction1')] = resampled['Direction1'].shift(1)\n",
    "    resampled[shifted('Direction2')] = resampled['Direction2'].shift(1)\n",
    "    \n",
    "    for f in ['MinPrice', 'MaxPrice', 'LastEndPrice', 'FirstStartPrice']:\n",
    "        resampled[shifted(f)] = resampled[f].shift(1)\n",
    "        \n",
    "    resampled['F1'] = - closer_to(resampled['LastEndPrice'], resampled['MaxPrice'], resampled['MinPrice'])\n",
    "    resampled['F2'] = - closer_to(resampled['MaxPrice'], resampled['LastEndPrice'], resampled['FirstStartPrice'])\n",
    "    resampled['F3'] = closer_to(resampled['MinPrice'], resampled['LastEndPrice'], resampled['FirstStartPrice'])\n",
    "    resampled['F4'] = - closer_to(resampled['LastEndPrice'], resampled['MaxPrice'], resampled[shifted('MaxPrice')])\n",
    "    resampled['F5'] = closer_to(resampled['LastEndPrice'], resampled['MinPrice'], resampled[shifted('MinPrice')])\n",
    "    \n",
    "    resampled['F6'] = - closer_to_or(resampled['LastEndPrice'], resampled['MaxPrice'],\n",
    "                                   resampled[shifted('LastEndPrice')], resampled[shifted('MaxPrice')])\n",
    "\n",
    "    resampled['F7'] = closer_to_or(resampled['LastEndPrice'], resampled['MinPrice'],\n",
    "                                   resampled[shifted('LastEndPrice')], resampled[shifted('MinPrice')])\n",
    "    \n",
    "    \n",
    "    resampled['F8'] = np.where(resampled['Direction2'] >= 0, resampled['F4'] , resampled['F5'])\n",
    "    \n",
    "    \n",
    "    for t in range(1, 5):    \n",
    "        # note: normalization is fixed\n",
    "        resampled[norm_feature('H1', t, 'MeanEndPrice')] = - closer_to_with_normalization(\n",
    "                                             resampled['LastEndPrice'].shift(t), \n",
    "                                             resampled['MaxPrice'].shift(t), \n",
    "                                             resampled['MinPrice'].shift(t),\n",
    "                                             resampled['MeanEndPrice'].shift(1))    \n",
    "        \n",
    "    \n",
    "    for f in indicator_features:\n",
    "        resampled[shifted(f)] = resampled[f].shift(1)\n",
    "        \n",
    "    for f in price_features:\n",
    "        pct_change_f = pct_change_of(f)\n",
    "        resampled[pct_change_f] = resampled[f].pct_change()\n",
    "        resampled[shifted(pct_change_f)] = resampled[pct_change_f].shift(1) \n",
    "    \n",
    "    for f in price_features:\n",
    "        for t in range(1, 5):\n",
    "            rev_pct_change_f_t = rev_pct_change_of_at_t(f, t)\n",
    "            resampled[rev_pct_change_f_t] = rev_pct_change(resampled[f], t)    \n",
    "        \n",
    "\n",
    "    for f in log_ret_features:\n",
    "        log_ret_f = log_return_f(f)\n",
    "        resampled[log_ret_f] = log_return(resampled[f])\n",
    "        resampled[shifted(log_ret_f)] = resampled[log_ret_f].shift(1) \n",
    "        \n",
    "    resampled = resampled[resampled['HasTrade'] == 1.0]\n",
    "    \n",
    "    return resampled\n",
    "\n",
    "def correlation_with_feature(single_stock, corr_feature):\n",
    "    pct_change_of_f = corr_feature\n",
    "    sh_dir1 = shifted('Direction1')\n",
    "    sh_dir2 = shifted('Direction2')    \n",
    "    d = {\n",
    "        pct_change_of_f: single_stock[pct_change_of_f],\n",
    "        sh_dir1: single_stock[sh_dir1],\n",
    "        sh_dir2: single_stock[sh_dir2]        \n",
    "    }\n",
    "    \n",
    "    for f in indicator_features:\n",
    "        d[shifted(f)] = single_stock[shifted(f)]\n",
    "        \n",
    "    d['AdjustedPctChange[t - 1]'] = single_stock['AdjustedPctChange[t - 1]']    \n",
    "    d[adj_log_return_f('LastEndPrice')] = single_stock[adj_log_return_f('LastEndPrice')]\n",
    "    d[shifted(adj_log_return_f('LastEndPrice'))] = single_stock[shifted(adj_log_return_f('LastEndPrice'))]\n",
    "    \n",
    "    d['MeanPctChangeV2'] = single_stock['MeanPctChangeV2']\n",
    "    \n",
    "    for f in price_features:\n",
    "        pct_change_shifted = shifted(pct_change_of(f))\n",
    "        d[pct_change_shifted] = single_stock[pct_change_shifted]\n",
    "        \n",
    "    for t in range(1, 5):    \n",
    "        # note: normalization is fixed\n",
    "        f = norm_feature('H1', t, 'MeanEndPrice')\n",
    "        d[f] = single_stock[f]\n",
    "            \n",
    "    for f in log_ret_features:\n",
    "        log_ret_f = log_return_f(f)\n",
    "        d[log_ret_f] = single_stock[log_ret_f]\n",
    "            \n",
    "    for f in price_features:\n",
    "        for t in range(1, 5):\n",
    "            rev_pct_change_f_t = rev_pct_change_of_at_t(f, t)\n",
    "            d[rev_pct_change_f_t] = single_stock[rev_pct_change_f_t]\n",
    "            \n",
    "    d[shifted('VolumeWeightedPctChange')] = single_stock[('VolumeWeightedPctChange')].shift(1)\n",
    "    corr = pd.DataFrame(d).corr()\n",
    "    row_id = np.argwhere(corr.index.values == pct_change_of_f)[0][0]\n",
    "    return corr.iloc[[row_id]].drop(columns=[pct_change_of_f])\n",
    "\n",
    "def corr_pct_change(single_stock, feature):\n",
    "    pct_change_of_f = pct_change_of(feature)\n",
    "    return correlation_with_feature(single_stock, pct_change_of_f)\n",
    "\n",
    "def corr_log_return(single_stock, feature):\n",
    "    return correlation_with_feature(single_stock, log_return_f(feature))\n",
    "\n",
    "def find_most_correlated_features(single_stock, feature):\n",
    "    corrs = correlation_with_feature(single_stock, feature).T\n",
    "    corrs['AbsCorr'] = np.absolute(corrs[feature])\n",
    "    sorted_corrs = corrs.sort_values('AbsCorr', ascending=False)\n",
    "    selected_names = filter(lambda n: n.find('[t - 1') >= 0 or n.find('[t - 2') >= 0, list(sorted_corrs.index.values))\n",
    "    sorted_corrs = sorted_corrs.T[selected_names].T\n",
    "    return sorted_corrs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We choose a single stock, for example 'SIE' (Siemens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "single_stock = prepare_single_stock('SIE', '30Min')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We look at the corrlations of all the other price features from previous time periods `(t - 1)`.\n",
    "We could also say that we investigate if a single feature from the previous time period `(t - 1)`\n",
    "would be predictive of the change in the next time period `(t)`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first study the `LastEndPrice` or the last end price in the interval `t` (here interval is 30 minutes).\n",
    "As it can be seen there are no strong correlations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>RevPctChange[t - 4]:Direction2</th>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:VolumeWeightedEndPrice</th>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:LastEndPrice</th>\n",
       "      <td>-0.014452</td>\n",
       "      <td>-0.031263</td>\n",
       "      <td>-0.028687</td>\n",
       "      <td>-0.014556</td>\n",
       "      <td>-0.014273</td>\n",
       "      <td>0.019518</td>\n",
       "      <td>-0.018305</td>\n",
       "      <td>-0.020372</td>\n",
       "      <td>-0.030215</td>\n",
       "      <td>-0.013344</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.025498</td>\n",
       "      <td>0.004233</td>\n",
       "      <td>-0.036114</td>\n",
       "      <td>0.031601</td>\n",
       "      <td>-0.015092</td>\n",
       "      <td>0.011779</td>\n",
       "      <td>-0.00563</td>\n",
       "      <td>-0.032617</td>\n",
       "      <td>-0.018977</td>\n",
       "      <td>-0.004466</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 97 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                        Direction1[t - 1]  Direction2[t - 1]  F1[t - 1]  \\\n",
       "PctChange:LastEndPrice          -0.014452          -0.031263  -0.028687   \n",
       "\n",
       "                        F2[t - 1]  F3[t - 1]  F4[t - 1]  F5[t - 1]  F6[t - 1]  \\\n",
       "PctChange:LastEndPrice  -0.014556  -0.014273   0.019518  -0.018305  -0.020372   \n",
       "\n",
       "                        F7[t - 1]  F8[t - 1]  ...  \\\n",
       "PctChange:LastEndPrice  -0.030215  -0.013344  ...   \n",
       "\n",
       "                        RevPctChange[t - 4]:Direction2  \\\n",
       "PctChange:LastEndPrice                       -0.025498   \n",
       "\n",
       "                        RevPctChange[t - 1]:StdEndPrice  \\\n",
       "PctChange:LastEndPrice                         0.004233   \n",
       "\n",
       "                        RevPctChange[t - 2]:StdEndPrice  \\\n",
       "PctChange:LastEndPrice                        -0.036114   \n",
       "\n",
       "                        RevPctChange[t - 3]:StdEndPrice  \\\n",
       "PctChange:LastEndPrice                         0.031601   \n",
       "\n",
       "                        RevPctChange[t - 4]:StdEndPrice  \\\n",
       "PctChange:LastEndPrice                        -0.015092   \n",
       "\n",
       "                        RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "PctChange:LastEndPrice                                    0.011779   \n",
       "\n",
       "                        RevPctChange[t - 2]:VolumeWeightedEndPrice  \\\n",
       "PctChange:LastEndPrice                                    -0.00563   \n",
       "\n",
       "                        RevPctChange[t - 3]:VolumeWeightedEndPrice  \\\n",
       "PctChange:LastEndPrice                                   -0.032617   \n",
       "\n",
       "                        RevPctChange[t - 4]:VolumeWeightedEndPrice  \\\n",
       "PctChange:LastEndPrice                                   -0.018977   \n",
       "\n",
       "                        VolumeWeightedPctChange[t - 1]  \n",
       "PctChange:LastEndPrice                       -0.004466  \n",
       "\n",
       "[1 rows x 97 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_pct_change(single_stock, 'LastEndPrice')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we look for correlations with the mean prices: \n",
    "- MeanAvg4Price: we averaged all 4 prices available within a minute and then averaged within an interval like 10Min\n",
    "- MeanMaxMinPrice: took the average of Min and Max prices and then averaged within an interval\n",
    "- MeanStartEndPrice: took the average of Start and End prices and then averaged within an interval\n",
    "\n",
    "# Conclusion:\n",
    "\n",
    "- The more averaging was done, the easier to predict\n",
    "- EndPrice is very difficult to predict, but the mean price is not \n",
    "- The median of the end price is harder to predict than the mean, but a lot easier than the end price\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>RevPctChange[t - 4]:Direction2</th>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:VolumeWeightedEndPrice</th>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanAvg4Price</th>\n",
       "      <td>0.499097</td>\n",
       "      <td>0.545182</td>\n",
       "      <td>0.594027</td>\n",
       "      <td>0.500112</td>\n",
       "      <td>0.497522</td>\n",
       "      <td>0.100365</td>\n",
       "      <td>0.098164</td>\n",
       "      <td>0.3572</td>\n",
       "      <td>0.337546</td>\n",
       "      <td>0.396958</td>\n",
       "      <td>...</td>\n",
       "      <td>0.007962</td>\n",
       "      <td>0.000075</td>\n",
       "      <td>-0.036812</td>\n",
       "      <td>0.007804</td>\n",
       "      <td>-0.017486</td>\n",
       "      <td>0.273278</td>\n",
       "      <td>0.173408</td>\n",
       "      <td>0.121242</td>\n",
       "      <td>0.090679</td>\n",
       "      <td>0.279725</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 97 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                         Direction1[t - 1]  Direction2[t - 1]  F1[t - 1]  \\\n",
       "PctChange:MeanAvg4Price           0.499097           0.545182   0.594027   \n",
       "\n",
       "                         F2[t - 1]  F3[t - 1]  F4[t - 1]  F5[t - 1]  \\\n",
       "PctChange:MeanAvg4Price   0.500112   0.497522   0.100365   0.098164   \n",
       "\n",
       "                         F6[t - 1]  F7[t - 1]  F8[t - 1]  ...  \\\n",
       "PctChange:MeanAvg4Price     0.3572   0.337546   0.396958  ...   \n",
       "\n",
       "                         RevPctChange[t - 4]:Direction2  \\\n",
       "PctChange:MeanAvg4Price                        0.007962   \n",
       "\n",
       "                         RevPctChange[t - 1]:StdEndPrice  \\\n",
       "PctChange:MeanAvg4Price                         0.000075   \n",
       "\n",
       "                         RevPctChange[t - 2]:StdEndPrice  \\\n",
       "PctChange:MeanAvg4Price                        -0.036812   \n",
       "\n",
       "                         RevPctChange[t - 3]:StdEndPrice  \\\n",
       "PctChange:MeanAvg4Price                         0.007804   \n",
       "\n",
       "                         RevPctChange[t - 4]:StdEndPrice  \\\n",
       "PctChange:MeanAvg4Price                        -0.017486   \n",
       "\n",
       "                         RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanAvg4Price                                    0.273278   \n",
       "\n",
       "                         RevPctChange[t - 2]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanAvg4Price                                    0.173408   \n",
       "\n",
       "                         RevPctChange[t - 3]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanAvg4Price                                    0.121242   \n",
       "\n",
       "                         RevPctChange[t - 4]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanAvg4Price                                    0.090679   \n",
       "\n",
       "                         VolumeWeightedPctChange[t - 1]  \n",
       "PctChange:MeanAvg4Price                        0.279725  \n",
       "\n",
       "[1 rows x 97 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_pct_change(single_stock, 'MeanAvg4Price')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PctChange:MeanAvg4Price</th>\n",
       "      <th>AbsCorr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AdjustedPctChange[t - 1]</th>\n",
       "      <td>0.648615</td>\n",
       "      <td>0.648615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AdjLogReturn:LastEndPrice[t - 1]</th>\n",
       "      <td>0.648611</td>\n",
       "      <td>0.648611</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <td>0.594027</td>\n",
       "      <td>0.594027</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@WithNorm(MeanEndPrice)[t - 1]</th>\n",
       "      <td>0.593925</td>\n",
       "      <td>0.593925</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:LastEndPrice</th>\n",
       "      <td>0.545623</td>\n",
       "      <td>0.545623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <td>0.545182</td>\n",
       "      <td>0.545182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <td>0.500112</td>\n",
       "      <td>0.500112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <td>0.499097</td>\n",
       "      <td>0.499097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <td>0.497522</td>\n",
       "      <td>0.497522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>G1[t - 1]</th>\n",
       "      <td>0.426857</td>\n",
       "      <td>0.426857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <td>0.396958</td>\n",
       "      <td>0.396958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:LastEndPrice</th>\n",
       "      <td>0.382551</td>\n",
       "      <td>0.382551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <td>0.357200</td>\n",
       "      <td>0.357200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>G3[t - 1]</th>\n",
       "      <td>0.345520</td>\n",
       "      <td>0.345520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>G2[t - 1]</th>\n",
       "      <td>0.340961</td>\n",
       "      <td>0.340961</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <td>0.337546</td>\n",
       "      <td>0.337546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "      <td>0.279725</td>\n",
       "      <td>0.279725</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <td>0.273278</td>\n",
       "      <td>0.273278</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>G4[t - 1]</th>\n",
       "      <td>0.262551</td>\n",
       "      <td>0.262551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:LastEndPrice[t - 1]</th>\n",
       "      <td>0.260451</td>\n",
       "      <td>0.260451</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MeanEndPrice</th>\n",
       "      <td>0.246160</td>\n",
       "      <td>0.246160</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MeanMaxMinPrice</th>\n",
       "      <td>0.237190</td>\n",
       "      <td>0.237190</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MeanAvg4Price</th>\n",
       "      <td>0.237016</td>\n",
       "      <td>0.237016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MeanStartEndPrice</th>\n",
       "      <td>0.236824</td>\n",
       "      <td>0.236824</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MedianEndPrice</th>\n",
       "      <td>0.223581</td>\n",
       "      <td>0.223581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MaxPrice</th>\n",
       "      <td>0.218768</td>\n",
       "      <td>0.218768</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MinPrice</th>\n",
       "      <td>0.214373</td>\n",
       "      <td>0.214373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <td>0.173408</td>\n",
       "      <td>0.173408</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MinPrice</th>\n",
       "      <td>0.166215</td>\n",
       "      <td>0.166215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MeanEndPrice</th>\n",
       "      <td>0.152325</td>\n",
       "      <td>0.152325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MeanMaxMinPrice</th>\n",
       "      <td>0.147126</td>\n",
       "      <td>0.147126</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MeanAvg4Price</th>\n",
       "      <td>0.146701</td>\n",
       "      <td>0.146701</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MeanStartEndPrice</th>\n",
       "      <td>0.146270</td>\n",
       "      <td>0.146270</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MedianEndPrice</th>\n",
       "      <td>0.145133</td>\n",
       "      <td>0.145133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:VolumeWeightedEndPrice[t - 1]</th>\n",
       "      <td>0.132772</td>\n",
       "      <td>0.132772</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MaxPrice</th>\n",
       "      <td>0.128987</td>\n",
       "      <td>0.128987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice[t - 1]</th>\n",
       "      <td>0.121603</td>\n",
       "      <td>0.121603</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanMaxMinPrice[t - 1]</th>\n",
       "      <td>0.116762</td>\n",
       "      <td>0.116762</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanAvg4Price[t - 1]</th>\n",
       "      <td>0.116700</td>\n",
       "      <td>0.116700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanStartEndPrice[t - 1]</th>\n",
       "      <td>0.116626</td>\n",
       "      <td>0.116626</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MaxPrice[t - 1]</th>\n",
       "      <td>0.114912</td>\n",
       "      <td>0.114912</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MedianEndPrice[t - 1]</th>\n",
       "      <td>0.114374</td>\n",
       "      <td>0.114374</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MinPrice[t - 1]</th>\n",
       "      <td>0.106973</td>\n",
       "      <td>0.106973</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <td>0.100365</td>\n",
       "      <td>0.100365</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <td>0.098164</td>\n",
       "      <td>0.098164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <td>-0.036812</td>\n",
       "      <td>0.036812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:FirstStartPrice</th>\n",
       "      <td>-0.021477</td>\n",
       "      <td>0.021477</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:Direction1[t - 1]</th>\n",
       "      <td>-0.021307</td>\n",
       "      <td>0.021307</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:Direction2[t - 1]</th>\n",
       "      <td>-0.019160</td>\n",
       "      <td>0.019160</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:Direction2</th>\n",
       "      <td>-0.018997</td>\n",
       "      <td>0.018997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@WithNorm(MeanEndPrice)[t - 2]</th>\n",
       "      <td>-0.016041</td>\n",
       "      <td>0.016041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:StdEndPrice[t - 1]</th>\n",
       "      <td>0.013559</td>\n",
       "      <td>0.013559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:FirstStartPrice</th>\n",
       "      <td>-0.011718</td>\n",
       "      <td>0.011718</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:Direction1</th>\n",
       "      <td>-0.006752</td>\n",
       "      <td>0.006752</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:FirstStartPrice[t - 1]</th>\n",
       "      <td>0.003496</td>\n",
       "      <td>0.003496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:Direction1</th>\n",
       "      <td>0.003026</td>\n",
       "      <td>0.003026</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:Direction2</th>\n",
       "      <td>0.002136</td>\n",
       "      <td>0.002136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <td>0.000075</td>\n",
       "      <td>0.000075</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            PctChange:MeanAvg4Price   AbsCorr\n",
       "AdjustedPctChange[t - 1]                                   0.648615  0.648615\n",
       "AdjLogReturn:LastEndPrice[t - 1]                           0.648611  0.648611\n",
       "F1[t - 1]                                                  0.594027  0.594027\n",
       "H1@WithNorm(MeanEndPrice)[t - 1]                           0.593925  0.593925\n",
       "RevPctChange[t - 1]:LastEndPrice                           0.545623  0.545623\n",
       "Direction2[t - 1]                                          0.545182  0.545182\n",
       "F2[t - 1]                                                  0.500112  0.500112\n",
       "Direction1[t - 1]                                          0.499097  0.499097\n",
       "F3[t - 1]                                                  0.497522  0.497522\n",
       "G1[t - 1]                                                  0.426857  0.426857\n",
       "F8[t - 1]                                                  0.396958  0.396958\n",
       "RevPctChange[t - 2]:LastEndPrice                           0.382551  0.382551\n",
       "F6[t - 1]                                                  0.357200  0.357200\n",
       "G3[t - 1]                                                  0.345520  0.345520\n",
       "G2[t - 1]                                                  0.340961  0.340961\n",
       "F7[t - 1]                                                  0.337546  0.337546\n",
       "VolumeWeightedPctChange[t - 1]                             0.279725  0.279725\n",
       "RevPctChange[t - 1]:VolumeWeightedEndPrice                 0.273278  0.273278\n",
       "G4[t - 1]                                                  0.262551  0.262551\n",
       "PctChange:LastEndPrice[t - 1]                              0.260451  0.260451\n",
       "RevPctChange[t - 1]:MeanEndPrice                           0.246160  0.246160\n",
       "RevPctChange[t - 1]:MeanMaxMinPrice                        0.237190  0.237190\n",
       "RevPctChange[t - 1]:MeanAvg4Price                          0.237016  0.237016\n",
       "RevPctChange[t - 1]:MeanStartEndPrice                      0.236824  0.236824\n",
       "RevPctChange[t - 1]:MedianEndPrice                         0.223581  0.223581\n",
       "RevPctChange[t - 1]:MaxPrice                               0.218768  0.218768\n",
       "RevPctChange[t - 1]:MinPrice                               0.214373  0.214373\n",
       "RevPctChange[t - 2]:VolumeWeightedEndPrice                 0.173408  0.173408\n",
       "RevPctChange[t - 2]:MinPrice                               0.166215  0.166215\n",
       "RevPctChange[t - 2]:MeanEndPrice                           0.152325  0.152325\n",
       "RevPctChange[t - 2]:MeanMaxMinPrice                        0.147126  0.147126\n",
       "RevPctChange[t - 2]:MeanAvg4Price                          0.146701  0.146701\n",
       "RevPctChange[t - 2]:MeanStartEndPrice                      0.146270  0.146270\n",
       "RevPctChange[t - 2]:MedianEndPrice                         0.145133  0.145133\n",
       "PctChange:VolumeWeightedEndPrice[t - 1]                    0.132772  0.132772\n",
       "RevPctChange[t - 2]:MaxPrice                               0.128987  0.128987\n",
       "PctChange:MeanEndPrice[t - 1]                              0.121603  0.121603\n",
       "PctChange:MeanMaxMinPrice[t - 1]                           0.116762  0.116762\n",
       "PctChange:MeanAvg4Price[t - 1]                             0.116700  0.116700\n",
       "PctChange:MeanStartEndPrice[t - 1]                         0.116626  0.116626\n",
       "PctChange:MaxPrice[t - 1]                                  0.114912  0.114912\n",
       "PctChange:MedianEndPrice[t - 1]                            0.114374  0.114374\n",
       "PctChange:MinPrice[t - 1]                                  0.106973  0.106973\n",
       "F4[t - 1]                                                  0.100365  0.100365\n",
       "F5[t - 1]                                                  0.098164  0.098164\n",
       "RevPctChange[t - 2]:StdEndPrice                           -0.036812  0.036812\n",
       "RevPctChange[t - 1]:FirstStartPrice                       -0.021477  0.021477\n",
       "PctChange:Direction1[t - 1]                               -0.021307  0.021307\n",
       "PctChange:Direction2[t - 1]                               -0.019160  0.019160\n",
       "RevPctChange[t - 1]:Direction2                            -0.018997  0.018997\n",
       "H1@WithNorm(MeanEndPrice)[t - 2]                          -0.016041  0.016041\n",
       "PctChange:StdEndPrice[t - 1]                               0.013559  0.013559\n",
       "RevPctChange[t - 2]:FirstStartPrice                       -0.011718  0.011718\n",
       "RevPctChange[t - 1]:Direction1                            -0.006752  0.006752\n",
       "PctChange:FirstStartPrice[t - 1]                           0.003496  0.003496\n",
       "RevPctChange[t - 2]:Direction1                             0.003026  0.003026\n",
       "RevPctChange[t - 2]:Direction2                             0.002136  0.002136\n",
       "RevPctChange[t - 1]:StdEndPrice                            0.000075  0.000075"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "find_most_correlated_features(single_stock, pct_change_of('MeanAvg4Price'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>RevPctChange[t - 4]:Direction2</th>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:VolumeWeightedEndPrice</th>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanMaxMinPrice</th>\n",
       "      <td>0.498263</td>\n",
       "      <td>0.544435</td>\n",
       "      <td>0.59389</td>\n",
       "      <td>0.499274</td>\n",
       "      <td>0.496686</td>\n",
       "      <td>0.100764</td>\n",
       "      <td>0.098704</td>\n",
       "      <td>0.356966</td>\n",
       "      <td>0.337818</td>\n",
       "      <td>0.397218</td>\n",
       "      <td>...</td>\n",
       "      <td>0.007842</td>\n",
       "      <td>0.000264</td>\n",
       "      <td>-0.036838</td>\n",
       "      <td>0.007853</td>\n",
       "      <td>-0.01757</td>\n",
       "      <td>0.272533</td>\n",
       "      <td>0.172921</td>\n",
       "      <td>0.120676</td>\n",
       "      <td>0.090292</td>\n",
       "      <td>0.278972</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 97 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                           Direction1[t - 1]  Direction2[t - 1]  F1[t - 1]  \\\n",
       "PctChange:MeanMaxMinPrice           0.498263           0.544435    0.59389   \n",
       "\n",
       "                           F2[t - 1]  F3[t - 1]  F4[t - 1]  F5[t - 1]  \\\n",
       "PctChange:MeanMaxMinPrice   0.499274   0.496686   0.100764   0.098704   \n",
       "\n",
       "                           F6[t - 1]  F7[t - 1]  F8[t - 1]  ...  \\\n",
       "PctChange:MeanMaxMinPrice   0.356966   0.337818   0.397218  ...   \n",
       "\n",
       "                           RevPctChange[t - 4]:Direction2  \\\n",
       "PctChange:MeanMaxMinPrice                        0.007842   \n",
       "\n",
       "                           RevPctChange[t - 1]:StdEndPrice  \\\n",
       "PctChange:MeanMaxMinPrice                         0.000264   \n",
       "\n",
       "                           RevPctChange[t - 2]:StdEndPrice  \\\n",
       "PctChange:MeanMaxMinPrice                        -0.036838   \n",
       "\n",
       "                           RevPctChange[t - 3]:StdEndPrice  \\\n",
       "PctChange:MeanMaxMinPrice                         0.007853   \n",
       "\n",
       "                           RevPctChange[t - 4]:StdEndPrice  \\\n",
       "PctChange:MeanMaxMinPrice                         -0.01757   \n",
       "\n",
       "                           RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanMaxMinPrice                                    0.272533   \n",
       "\n",
       "                           RevPctChange[t - 2]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanMaxMinPrice                                    0.172921   \n",
       "\n",
       "                           RevPctChange[t - 3]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanMaxMinPrice                                    0.120676   \n",
       "\n",
       "                           RevPctChange[t - 4]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanMaxMinPrice                                    0.090292   \n",
       "\n",
       "                           VolumeWeightedPctChange[t - 1]  \n",
       "PctChange:MeanMaxMinPrice                        0.278972  \n",
       "\n",
       "[1 rows x 97 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_pct_change(single_stock, 'MeanMaxMinPrice')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>RevPctChange[t - 4]:Direction2</th>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:VolumeWeightedEndPrice</th>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanStartEndPrice</th>\n",
       "      <td>0.499896</td>\n",
       "      <td>0.545891</td>\n",
       "      <td>0.59412</td>\n",
       "      <td>0.500914</td>\n",
       "      <td>0.498324</td>\n",
       "      <td>0.099959</td>\n",
       "      <td>0.097617</td>\n",
       "      <td>0.35741</td>\n",
       "      <td>0.337251</td>\n",
       "      <td>0.396669</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008082</td>\n",
       "      <td>-0.000114</td>\n",
       "      <td>-0.036783</td>\n",
       "      <td>0.007755</td>\n",
       "      <td>-0.017401</td>\n",
       "      <td>0.274004</td>\n",
       "      <td>0.173883</td>\n",
       "      <td>0.121801</td>\n",
       "      <td>0.091061</td>\n",
       "      <td>0.280448</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 97 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                             Direction1[t - 1]  Direction2[t - 1]  F1[t - 1]  \\\n",
       "PctChange:MeanStartEndPrice           0.499896           0.545891    0.59412   \n",
       "\n",
       "                             F2[t - 1]  F3[t - 1]  F4[t - 1]  F5[t - 1]  \\\n",
       "PctChange:MeanStartEndPrice   0.500914   0.498324   0.099959   0.097617   \n",
       "\n",
       "                             F6[t - 1]  F7[t - 1]  F8[t - 1]  ...  \\\n",
       "PctChange:MeanStartEndPrice    0.35741   0.337251   0.396669  ...   \n",
       "\n",
       "                             RevPctChange[t - 4]:Direction2  \\\n",
       "PctChange:MeanStartEndPrice                        0.008082   \n",
       "\n",
       "                             RevPctChange[t - 1]:StdEndPrice  \\\n",
       "PctChange:MeanStartEndPrice                        -0.000114   \n",
       "\n",
       "                             RevPctChange[t - 2]:StdEndPrice  \\\n",
       "PctChange:MeanStartEndPrice                        -0.036783   \n",
       "\n",
       "                             RevPctChange[t - 3]:StdEndPrice  \\\n",
       "PctChange:MeanStartEndPrice                         0.007755   \n",
       "\n",
       "                             RevPctChange[t - 4]:StdEndPrice  \\\n",
       "PctChange:MeanStartEndPrice                        -0.017401   \n",
       "\n",
       "                             RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanStartEndPrice                                    0.274004   \n",
       "\n",
       "                             RevPctChange[t - 2]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanStartEndPrice                                    0.173883   \n",
       "\n",
       "                             RevPctChange[t - 3]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanStartEndPrice                                    0.121801   \n",
       "\n",
       "                             RevPctChange[t - 4]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanStartEndPrice                                    0.091061   \n",
       "\n",
       "                             VolumeWeightedPctChange[t - 1]  \n",
       "PctChange:MeanStartEndPrice                        0.280448  \n",
       "\n",
       "[1 rows x 97 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_pct_change(single_stock, 'MeanStartEndPrice')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>RevPctChange[t - 4]:Direction2</th>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:VolumeWeightedEndPrice</th>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice</th>\n",
       "      <td>0.483451</td>\n",
       "      <td>0.526626</td>\n",
       "      <td>0.578251</td>\n",
       "      <td>0.484426</td>\n",
       "      <td>0.481941</td>\n",
       "      <td>0.100102</td>\n",
       "      <td>0.096647</td>\n",
       "      <td>0.346744</td>\n",
       "      <td>0.328076</td>\n",
       "      <td>0.386489</td>\n",
       "      <td>...</td>\n",
       "      <td>0.00797</td>\n",
       "      <td>0.001267</td>\n",
       "      <td>-0.03703</td>\n",
       "      <td>0.009331</td>\n",
       "      <td>-0.017346</td>\n",
       "      <td>0.261489</td>\n",
       "      <td>0.165408</td>\n",
       "      <td>0.114751</td>\n",
       "      <td>0.08554</td>\n",
       "      <td>0.267774</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 97 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                        Direction1[t - 1]  Direction2[t - 1]  F1[t - 1]  \\\n",
       "PctChange:MeanEndPrice           0.483451           0.526626   0.578251   \n",
       "\n",
       "                        F2[t - 1]  F3[t - 1]  F4[t - 1]  F5[t - 1]  F6[t - 1]  \\\n",
       "PctChange:MeanEndPrice   0.484426   0.481941   0.100102   0.096647   0.346744   \n",
       "\n",
       "                        F7[t - 1]  F8[t - 1]  ...  \\\n",
       "PctChange:MeanEndPrice   0.328076   0.386489  ...   \n",
       "\n",
       "                        RevPctChange[t - 4]:Direction2  \\\n",
       "PctChange:MeanEndPrice                         0.00797   \n",
       "\n",
       "                        RevPctChange[t - 1]:StdEndPrice  \\\n",
       "PctChange:MeanEndPrice                         0.001267   \n",
       "\n",
       "                        RevPctChange[t - 2]:StdEndPrice  \\\n",
       "PctChange:MeanEndPrice                         -0.03703   \n",
       "\n",
       "                        RevPctChange[t - 3]:StdEndPrice  \\\n",
       "PctChange:MeanEndPrice                         0.009331   \n",
       "\n",
       "                        RevPctChange[t - 4]:StdEndPrice  \\\n",
       "PctChange:MeanEndPrice                        -0.017346   \n",
       "\n",
       "                        RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanEndPrice                                    0.261489   \n",
       "\n",
       "                        RevPctChange[t - 2]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanEndPrice                                    0.165408   \n",
       "\n",
       "                        RevPctChange[t - 3]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanEndPrice                                    0.114751   \n",
       "\n",
       "                        RevPctChange[t - 4]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanEndPrice                                     0.08554   \n",
       "\n",
       "                        VolumeWeightedPctChange[t - 1]  \n",
       "PctChange:MeanEndPrice                        0.267774  \n",
       "\n",
       "[1 rows x 97 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_pct_change(single_stock, 'MeanEndPrice')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>RevPctChange[t - 4]:Direction2</th>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:VolumeWeightedEndPrice</th>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:MedianEndPrice</th>\n",
       "      <td>0.43313</td>\n",
       "      <td>0.495696</td>\n",
       "      <td>0.531789</td>\n",
       "      <td>0.434237</td>\n",
       "      <td>0.431455</td>\n",
       "      <td>0.103161</td>\n",
       "      <td>0.099751</td>\n",
       "      <td>0.333383</td>\n",
       "      <td>0.314047</td>\n",
       "      <td>0.375251</td>\n",
       "      <td>...</td>\n",
       "      <td>0.003766</td>\n",
       "      <td>-0.001568</td>\n",
       "      <td>-0.036419</td>\n",
       "      <td>0.017188</td>\n",
       "      <td>-0.017327</td>\n",
       "      <td>0.231907</td>\n",
       "      <td>0.146785</td>\n",
       "      <td>0.102032</td>\n",
       "      <td>0.075265</td>\n",
       "      <td>0.25105</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 97 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                          Direction1[t - 1]  Direction2[t - 1]  F1[t - 1]  \\\n",
       "PctChange:MedianEndPrice            0.43313           0.495696   0.531789   \n",
       "\n",
       "                          F2[t - 1]  F3[t - 1]  F4[t - 1]  F5[t - 1]  \\\n",
       "PctChange:MedianEndPrice   0.434237   0.431455   0.103161   0.099751   \n",
       "\n",
       "                          F6[t - 1]  F7[t - 1]  F8[t - 1]  ...  \\\n",
       "PctChange:MedianEndPrice   0.333383   0.314047   0.375251  ...   \n",
       "\n",
       "                          RevPctChange[t - 4]:Direction2  \\\n",
       "PctChange:MedianEndPrice                        0.003766   \n",
       "\n",
       "                          RevPctChange[t - 1]:StdEndPrice  \\\n",
       "PctChange:MedianEndPrice                        -0.001568   \n",
       "\n",
       "                          RevPctChange[t - 2]:StdEndPrice  \\\n",
       "PctChange:MedianEndPrice                        -0.036419   \n",
       "\n",
       "                          RevPctChange[t - 3]:StdEndPrice  \\\n",
       "PctChange:MedianEndPrice                         0.017188   \n",
       "\n",
       "                          RevPctChange[t - 4]:StdEndPrice  \\\n",
       "PctChange:MedianEndPrice                        -0.017327   \n",
       "\n",
       "                          RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MedianEndPrice                                    0.231907   \n",
       "\n",
       "                          RevPctChange[t - 2]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MedianEndPrice                                    0.146785   \n",
       "\n",
       "                          RevPctChange[t - 3]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MedianEndPrice                                    0.102032   \n",
       "\n",
       "                          RevPctChange[t - 4]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MedianEndPrice                                    0.075265   \n",
       "\n",
       "                          VolumeWeightedPctChange[t - 1]  \n",
       "PctChange:MedianEndPrice                         0.25105  \n",
       "\n",
       "[1 rows x 97 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_pct_change(single_stock, 'MedianEndPrice')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>RevPctChange[t - 4]:Direction2</th>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:VolumeWeightedEndPrice</th>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:VolumeWeightedEndPrice</th>\n",
       "      <td>0.40708</td>\n",
       "      <td>0.465904</td>\n",
       "      <td>0.517969</td>\n",
       "      <td>0.408149</td>\n",
       "      <td>0.405454</td>\n",
       "      <td>0.102325</td>\n",
       "      <td>0.092331</td>\n",
       "      <td>0.321097</td>\n",
       "      <td>0.313636</td>\n",
       "      <td>0.355343</td>\n",
       "      <td>...</td>\n",
       "      <td>0.007447</td>\n",
       "      <td>0.005583</td>\n",
       "      <td>-0.036474</td>\n",
       "      <td>0.014437</td>\n",
       "      <td>-0.010864</td>\n",
       "      <td>0.201354</td>\n",
       "      <td>0.123107</td>\n",
       "      <td>0.076505</td>\n",
       "      <td>0.054892</td>\n",
       "      <td>0.223231</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 97 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  Direction1[t - 1]  Direction2[t - 1]  \\\n",
       "PctChange:VolumeWeightedEndPrice            0.40708           0.465904   \n",
       "\n",
       "                                  F1[t - 1]  F2[t - 1]  F3[t - 1]  F4[t - 1]  \\\n",
       "PctChange:VolumeWeightedEndPrice   0.517969   0.408149   0.405454   0.102325   \n",
       "\n",
       "                                  F5[t - 1]  F6[t - 1]  F7[t - 1]  F8[t - 1]  \\\n",
       "PctChange:VolumeWeightedEndPrice   0.092331   0.321097   0.313636   0.355343   \n",
       "\n",
       "                                  ...  RevPctChange[t - 4]:Direction2  \\\n",
       "PctChange:VolumeWeightedEndPrice  ...                        0.007447   \n",
       "\n",
       "                                  RevPctChange[t - 1]:StdEndPrice  \\\n",
       "PctChange:VolumeWeightedEndPrice                         0.005583   \n",
       "\n",
       "                                  RevPctChange[t - 2]:StdEndPrice  \\\n",
       "PctChange:VolumeWeightedEndPrice                        -0.036474   \n",
       "\n",
       "                                  RevPctChange[t - 3]:StdEndPrice  \\\n",
       "PctChange:VolumeWeightedEndPrice                         0.014437   \n",
       "\n",
       "                                  RevPctChange[t - 4]:StdEndPrice  \\\n",
       "PctChange:VolumeWeightedEndPrice                        -0.010864   \n",
       "\n",
       "                                  RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "PctChange:VolumeWeightedEndPrice                                    0.201354   \n",
       "\n",
       "                                  RevPctChange[t - 2]:VolumeWeightedEndPrice  \\\n",
       "PctChange:VolumeWeightedEndPrice                                    0.123107   \n",
       "\n",
       "                                  RevPctChange[t - 3]:VolumeWeightedEndPrice  \\\n",
       "PctChange:VolumeWeightedEndPrice                                    0.076505   \n",
       "\n",
       "                                  RevPctChange[t - 4]:VolumeWeightedEndPrice  \\\n",
       "PctChange:VolumeWeightedEndPrice                                    0.054892   \n",
       "\n",
       "                                  VolumeWeightedPctChange[t - 1]  \n",
       "PctChange:VolumeWeightedEndPrice                        0.223231  \n",
       "\n",
       "[1 rows x 97 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_pct_change(single_stock, 'VolumeWeightedEndPrice')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>RevPctChange[t - 4]:Direction2</th>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:VolumeWeightedEndPrice</th>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>MeanPctChangeV2</th>\n",
       "      <td>-0.031991</td>\n",
       "      <td>-0.045596</td>\n",
       "      <td>-0.04144</td>\n",
       "      <td>-0.032153</td>\n",
       "      <td>-0.031811</td>\n",
       "      <td>0.023188</td>\n",
       "      <td>-0.014245</td>\n",
       "      <td>-0.022885</td>\n",
       "      <td>-0.022381</td>\n",
       "      <td>-0.019361</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.009426</td>\n",
       "      <td>0.016292</td>\n",
       "      <td>-0.040286</td>\n",
       "      <td>0.028166</td>\n",
       "      <td>-0.019766</td>\n",
       "      <td>-0.009845</td>\n",
       "      <td>-0.026365</td>\n",
       "      <td>-0.041061</td>\n",
       "      <td>-0.028729</td>\n",
       "      <td>-0.024427</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 96 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Direction1[t - 1]  Direction2[t - 1]  F1[t - 1]  F2[t - 1]  \\\n",
       "MeanPctChangeV2          -0.031991          -0.045596   -0.04144  -0.032153   \n",
       "\n",
       "                 F3[t - 1]  F4[t - 1]  F5[t - 1]  F6[t - 1]  F7[t - 1]  \\\n",
       "MeanPctChangeV2  -0.031811   0.023188  -0.014245  -0.022885  -0.022381   \n",
       "\n",
       "                 F8[t - 1]  ...  RevPctChange[t - 4]:Direction2  \\\n",
       "MeanPctChangeV2  -0.019361  ...                       -0.009426   \n",
       "\n",
       "                 RevPctChange[t - 1]:StdEndPrice  \\\n",
       "MeanPctChangeV2                         0.016292   \n",
       "\n",
       "                 RevPctChange[t - 2]:StdEndPrice  \\\n",
       "MeanPctChangeV2                        -0.040286   \n",
       "\n",
       "                 RevPctChange[t - 3]:StdEndPrice  \\\n",
       "MeanPctChangeV2                         0.028166   \n",
       "\n",
       "                 RevPctChange[t - 4]:StdEndPrice  \\\n",
       "MeanPctChangeV2                        -0.019766   \n",
       "\n",
       "                 RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "MeanPctChangeV2                                   -0.009845   \n",
       "\n",
       "                 RevPctChange[t - 2]:VolumeWeightedEndPrice  \\\n",
       "MeanPctChangeV2                                   -0.026365   \n",
       "\n",
       "                 RevPctChange[t - 3]:VolumeWeightedEndPrice  \\\n",
       "MeanPctChangeV2                                   -0.041061   \n",
       "\n",
       "                 RevPctChange[t - 4]:VolumeWeightedEndPrice  \\\n",
       "MeanPctChangeV2                                   -0.028729   \n",
       "\n",
       "                 VolumeWeightedPctChange[t - 1]  \n",
       "MeanPctChangeV2                       -0.024427  \n",
       "\n",
       "[1 rows x 96 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "correlation_with_feature(single_stock, 'MeanPctChangeV2')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MeanPctChangeV2</th>\n",
       "      <th>AbsCorr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>G4[t - 1]</th>\n",
       "      <td>-0.063230</td>\n",
       "      <td>0.063230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>G2[t - 1]</th>\n",
       "      <td>-0.062256</td>\n",
       "      <td>0.062256</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>G3[t - 1]</th>\n",
       "      <td>-0.059414</td>\n",
       "      <td>0.059414</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AdjLogReturn:LastEndPrice[t - 1]</th>\n",
       "      <td>-0.056929</td>\n",
       "      <td>0.056929</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AdjustedPctChange[t - 1]</th>\n",
       "      <td>-0.056923</td>\n",
       "      <td>0.056923</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>G1[t - 1]</th>\n",
       "      <td>-0.053496</td>\n",
       "      <td>0.053496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <td>-0.045596</td>\n",
       "      <td>0.045596</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:LastEndPrice</th>\n",
       "      <td>-0.043364</td>\n",
       "      <td>0.043364</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <td>-0.041440</td>\n",
       "      <td>0.041440</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@WithNorm(MeanEndPrice)[t - 1]</th>\n",
       "      <td>-0.041382</td>\n",
       "      <td>0.041382</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <td>-0.040286</td>\n",
       "      <td>0.040286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:LastEndPrice</th>\n",
       "      <td>-0.039042</td>\n",
       "      <td>0.039042</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MaxPrice</th>\n",
       "      <td>-0.037298</td>\n",
       "      <td>0.037298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MedianEndPrice[t - 1]</th>\n",
       "      <td>0.035520</td>\n",
       "      <td>0.035520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice[t - 1]</th>\n",
       "      <td>0.032675</td>\n",
       "      <td>0.032675</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <td>-0.032153</td>\n",
       "      <td>0.032153</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <td>-0.031991</td>\n",
       "      <td>0.031991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:StdEndPrice[t - 1]</th>\n",
       "      <td>0.031986</td>\n",
       "      <td>0.031986</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <td>-0.031811</td>\n",
       "      <td>0.031811</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanStartEndPrice[t - 1]</th>\n",
       "      <td>0.031624</td>\n",
       "      <td>0.031624</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanAvg4Price[t - 1]</th>\n",
       "      <td>0.031601</td>\n",
       "      <td>0.031601</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanMaxMinPrice[t - 1]</th>\n",
       "      <td>0.031575</td>\n",
       "      <td>0.031575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:VolumeWeightedEndPrice[t - 1]</th>\n",
       "      <td>0.031453</td>\n",
       "      <td>0.031453</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:FirstStartPrice</th>\n",
       "      <td>-0.028884</td>\n",
       "      <td>0.028884</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <td>-0.026365</td>\n",
       "      <td>0.026365</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "      <td>-0.024427</td>\n",
       "      <td>0.024427</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MeanEndPrice</th>\n",
       "      <td>-0.023796</td>\n",
       "      <td>0.023796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MeanStartEndPrice</th>\n",
       "      <td>-0.023460</td>\n",
       "      <td>0.023460</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MeanAvg4Price</th>\n",
       "      <td>-0.023324</td>\n",
       "      <td>0.023324</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <td>0.023188</td>\n",
       "      <td>0.023188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MeanMaxMinPrice</th>\n",
       "      <td>-0.023187</td>\n",
       "      <td>0.023187</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MaxPrice</th>\n",
       "      <td>-0.022937</td>\n",
       "      <td>0.022937</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <td>-0.022885</td>\n",
       "      <td>0.022885</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <td>-0.022381</td>\n",
       "      <td>0.022381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:Direction1[t - 1]</th>\n",
       "      <td>0.022045</td>\n",
       "      <td>0.022045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MaxPrice[t - 1]</th>\n",
       "      <td>0.021680</td>\n",
       "      <td>0.021680</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MinPrice</th>\n",
       "      <td>-0.021157</td>\n",
       "      <td>0.021157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MedianEndPrice</th>\n",
       "      <td>-0.020154</td>\n",
       "      <td>0.020154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <td>-0.019361</td>\n",
       "      <td>0.019361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:Direction2[t - 1]</th>\n",
       "      <td>0.018039</td>\n",
       "      <td>0.018039</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <td>0.016292</td>\n",
       "      <td>0.016292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <td>-0.014245</td>\n",
       "      <td>0.014245</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MinPrice[t - 1]</th>\n",
       "      <td>0.013596</td>\n",
       "      <td>0.013596</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:Direction1</th>\n",
       "      <td>-0.013045</td>\n",
       "      <td>0.013045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:Direction2</th>\n",
       "      <td>-0.010306</td>\n",
       "      <td>0.010306</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@WithNorm(MeanEndPrice)[t - 2]</th>\n",
       "      <td>-0.010220</td>\n",
       "      <td>0.010220</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <td>-0.009845</td>\n",
       "      <td>0.009845</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MinPrice</th>\n",
       "      <td>-0.008578</td>\n",
       "      <td>0.008578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:Direction2</th>\n",
       "      <td>0.008359</td>\n",
       "      <td>0.008359</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MeanEndPrice</th>\n",
       "      <td>-0.007801</td>\n",
       "      <td>0.007801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MeanMaxMinPrice</th>\n",
       "      <td>-0.007215</td>\n",
       "      <td>0.007215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MeanAvg4Price</th>\n",
       "      <td>-0.007151</td>\n",
       "      <td>0.007151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MeanStartEndPrice</th>\n",
       "      <td>-0.007086</td>\n",
       "      <td>0.007086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:Direction1</th>\n",
       "      <td>0.006967</td>\n",
       "      <td>0.006967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:FirstStartPrice</th>\n",
       "      <td>-0.005815</td>\n",
       "      <td>0.005815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:FirstStartPrice[t - 1]</th>\n",
       "      <td>0.005598</td>\n",
       "      <td>0.005598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:LastEndPrice[t - 1]</th>\n",
       "      <td>0.005375</td>\n",
       "      <td>0.005375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MedianEndPrice</th>\n",
       "      <td>-0.000717</td>\n",
       "      <td>0.000717</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            MeanPctChangeV2   AbsCorr\n",
       "G4[t - 1]                                         -0.063230  0.063230\n",
       "G2[t - 1]                                         -0.062256  0.062256\n",
       "G3[t - 1]                                         -0.059414  0.059414\n",
       "AdjLogReturn:LastEndPrice[t - 1]                  -0.056929  0.056929\n",
       "AdjustedPctChange[t - 1]                          -0.056923  0.056923\n",
       "G1[t - 1]                                         -0.053496  0.053496\n",
       "Direction2[t - 1]                                 -0.045596  0.045596\n",
       "RevPctChange[t - 1]:LastEndPrice                  -0.043364  0.043364\n",
       "F1[t - 1]                                         -0.041440  0.041440\n",
       "H1@WithNorm(MeanEndPrice)[t - 1]                  -0.041382  0.041382\n",
       "RevPctChange[t - 2]:StdEndPrice                   -0.040286  0.040286\n",
       "RevPctChange[t - 2]:LastEndPrice                  -0.039042  0.039042\n",
       "RevPctChange[t - 2]:MaxPrice                      -0.037298  0.037298\n",
       "PctChange:MedianEndPrice[t - 1]                    0.035520  0.035520\n",
       "PctChange:MeanEndPrice[t - 1]                      0.032675  0.032675\n",
       "F2[t - 1]                                         -0.032153  0.032153\n",
       "Direction1[t - 1]                                 -0.031991  0.031991\n",
       "PctChange:StdEndPrice[t - 1]                       0.031986  0.031986\n",
       "F3[t - 1]                                         -0.031811  0.031811\n",
       "PctChange:MeanStartEndPrice[t - 1]                 0.031624  0.031624\n",
       "PctChange:MeanAvg4Price[t - 1]                     0.031601  0.031601\n",
       "PctChange:MeanMaxMinPrice[t - 1]                   0.031575  0.031575\n",
       "PctChange:VolumeWeightedEndPrice[t - 1]            0.031453  0.031453\n",
       "RevPctChange[t - 1]:FirstStartPrice               -0.028884  0.028884\n",
       "RevPctChange[t - 2]:VolumeWeightedEndPrice        -0.026365  0.026365\n",
       "VolumeWeightedPctChange[t - 1]                    -0.024427  0.024427\n",
       "RevPctChange[t - 2]:MeanEndPrice                  -0.023796  0.023796\n",
       "RevPctChange[t - 2]:MeanStartEndPrice             -0.023460  0.023460\n",
       "RevPctChange[t - 2]:MeanAvg4Price                 -0.023324  0.023324\n",
       "F4[t - 1]                                          0.023188  0.023188\n",
       "RevPctChange[t - 2]:MeanMaxMinPrice               -0.023187  0.023187\n",
       "RevPctChange[t - 1]:MaxPrice                      -0.022937  0.022937\n",
       "F6[t - 1]                                         -0.022885  0.022885\n",
       "F7[t - 1]                                         -0.022381  0.022381\n",
       "PctChange:Direction1[t - 1]                        0.022045  0.022045\n",
       "PctChange:MaxPrice[t - 1]                          0.021680  0.021680\n",
       "RevPctChange[t - 1]:MinPrice                      -0.021157  0.021157\n",
       "RevPctChange[t - 2]:MedianEndPrice                -0.020154  0.020154\n",
       "F8[t - 1]                                         -0.019361  0.019361\n",
       "PctChange:Direction2[t - 1]                        0.018039  0.018039\n",
       "RevPctChange[t - 1]:StdEndPrice                    0.016292  0.016292\n",
       "F5[t - 1]                                         -0.014245  0.014245\n",
       "PctChange:MinPrice[t - 1]                          0.013596  0.013596\n",
       "RevPctChange[t - 1]:Direction1                    -0.013045  0.013045\n",
       "RevPctChange[t - 1]:Direction2                    -0.010306  0.010306\n",
       "H1@WithNorm(MeanEndPrice)[t - 2]                  -0.010220  0.010220\n",
       "RevPctChange[t - 1]:VolumeWeightedEndPrice        -0.009845  0.009845\n",
       "RevPctChange[t - 2]:MinPrice                      -0.008578  0.008578\n",
       "RevPctChange[t - 2]:Direction2                     0.008359  0.008359\n",
       "RevPctChange[t - 1]:MeanEndPrice                  -0.007801  0.007801\n",
       "RevPctChange[t - 1]:MeanMaxMinPrice               -0.007215  0.007215\n",
       "RevPctChange[t - 1]:MeanAvg4Price                 -0.007151  0.007151\n",
       "RevPctChange[t - 1]:MeanStartEndPrice             -0.007086  0.007086\n",
       "RevPctChange[t - 2]:Direction1                     0.006967  0.006967\n",
       "RevPctChange[t - 2]:FirstStartPrice               -0.005815  0.005815\n",
       "PctChange:FirstStartPrice[t - 1]                   0.005598  0.005598\n",
       "PctChange:LastEndPrice[t - 1]                      0.005375  0.005375\n",
       "RevPctChange[t - 1]:MedianEndPrice                -0.000717  0.000717"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "find_most_correlated_features(single_stock, 'MeanPctChangeV2')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us settle for the moment on the 'MeanEndPrice', and compute the correlations over multiple intervals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# some of the better features\n",
    "x_features = [\n",
    "    'AdjustedPctChange[t - 1]',\n",
    "    'F1[t - 1]',\n",
    "    'H1@WithNorm(MeanEndPrice)[t - 1]',\n",
    "    'RevPctChange[t - 1]:LastEndPrice',\n",
    "    'F2[t - 1]',\n",
    "    'Direction2[t - 1]',\n",
    "    'F3[t - 1]',\n",
    "    'G1[t - 1]',\n",
    "    'G3[t - 1]',\n",
    "    'PctChange:LastEndPrice[t - 1]',\n",
    "    'RevPctChange[t - 2]:LastEndPrice',\n",
    "    'F6[t - 1]',\n",
    "    'G2[t - 1]',\n",
    "    'F7[t - 1]',\n",
    "    'G4[t - 1]',\n",
    "    'RevPctChange[t - 1]:VolumeWeightedEndPrice',\n",
    "    'RevPctChange[t - 1]:MinPrice',\n",
    "    'RevPctChange[t - 1]:MeanEndPrice',\n",
    "    'RevPctChange[t - 1]:MeanMaxMinPrice',\n",
    "    'RevPctChange[t - 1]:MeanAvg4Price',\n",
    "    'RevPctChange[t - 1]:MeanStartEndPrice',\n",
    "    'RevPctChange[t - 1]:MaxPrice']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "intervals = ['1Min', '2Min', '3Min', '4Min', '5Min', '6Min', '7Min', '8Min', '9Min', '10Min', \n",
    "             '15Min', '30Min', '60Min', '120Min', '180Min', '240Min',\n",
    "             '1D', '2D', '1W', '2W']\n",
    "def corr_over_intervals(stock):\n",
    "    results = []\n",
    "    for interval in intervals:\n",
    "        single_stock = prepare_single_stock(stock, interval)\n",
    "        cr = corr_pct_change(single_stock, 'MeanEndPrice')\n",
    "        cr.index = [cr.index[0] + \"@\" + interval]\n",
    "        results.append(cr)\n",
    "    return pd.concat(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AdjustedPctChange[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>H1@WithNorm(MeanEndPrice)[t - 1]</th>\n",
       "      <th>RevPctChange[t - 1]:LastEndPrice</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>G1[t - 1]</th>\n",
       "      <th>G3[t - 1]</th>\n",
       "      <th>PctChange:LastEndPrice[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>G2[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>G4[t - 1]</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:MinPrice</th>\n",
       "      <th>RevPctChange[t - 1]:MeanEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:MeanMaxMinPrice</th>\n",
       "      <th>RevPctChange[t - 1]:MeanAvg4Price</th>\n",
       "      <th>RevPctChange[t - 1]:MeanStartEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:MaxPrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@1Min</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.040016</td>\n",
       "      <td>-0.040016</td>\n",
       "      <td>-0.057669</td>\n",
       "      <td>-0.044060</td>\n",
       "      <td>-0.044253</td>\n",
       "      <td>-0.044180</td>\n",
       "      <td>-0.044115</td>\n",
       "      <td>-0.040016</td>\n",
       "      <td>-0.042200</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.025962</td>\n",
       "      <td>-0.027606</td>\n",
       "      <td>-0.024194</td>\n",
       "      <td>-0.057669</td>\n",
       "      <td>-0.038706</td>\n",
       "      <td>-0.057669</td>\n",
       "      <td>-0.041413</td>\n",
       "      <td>-0.041558</td>\n",
       "      <td>-0.040399</td>\n",
       "      <td>-0.035641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@2Min</th>\n",
       "      <td>0.342471</td>\n",
       "      <td>0.289739</td>\n",
       "      <td>0.289734</td>\n",
       "      <td>0.225450</td>\n",
       "      <td>0.244871</td>\n",
       "      <td>0.268131</td>\n",
       "      <td>0.244607</td>\n",
       "      <td>0.220126</td>\n",
       "      <td>0.188707</td>\n",
       "      <td>0.136605</td>\n",
       "      <td>...</td>\n",
       "      <td>0.167881</td>\n",
       "      <td>0.204659</td>\n",
       "      <td>0.136711</td>\n",
       "      <td>0.116428</td>\n",
       "      <td>0.065166</td>\n",
       "      <td>0.107801</td>\n",
       "      <td>0.054954</td>\n",
       "      <td>0.051866</td>\n",
       "      <td>0.048316</td>\n",
       "      <td>0.076487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@3Min</th>\n",
       "      <td>0.551271</td>\n",
       "      <td>0.495723</td>\n",
       "      <td>0.495199</td>\n",
       "      <td>0.368611</td>\n",
       "      <td>0.441762</td>\n",
       "      <td>0.378296</td>\n",
       "      <td>0.442690</td>\n",
       "      <td>0.332429</td>\n",
       "      <td>0.284448</td>\n",
       "      <td>0.282670</td>\n",
       "      <td>...</td>\n",
       "      <td>0.243943</td>\n",
       "      <td>0.273013</td>\n",
       "      <td>0.202095</td>\n",
       "      <td>0.196265</td>\n",
       "      <td>0.134186</td>\n",
       "      <td>0.156402</td>\n",
       "      <td>0.102394</td>\n",
       "      <td>0.099780</td>\n",
       "      <td>0.096735</td>\n",
       "      <td>0.129668</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@4Min</th>\n",
       "      <td>0.566814</td>\n",
       "      <td>0.525967</td>\n",
       "      <td>0.525993</td>\n",
       "      <td>0.430283</td>\n",
       "      <td>0.462232</td>\n",
       "      <td>0.433443</td>\n",
       "      <td>0.461978</td>\n",
       "      <td>0.378099</td>\n",
       "      <td>0.324907</td>\n",
       "      <td>0.290082</td>\n",
       "      <td>...</td>\n",
       "      <td>0.286096</td>\n",
       "      <td>0.308451</td>\n",
       "      <td>0.233498</td>\n",
       "      <td>0.216985</td>\n",
       "      <td>0.164556</td>\n",
       "      <td>0.183966</td>\n",
       "      <td>0.133441</td>\n",
       "      <td>0.131541</td>\n",
       "      <td>0.129296</td>\n",
       "      <td>0.145654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@5Min</th>\n",
       "      <td>0.565416</td>\n",
       "      <td>0.531774</td>\n",
       "      <td>0.531898</td>\n",
       "      <td>0.430781</td>\n",
       "      <td>0.455919</td>\n",
       "      <td>0.440793</td>\n",
       "      <td>0.454894</td>\n",
       "      <td>0.387331</td>\n",
       "      <td>0.344720</td>\n",
       "      <td>0.273967</td>\n",
       "      <td>...</td>\n",
       "      <td>0.300524</td>\n",
       "      <td>0.325587</td>\n",
       "      <td>0.250678</td>\n",
       "      <td>0.209390</td>\n",
       "      <td>0.150874</td>\n",
       "      <td>0.196675</td>\n",
       "      <td>0.152193</td>\n",
       "      <td>0.151793</td>\n",
       "      <td>0.151141</td>\n",
       "      <td>0.161078</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@6Min</th>\n",
       "      <td>0.590399</td>\n",
       "      <td>0.555696</td>\n",
       "      <td>0.555851</td>\n",
       "      <td>0.467702</td>\n",
       "      <td>0.477667</td>\n",
       "      <td>0.472083</td>\n",
       "      <td>0.476358</td>\n",
       "      <td>0.421592</td>\n",
       "      <td>0.366387</td>\n",
       "      <td>0.289028</td>\n",
       "      <td>...</td>\n",
       "      <td>0.313906</td>\n",
       "      <td>0.340947</td>\n",
       "      <td>0.262960</td>\n",
       "      <td>0.227110</td>\n",
       "      <td>0.166189</td>\n",
       "      <td>0.205975</td>\n",
       "      <td>0.165595</td>\n",
       "      <td>0.164571</td>\n",
       "      <td>0.163352</td>\n",
       "      <td>0.183919</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@7Min</th>\n",
       "      <td>0.625876</td>\n",
       "      <td>0.591994</td>\n",
       "      <td>0.592208</td>\n",
       "      <td>0.510106</td>\n",
       "      <td>0.523547</td>\n",
       "      <td>0.515200</td>\n",
       "      <td>0.522686</td>\n",
       "      <td>0.431648</td>\n",
       "      <td>0.380510</td>\n",
       "      <td>0.317882</td>\n",
       "      <td>...</td>\n",
       "      <td>0.323291</td>\n",
       "      <td>0.374064</td>\n",
       "      <td>0.270319</td>\n",
       "      <td>0.250972</td>\n",
       "      <td>0.193256</td>\n",
       "      <td>0.213155</td>\n",
       "      <td>0.173343</td>\n",
       "      <td>0.172789</td>\n",
       "      <td>0.172079</td>\n",
       "      <td>0.196120</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@8Min</th>\n",
       "      <td>0.630653</td>\n",
       "      <td>0.594318</td>\n",
       "      <td>0.594350</td>\n",
       "      <td>0.515764</td>\n",
       "      <td>0.515287</td>\n",
       "      <td>0.520891</td>\n",
       "      <td>0.513944</td>\n",
       "      <td>0.467379</td>\n",
       "      <td>0.406207</td>\n",
       "      <td>0.307911</td>\n",
       "      <td>...</td>\n",
       "      <td>0.334354</td>\n",
       "      <td>0.344175</td>\n",
       "      <td>0.273827</td>\n",
       "      <td>0.249791</td>\n",
       "      <td>0.210597</td>\n",
       "      <td>0.210900</td>\n",
       "      <td>0.177758</td>\n",
       "      <td>0.176773</td>\n",
       "      <td>0.175667</td>\n",
       "      <td>0.181611</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@9Min</th>\n",
       "      <td>0.618434</td>\n",
       "      <td>0.579562</td>\n",
       "      <td>0.579671</td>\n",
       "      <td>0.496090</td>\n",
       "      <td>0.501663</td>\n",
       "      <td>0.499425</td>\n",
       "      <td>0.500130</td>\n",
       "      <td>0.435211</td>\n",
       "      <td>0.370102</td>\n",
       "      <td>0.301639</td>\n",
       "      <td>...</td>\n",
       "      <td>0.329688</td>\n",
       "      <td>0.335445</td>\n",
       "      <td>0.268693</td>\n",
       "      <td>0.237636</td>\n",
       "      <td>0.203600</td>\n",
       "      <td>0.207381</td>\n",
       "      <td>0.181254</td>\n",
       "      <td>0.180439</td>\n",
       "      <td>0.179524</td>\n",
       "      <td>0.184723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@10Min</th>\n",
       "      <td>0.614705</td>\n",
       "      <td>0.574660</td>\n",
       "      <td>0.574693</td>\n",
       "      <td>0.483943</td>\n",
       "      <td>0.481906</td>\n",
       "      <td>0.487570</td>\n",
       "      <td>0.480498</td>\n",
       "      <td>0.428700</td>\n",
       "      <td>0.375200</td>\n",
       "      <td>0.286715</td>\n",
       "      <td>...</td>\n",
       "      <td>0.332562</td>\n",
       "      <td>0.330276</td>\n",
       "      <td>0.265342</td>\n",
       "      <td>0.219296</td>\n",
       "      <td>0.192938</td>\n",
       "      <td>0.198068</td>\n",
       "      <td>0.174794</td>\n",
       "      <td>0.174223</td>\n",
       "      <td>0.173575</td>\n",
       "      <td>0.169566</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@15Min</th>\n",
       "      <td>0.654117</td>\n",
       "      <td>0.602978</td>\n",
       "      <td>0.602929</td>\n",
       "      <td>0.543033</td>\n",
       "      <td>0.508436</td>\n",
       "      <td>0.544789</td>\n",
       "      <td>0.506506</td>\n",
       "      <td>0.478727</td>\n",
       "      <td>0.423081</td>\n",
       "      <td>0.292065</td>\n",
       "      <td>...</td>\n",
       "      <td>0.349454</td>\n",
       "      <td>0.344299</td>\n",
       "      <td>0.276030</td>\n",
       "      <td>0.258986</td>\n",
       "      <td>0.232437</td>\n",
       "      <td>0.222886</td>\n",
       "      <td>0.205639</td>\n",
       "      <td>0.205112</td>\n",
       "      <td>0.204543</td>\n",
       "      <td>0.202689</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@30Min</th>\n",
       "      <td>0.658903</td>\n",
       "      <td>0.613942</td>\n",
       "      <td>0.613955</td>\n",
       "      <td>0.525468</td>\n",
       "      <td>0.507088</td>\n",
       "      <td>0.527821</td>\n",
       "      <td>0.505324</td>\n",
       "      <td>0.466055</td>\n",
       "      <td>0.399106</td>\n",
       "      <td>0.291494</td>\n",
       "      <td>...</td>\n",
       "      <td>0.337967</td>\n",
       "      <td>0.337137</td>\n",
       "      <td>0.251287</td>\n",
       "      <td>0.240973</td>\n",
       "      <td>0.197967</td>\n",
       "      <td>0.214357</td>\n",
       "      <td>0.206402</td>\n",
       "      <td>0.206200</td>\n",
       "      <td>0.205986</td>\n",
       "      <td>0.207997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@60Min</th>\n",
       "      <td>0.694655</td>\n",
       "      <td>0.649276</td>\n",
       "      <td>0.649291</td>\n",
       "      <td>0.582784</td>\n",
       "      <td>0.567457</td>\n",
       "      <td>0.585198</td>\n",
       "      <td>0.565809</td>\n",
       "      <td>0.524512</td>\n",
       "      <td>0.437473</td>\n",
       "      <td>0.307664</td>\n",
       "      <td>...</td>\n",
       "      <td>0.368559</td>\n",
       "      <td>0.308067</td>\n",
       "      <td>0.262769</td>\n",
       "      <td>0.275303</td>\n",
       "      <td>0.267886</td>\n",
       "      <td>0.239133</td>\n",
       "      <td>0.233871</td>\n",
       "      <td>0.233797</td>\n",
       "      <td>0.233719</td>\n",
       "      <td>0.217697</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@120Min</th>\n",
       "      <td>0.701747</td>\n",
       "      <td>0.636728</td>\n",
       "      <td>0.637069</td>\n",
       "      <td>0.605349</td>\n",
       "      <td>0.557808</td>\n",
       "      <td>0.607802</td>\n",
       "      <td>0.556109</td>\n",
       "      <td>0.482065</td>\n",
       "      <td>0.371293</td>\n",
       "      <td>0.283848</td>\n",
       "      <td>...</td>\n",
       "      <td>0.313248</td>\n",
       "      <td>0.344561</td>\n",
       "      <td>0.215147</td>\n",
       "      <td>0.269903</td>\n",
       "      <td>0.253878</td>\n",
       "      <td>0.213852</td>\n",
       "      <td>0.211203</td>\n",
       "      <td>0.210943</td>\n",
       "      <td>0.210682</td>\n",
       "      <td>0.177215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@180Min</th>\n",
       "      <td>0.695476</td>\n",
       "      <td>0.648003</td>\n",
       "      <td>0.648012</td>\n",
       "      <td>0.600194</td>\n",
       "      <td>0.548924</td>\n",
       "      <td>0.595142</td>\n",
       "      <td>0.547812</td>\n",
       "      <td>0.432629</td>\n",
       "      <td>0.326240</td>\n",
       "      <td>0.295896</td>\n",
       "      <td>...</td>\n",
       "      <td>0.285098</td>\n",
       "      <td>0.439516</td>\n",
       "      <td>0.190657</td>\n",
       "      <td>0.186879</td>\n",
       "      <td>0.190958</td>\n",
       "      <td>0.130026</td>\n",
       "      <td>0.127243</td>\n",
       "      <td>0.127071</td>\n",
       "      <td>0.126899</td>\n",
       "      <td>0.146938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@240Min</th>\n",
       "      <td>0.733977</td>\n",
       "      <td>0.632396</td>\n",
       "      <td>0.632631</td>\n",
       "      <td>0.657055</td>\n",
       "      <td>0.476532</td>\n",
       "      <td>0.642651</td>\n",
       "      <td>0.475621</td>\n",
       "      <td>0.440196</td>\n",
       "      <td>0.291781</td>\n",
       "      <td>0.197493</td>\n",
       "      <td>...</td>\n",
       "      <td>0.269049</td>\n",
       "      <td>0.438542</td>\n",
       "      <td>0.154332</td>\n",
       "      <td>0.252468</td>\n",
       "      <td>0.183441</td>\n",
       "      <td>0.129210</td>\n",
       "      <td>0.127184</td>\n",
       "      <td>0.126991</td>\n",
       "      <td>0.126798</td>\n",
       "      <td>0.125898</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@1D</th>\n",
       "      <td>0.426504</td>\n",
       "      <td>0.396335</td>\n",
       "      <td>0.396102</td>\n",
       "      <td>0.237291</td>\n",
       "      <td>0.269173</td>\n",
       "      <td>0.401603</td>\n",
       "      <td>0.267961</td>\n",
       "      <td>0.130971</td>\n",
       "      <td>0.119244</td>\n",
       "      <td>0.144356</td>\n",
       "      <td>...</td>\n",
       "      <td>0.129605</td>\n",
       "      <td>0.222243</td>\n",
       "      <td>0.116906</td>\n",
       "      <td>0.050948</td>\n",
       "      <td>0.021564</td>\n",
       "      <td>0.074884</td>\n",
       "      <td>0.074113</td>\n",
       "      <td>0.074368</td>\n",
       "      <td>0.074622</td>\n",
       "      <td>0.016199</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@2D</th>\n",
       "      <td>0.559142</td>\n",
       "      <td>0.528476</td>\n",
       "      <td>0.527267</td>\n",
       "      <td>0.420302</td>\n",
       "      <td>0.473881</td>\n",
       "      <td>0.484741</td>\n",
       "      <td>0.474807</td>\n",
       "      <td>0.269731</td>\n",
       "      <td>0.192097</td>\n",
       "      <td>0.377776</td>\n",
       "      <td>...</td>\n",
       "      <td>0.152933</td>\n",
       "      <td>-0.008486</td>\n",
       "      <td>0.141785</td>\n",
       "      <td>0.330986</td>\n",
       "      <td>0.372762</td>\n",
       "      <td>0.368392</td>\n",
       "      <td>0.367833</td>\n",
       "      <td>0.367627</td>\n",
       "      <td>0.367420</td>\n",
       "      <td>0.220406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@1W</th>\n",
       "      <td>0.672781</td>\n",
       "      <td>0.644818</td>\n",
       "      <td>0.643178</td>\n",
       "      <td>0.499140</td>\n",
       "      <td>0.571495</td>\n",
       "      <td>0.584088</td>\n",
       "      <td>0.572422</td>\n",
       "      <td>0.130896</td>\n",
       "      <td>0.177781</td>\n",
       "      <td>0.496672</td>\n",
       "      <td>...</td>\n",
       "      <td>0.058503</td>\n",
       "      <td>0.433556</td>\n",
       "      <td>0.149605</td>\n",
       "      <td>0.210228</td>\n",
       "      <td>0.113029</td>\n",
       "      <td>0.164668</td>\n",
       "      <td>0.164904</td>\n",
       "      <td>0.164882</td>\n",
       "      <td>0.164859</td>\n",
       "      <td>0.184723</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@2W</th>\n",
       "      <td>0.766301</td>\n",
       "      <td>0.780059</td>\n",
       "      <td>0.775115</td>\n",
       "      <td>0.592554</td>\n",
       "      <td>0.629207</td>\n",
       "      <td>0.642983</td>\n",
       "      <td>0.629962</td>\n",
       "      <td>0.360310</td>\n",
       "      <td>0.380075</td>\n",
       "      <td>0.581410</td>\n",
       "      <td>...</td>\n",
       "      <td>0.399271</td>\n",
       "      <td>0.456741</td>\n",
       "      <td>0.406928</td>\n",
       "      <td>0.040768</td>\n",
       "      <td>0.133575</td>\n",
       "      <td>0.010775</td>\n",
       "      <td>0.010645</td>\n",
       "      <td>0.010675</td>\n",
       "      <td>0.010706</td>\n",
       "      <td>0.093234</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>20 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                               AdjustedPctChange[t - 1]  F1[t - 1]  \\\n",
       "PctChange:MeanEndPrice@1Min                         NaN  -0.040016   \n",
       "PctChange:MeanEndPrice@2Min                    0.342471   0.289739   \n",
       "PctChange:MeanEndPrice@3Min                    0.551271   0.495723   \n",
       "PctChange:MeanEndPrice@4Min                    0.566814   0.525967   \n",
       "PctChange:MeanEndPrice@5Min                    0.565416   0.531774   \n",
       "PctChange:MeanEndPrice@6Min                    0.590399   0.555696   \n",
       "PctChange:MeanEndPrice@7Min                    0.625876   0.591994   \n",
       "PctChange:MeanEndPrice@8Min                    0.630653   0.594318   \n",
       "PctChange:MeanEndPrice@9Min                    0.618434   0.579562   \n",
       "PctChange:MeanEndPrice@10Min                   0.614705   0.574660   \n",
       "PctChange:MeanEndPrice@15Min                   0.654117   0.602978   \n",
       "PctChange:MeanEndPrice@30Min                   0.658903   0.613942   \n",
       "PctChange:MeanEndPrice@60Min                   0.694655   0.649276   \n",
       "PctChange:MeanEndPrice@120Min                  0.701747   0.636728   \n",
       "PctChange:MeanEndPrice@180Min                  0.695476   0.648003   \n",
       "PctChange:MeanEndPrice@240Min                  0.733977   0.632396   \n",
       "PctChange:MeanEndPrice@1D                      0.426504   0.396335   \n",
       "PctChange:MeanEndPrice@2D                      0.559142   0.528476   \n",
       "PctChange:MeanEndPrice@1W                      0.672781   0.644818   \n",
       "PctChange:MeanEndPrice@2W                      0.766301   0.780059   \n",
       "\n",
       "                               H1@WithNorm(MeanEndPrice)[t - 1]  \\\n",
       "PctChange:MeanEndPrice@1Min                           -0.040016   \n",
       "PctChange:MeanEndPrice@2Min                            0.289734   \n",
       "PctChange:MeanEndPrice@3Min                            0.495199   \n",
       "PctChange:MeanEndPrice@4Min                            0.525993   \n",
       "PctChange:MeanEndPrice@5Min                            0.531898   \n",
       "PctChange:MeanEndPrice@6Min                            0.555851   \n",
       "PctChange:MeanEndPrice@7Min                            0.592208   \n",
       "PctChange:MeanEndPrice@8Min                            0.594350   \n",
       "PctChange:MeanEndPrice@9Min                            0.579671   \n",
       "PctChange:MeanEndPrice@10Min                           0.574693   \n",
       "PctChange:MeanEndPrice@15Min                           0.602929   \n",
       "PctChange:MeanEndPrice@30Min                           0.613955   \n",
       "PctChange:MeanEndPrice@60Min                           0.649291   \n",
       "PctChange:MeanEndPrice@120Min                          0.637069   \n",
       "PctChange:MeanEndPrice@180Min                          0.648012   \n",
       "PctChange:MeanEndPrice@240Min                          0.632631   \n",
       "PctChange:MeanEndPrice@1D                              0.396102   \n",
       "PctChange:MeanEndPrice@2D                              0.527267   \n",
       "PctChange:MeanEndPrice@1W                              0.643178   \n",
       "PctChange:MeanEndPrice@2W                              0.775115   \n",
       "\n",
       "                               RevPctChange[t - 1]:LastEndPrice  F2[t - 1]  \\\n",
       "PctChange:MeanEndPrice@1Min                           -0.057669  -0.044060   \n",
       "PctChange:MeanEndPrice@2Min                            0.225450   0.244871   \n",
       "PctChange:MeanEndPrice@3Min                            0.368611   0.441762   \n",
       "PctChange:MeanEndPrice@4Min                            0.430283   0.462232   \n",
       "PctChange:MeanEndPrice@5Min                            0.430781   0.455919   \n",
       "PctChange:MeanEndPrice@6Min                            0.467702   0.477667   \n",
       "PctChange:MeanEndPrice@7Min                            0.510106   0.523547   \n",
       "PctChange:MeanEndPrice@8Min                            0.515764   0.515287   \n",
       "PctChange:MeanEndPrice@9Min                            0.496090   0.501663   \n",
       "PctChange:MeanEndPrice@10Min                           0.483943   0.481906   \n",
       "PctChange:MeanEndPrice@15Min                           0.543033   0.508436   \n",
       "PctChange:MeanEndPrice@30Min                           0.525468   0.507088   \n",
       "PctChange:MeanEndPrice@60Min                           0.582784   0.567457   \n",
       "PctChange:MeanEndPrice@120Min                          0.605349   0.557808   \n",
       "PctChange:MeanEndPrice@180Min                          0.600194   0.548924   \n",
       "PctChange:MeanEndPrice@240Min                          0.657055   0.476532   \n",
       "PctChange:MeanEndPrice@1D                              0.237291   0.269173   \n",
       "PctChange:MeanEndPrice@2D                              0.420302   0.473881   \n",
       "PctChange:MeanEndPrice@1W                              0.499140   0.571495   \n",
       "PctChange:MeanEndPrice@2W                              0.592554   0.629207   \n",
       "\n",
       "                               Direction2[t - 1]  F3[t - 1]  G1[t - 1]  \\\n",
       "PctChange:MeanEndPrice@1Min            -0.044253  -0.044180  -0.044115   \n",
       "PctChange:MeanEndPrice@2Min             0.268131   0.244607   0.220126   \n",
       "PctChange:MeanEndPrice@3Min             0.378296   0.442690   0.332429   \n",
       "PctChange:MeanEndPrice@4Min             0.433443   0.461978   0.378099   \n",
       "PctChange:MeanEndPrice@5Min             0.440793   0.454894   0.387331   \n",
       "PctChange:MeanEndPrice@6Min             0.472083   0.476358   0.421592   \n",
       "PctChange:MeanEndPrice@7Min             0.515200   0.522686   0.431648   \n",
       "PctChange:MeanEndPrice@8Min             0.520891   0.513944   0.467379   \n",
       "PctChange:MeanEndPrice@9Min             0.499425   0.500130   0.435211   \n",
       "PctChange:MeanEndPrice@10Min            0.487570   0.480498   0.428700   \n",
       "PctChange:MeanEndPrice@15Min            0.544789   0.506506   0.478727   \n",
       "PctChange:MeanEndPrice@30Min            0.527821   0.505324   0.466055   \n",
       "PctChange:MeanEndPrice@60Min            0.585198   0.565809   0.524512   \n",
       "PctChange:MeanEndPrice@120Min           0.607802   0.556109   0.482065   \n",
       "PctChange:MeanEndPrice@180Min           0.595142   0.547812   0.432629   \n",
       "PctChange:MeanEndPrice@240Min           0.642651   0.475621   0.440196   \n",
       "PctChange:MeanEndPrice@1D               0.401603   0.267961   0.130971   \n",
       "PctChange:MeanEndPrice@2D               0.484741   0.474807   0.269731   \n",
       "PctChange:MeanEndPrice@1W               0.584088   0.572422   0.130896   \n",
       "PctChange:MeanEndPrice@2W               0.642983   0.629962   0.360310   \n",
       "\n",
       "                               G3[t - 1]  PctChange:LastEndPrice[t - 1]  ...  \\\n",
       "PctChange:MeanEndPrice@1Min    -0.040016                      -0.042200  ...   \n",
       "PctChange:MeanEndPrice@2Min     0.188707                       0.136605  ...   \n",
       "PctChange:MeanEndPrice@3Min     0.284448                       0.282670  ...   \n",
       "PctChange:MeanEndPrice@4Min     0.324907                       0.290082  ...   \n",
       "PctChange:MeanEndPrice@5Min     0.344720                       0.273967  ...   \n",
       "PctChange:MeanEndPrice@6Min     0.366387                       0.289028  ...   \n",
       "PctChange:MeanEndPrice@7Min     0.380510                       0.317882  ...   \n",
       "PctChange:MeanEndPrice@8Min     0.406207                       0.307911  ...   \n",
       "PctChange:MeanEndPrice@9Min     0.370102                       0.301639  ...   \n",
       "PctChange:MeanEndPrice@10Min    0.375200                       0.286715  ...   \n",
       "PctChange:MeanEndPrice@15Min    0.423081                       0.292065  ...   \n",
       "PctChange:MeanEndPrice@30Min    0.399106                       0.291494  ...   \n",
       "PctChange:MeanEndPrice@60Min    0.437473                       0.307664  ...   \n",
       "PctChange:MeanEndPrice@120Min   0.371293                       0.283848  ...   \n",
       "PctChange:MeanEndPrice@180Min   0.326240                       0.295896  ...   \n",
       "PctChange:MeanEndPrice@240Min   0.291781                       0.197493  ...   \n",
       "PctChange:MeanEndPrice@1D       0.119244                       0.144356  ...   \n",
       "PctChange:MeanEndPrice@2D       0.192097                       0.377776  ...   \n",
       "PctChange:MeanEndPrice@1W       0.177781                       0.496672  ...   \n",
       "PctChange:MeanEndPrice@2W       0.380075                       0.581410  ...   \n",
       "\n",
       "                               G2[t - 1]  F7[t - 1]  G4[t - 1]  \\\n",
       "PctChange:MeanEndPrice@1Min    -0.025962  -0.027606  -0.024194   \n",
       "PctChange:MeanEndPrice@2Min     0.167881   0.204659   0.136711   \n",
       "PctChange:MeanEndPrice@3Min     0.243943   0.273013   0.202095   \n",
       "PctChange:MeanEndPrice@4Min     0.286096   0.308451   0.233498   \n",
       "PctChange:MeanEndPrice@5Min     0.300524   0.325587   0.250678   \n",
       "PctChange:MeanEndPrice@6Min     0.313906   0.340947   0.262960   \n",
       "PctChange:MeanEndPrice@7Min     0.323291   0.374064   0.270319   \n",
       "PctChange:MeanEndPrice@8Min     0.334354   0.344175   0.273827   \n",
       "PctChange:MeanEndPrice@9Min     0.329688   0.335445   0.268693   \n",
       "PctChange:MeanEndPrice@10Min    0.332562   0.330276   0.265342   \n",
       "PctChange:MeanEndPrice@15Min    0.349454   0.344299   0.276030   \n",
       "PctChange:MeanEndPrice@30Min    0.337967   0.337137   0.251287   \n",
       "PctChange:MeanEndPrice@60Min    0.368559   0.308067   0.262769   \n",
       "PctChange:MeanEndPrice@120Min   0.313248   0.344561   0.215147   \n",
       "PctChange:MeanEndPrice@180Min   0.285098   0.439516   0.190657   \n",
       "PctChange:MeanEndPrice@240Min   0.269049   0.438542   0.154332   \n",
       "PctChange:MeanEndPrice@1D       0.129605   0.222243   0.116906   \n",
       "PctChange:MeanEndPrice@2D       0.152933  -0.008486   0.141785   \n",
       "PctChange:MeanEndPrice@1W       0.058503   0.433556   0.149605   \n",
       "PctChange:MeanEndPrice@2W       0.399271   0.456741   0.406928   \n",
       "\n",
       "                               RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanEndPrice@1Min                                     -0.057669   \n",
       "PctChange:MeanEndPrice@2Min                                      0.116428   \n",
       "PctChange:MeanEndPrice@3Min                                      0.196265   \n",
       "PctChange:MeanEndPrice@4Min                                      0.216985   \n",
       "PctChange:MeanEndPrice@5Min                                      0.209390   \n",
       "PctChange:MeanEndPrice@6Min                                      0.227110   \n",
       "PctChange:MeanEndPrice@7Min                                      0.250972   \n",
       "PctChange:MeanEndPrice@8Min                                      0.249791   \n",
       "PctChange:MeanEndPrice@9Min                                      0.237636   \n",
       "PctChange:MeanEndPrice@10Min                                     0.219296   \n",
       "PctChange:MeanEndPrice@15Min                                     0.258986   \n",
       "PctChange:MeanEndPrice@30Min                                     0.240973   \n",
       "PctChange:MeanEndPrice@60Min                                     0.275303   \n",
       "PctChange:MeanEndPrice@120Min                                    0.269903   \n",
       "PctChange:MeanEndPrice@180Min                                    0.186879   \n",
       "PctChange:MeanEndPrice@240Min                                    0.252468   \n",
       "PctChange:MeanEndPrice@1D                                        0.050948   \n",
       "PctChange:MeanEndPrice@2D                                        0.330986   \n",
       "PctChange:MeanEndPrice@1W                                        0.210228   \n",
       "PctChange:MeanEndPrice@2W                                        0.040768   \n",
       "\n",
       "                               RevPctChange[t - 1]:MinPrice  \\\n",
       "PctChange:MeanEndPrice@1Min                       -0.038706   \n",
       "PctChange:MeanEndPrice@2Min                        0.065166   \n",
       "PctChange:MeanEndPrice@3Min                        0.134186   \n",
       "PctChange:MeanEndPrice@4Min                        0.164556   \n",
       "PctChange:MeanEndPrice@5Min                        0.150874   \n",
       "PctChange:MeanEndPrice@6Min                        0.166189   \n",
       "PctChange:MeanEndPrice@7Min                        0.193256   \n",
       "PctChange:MeanEndPrice@8Min                        0.210597   \n",
       "PctChange:MeanEndPrice@9Min                        0.203600   \n",
       "PctChange:MeanEndPrice@10Min                       0.192938   \n",
       "PctChange:MeanEndPrice@15Min                       0.232437   \n",
       "PctChange:MeanEndPrice@30Min                       0.197967   \n",
       "PctChange:MeanEndPrice@60Min                       0.267886   \n",
       "PctChange:MeanEndPrice@120Min                      0.253878   \n",
       "PctChange:MeanEndPrice@180Min                      0.190958   \n",
       "PctChange:MeanEndPrice@240Min                      0.183441   \n",
       "PctChange:MeanEndPrice@1D                          0.021564   \n",
       "PctChange:MeanEndPrice@2D                          0.372762   \n",
       "PctChange:MeanEndPrice@1W                          0.113029   \n",
       "PctChange:MeanEndPrice@2W                          0.133575   \n",
       "\n",
       "                               RevPctChange[t - 1]:MeanEndPrice  \\\n",
       "PctChange:MeanEndPrice@1Min                           -0.057669   \n",
       "PctChange:MeanEndPrice@2Min                            0.107801   \n",
       "PctChange:MeanEndPrice@3Min                            0.156402   \n",
       "PctChange:MeanEndPrice@4Min                            0.183966   \n",
       "PctChange:MeanEndPrice@5Min                            0.196675   \n",
       "PctChange:MeanEndPrice@6Min                            0.205975   \n",
       "PctChange:MeanEndPrice@7Min                            0.213155   \n",
       "PctChange:MeanEndPrice@8Min                            0.210900   \n",
       "PctChange:MeanEndPrice@9Min                            0.207381   \n",
       "PctChange:MeanEndPrice@10Min                           0.198068   \n",
       "PctChange:MeanEndPrice@15Min                           0.222886   \n",
       "PctChange:MeanEndPrice@30Min                           0.214357   \n",
       "PctChange:MeanEndPrice@60Min                           0.239133   \n",
       "PctChange:MeanEndPrice@120Min                          0.213852   \n",
       "PctChange:MeanEndPrice@180Min                          0.130026   \n",
       "PctChange:MeanEndPrice@240Min                          0.129210   \n",
       "PctChange:MeanEndPrice@1D                              0.074884   \n",
       "PctChange:MeanEndPrice@2D                              0.368392   \n",
       "PctChange:MeanEndPrice@1W                              0.164668   \n",
       "PctChange:MeanEndPrice@2W                              0.010775   \n",
       "\n",
       "                               RevPctChange[t - 1]:MeanMaxMinPrice  \\\n",
       "PctChange:MeanEndPrice@1Min                              -0.041413   \n",
       "PctChange:MeanEndPrice@2Min                               0.054954   \n",
       "PctChange:MeanEndPrice@3Min                               0.102394   \n",
       "PctChange:MeanEndPrice@4Min                               0.133441   \n",
       "PctChange:MeanEndPrice@5Min                               0.152193   \n",
       "PctChange:MeanEndPrice@6Min                               0.165595   \n",
       "PctChange:MeanEndPrice@7Min                               0.173343   \n",
       "PctChange:MeanEndPrice@8Min                               0.177758   \n",
       "PctChange:MeanEndPrice@9Min                               0.181254   \n",
       "PctChange:MeanEndPrice@10Min                              0.174794   \n",
       "PctChange:MeanEndPrice@15Min                              0.205639   \n",
       "PctChange:MeanEndPrice@30Min                              0.206402   \n",
       "PctChange:MeanEndPrice@60Min                              0.233871   \n",
       "PctChange:MeanEndPrice@120Min                             0.211203   \n",
       "PctChange:MeanEndPrice@180Min                             0.127243   \n",
       "PctChange:MeanEndPrice@240Min                             0.127184   \n",
       "PctChange:MeanEndPrice@1D                                 0.074113   \n",
       "PctChange:MeanEndPrice@2D                                 0.367833   \n",
       "PctChange:MeanEndPrice@1W                                 0.164904   \n",
       "PctChange:MeanEndPrice@2W                                 0.010645   \n",
       "\n",
       "                               RevPctChange[t - 1]:MeanAvg4Price  \\\n",
       "PctChange:MeanEndPrice@1Min                            -0.041558   \n",
       "PctChange:MeanEndPrice@2Min                             0.051866   \n",
       "PctChange:MeanEndPrice@3Min                             0.099780   \n",
       "PctChange:MeanEndPrice@4Min                             0.131541   \n",
       "PctChange:MeanEndPrice@5Min                             0.151793   \n",
       "PctChange:MeanEndPrice@6Min                             0.164571   \n",
       "PctChange:MeanEndPrice@7Min                             0.172789   \n",
       "PctChange:MeanEndPrice@8Min                             0.176773   \n",
       "PctChange:MeanEndPrice@9Min                             0.180439   \n",
       "PctChange:MeanEndPrice@10Min                            0.174223   \n",
       "PctChange:MeanEndPrice@15Min                            0.205112   \n",
       "PctChange:MeanEndPrice@30Min                            0.206200   \n",
       "PctChange:MeanEndPrice@60Min                            0.233797   \n",
       "PctChange:MeanEndPrice@120Min                           0.210943   \n",
       "PctChange:MeanEndPrice@180Min                           0.127071   \n",
       "PctChange:MeanEndPrice@240Min                           0.126991   \n",
       "PctChange:MeanEndPrice@1D                               0.074368   \n",
       "PctChange:MeanEndPrice@2D                               0.367627   \n",
       "PctChange:MeanEndPrice@1W                               0.164882   \n",
       "PctChange:MeanEndPrice@2W                               0.010675   \n",
       "\n",
       "                               RevPctChange[t - 1]:MeanStartEndPrice  \\\n",
       "PctChange:MeanEndPrice@1Min                                -0.040399   \n",
       "PctChange:MeanEndPrice@2Min                                 0.048316   \n",
       "PctChange:MeanEndPrice@3Min                                 0.096735   \n",
       "PctChange:MeanEndPrice@4Min                                 0.129296   \n",
       "PctChange:MeanEndPrice@5Min                                 0.151141   \n",
       "PctChange:MeanEndPrice@6Min                                 0.163352   \n",
       "PctChange:MeanEndPrice@7Min                                 0.172079   \n",
       "PctChange:MeanEndPrice@8Min                                 0.175667   \n",
       "PctChange:MeanEndPrice@9Min                                 0.179524   \n",
       "PctChange:MeanEndPrice@10Min                                0.173575   \n",
       "PctChange:MeanEndPrice@15Min                                0.204543   \n",
       "PctChange:MeanEndPrice@30Min                                0.205986   \n",
       "PctChange:MeanEndPrice@60Min                                0.233719   \n",
       "PctChange:MeanEndPrice@120Min                               0.210682   \n",
       "PctChange:MeanEndPrice@180Min                               0.126899   \n",
       "PctChange:MeanEndPrice@240Min                               0.126798   \n",
       "PctChange:MeanEndPrice@1D                                   0.074622   \n",
       "PctChange:MeanEndPrice@2D                                   0.367420   \n",
       "PctChange:MeanEndPrice@1W                                   0.164859   \n",
       "PctChange:MeanEndPrice@2W                                   0.010706   \n",
       "\n",
       "                               RevPctChange[t - 1]:MaxPrice  \n",
       "PctChange:MeanEndPrice@1Min                       -0.035641  \n",
       "PctChange:MeanEndPrice@2Min                        0.076487  \n",
       "PctChange:MeanEndPrice@3Min                        0.129668  \n",
       "PctChange:MeanEndPrice@4Min                        0.145654  \n",
       "PctChange:MeanEndPrice@5Min                        0.161078  \n",
       "PctChange:MeanEndPrice@6Min                        0.183919  \n",
       "PctChange:MeanEndPrice@7Min                        0.196120  \n",
       "PctChange:MeanEndPrice@8Min                        0.181611  \n",
       "PctChange:MeanEndPrice@9Min                        0.184723  \n",
       "PctChange:MeanEndPrice@10Min                       0.169566  \n",
       "PctChange:MeanEndPrice@15Min                       0.202689  \n",
       "PctChange:MeanEndPrice@30Min                       0.207997  \n",
       "PctChange:MeanEndPrice@60Min                       0.217697  \n",
       "PctChange:MeanEndPrice@120Min                      0.177215  \n",
       "PctChange:MeanEndPrice@180Min                      0.146938  \n",
       "PctChange:MeanEndPrice@240Min                      0.125898  \n",
       "PctChange:MeanEndPrice@1D                          0.016199  \n",
       "PctChange:MeanEndPrice@2D                          0.220406  \n",
       "PctChange:MeanEndPrice@1W                          0.184723  \n",
       "PctChange:MeanEndPrice@2W                          0.093234  \n",
       "\n",
       "[20 rows x 22 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_over_intervals('BMW')[x_features]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AdjustedPctChange[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>H1@WithNorm(MeanEndPrice)[t - 1]</th>\n",
       "      <th>RevPctChange[t - 1]:LastEndPrice</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>G1[t - 1]</th>\n",
       "      <th>G3[t - 1]</th>\n",
       "      <th>PctChange:LastEndPrice[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>G2[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>G4[t - 1]</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:MinPrice</th>\n",
       "      <th>RevPctChange[t - 1]:MeanEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:MeanMaxMinPrice</th>\n",
       "      <th>RevPctChange[t - 1]:MeanAvg4Price</th>\n",
       "      <th>RevPctChange[t - 1]:MeanStartEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:MaxPrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@1Min</th>\n",
       "      <td>NaN</td>\n",
       "      <td>-0.090224</td>\n",
       "      <td>-0.090224</td>\n",
       "      <td>-0.085911</td>\n",
       "      <td>-0.082159</td>\n",
       "      <td>-0.082471</td>\n",
       "      <td>-0.082134</td>\n",
       "      <td>-0.082144</td>\n",
       "      <td>-0.090224</td>\n",
       "      <td>-0.063377</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.078285</td>\n",
       "      <td>-0.051484</td>\n",
       "      <td>-0.088564</td>\n",
       "      <td>-0.085911</td>\n",
       "      <td>-0.049432</td>\n",
       "      <td>-0.085911</td>\n",
       "      <td>-0.052119</td>\n",
       "      <td>-0.059140</td>\n",
       "      <td>-0.064718</td>\n",
       "      <td>-0.044420</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@2Min</th>\n",
       "      <td>0.301682</td>\n",
       "      <td>0.230909</td>\n",
       "      <td>0.230919</td>\n",
       "      <td>0.193584</td>\n",
       "      <td>0.202880</td>\n",
       "      <td>0.233511</td>\n",
       "      <td>0.202869</td>\n",
       "      <td>0.144197</td>\n",
       "      <td>0.111798</td>\n",
       "      <td>0.112306</td>\n",
       "      <td>...</td>\n",
       "      <td>0.109725</td>\n",
       "      <td>0.173531</td>\n",
       "      <td>0.069823</td>\n",
       "      <td>0.093480</td>\n",
       "      <td>0.059237</td>\n",
       "      <td>0.083069</td>\n",
       "      <td>0.046689</td>\n",
       "      <td>0.043832</td>\n",
       "      <td>0.040555</td>\n",
       "      <td>0.063550</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@3Min</th>\n",
       "      <td>0.567136</td>\n",
       "      <td>0.489287</td>\n",
       "      <td>0.489356</td>\n",
       "      <td>0.342411</td>\n",
       "      <td>0.465839</td>\n",
       "      <td>0.355991</td>\n",
       "      <td>0.467812</td>\n",
       "      <td>0.239553</td>\n",
       "      <td>0.194866</td>\n",
       "      <td>0.280627</td>\n",
       "      <td>...</td>\n",
       "      <td>0.193716</td>\n",
       "      <td>0.243119</td>\n",
       "      <td>0.142012</td>\n",
       "      <td>0.179563</td>\n",
       "      <td>0.119608</td>\n",
       "      <td>0.144509</td>\n",
       "      <td>0.098368</td>\n",
       "      <td>0.096647</td>\n",
       "      <td>0.094445</td>\n",
       "      <td>0.121027</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@4Min</th>\n",
       "      <td>0.590153</td>\n",
       "      <td>0.530668</td>\n",
       "      <td>0.530669</td>\n",
       "      <td>0.409752</td>\n",
       "      <td>0.492846</td>\n",
       "      <td>0.421914</td>\n",
       "      <td>0.493727</td>\n",
       "      <td>0.292073</td>\n",
       "      <td>0.236732</td>\n",
       "      <td>0.281313</td>\n",
       "      <td>...</td>\n",
       "      <td>0.237012</td>\n",
       "      <td>0.272101</td>\n",
       "      <td>0.174166</td>\n",
       "      <td>0.214838</td>\n",
       "      <td>0.157742</td>\n",
       "      <td>0.172412</td>\n",
       "      <td>0.133051</td>\n",
       "      <td>0.131110</td>\n",
       "      <td>0.128759</td>\n",
       "      <td>0.148077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@5Min</th>\n",
       "      <td>0.574304</td>\n",
       "      <td>0.531847</td>\n",
       "      <td>0.531773</td>\n",
       "      <td>0.430875</td>\n",
       "      <td>0.493123</td>\n",
       "      <td>0.443913</td>\n",
       "      <td>0.493072</td>\n",
       "      <td>0.322261</td>\n",
       "      <td>0.268725</td>\n",
       "      <td>0.271183</td>\n",
       "      <td>...</td>\n",
       "      <td>0.267913</td>\n",
       "      <td>0.292105</td>\n",
       "      <td>0.202738</td>\n",
       "      <td>0.219473</td>\n",
       "      <td>0.158246</td>\n",
       "      <td>0.186510</td>\n",
       "      <td>0.147531</td>\n",
       "      <td>0.146439</td>\n",
       "      <td>0.145044</td>\n",
       "      <td>0.147374</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@6Min</th>\n",
       "      <td>0.598595</td>\n",
       "      <td>0.557032</td>\n",
       "      <td>0.556929</td>\n",
       "      <td>0.463866</td>\n",
       "      <td>0.502388</td>\n",
       "      <td>0.472940</td>\n",
       "      <td>0.501710</td>\n",
       "      <td>0.347485</td>\n",
       "      <td>0.291706</td>\n",
       "      <td>0.277257</td>\n",
       "      <td>...</td>\n",
       "      <td>0.287080</td>\n",
       "      <td>0.300238</td>\n",
       "      <td>0.223415</td>\n",
       "      <td>0.230445</td>\n",
       "      <td>0.182982</td>\n",
       "      <td>0.198046</td>\n",
       "      <td>0.164598</td>\n",
       "      <td>0.163655</td>\n",
       "      <td>0.162469</td>\n",
       "      <td>0.162552</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@7Min</th>\n",
       "      <td>0.580801</td>\n",
       "      <td>0.540116</td>\n",
       "      <td>0.540132</td>\n",
       "      <td>0.475107</td>\n",
       "      <td>0.483449</td>\n",
       "      <td>0.499448</td>\n",
       "      <td>0.482672</td>\n",
       "      <td>0.327988</td>\n",
       "      <td>0.272410</td>\n",
       "      <td>0.272062</td>\n",
       "      <td>...</td>\n",
       "      <td>0.284170</td>\n",
       "      <td>0.308659</td>\n",
       "      <td>0.206439</td>\n",
       "      <td>0.237626</td>\n",
       "      <td>0.201766</td>\n",
       "      <td>0.204463</td>\n",
       "      <td>0.175912</td>\n",
       "      <td>0.175371</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.176317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@8Min</th>\n",
       "      <td>0.620121</td>\n",
       "      <td>0.581521</td>\n",
       "      <td>0.581395</td>\n",
       "      <td>0.508523</td>\n",
       "      <td>0.514442</td>\n",
       "      <td>0.512527</td>\n",
       "      <td>0.512973</td>\n",
       "      <td>0.388415</td>\n",
       "      <td>0.323499</td>\n",
       "      <td>0.280601</td>\n",
       "      <td>...</td>\n",
       "      <td>0.318103</td>\n",
       "      <td>0.313624</td>\n",
       "      <td>0.242819</td>\n",
       "      <td>0.257305</td>\n",
       "      <td>0.215078</td>\n",
       "      <td>0.220813</td>\n",
       "      <td>0.194119</td>\n",
       "      <td>0.193123</td>\n",
       "      <td>0.191955</td>\n",
       "      <td>0.186707</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@9Min</th>\n",
       "      <td>0.623222</td>\n",
       "      <td>0.582490</td>\n",
       "      <td>0.582389</td>\n",
       "      <td>0.498934</td>\n",
       "      <td>0.526482</td>\n",
       "      <td>0.504428</td>\n",
       "      <td>0.525521</td>\n",
       "      <td>0.364460</td>\n",
       "      <td>0.304915</td>\n",
       "      <td>0.290548</td>\n",
       "      <td>...</td>\n",
       "      <td>0.312963</td>\n",
       "      <td>0.304046</td>\n",
       "      <td>0.244559</td>\n",
       "      <td>0.254540</td>\n",
       "      <td>0.220449</td>\n",
       "      <td>0.216911</td>\n",
       "      <td>0.193835</td>\n",
       "      <td>0.193237</td>\n",
       "      <td>0.192508</td>\n",
       "      <td>0.182340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@10Min</th>\n",
       "      <td>0.615522</td>\n",
       "      <td>0.566007</td>\n",
       "      <td>0.565975</td>\n",
       "      <td>0.499654</td>\n",
       "      <td>0.492738</td>\n",
       "      <td>0.504239</td>\n",
       "      <td>0.490857</td>\n",
       "      <td>0.381413</td>\n",
       "      <td>0.322332</td>\n",
       "      <td>0.264453</td>\n",
       "      <td>...</td>\n",
       "      <td>0.315689</td>\n",
       "      <td>0.305738</td>\n",
       "      <td>0.244638</td>\n",
       "      <td>0.241467</td>\n",
       "      <td>0.210238</td>\n",
       "      <td>0.213795</td>\n",
       "      <td>0.192759</td>\n",
       "      <td>0.192372</td>\n",
       "      <td>0.191879</td>\n",
       "      <td>0.177824</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@15Min</th>\n",
       "      <td>0.627423</td>\n",
       "      <td>0.571155</td>\n",
       "      <td>0.571099</td>\n",
       "      <td>0.523224</td>\n",
       "      <td>0.488795</td>\n",
       "      <td>0.523973</td>\n",
       "      <td>0.486416</td>\n",
       "      <td>0.405021</td>\n",
       "      <td>0.332411</td>\n",
       "      <td>0.250413</td>\n",
       "      <td>...</td>\n",
       "      <td>0.328458</td>\n",
       "      <td>0.326040</td>\n",
       "      <td>0.252966</td>\n",
       "      <td>0.245331</td>\n",
       "      <td>0.211400</td>\n",
       "      <td>0.220279</td>\n",
       "      <td>0.204190</td>\n",
       "      <td>0.203853</td>\n",
       "      <td>0.203463</td>\n",
       "      <td>0.193396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@30Min</th>\n",
       "      <td>0.632558</td>\n",
       "      <td>0.578251</td>\n",
       "      <td>0.578156</td>\n",
       "      <td>0.527310</td>\n",
       "      <td>0.484426</td>\n",
       "      <td>0.526626</td>\n",
       "      <td>0.481941</td>\n",
       "      <td>0.404225</td>\n",
       "      <td>0.322880</td>\n",
       "      <td>0.253447</td>\n",
       "      <td>...</td>\n",
       "      <td>0.322386</td>\n",
       "      <td>0.328076</td>\n",
       "      <td>0.244377</td>\n",
       "      <td>0.261489</td>\n",
       "      <td>0.205414</td>\n",
       "      <td>0.235247</td>\n",
       "      <td>0.226809</td>\n",
       "      <td>0.226606</td>\n",
       "      <td>0.226386</td>\n",
       "      <td>0.209871</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@60Min</th>\n",
       "      <td>0.686364</td>\n",
       "      <td>0.607140</td>\n",
       "      <td>0.606862</td>\n",
       "      <td>0.604648</td>\n",
       "      <td>0.512574</td>\n",
       "      <td>0.606072</td>\n",
       "      <td>0.509169</td>\n",
       "      <td>0.424507</td>\n",
       "      <td>0.324127</td>\n",
       "      <td>0.257177</td>\n",
       "      <td>...</td>\n",
       "      <td>0.350929</td>\n",
       "      <td>0.340335</td>\n",
       "      <td>0.259942</td>\n",
       "      <td>0.301226</td>\n",
       "      <td>0.275617</td>\n",
       "      <td>0.246020</td>\n",
       "      <td>0.241011</td>\n",
       "      <td>0.240642</td>\n",
       "      <td>0.240268</td>\n",
       "      <td>0.217155</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@120Min</th>\n",
       "      <td>0.717188</td>\n",
       "      <td>0.644845</td>\n",
       "      <td>0.644335</td>\n",
       "      <td>0.623113</td>\n",
       "      <td>0.544067</td>\n",
       "      <td>0.623234</td>\n",
       "      <td>0.541958</td>\n",
       "      <td>0.332287</td>\n",
       "      <td>0.239049</td>\n",
       "      <td>0.250666</td>\n",
       "      <td>...</td>\n",
       "      <td>0.302817</td>\n",
       "      <td>0.367969</td>\n",
       "      <td>0.224962</td>\n",
       "      <td>0.251374</td>\n",
       "      <td>0.214364</td>\n",
       "      <td>0.191435</td>\n",
       "      <td>0.187258</td>\n",
       "      <td>0.187211</td>\n",
       "      <td>0.187163</td>\n",
       "      <td>0.241888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@180Min</th>\n",
       "      <td>0.641708</td>\n",
       "      <td>0.565835</td>\n",
       "      <td>0.565247</td>\n",
       "      <td>0.577326</td>\n",
       "      <td>0.457533</td>\n",
       "      <td>0.578537</td>\n",
       "      <td>0.455081</td>\n",
       "      <td>0.213339</td>\n",
       "      <td>0.131524</td>\n",
       "      <td>0.232633</td>\n",
       "      <td>...</td>\n",
       "      <td>0.199041</td>\n",
       "      <td>0.385694</td>\n",
       "      <td>0.114321</td>\n",
       "      <td>0.229021</td>\n",
       "      <td>0.209580</td>\n",
       "      <td>0.200713</td>\n",
       "      <td>0.197377</td>\n",
       "      <td>0.197268</td>\n",
       "      <td>0.197159</td>\n",
       "      <td>0.231707</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@240Min</th>\n",
       "      <td>0.748678</td>\n",
       "      <td>0.652954</td>\n",
       "      <td>0.652171</td>\n",
       "      <td>0.718147</td>\n",
       "      <td>0.522449</td>\n",
       "      <td>0.719841</td>\n",
       "      <td>0.519891</td>\n",
       "      <td>0.107307</td>\n",
       "      <td>0.025242</td>\n",
       "      <td>0.190099</td>\n",
       "      <td>...</td>\n",
       "      <td>0.122331</td>\n",
       "      <td>0.357899</td>\n",
       "      <td>0.050281</td>\n",
       "      <td>0.387777</td>\n",
       "      <td>0.204521</td>\n",
       "      <td>0.343719</td>\n",
       "      <td>0.342207</td>\n",
       "      <td>0.342153</td>\n",
       "      <td>0.342099</td>\n",
       "      <td>0.435201</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@1D</th>\n",
       "      <td>0.389065</td>\n",
       "      <td>0.370336</td>\n",
       "      <td>0.369850</td>\n",
       "      <td>0.298933</td>\n",
       "      <td>0.309640</td>\n",
       "      <td>0.437879</td>\n",
       "      <td>0.308638</td>\n",
       "      <td>0.072500</td>\n",
       "      <td>0.002706</td>\n",
       "      <td>0.203752</td>\n",
       "      <td>...</td>\n",
       "      <td>0.173451</td>\n",
       "      <td>0.222744</td>\n",
       "      <td>0.008701</td>\n",
       "      <td>0.150071</td>\n",
       "      <td>0.142733</td>\n",
       "      <td>0.180653</td>\n",
       "      <td>0.181540</td>\n",
       "      <td>0.180986</td>\n",
       "      <td>0.180429</td>\n",
       "      <td>0.023143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@2D</th>\n",
       "      <td>0.591899</td>\n",
       "      <td>0.511921</td>\n",
       "      <td>0.509516</td>\n",
       "      <td>0.464655</td>\n",
       "      <td>0.463501</td>\n",
       "      <td>0.565339</td>\n",
       "      <td>0.463288</td>\n",
       "      <td>0.112312</td>\n",
       "      <td>0.060119</td>\n",
       "      <td>0.320824</td>\n",
       "      <td>...</td>\n",
       "      <td>0.162812</td>\n",
       "      <td>0.037132</td>\n",
       "      <td>0.051152</td>\n",
       "      <td>0.266031</td>\n",
       "      <td>0.432768</td>\n",
       "      <td>0.234967</td>\n",
       "      <td>0.234851</td>\n",
       "      <td>0.234225</td>\n",
       "      <td>0.233597</td>\n",
       "      <td>0.107803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@1W</th>\n",
       "      <td>0.756558</td>\n",
       "      <td>0.737449</td>\n",
       "      <td>0.737329</td>\n",
       "      <td>0.641032</td>\n",
       "      <td>0.679970</td>\n",
       "      <td>0.678081</td>\n",
       "      <td>0.678996</td>\n",
       "      <td>0.043031</td>\n",
       "      <td>0.059629</td>\n",
       "      <td>0.640412</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.089366</td>\n",
       "      <td>0.481892</td>\n",
       "      <td>-0.021781</td>\n",
       "      <td>0.327005</td>\n",
       "      <td>0.399118</td>\n",
       "      <td>0.275935</td>\n",
       "      <td>0.275584</td>\n",
       "      <td>0.275723</td>\n",
       "      <td>0.275861</td>\n",
       "      <td>0.176603</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice@2W</th>\n",
       "      <td>0.833048</td>\n",
       "      <td>0.831545</td>\n",
       "      <td>0.827141</td>\n",
       "      <td>0.701372</td>\n",
       "      <td>0.717937</td>\n",
       "      <td>0.720215</td>\n",
       "      <td>0.719283</td>\n",
       "      <td>0.057052</td>\n",
       "      <td>-0.009206</td>\n",
       "      <td>0.687632</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.178759</td>\n",
       "      <td>0.500015</td>\n",
       "      <td>-0.144331</td>\n",
       "      <td>0.215234</td>\n",
       "      <td>0.375985</td>\n",
       "      <td>0.170567</td>\n",
       "      <td>0.171013</td>\n",
       "      <td>0.170979</td>\n",
       "      <td>0.170945</td>\n",
       "      <td>0.162010</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>20 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                               AdjustedPctChange[t - 1]  F1[t - 1]  \\\n",
       "PctChange:MeanEndPrice@1Min                         NaN  -0.090224   \n",
       "PctChange:MeanEndPrice@2Min                    0.301682   0.230909   \n",
       "PctChange:MeanEndPrice@3Min                    0.567136   0.489287   \n",
       "PctChange:MeanEndPrice@4Min                    0.590153   0.530668   \n",
       "PctChange:MeanEndPrice@5Min                    0.574304   0.531847   \n",
       "PctChange:MeanEndPrice@6Min                    0.598595   0.557032   \n",
       "PctChange:MeanEndPrice@7Min                    0.580801   0.540116   \n",
       "PctChange:MeanEndPrice@8Min                    0.620121   0.581521   \n",
       "PctChange:MeanEndPrice@9Min                    0.623222   0.582490   \n",
       "PctChange:MeanEndPrice@10Min                   0.615522   0.566007   \n",
       "PctChange:MeanEndPrice@15Min                   0.627423   0.571155   \n",
       "PctChange:MeanEndPrice@30Min                   0.632558   0.578251   \n",
       "PctChange:MeanEndPrice@60Min                   0.686364   0.607140   \n",
       "PctChange:MeanEndPrice@120Min                  0.717188   0.644845   \n",
       "PctChange:MeanEndPrice@180Min                  0.641708   0.565835   \n",
       "PctChange:MeanEndPrice@240Min                  0.748678   0.652954   \n",
       "PctChange:MeanEndPrice@1D                      0.389065   0.370336   \n",
       "PctChange:MeanEndPrice@2D                      0.591899   0.511921   \n",
       "PctChange:MeanEndPrice@1W                      0.756558   0.737449   \n",
       "PctChange:MeanEndPrice@2W                      0.833048   0.831545   \n",
       "\n",
       "                               H1@WithNorm(MeanEndPrice)[t - 1]  \\\n",
       "PctChange:MeanEndPrice@1Min                           -0.090224   \n",
       "PctChange:MeanEndPrice@2Min                            0.230919   \n",
       "PctChange:MeanEndPrice@3Min                            0.489356   \n",
       "PctChange:MeanEndPrice@4Min                            0.530669   \n",
       "PctChange:MeanEndPrice@5Min                            0.531773   \n",
       "PctChange:MeanEndPrice@6Min                            0.556929   \n",
       "PctChange:MeanEndPrice@7Min                            0.540132   \n",
       "PctChange:MeanEndPrice@8Min                            0.581395   \n",
       "PctChange:MeanEndPrice@9Min                            0.582389   \n",
       "PctChange:MeanEndPrice@10Min                           0.565975   \n",
       "PctChange:MeanEndPrice@15Min                           0.571099   \n",
       "PctChange:MeanEndPrice@30Min                           0.578156   \n",
       "PctChange:MeanEndPrice@60Min                           0.606862   \n",
       "PctChange:MeanEndPrice@120Min                          0.644335   \n",
       "PctChange:MeanEndPrice@180Min                          0.565247   \n",
       "PctChange:MeanEndPrice@240Min                          0.652171   \n",
       "PctChange:MeanEndPrice@1D                              0.369850   \n",
       "PctChange:MeanEndPrice@2D                              0.509516   \n",
       "PctChange:MeanEndPrice@1W                              0.737329   \n",
       "PctChange:MeanEndPrice@2W                              0.827141   \n",
       "\n",
       "                               RevPctChange[t - 1]:LastEndPrice  F2[t - 1]  \\\n",
       "PctChange:MeanEndPrice@1Min                           -0.085911  -0.082159   \n",
       "PctChange:MeanEndPrice@2Min                            0.193584   0.202880   \n",
       "PctChange:MeanEndPrice@3Min                            0.342411   0.465839   \n",
       "PctChange:MeanEndPrice@4Min                            0.409752   0.492846   \n",
       "PctChange:MeanEndPrice@5Min                            0.430875   0.493123   \n",
       "PctChange:MeanEndPrice@6Min                            0.463866   0.502388   \n",
       "PctChange:MeanEndPrice@7Min                            0.475107   0.483449   \n",
       "PctChange:MeanEndPrice@8Min                            0.508523   0.514442   \n",
       "PctChange:MeanEndPrice@9Min                            0.498934   0.526482   \n",
       "PctChange:MeanEndPrice@10Min                           0.499654   0.492738   \n",
       "PctChange:MeanEndPrice@15Min                           0.523224   0.488795   \n",
       "PctChange:MeanEndPrice@30Min                           0.527310   0.484426   \n",
       "PctChange:MeanEndPrice@60Min                           0.604648   0.512574   \n",
       "PctChange:MeanEndPrice@120Min                          0.623113   0.544067   \n",
       "PctChange:MeanEndPrice@180Min                          0.577326   0.457533   \n",
       "PctChange:MeanEndPrice@240Min                          0.718147   0.522449   \n",
       "PctChange:MeanEndPrice@1D                              0.298933   0.309640   \n",
       "PctChange:MeanEndPrice@2D                              0.464655   0.463501   \n",
       "PctChange:MeanEndPrice@1W                              0.641032   0.679970   \n",
       "PctChange:MeanEndPrice@2W                              0.701372   0.717937   \n",
       "\n",
       "                               Direction2[t - 1]  F3[t - 1]  G1[t - 1]  \\\n",
       "PctChange:MeanEndPrice@1Min            -0.082471  -0.082134  -0.082144   \n",
       "PctChange:MeanEndPrice@2Min             0.233511   0.202869   0.144197   \n",
       "PctChange:MeanEndPrice@3Min             0.355991   0.467812   0.239553   \n",
       "PctChange:MeanEndPrice@4Min             0.421914   0.493727   0.292073   \n",
       "PctChange:MeanEndPrice@5Min             0.443913   0.493072   0.322261   \n",
       "PctChange:MeanEndPrice@6Min             0.472940   0.501710   0.347485   \n",
       "PctChange:MeanEndPrice@7Min             0.499448   0.482672   0.327988   \n",
       "PctChange:MeanEndPrice@8Min             0.512527   0.512973   0.388415   \n",
       "PctChange:MeanEndPrice@9Min             0.504428   0.525521   0.364460   \n",
       "PctChange:MeanEndPrice@10Min            0.504239   0.490857   0.381413   \n",
       "PctChange:MeanEndPrice@15Min            0.523973   0.486416   0.405021   \n",
       "PctChange:MeanEndPrice@30Min            0.526626   0.481941   0.404225   \n",
       "PctChange:MeanEndPrice@60Min            0.606072   0.509169   0.424507   \n",
       "PctChange:MeanEndPrice@120Min           0.623234   0.541958   0.332287   \n",
       "PctChange:MeanEndPrice@180Min           0.578537   0.455081   0.213339   \n",
       "PctChange:MeanEndPrice@240Min           0.719841   0.519891   0.107307   \n",
       "PctChange:MeanEndPrice@1D               0.437879   0.308638   0.072500   \n",
       "PctChange:MeanEndPrice@2D               0.565339   0.463288   0.112312   \n",
       "PctChange:MeanEndPrice@1W               0.678081   0.678996   0.043031   \n",
       "PctChange:MeanEndPrice@2W               0.720215   0.719283   0.057052   \n",
       "\n",
       "                               G3[t - 1]  PctChange:LastEndPrice[t - 1]  ...  \\\n",
       "PctChange:MeanEndPrice@1Min    -0.090224                      -0.063377  ...   \n",
       "PctChange:MeanEndPrice@2Min     0.111798                       0.112306  ...   \n",
       "PctChange:MeanEndPrice@3Min     0.194866                       0.280627  ...   \n",
       "PctChange:MeanEndPrice@4Min     0.236732                       0.281313  ...   \n",
       "PctChange:MeanEndPrice@5Min     0.268725                       0.271183  ...   \n",
       "PctChange:MeanEndPrice@6Min     0.291706                       0.277257  ...   \n",
       "PctChange:MeanEndPrice@7Min     0.272410                       0.272062  ...   \n",
       "PctChange:MeanEndPrice@8Min     0.323499                       0.280601  ...   \n",
       "PctChange:MeanEndPrice@9Min     0.304915                       0.290548  ...   \n",
       "PctChange:MeanEndPrice@10Min    0.322332                       0.264453  ...   \n",
       "PctChange:MeanEndPrice@15Min    0.332411                       0.250413  ...   \n",
       "PctChange:MeanEndPrice@30Min    0.322880                       0.253447  ...   \n",
       "PctChange:MeanEndPrice@60Min    0.324127                       0.257177  ...   \n",
       "PctChange:MeanEndPrice@120Min   0.239049                       0.250666  ...   \n",
       "PctChange:MeanEndPrice@180Min   0.131524                       0.232633  ...   \n",
       "PctChange:MeanEndPrice@240Min   0.025242                       0.190099  ...   \n",
       "PctChange:MeanEndPrice@1D       0.002706                       0.203752  ...   \n",
       "PctChange:MeanEndPrice@2D       0.060119                       0.320824  ...   \n",
       "PctChange:MeanEndPrice@1W       0.059629                       0.640412  ...   \n",
       "PctChange:MeanEndPrice@2W      -0.009206                       0.687632  ...   \n",
       "\n",
       "                               G2[t - 1]  F7[t - 1]  G4[t - 1]  \\\n",
       "PctChange:MeanEndPrice@1Min    -0.078285  -0.051484  -0.088564   \n",
       "PctChange:MeanEndPrice@2Min     0.109725   0.173531   0.069823   \n",
       "PctChange:MeanEndPrice@3Min     0.193716   0.243119   0.142012   \n",
       "PctChange:MeanEndPrice@4Min     0.237012   0.272101   0.174166   \n",
       "PctChange:MeanEndPrice@5Min     0.267913   0.292105   0.202738   \n",
       "PctChange:MeanEndPrice@6Min     0.287080   0.300238   0.223415   \n",
       "PctChange:MeanEndPrice@7Min     0.284170   0.308659   0.206439   \n",
       "PctChange:MeanEndPrice@8Min     0.318103   0.313624   0.242819   \n",
       "PctChange:MeanEndPrice@9Min     0.312963   0.304046   0.244559   \n",
       "PctChange:MeanEndPrice@10Min    0.315689   0.305738   0.244638   \n",
       "PctChange:MeanEndPrice@15Min    0.328458   0.326040   0.252966   \n",
       "PctChange:MeanEndPrice@30Min    0.322386   0.328076   0.244377   \n",
       "PctChange:MeanEndPrice@60Min    0.350929   0.340335   0.259942   \n",
       "PctChange:MeanEndPrice@120Min   0.302817   0.367969   0.224962   \n",
       "PctChange:MeanEndPrice@180Min   0.199041   0.385694   0.114321   \n",
       "PctChange:MeanEndPrice@240Min   0.122331   0.357899   0.050281   \n",
       "PctChange:MeanEndPrice@1D       0.173451   0.222744   0.008701   \n",
       "PctChange:MeanEndPrice@2D       0.162812   0.037132   0.051152   \n",
       "PctChange:MeanEndPrice@1W      -0.089366   0.481892  -0.021781   \n",
       "PctChange:MeanEndPrice@2W      -0.178759   0.500015  -0.144331   \n",
       "\n",
       "                               RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "PctChange:MeanEndPrice@1Min                                     -0.085911   \n",
       "PctChange:MeanEndPrice@2Min                                      0.093480   \n",
       "PctChange:MeanEndPrice@3Min                                      0.179563   \n",
       "PctChange:MeanEndPrice@4Min                                      0.214838   \n",
       "PctChange:MeanEndPrice@5Min                                      0.219473   \n",
       "PctChange:MeanEndPrice@6Min                                      0.230445   \n",
       "PctChange:MeanEndPrice@7Min                                      0.237626   \n",
       "PctChange:MeanEndPrice@8Min                                      0.257305   \n",
       "PctChange:MeanEndPrice@9Min                                      0.254540   \n",
       "PctChange:MeanEndPrice@10Min                                     0.241467   \n",
       "PctChange:MeanEndPrice@15Min                                     0.245331   \n",
       "PctChange:MeanEndPrice@30Min                                     0.261489   \n",
       "PctChange:MeanEndPrice@60Min                                     0.301226   \n",
       "PctChange:MeanEndPrice@120Min                                    0.251374   \n",
       "PctChange:MeanEndPrice@180Min                                    0.229021   \n",
       "PctChange:MeanEndPrice@240Min                                    0.387777   \n",
       "PctChange:MeanEndPrice@1D                                        0.150071   \n",
       "PctChange:MeanEndPrice@2D                                        0.266031   \n",
       "PctChange:MeanEndPrice@1W                                        0.327005   \n",
       "PctChange:MeanEndPrice@2W                                        0.215234   \n",
       "\n",
       "                               RevPctChange[t - 1]:MinPrice  \\\n",
       "PctChange:MeanEndPrice@1Min                       -0.049432   \n",
       "PctChange:MeanEndPrice@2Min                        0.059237   \n",
       "PctChange:MeanEndPrice@3Min                        0.119608   \n",
       "PctChange:MeanEndPrice@4Min                        0.157742   \n",
       "PctChange:MeanEndPrice@5Min                        0.158246   \n",
       "PctChange:MeanEndPrice@6Min                        0.182982   \n",
       "PctChange:MeanEndPrice@7Min                        0.201766   \n",
       "PctChange:MeanEndPrice@8Min                        0.215078   \n",
       "PctChange:MeanEndPrice@9Min                        0.220449   \n",
       "PctChange:MeanEndPrice@10Min                       0.210238   \n",
       "PctChange:MeanEndPrice@15Min                       0.211400   \n",
       "PctChange:MeanEndPrice@30Min                       0.205414   \n",
       "PctChange:MeanEndPrice@60Min                       0.275617   \n",
       "PctChange:MeanEndPrice@120Min                      0.214364   \n",
       "PctChange:MeanEndPrice@180Min                      0.209580   \n",
       "PctChange:MeanEndPrice@240Min                      0.204521   \n",
       "PctChange:MeanEndPrice@1D                          0.142733   \n",
       "PctChange:MeanEndPrice@2D                          0.432768   \n",
       "PctChange:MeanEndPrice@1W                          0.399118   \n",
       "PctChange:MeanEndPrice@2W                          0.375985   \n",
       "\n",
       "                               RevPctChange[t - 1]:MeanEndPrice  \\\n",
       "PctChange:MeanEndPrice@1Min                           -0.085911   \n",
       "PctChange:MeanEndPrice@2Min                            0.083069   \n",
       "PctChange:MeanEndPrice@3Min                            0.144509   \n",
       "PctChange:MeanEndPrice@4Min                            0.172412   \n",
       "PctChange:MeanEndPrice@5Min                            0.186510   \n",
       "PctChange:MeanEndPrice@6Min                            0.198046   \n",
       "PctChange:MeanEndPrice@7Min                            0.204463   \n",
       "PctChange:MeanEndPrice@8Min                            0.220813   \n",
       "PctChange:MeanEndPrice@9Min                            0.216911   \n",
       "PctChange:MeanEndPrice@10Min                           0.213795   \n",
       "PctChange:MeanEndPrice@15Min                           0.220279   \n",
       "PctChange:MeanEndPrice@30Min                           0.235247   \n",
       "PctChange:MeanEndPrice@60Min                           0.246020   \n",
       "PctChange:MeanEndPrice@120Min                          0.191435   \n",
       "PctChange:MeanEndPrice@180Min                          0.200713   \n",
       "PctChange:MeanEndPrice@240Min                          0.343719   \n",
       "PctChange:MeanEndPrice@1D                              0.180653   \n",
       "PctChange:MeanEndPrice@2D                              0.234967   \n",
       "PctChange:MeanEndPrice@1W                              0.275935   \n",
       "PctChange:MeanEndPrice@2W                              0.170567   \n",
       "\n",
       "                               RevPctChange[t - 1]:MeanMaxMinPrice  \\\n",
       "PctChange:MeanEndPrice@1Min                              -0.052119   \n",
       "PctChange:MeanEndPrice@2Min                               0.046689   \n",
       "PctChange:MeanEndPrice@3Min                               0.098368   \n",
       "PctChange:MeanEndPrice@4Min                               0.133051   \n",
       "PctChange:MeanEndPrice@5Min                               0.147531   \n",
       "PctChange:MeanEndPrice@6Min                               0.164598   \n",
       "PctChange:MeanEndPrice@7Min                               0.175912   \n",
       "PctChange:MeanEndPrice@8Min                               0.194119   \n",
       "PctChange:MeanEndPrice@9Min                               0.193835   \n",
       "PctChange:MeanEndPrice@10Min                              0.192759   \n",
       "PctChange:MeanEndPrice@15Min                              0.204190   \n",
       "PctChange:MeanEndPrice@30Min                              0.226809   \n",
       "PctChange:MeanEndPrice@60Min                              0.241011   \n",
       "PctChange:MeanEndPrice@120Min                             0.187258   \n",
       "PctChange:MeanEndPrice@180Min                             0.197377   \n",
       "PctChange:MeanEndPrice@240Min                             0.342207   \n",
       "PctChange:MeanEndPrice@1D                                 0.181540   \n",
       "PctChange:MeanEndPrice@2D                                 0.234851   \n",
       "PctChange:MeanEndPrice@1W                                 0.275584   \n",
       "PctChange:MeanEndPrice@2W                                 0.171013   \n",
       "\n",
       "                               RevPctChange[t - 1]:MeanAvg4Price  \\\n",
       "PctChange:MeanEndPrice@1Min                            -0.059140   \n",
       "PctChange:MeanEndPrice@2Min                             0.043832   \n",
       "PctChange:MeanEndPrice@3Min                             0.096647   \n",
       "PctChange:MeanEndPrice@4Min                             0.131110   \n",
       "PctChange:MeanEndPrice@5Min                             0.146439   \n",
       "PctChange:MeanEndPrice@6Min                             0.163655   \n",
       "PctChange:MeanEndPrice@7Min                             0.175371   \n",
       "PctChange:MeanEndPrice@8Min                             0.193123   \n",
       "PctChange:MeanEndPrice@9Min                             0.193237   \n",
       "PctChange:MeanEndPrice@10Min                            0.192372   \n",
       "PctChange:MeanEndPrice@15Min                            0.203853   \n",
       "PctChange:MeanEndPrice@30Min                            0.226606   \n",
       "PctChange:MeanEndPrice@60Min                            0.240642   \n",
       "PctChange:MeanEndPrice@120Min                           0.187211   \n",
       "PctChange:MeanEndPrice@180Min                           0.197268   \n",
       "PctChange:MeanEndPrice@240Min                           0.342153   \n",
       "PctChange:MeanEndPrice@1D                               0.180986   \n",
       "PctChange:MeanEndPrice@2D                               0.234225   \n",
       "PctChange:MeanEndPrice@1W                               0.275723   \n",
       "PctChange:MeanEndPrice@2W                               0.170979   \n",
       "\n",
       "                               RevPctChange[t - 1]:MeanStartEndPrice  \\\n",
       "PctChange:MeanEndPrice@1Min                                -0.064718   \n",
       "PctChange:MeanEndPrice@2Min                                 0.040555   \n",
       "PctChange:MeanEndPrice@3Min                                 0.094445   \n",
       "PctChange:MeanEndPrice@4Min                                 0.128759   \n",
       "PctChange:MeanEndPrice@5Min                                 0.145044   \n",
       "PctChange:MeanEndPrice@6Min                                 0.162469   \n",
       "PctChange:MeanEndPrice@7Min                                 0.174649   \n",
       "PctChange:MeanEndPrice@8Min                                 0.191955   \n",
       "PctChange:MeanEndPrice@9Min                                 0.192508   \n",
       "PctChange:MeanEndPrice@10Min                                0.191879   \n",
       "PctChange:MeanEndPrice@15Min                                0.203463   \n",
       "PctChange:MeanEndPrice@30Min                                0.226386   \n",
       "PctChange:MeanEndPrice@60Min                                0.240268   \n",
       "PctChange:MeanEndPrice@120Min                               0.187163   \n",
       "PctChange:MeanEndPrice@180Min                               0.197159   \n",
       "PctChange:MeanEndPrice@240Min                               0.342099   \n",
       "PctChange:MeanEndPrice@1D                                   0.180429   \n",
       "PctChange:MeanEndPrice@2D                                   0.233597   \n",
       "PctChange:MeanEndPrice@1W                                   0.275861   \n",
       "PctChange:MeanEndPrice@2W                                   0.170945   \n",
       "\n",
       "                               RevPctChange[t - 1]:MaxPrice  \n",
       "PctChange:MeanEndPrice@1Min                       -0.044420  \n",
       "PctChange:MeanEndPrice@2Min                        0.063550  \n",
       "PctChange:MeanEndPrice@3Min                        0.121027  \n",
       "PctChange:MeanEndPrice@4Min                        0.148077  \n",
       "PctChange:MeanEndPrice@5Min                        0.147374  \n",
       "PctChange:MeanEndPrice@6Min                        0.162552  \n",
       "PctChange:MeanEndPrice@7Min                        0.176317  \n",
       "PctChange:MeanEndPrice@8Min                        0.186707  \n",
       "PctChange:MeanEndPrice@9Min                        0.182340  \n",
       "PctChange:MeanEndPrice@10Min                       0.177824  \n",
       "PctChange:MeanEndPrice@15Min                       0.193396  \n",
       "PctChange:MeanEndPrice@30Min                       0.209871  \n",
       "PctChange:MeanEndPrice@60Min                       0.217155  \n",
       "PctChange:MeanEndPrice@120Min                      0.241888  \n",
       "PctChange:MeanEndPrice@180Min                      0.231707  \n",
       "PctChange:MeanEndPrice@240Min                      0.435201  \n",
       "PctChange:MeanEndPrice@1D                          0.023143  \n",
       "PctChange:MeanEndPrice@2D                          0.107803  \n",
       "PctChange:MeanEndPrice@1W                          0.176603  \n",
       "PctChange:MeanEndPrice@2W                          0.162010  \n",
       "\n",
       "[20 rows x 22 columns]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_over_intervals('SIE')[x_features]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# corr_over_intervals('SAP')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The reasons for the strong correlations is due to averaging.\n",
    "We make an experiment to check the correlation with `MaxPrice` which uses only `MaxPrice`\n",
    "and `Direction` which uses only `EndPrice` and `StartPrice`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PctChange:MaxPrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:MaxPrice</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t + 1]</th>\n",
       "      <td>-0.040965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 0]</th>\n",
       "      <td>0.457563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <td>0.522053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 2]</th>\n",
       "      <td>0.103093</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 3]</th>\n",
       "      <td>0.060559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 4]</th>\n",
       "      <td>-0.088948</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    PctChange:MaxPrice\n",
       "PctChange:MaxPrice            1.000000\n",
       "Direction2[t + 1]            -0.040965\n",
       "Direction2[t - 0]             0.457563\n",
       "Direction2[t - 1]             0.522053\n",
       "Direction2[t - 2]             0.103093\n",
       "Direction2[t - 3]             0.060559\n",
       "Direction2[t - 4]            -0.088948"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_stock = prepare_single_stock('BMW', '1D')\n",
    "pd.DataFrame({\n",
    "    'PctChange:MaxPrice': single_stock['PctChange:MaxPrice'],\n",
    "    'Direction2[t + 1]': single_stock['Direction2'].shift(-1),   \n",
    "    'Direction2[t - 0]': single_stock['Direction2'],        \n",
    "    'Direction2[t - 1]': single_stock['Direction2[t - 1]'],\n",
    "    'Direction2[t - 2]': single_stock['Direction2[t - 1]'].shift(1),\n",
    "    'Direction2[t - 3]': single_stock['Direction2[t - 1]'].shift(2),\n",
    "    'Direction2[t - 4]': single_stock['Direction2[t - 1]'].shift(3),\n",
    "    'Direction2[t - 4]': single_stock['Direction2[t - 1]'].shift(4)    \n",
    "}).corr()[['PctChange:MaxPrice']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PctChange:LastEndPrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:LastEndPrice</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t + 1]</th>\n",
       "      <td>0.037161</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 0]</th>\n",
       "      <td>0.764989</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <td>0.165309</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 2]</th>\n",
       "      <td>0.171774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 3]</th>\n",
       "      <td>0.015430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction2[t - 4]</th>\n",
       "      <td>-0.084145</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        PctChange:LastEndPrice\n",
       "PctChange:LastEndPrice                1.000000\n",
       "Direction2[t + 1]                     0.037161\n",
       "Direction2[t - 0]                     0.764989\n",
       "Direction2[t - 1]                     0.165309\n",
       "Direction2[t - 2]                     0.171774\n",
       "Direction2[t - 3]                     0.015430\n",
       "Direction2[t - 4]                    -0.084145"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_stock = prepare_single_stock('BMW', '1D')\n",
    "pd.DataFrame({\n",
    "    'PctChange:LastEndPrice': single_stock['PctChange:LastEndPrice'],\n",
    "    'Direction2[t + 1]': single_stock['Direction2'].shift(-1),   \n",
    "    'Direction2[t - 0]': single_stock['Direction2'],        \n",
    "    'Direction2[t - 1]': single_stock['Direction2[t - 1]'],\n",
    "    'Direction2[t - 2]': single_stock['Direction2[t - 1]'].shift(1),\n",
    "    'Direction2[t - 3]': single_stock['Direction2[t - 1]'].shift(2),\n",
    "    'Direction2[t - 4]': single_stock['Direction2[t - 1]'].shift(3),\n",
    "    'Direction2[t - 4]': single_stock['Direction2[t - 1]'].shift(4)    \n",
    "}).corr()[['PctChange:LastEndPrice']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The reason that there is no correlation between period `(t)` and `(t - 2)` is related to the normalization we do in percent change. We can verify this independently next."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PctChange:LastEndPrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:LastEndPrice</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:LastEndPrice[t - 1]</th>\n",
       "      <td>0.069811</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:LastEndPrice[t - 2]</th>\n",
       "      <td>0.061999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:LastEndPrice[t - 3]</th>\n",
       "      <td>0.037790</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PctChange:LastEndPrice[t - 4]</th>\n",
       "      <td>-0.093435</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                               PctChange:LastEndPrice\n",
       "PctChange:LastEndPrice                       1.000000\n",
       "PctChange:LastEndPrice[t - 1]                0.069811\n",
       "PctChange:LastEndPrice[t - 2]                0.061999\n",
       "PctChange:LastEndPrice[t - 3]                0.037790\n",
       "PctChange:LastEndPrice[t - 4]               -0.093435"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_stock = prepare_single_stock('BMW', '1D')\n",
    "last_end_price = single_stock['PctChange:LastEndPrice']\n",
    "pd.DataFrame({\n",
    "    'PctChange:LastEndPrice': last_end_price,       \n",
    "    'PctChange:LastEndPrice[t - 1]': last_end_price.shift(1),\n",
    "    'PctChange:LastEndPrice[t - 2]': last_end_price.shift(2),\n",
    "    'PctChange:LastEndPrice[t - 3]': last_end_price.shift(3),\n",
    "    'PctChange:LastEndPrice[t - 4]': last_end_price.shift(4)    \n",
    "}).corr()[['PctChange:LastEndPrice']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we start normalizing the features in the same way, we find that past periods also exibit\n",
    "correlations to the present"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PctChange:LastEndPrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:LastEndPrice</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:LastEndPrice</th>\n",
       "      <td>0.082298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:LastEndPrice</th>\n",
       "      <td>0.138807</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 3]:LastEndPrice</th>\n",
       "      <td>0.116341</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  PctChange:LastEndPrice\n",
       "PctChange:LastEndPrice                          1.000000\n",
       "RevPctChange[t - 1]:LastEndPrice                0.082298\n",
       "RevPctChange[t - 2]:LastEndPrice                0.138807\n",
       "RevPctChange[t - 3]:LastEndPrice                0.116341"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_stock = prepare_single_stock('BMW', '1D')\n",
    "last_end_price = single_stock['PctChange:LastEndPrice']\n",
    "pd.DataFrame({\n",
    "    'PctChange:LastEndPrice': last_end_price,       \n",
    "    rev_pct_change_of_at_t('LastEndPrice', 1): single_stock[rev_pct_change_of_at_t('LastEndPrice', 1)],\n",
    "    rev_pct_change_of_at_t('LastEndPrice', 2): single_stock[rev_pct_change_of_at_t('LastEndPrice', 2)],\n",
    "    rev_pct_change_of_at_t('LastEndPrice', 3): single_stock[rev_pct_change_of_at_t('LastEndPrice', 3)]\n",
    "}).corr()[['PctChange:LastEndPrice']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PctChange:MeanEndPrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 1]:MeanEndPrice</th>\n",
       "      <td>0.074884</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 2]:MeanEndPrice</th>\n",
       "      <td>0.098395</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RevPctChange[t - 3]:MeanEndPrice</th>\n",
       "      <td>0.170181</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  PctChange:MeanEndPrice\n",
       "PctChange:MeanEndPrice                          1.000000\n",
       "RevPctChange[t - 1]:MeanEndPrice                0.074884\n",
       "RevPctChange[t - 2]:MeanEndPrice                0.098395\n",
       "RevPctChange[t - 3]:MeanEndPrice                0.170181"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_stock = prepare_single_stock('BMW', '1D')\n",
    "mean_end_price = single_stock['PctChange:MeanEndPrice']\n",
    "pd.DataFrame({\n",
    "    'PctChange:MeanEndPrice': mean_end_price,       \n",
    "    rev_pct_change_of_at_t('MeanEndPrice', 1): single_stock[rev_pct_change_of_at_t('MeanEndPrice', 1)],\n",
    "    rev_pct_change_of_at_t('MeanEndPrice', 2): single_stock[rev_pct_change_of_at_t('MeanEndPrice', 2)],\n",
    "    rev_pct_change_of_at_t('MeanEndPrice', 3): single_stock[rev_pct_change_of_at_t('MeanEndPrice', 3)]\n",
    "}).corr()[['PctChange:MeanEndPrice']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we experiment with the same feature, but normalized differently we:\n",
    "- are not going to see correlations beyond `(t - 1)` when different normalization is used\n",
    "- we may see correlations with the past when the same normalization is used"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PctChange:MeanEndPrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PctChange:MeanEndPrice</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@WithNorm(MeanEndPrice)[t - 1]</th>\n",
       "      <td>0.396102</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@WithNorm(MeanEndPrice)[t - 2]</th>\n",
       "      <td>0.024512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@WithNorm(MeanEndPrice)[t - 3]</th>\n",
       "      <td>-0.083978</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@WithNorm(MeanEndPrice)[t - 4]</th>\n",
       "      <td>0.258168</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <td>0.396335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F1[t - 2]</th>\n",
       "      <td>0.041163</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F1[t - 3]</th>\n",
       "      <td>-0.014884</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>F1[t - 4]</th>\n",
       "      <td>0.059665</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  PctChange:MeanEndPrice\n",
       "PctChange:MeanEndPrice                          1.000000\n",
       "H1@WithNorm(MeanEndPrice)[t - 1]                0.396102\n",
       "H1@WithNorm(MeanEndPrice)[t - 2]                0.024512\n",
       "H1@WithNorm(MeanEndPrice)[t - 3]               -0.083978\n",
       "H1@WithNorm(MeanEndPrice)[t - 4]                0.258168\n",
       "F1[t - 1]                                       0.396335\n",
       "F1[t - 2]                                       0.041163\n",
       "F1[t - 3]                                      -0.014884\n",
       "F1[t - 4]                                       0.059665"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_stock = prepare_single_stock('BMW', '1D')\n",
    "mean_end_price = single_stock['PctChange:MeanEndPrice']\n",
    "pd.DataFrame({\n",
    "    'PctChange:MeanEndPrice': mean_end_price,       \n",
    "    norm_feature('H1', 1, 'MeanEndPrice'): single_stock[norm_feature('H1', 1, 'MeanEndPrice')],\n",
    "    norm_feature('H1', 2, 'MeanEndPrice'): single_stock[norm_feature('H1', 2, 'MeanEndPrice')],\n",
    "    norm_feature('H1', 3, 'MeanEndPrice'): single_stock[norm_feature('H1', 3, 'MeanEndPrice')],\n",
    "    norm_feature('H1', 4, 'MeanEndPrice'): single_stock[norm_feature('H1', 4, 'MeanEndPrice')],\n",
    "    'F1[t - 1]': single_stock['F1[t - 1]'],\n",
    "    'F1[t - 2]': single_stock['F1[t - 1]'].shift(1),\n",
    "    'F1[t - 3]': single_stock['F1[t - 1]'].shift(2),\n",
    "    'F1[t - 4]': single_stock['F1[t - 1]'].shift(3)  \n",
    "}).corr()[['PctChange:MeanEndPrice']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Log of return\n",
    "In their [demo notebook](https://github.com/googledatalab/notebooks/blob/master/samples/TensorFlow/Machine%20Learning%20with%20Financial%20Data.ipynb) Google Cloud computes a feature that they call log return. This feature is:\n",
    "\n",
    "```\n",
    "log(Price:X[t]/Price:X[t - 1])\n",
    "```\n",
    "for some version of price X.\n",
    "\n",
    "We explore this feature below. We find that when `Mean` of price is used, this feature correlates well\n",
    "with various predictors, but when the `Last` of price is used, this feature does not correlate very well"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>RevPctChange[t - 4]:Direction2</th>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:VolumeWeightedEndPrice</th>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>LogReturn:MeanEndPrice</th>\n",
       "      <td>0.483406</td>\n",
       "      <td>0.526596</td>\n",
       "      <td>0.578146</td>\n",
       "      <td>0.484382</td>\n",
       "      <td>0.481897</td>\n",
       "      <td>0.099758</td>\n",
       "      <td>0.096859</td>\n",
       "      <td>0.346751</td>\n",
       "      <td>0.32796</td>\n",
       "      <td>0.386403</td>\n",
       "      <td>...</td>\n",
       "      <td>0.007901</td>\n",
       "      <td>0.001222</td>\n",
       "      <td>-0.03723</td>\n",
       "      <td>0.009178</td>\n",
       "      <td>-0.017552</td>\n",
       "      <td>0.261486</td>\n",
       "      <td>0.165445</td>\n",
       "      <td>0.114805</td>\n",
       "      <td>0.085632</td>\n",
       "      <td>0.447735</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 96 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                        Direction1[t - 1]  Direction2[t - 1]  F1[t - 1]  \\\n",
       "LogReturn:MeanEndPrice           0.483406           0.526596   0.578146   \n",
       "\n",
       "                        F2[t - 1]  F3[t - 1]  F4[t - 1]  F5[t - 1]  F6[t - 1]  \\\n",
       "LogReturn:MeanEndPrice   0.484382   0.481897   0.099758   0.096859   0.346751   \n",
       "\n",
       "                        F7[t - 1]  F8[t - 1]  ...  \\\n",
       "LogReturn:MeanEndPrice    0.32796   0.386403  ...   \n",
       "\n",
       "                        RevPctChange[t - 4]:Direction2  \\\n",
       "LogReturn:MeanEndPrice                        0.007901   \n",
       "\n",
       "                        RevPctChange[t - 1]:StdEndPrice  \\\n",
       "LogReturn:MeanEndPrice                         0.001222   \n",
       "\n",
       "                        RevPctChange[t - 2]:StdEndPrice  \\\n",
       "LogReturn:MeanEndPrice                         -0.03723   \n",
       "\n",
       "                        RevPctChange[t - 3]:StdEndPrice  \\\n",
       "LogReturn:MeanEndPrice                         0.009178   \n",
       "\n",
       "                        RevPctChange[t - 4]:StdEndPrice  \\\n",
       "LogReturn:MeanEndPrice                        -0.017552   \n",
       "\n",
       "                        RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "LogReturn:MeanEndPrice                                    0.261486   \n",
       "\n",
       "                        RevPctChange[t - 2]:VolumeWeightedEndPrice  \\\n",
       "LogReturn:MeanEndPrice                                    0.165445   \n",
       "\n",
       "                        RevPctChange[t - 3]:VolumeWeightedEndPrice  \\\n",
       "LogReturn:MeanEndPrice                                    0.114805   \n",
       "\n",
       "                        RevPctChange[t - 4]:VolumeWeightedEndPrice  \\\n",
       "LogReturn:MeanEndPrice                                    0.085632   \n",
       "\n",
       "                        VolumeWeightedPctChange[t - 1]  \n",
       "LogReturn:MeanEndPrice                        0.447735  \n",
       "\n",
       "[1 rows x 96 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_stock = prepare_single_stock('SIE', '30Min')\n",
    "corr_log_return(single_stock, 'MeanEndPrice')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Direction1[t - 1]</th>\n",
       "      <th>Direction2[t - 1]</th>\n",
       "      <th>F1[t - 1]</th>\n",
       "      <th>F2[t - 1]</th>\n",
       "      <th>F3[t - 1]</th>\n",
       "      <th>F4[t - 1]</th>\n",
       "      <th>F5[t - 1]</th>\n",
       "      <th>F6[t - 1]</th>\n",
       "      <th>F7[t - 1]</th>\n",
       "      <th>F8[t - 1]</th>\n",
       "      <th>...</th>\n",
       "      <th>RevPctChange[t - 4]:Direction2</th>\n",
       "      <th>RevPctChange[t - 1]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:StdEndPrice</th>\n",
       "      <th>RevPctChange[t - 1]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 2]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 3]:VolumeWeightedEndPrice</th>\n",
       "      <th>RevPctChange[t - 4]:VolumeWeightedEndPrice</th>\n",
       "      <th>VolumeWeightedPctChange[t - 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>LogReturn:LastEndPrice</th>\n",
       "      <td>-0.014406</td>\n",
       "      <td>-0.0312</td>\n",
       "      <td>-0.028672</td>\n",
       "      <td>-0.01451</td>\n",
       "      <td>-0.014227</td>\n",
       "      <td>0.019149</td>\n",
       "      <td>-0.018149</td>\n",
       "      <td>-0.020512</td>\n",
       "      <td>-0.030164</td>\n",
       "      <td>-0.013356</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.025579</td>\n",
       "      <td>0.004343</td>\n",
       "      <td>-0.036127</td>\n",
       "      <td>0.031563</td>\n",
       "      <td>-0.015164</td>\n",
       "      <td>0.01188</td>\n",
       "      <td>-0.005545</td>\n",
       "      <td>-0.032484</td>\n",
       "      <td>-0.018885</td>\n",
       "      <td>-0.009704</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 96 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                        Direction1[t - 1]  Direction2[t - 1]  F1[t - 1]  \\\n",
       "LogReturn:LastEndPrice          -0.014406            -0.0312  -0.028672   \n",
       "\n",
       "                        F2[t - 1]  F3[t - 1]  F4[t - 1]  F5[t - 1]  F6[t - 1]  \\\n",
       "LogReturn:LastEndPrice   -0.01451  -0.014227   0.019149  -0.018149  -0.020512   \n",
       "\n",
       "                        F7[t - 1]  F8[t - 1]  ...  \\\n",
       "LogReturn:LastEndPrice  -0.030164  -0.013356  ...   \n",
       "\n",
       "                        RevPctChange[t - 4]:Direction2  \\\n",
       "LogReturn:LastEndPrice                       -0.025579   \n",
       "\n",
       "                        RevPctChange[t - 1]:StdEndPrice  \\\n",
       "LogReturn:LastEndPrice                         0.004343   \n",
       "\n",
       "                        RevPctChange[t - 2]:StdEndPrice  \\\n",
       "LogReturn:LastEndPrice                        -0.036127   \n",
       "\n",
       "                        RevPctChange[t - 3]:StdEndPrice  \\\n",
       "LogReturn:LastEndPrice                         0.031563   \n",
       "\n",
       "                        RevPctChange[t - 4]:StdEndPrice  \\\n",
       "LogReturn:LastEndPrice                        -0.015164   \n",
       "\n",
       "                        RevPctChange[t - 1]:VolumeWeightedEndPrice  \\\n",
       "LogReturn:LastEndPrice                                     0.01188   \n",
       "\n",
       "                        RevPctChange[t - 2]:VolumeWeightedEndPrice  \\\n",
       "LogReturn:LastEndPrice                                   -0.005545   \n",
       "\n",
       "                        RevPctChange[t - 3]:VolumeWeightedEndPrice  \\\n",
       "LogReturn:LastEndPrice                                   -0.032484   \n",
       "\n",
       "                        RevPctChange[t - 4]:VolumeWeightedEndPrice  \\\n",
       "LogReturn:LastEndPrice                                   -0.018885   \n",
       "\n",
       "                        VolumeWeightedPctChange[t - 1]  \n",
       "LogReturn:LastEndPrice                       -0.009704  \n",
       "\n",
       "[1 rows x 96 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_stock = prepare_single_stock('SIE', '30Min')\n",
    "corr_log_return(single_stock, 'LastEndPrice')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we examine correlations of LogReturn we find that beyond `(t - 1)` there are no correlations.\n",
    "Google cloud also reports the same. As we explained, the reason is due to different normalizaiton\n",
    "in different periods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>LogReturn:MeanEndPrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>LogReturn:MeanEndPrice</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LogReturn:MeanEndPrice[t - 1]</th>\n",
       "      <td>0.235328</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LogReturn:MeanEndPrice[t - 2]</th>\n",
       "      <td>-0.006197</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LogReturn:MeanEndPrice[t - 3]</th>\n",
       "      <td>-0.032370</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LogReturn:MeanEndPrice[t - 4]</th>\n",
       "      <td>-0.021452</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                               LogReturn:MeanEndPrice\n",
       "LogReturn:MeanEndPrice                       1.000000\n",
       "LogReturn:MeanEndPrice[t - 1]                0.235328\n",
       "LogReturn:MeanEndPrice[t - 2]               -0.006197\n",
       "LogReturn:MeanEndPrice[t - 3]               -0.032370\n",
       "LogReturn:MeanEndPrice[t - 4]               -0.021452"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "single_stock = prepare_single_stock('SIE', '30Min')\n",
    "mean_end_price = single_stock['LogReturn:MeanEndPrice']\n",
    "pd.DataFrame({\n",
    "    'LogReturn:MeanEndPrice': mean_end_price,     \n",
    "    'LogReturn:MeanEndPrice[t - 1]': single_stock['LogReturn:MeanEndPrice[t - 1]'],\n",
    "    'LogReturn:MeanEndPrice[t - 2]': single_stock['LogReturn:MeanEndPrice[t - 1]'].shift(1),\n",
    "    'LogReturn:MeanEndPrice[t - 3]': single_stock['LogReturn:MeanEndPrice[t - 1]'].shift(2),\n",
    "    'LogReturn:MeanEndPrice[t - 4]': single_stock['LogReturn:MeanEndPrice[t - 1]'].shift(3)  \n",
    "}).corr()[['LogReturn:MeanEndPrice']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## On the relationship between percent change and log return"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some people prefer to model percent change, while others prefer to model log return.\n",
    "We find that both a suspiciously correlated. It turns that there is an approximate\n",
    "mathematical equality that relates them\n",
    "\n",
    "```\n",
    "log(a/b) = log[ (b + (a - b))/b ] = log(1 + pct_change) = pct_change, \n",
    "where pct_change = (a - b)/b\n",
    "\n",
    "log(1 + x) = x is an approximate equality when x is close to zero\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The variance of predicted features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we study the variance of a number of functions:\n",
    "\n",
    "```\n",
    "func = (price - anchor)/anchor\n",
    "```\n",
    "for different choices of price and anchor.\n",
    "\n",
    "```\n",
    "end to end: (end_price[t] - end_price[t - 1])/end_price[t - 1]\n",
    "end to mean: (end_price[t] - mean_price[t - 1])/mean_price[t - 1]\n",
    "mean to end: (mean_price[t] - end_price[t - 1])/end_price[t - 1]\n",
    "mean to mean: (mean_price[t] - mean_price[t - 1])/mean_price[t - 1]\n",
    "mean to prev mean: (mean_price[t] - mean_price[t - 2])/mean_price[t - 2]\n",
    "```\n",
    "\n",
    "The idea is that we use the following method to estimate mean price in minutes 15 to 20.\n",
    "- estimate the mean price in minutes 10 to 15, call it `M[1]`\n",
    "- estimate the mean price in minutes 10 to 20, call it `M[2]`\n",
    "- compute `M[2] - M[1]`, the second minus the first\n",
    "\n",
    "## Some observations\n",
    "\n",
    "- As the anchor goes further back in time from the predicted price, the variance increases\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "intervals = ['1Min', '2Min', '3Min', '4Min', '5Min', '6Min', '7Min', '8Min', '9Min', '10Min', \n",
    "             '15Min', '30Min', '60Min', '120Min', '180Min', '240Min',\n",
    "             '1D', '2D', '1W', '2W', '1M', '2M', '3M', '6M']\n",
    "rows = []\n",
    "for i in intervals:\n",
    "    single_stock = prepare_single_stock('SIE', i)\n",
    "    e = single_stock['LastEndPrice']\n",
    "    n = single_stock['LastEndPrice'].shift(1)\n",
    "    end_to_end = (e - n)/n\n",
    "    \n",
    "    m = single_stock['MeanAvg4Price']\n",
    "    end_to_mean = (e - m.shift(1))/m.shift(1)  \n",
    "    mean_to_mean = (m - m.shift(1))/m.shift(1)\n",
    "    mean_to_prev_mean = (m - m.shift(2))/m.shift(2) \n",
    "    mean_to_end = (m - e)/e\n",
    "    rows.append((i, end_to_end.std(), end_to_mean.std(), mean_to_mean.std(), mean_to_prev_mean.std(), mean_to_end.std()))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>interval</th>\n",
       "      <th>end_to_end</th>\n",
       "      <th>end to mean</th>\n",
       "      <th>mean to mean</th>\n",
       "      <th>mean to prev mean</th>\n",
       "      <th>mean to end</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1Min</td>\n",
       "      <td>0.000632</td>\n",
       "      <td>0.000653</td>\n",
       "      <td>0.000553</td>\n",
       "      <td>0.000813</td>\n",
       "      <td>0.000217</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2Min</td>\n",
       "      <td>0.000842</td>\n",
       "      <td>0.000889</td>\n",
       "      <td>0.000754</td>\n",
       "      <td>0.001113</td>\n",
       "      <td>0.000335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3Min</td>\n",
       "      <td>0.001027</td>\n",
       "      <td>0.001115</td>\n",
       "      <td>0.000872</td>\n",
       "      <td>0.001320</td>\n",
       "      <td>0.000492</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4Min</td>\n",
       "      <td>0.001163</td>\n",
       "      <td>0.001258</td>\n",
       "      <td>0.000994</td>\n",
       "      <td>0.001508</td>\n",
       "      <td>0.000522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5Min</td>\n",
       "      <td>0.001312</td>\n",
       "      <td>0.001403</td>\n",
       "      <td>0.001117</td>\n",
       "      <td>0.001685</td>\n",
       "      <td>0.000570</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6Min</td>\n",
       "      <td>0.001412</td>\n",
       "      <td>0.001515</td>\n",
       "      <td>0.001225</td>\n",
       "      <td>0.001845</td>\n",
       "      <td>0.000608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7Min</td>\n",
       "      <td>0.001523</td>\n",
       "      <td>0.001634</td>\n",
       "      <td>0.001328</td>\n",
       "      <td>0.001993</td>\n",
       "      <td>0.000658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8Min</td>\n",
       "      <td>0.001619</td>\n",
       "      <td>0.001738</td>\n",
       "      <td>0.001406</td>\n",
       "      <td>0.002123</td>\n",
       "      <td>0.000683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9Min</td>\n",
       "      <td>0.001708</td>\n",
       "      <td>0.001831</td>\n",
       "      <td>0.001481</td>\n",
       "      <td>0.002241</td>\n",
       "      <td>0.000727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10Min</td>\n",
       "      <td>0.001803</td>\n",
       "      <td>0.001931</td>\n",
       "      <td>0.001590</td>\n",
       "      <td>0.002389</td>\n",
       "      <td>0.000743</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>15Min</td>\n",
       "      <td>0.002198</td>\n",
       "      <td>0.002355</td>\n",
       "      <td>0.001960</td>\n",
       "      <td>0.002921</td>\n",
       "      <td>0.000885</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>30Min</td>\n",
       "      <td>0.003097</td>\n",
       "      <td>0.003299</td>\n",
       "      <td>0.002777</td>\n",
       "      <td>0.004142</td>\n",
       "      <td>0.001228</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>60Min</td>\n",
       "      <td>0.004356</td>\n",
       "      <td>0.004649</td>\n",
       "      <td>0.003839</td>\n",
       "      <td>0.005718</td>\n",
       "      <td>0.001668</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>120Min</td>\n",
       "      <td>0.006029</td>\n",
       "      <td>0.006358</td>\n",
       "      <td>0.005313</td>\n",
       "      <td>0.007936</td>\n",
       "      <td>0.002311</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>180Min</td>\n",
       "      <td>0.006252</td>\n",
       "      <td>0.006558</td>\n",
       "      <td>0.005570</td>\n",
       "      <td>0.008353</td>\n",
       "      <td>0.002267</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>240Min</td>\n",
       "      <td>0.007854</td>\n",
       "      <td>0.008387</td>\n",
       "      <td>0.007248</td>\n",
       "      <td>0.010845</td>\n",
       "      <td>0.002862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1D</td>\n",
       "      <td>0.012914</td>\n",
       "      <td>0.013554</td>\n",
       "      <td>0.011700</td>\n",
       "      <td>0.017802</td>\n",
       "      <td>0.003518</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>2D</td>\n",
       "      <td>0.016131</td>\n",
       "      <td>0.017689</td>\n",
       "      <td>0.014470</td>\n",
       "      <td>0.022538</td>\n",
       "      <td>0.006585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1W</td>\n",
       "      <td>0.030696</td>\n",
       "      <td>0.036558</td>\n",
       "      <td>0.024981</td>\n",
       "      <td>0.039890</td>\n",
       "      <td>0.017313</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>2W</td>\n",
       "      <td>0.038304</td>\n",
       "      <td>0.046842</td>\n",
       "      <td>0.035825</td>\n",
       "      <td>0.054845</td>\n",
       "      <td>0.026559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>1M</td>\n",
       "      <td>0.064925</td>\n",
       "      <td>0.068857</td>\n",
       "      <td>0.051602</td>\n",
       "      <td>0.067335</td>\n",
       "      <td>0.022229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>2M</td>\n",
       "      <td>0.103959</td>\n",
       "      <td>0.076894</td>\n",
       "      <td>0.053693</td>\n",
       "      <td>0.070610</td>\n",
       "      <td>0.048815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>3M</td>\n",
       "      <td>0.102349</td>\n",
       "      <td>0.088473</td>\n",
       "      <td>0.067846</td>\n",
       "      <td>0.034848</td>\n",
       "      <td>0.033988</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>6M</td>\n",
       "      <td>0.081347</td>\n",
       "      <td>0.022896</td>\n",
       "      <td>0.042427</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.057608</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   interval  end_to_end  end to mean  mean to mean  mean to prev mean  \\\n",
       "0      1Min    0.000632     0.000653      0.000553           0.000813   \n",
       "1      2Min    0.000842     0.000889      0.000754           0.001113   \n",
       "2      3Min    0.001027     0.001115      0.000872           0.001320   \n",
       "3      4Min    0.001163     0.001258      0.000994           0.001508   \n",
       "4      5Min    0.001312     0.001403      0.001117           0.001685   \n",
       "5      6Min    0.001412     0.001515      0.001225           0.001845   \n",
       "6      7Min    0.001523     0.001634      0.001328           0.001993   \n",
       "7      8Min    0.001619     0.001738      0.001406           0.002123   \n",
       "8      9Min    0.001708     0.001831      0.001481           0.002241   \n",
       "9     10Min    0.001803     0.001931      0.001590           0.002389   \n",
       "10    15Min    0.002198     0.002355      0.001960           0.002921   \n",
       "11    30Min    0.003097     0.003299      0.002777           0.004142   \n",
       "12    60Min    0.004356     0.004649      0.003839           0.005718   \n",
       "13   120Min    0.006029     0.006358      0.005313           0.007936   \n",
       "14   180Min    0.006252     0.006558      0.005570           0.008353   \n",
       "15   240Min    0.007854     0.008387      0.007248           0.010845   \n",
       "16       1D    0.012914     0.013554      0.011700           0.017802   \n",
       "17       2D    0.016131     0.017689      0.014470           0.022538   \n",
       "18       1W    0.030696     0.036558      0.024981           0.039890   \n",
       "19       2W    0.038304     0.046842      0.035825           0.054845   \n",
       "20       1M    0.064925     0.068857      0.051602           0.067335   \n",
       "21       2M    0.103959     0.076894      0.053693           0.070610   \n",
       "22       3M    0.102349     0.088473      0.067846           0.034848   \n",
       "23       6M    0.081347     0.022896      0.042427                NaN   \n",
       "\n",
       "    mean to end  \n",
       "0      0.000217  \n",
       "1      0.000335  \n",
       "2      0.000492  \n",
       "3      0.000522  \n",
       "4      0.000570  \n",
       "5      0.000608  \n",
       "6      0.000658  \n",
       "7      0.000683  \n",
       "8      0.000727  \n",
       "9      0.000743  \n",
       "10     0.000885  \n",
       "11     0.001228  \n",
       "12     0.001668  \n",
       "13     0.002311  \n",
       "14     0.002267  \n",
       "15     0.002862  \n",
       "16     0.003518  \n",
       "17     0.006585  \n",
       "18     0.017313  \n",
       "19     0.026559  \n",
       "20     0.022229  \n",
       "21     0.048815  \n",
       "22     0.033988  \n",
       "23     0.057608  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(rows, columns = [\n",
    "    \"interval\", \"end_to_end\", \"end to mean\", \"mean to mean\", \"mean to prev mean\", \"mean to end\"\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predicting 60 min ahead using 10, 15, 20 and 30 minutes windows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def resample_single_stock(single_stock, interval):\n",
    "    return pd.DataFrame({\n",
    "        'MaxPrice': single_stock['MaxPrice'].resample(interval).max(),\n",
    "        'MinPrice': single_stock['MinPrice'].resample(interval).min(),\n",
    "        'LastEndPrice': single_stock['EndPrice'].resample(interval).last(),\n",
    "        'FirstStartPrice': single_stock['StartPrice'].resample(interval).first(),         \n",
    "        'MeanEndPrice': single_stock['EndPrice'].resample(interval).mean(),        \n",
    "        'HasTrade': single_stock['HasTrade'].resample(interval).max(),\n",
    "    })\n",
    "\n",
    "def prepare_single_stock_multi_intervals(mnemonic, predicted_price, main_interval, intervals):\n",
    "    single_stock = df[df.Mnemonic == mnemonic].copy()\n",
    "        \n",
    "    main = resample_single_stock(single_stock, main_interval)\n",
    "    # we use the same anchor\n",
    "    anchor = main['MeanEndPrice']\n",
    "    future_mean_price = main[predicted_price].shift(-1)\n",
    "    main['AdjustedPctChange[t + 1]'] = (future_mean_price - anchor)/anchor\n",
    "    \n",
    "    all_intervals = [main_interval] + intervals\n",
    "    \n",
    "    for interval in all_intervals:\n",
    "        sub = resample_single_stock(single_stock, interval)\n",
    "        resampled = sub.resample(main_interval).last() \n",
    "\n",
    "        main['Direction@' + interval] = \\\n",
    "            2.0*(resampled['LastEndPrice'] - resampled['FirstStartPrice'])/ \\\n",
    "            anchor\n",
    "\n",
    "        main['H1@' + interval] = - closer_to_with_normalization(\n",
    "                                                 resampled['LastEndPrice'], \n",
    "                                                 resampled['MaxPrice'], \n",
    "                                                 resampled['MinPrice'],\n",
    "                                                 anchor)    \n",
    "        \n",
    "        main['EndToMean@' + interval] = (resampled['LastEndPrice'] - resampled['MeanEndPrice'])/anchor\n",
    "        \n",
    "    main = main[main['HasTrade'] == 1.0]\n",
    "    main = main.drop(columns = [\n",
    "        'MaxPrice',\n",
    "        'MinPrice',\n",
    "        'LastEndPrice',\n",
    "        'FirstStartPrice',         \n",
    "        'MeanEndPrice',     \n",
    "        'HasTrade'       \n",
    "    ])\n",
    "    return main"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AdjustedPctChange[t + 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AdjustedPctChange[t + 1]</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@60Min</th>\n",
       "      <td>0.694290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@30Min</th>\n",
       "      <td>0.633703</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@60Min</th>\n",
       "      <td>0.614180</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@30Min</th>\n",
       "      <td>0.588048</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@30Min</th>\n",
       "      <td>0.582287</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@20Min</th>\n",
       "      <td>0.551924</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@60Min</th>\n",
       "      <td>0.519582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@15Min</th>\n",
       "      <td>0.493588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@20Min</th>\n",
       "      <td>0.486650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@20Min</th>\n",
       "      <td>0.473604</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@15Min</th>\n",
       "      <td>0.413438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@15Min</th>\n",
       "      <td>0.400779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@10Min</th>\n",
       "      <td>0.392419</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@10Min</th>\n",
       "      <td>0.323514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@10Min</th>\n",
       "      <td>0.309173</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@5Min</th>\n",
       "      <td>0.247695</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@5Min</th>\n",
       "      <td>0.231562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@5Min</th>\n",
       "      <td>0.210880</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@2Min</th>\n",
       "      <td>0.192312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@2Min</th>\n",
       "      <td>0.147793</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@2Min</th>\n",
       "      <td>0.124329</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          AdjustedPctChange[t + 1]\n",
       "AdjustedPctChange[t + 1]                  1.000000\n",
       "EndToMean@60Min                           0.694290\n",
       "Direction@30Min                           0.633703\n",
       "H1@60Min                                  0.614180\n",
       "H1@30Min                                  0.588048\n",
       "EndToMean@30Min                           0.582287\n",
       "Direction@20Min                           0.551924\n",
       "Direction@60Min                           0.519582\n",
       "Direction@15Min                           0.493588\n",
       "EndToMean@20Min                           0.486650\n",
       "H1@20Min                                  0.473604\n",
       "EndToMean@15Min                           0.413438\n",
       "H1@15Min                                  0.400779\n",
       "Direction@10Min                           0.392419\n",
       "EndToMean@10Min                           0.323514\n",
       "H1@10Min                                  0.309173\n",
       "Direction@5Min                            0.247695\n",
       "EndToMean@5Min                            0.231562\n",
       "H1@5Min                                   0.210880\n",
       "Direction@2Min                            0.192312\n",
       "H1@2Min                                   0.147793\n",
       "EndToMean@2Min                            0.124329"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "main_interval = '60Min'\n",
    "intervals = ['2Min', '5Min', '10Min', '15Min', '20Min', '30Min']\n",
    "\n",
    "single_stock = prepare_single_stock_multi_intervals('SIE', 'MeanEndPrice', main_interval, intervals)\n",
    "\n",
    "k = 'AdjustedPctChange[t + 1]'\n",
    "single_stock.corr()[[k]].sort_values(k, ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AdjustedPctChange[t + 1]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AdjustedPctChange[t + 1]</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@60Min</th>\n",
       "      <td>0.504806</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@30Min</th>\n",
       "      <td>0.468883</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@60Min</th>\n",
       "      <td>0.449401</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@30Min</th>\n",
       "      <td>0.430578</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@30Min</th>\n",
       "      <td>0.429825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@20Min</th>\n",
       "      <td>0.410920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@60Min</th>\n",
       "      <td>0.388155</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@15Min</th>\n",
       "      <td>0.359810</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@20Min</th>\n",
       "      <td>0.355766</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@20Min</th>\n",
       "      <td>0.345523</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@15Min</th>\n",
       "      <td>0.298978</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@15Min</th>\n",
       "      <td>0.287264</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@10Min</th>\n",
       "      <td>0.285013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@10Min</th>\n",
       "      <td>0.232282</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@10Min</th>\n",
       "      <td>0.218876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@5Min</th>\n",
       "      <td>0.173569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@5Min</th>\n",
       "      <td>0.162471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@5Min</th>\n",
       "      <td>0.150226</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Direction@2Min</th>\n",
       "      <td>0.138976</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>H1@2Min</th>\n",
       "      <td>0.098520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>EndToMean@2Min</th>\n",
       "      <td>0.078964</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          AdjustedPctChange[t + 1]\n",
       "AdjustedPctChange[t + 1]                  1.000000\n",
       "EndToMean@60Min                           0.504806\n",
       "Direction@30Min                           0.468883\n",
       "H1@60Min                                  0.449401\n",
       "EndToMean@30Min                           0.430578\n",
       "H1@30Min                                  0.429825\n",
       "Direction@20Min                           0.410920\n",
       "Direction@60Min                           0.388155\n",
       "Direction@15Min                           0.359810\n",
       "EndToMean@20Min                           0.355766\n",
       "H1@20Min                                  0.345523\n",
       "EndToMean@15Min                           0.298978\n",
       "H1@15Min                                  0.287264\n",
       "Direction@10Min                           0.285013\n",
       "EndToMean@10Min                           0.232282\n",
       "H1@10Min                                  0.218876\n",
       "Direction@5Min                            0.173569\n",
       "EndToMean@5Min                            0.162471\n",
       "H1@5Min                                   0.150226\n",
       "Direction@2Min                            0.138976\n",
       "H1@2Min                                   0.098520\n",
       "EndToMean@2Min                            0.078964"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "main_interval = '60Min'\n",
    "intervals = ['2Min', '5Min', '10Min', '15Min', '20Min', '30Min']\n",
    "\n",
    "single_stock = prepare_single_stock_multi_intervals('SIE', 'LastEndPrice', main_interval, intervals)\n",
    "\n",
    "k = 'AdjustedPctChange[t + 1]'\n",
    "single_stock.corr()[[k]].sort_values(k, ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last run on Wed Jan 22 18:30:31 UTC 2020\r\n"
     ]
    }
   ],
   "source": [
    "!echo \"Last run on `date`\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}