Temas/Módulo 06 - Redes recurrentes/Cuadernos/rnr_Long_Time_series.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<figure>\n",
    "<img src=\"../Imagenes/logo-final-ap.png\"  width=\"80\" height=\"80\" align=\"left\"/> \n",
    "</figure>\n",
    "\n",
    "# <span style=\"color:blue\"><left>Aprendizaje Profundo</left></span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <span style=\"color:red\"><center>Series de tiempo largas</center></span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center> Ejemplo : Consumo de energía en los hogares</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##   <span style=\"color:blue\">Profesores</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Alvaro Mauricio Montenegro Díaz, ammontenegrod@unal.edu.co\n",
    "2. Daniel Mauricio Montenegro Reyes, dextronomo@gmail.com \n",
    "3. Campo Elías Pardo Turriago, cepardot@unal.edu.co "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##   <span style=\"color:blue\">Asesora Medios y Marketing digital</span>\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Maria del Pilar Montenegro, pmontenegro88@gmail.com "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Asistentes</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Oleg Jarma, ojarmam@unal.edu.co \n",
    "6. Laura Lizarazo, ljlizarazore@unal.edu.co "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Referencias</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. [Introducción a Redes LSTM](Intro_LSTM.ipynb)\n",
    "1. [Time Series Forecasting with LSTMs using TensorFlow 2 and Keras in Python](https://towardsdatascience.com/time-series-forecasting-with-lstms-using-tensorflow-2-and-keras-in-python-6ceee9c6c651/)\n",
    "1. [Dive into Deep Learnig](https://d2l.ai/)\n",
    "1. [Introducción a Redes LSTM](Intro_LSTM.ipynb)\n",
    "1. [Time Series Forecasting with LSTMs using TensorFlow 2 and Keras in Python](https://towardsdatascience.com/time-series-forecasting-with-lstms-using-tensorflow-2-and-keras-in-python-6ceee9c6c651/)\n",
    "1. [Tensoflow-Time series forecasting](https://www.tensorflow.org/tutorials/structured_data/time_series)\n",
    "1. Adaptado de [3 Steps to Time Series Forecasting: LSTM with TensorFlow Keras\n",
    "](https://www.justintodata.com/forecast-time-series-lstm-with-tensorflow-keras/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Contenido</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* [Introducción](#Introducción)\n",
    "* [Importar las librería requeridas](#Importar-las-librería-requeridas)\n",
    "* [Conjunto de datos household power consumption](#Conjunto-de-datos-household-power-consumption)\n",
    "* [Preprocesamiento](#Preprocesamiento)\n",
    "* [Clase create_ts_files](#Clase-create_ts_files)\n",
    "* [Clase TimeSeriesLoader](#Clase-TimeSeriesLoader)\n",
    "* [Crea el modelo LSTM](#Crea-el-modelo-LSTM)\n",
    "* [Entrenamiento](#Entrenamiento)\n",
    "* [Evaluación del modelo](#Evaluación-del-modelo)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Introducción</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El conjunto de datos que vamos a utilizar es el consumo de energía eléctrica de los hogares de Kaggle. Para bajar los datos vaya [aquí](https://www.kaggle.com/sagnikseal/household-power-consumption). Los datos proporcionan mediciones del consumo de energía eléctrica en un hogar con una frecuencia de muestreo de un minuto.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importar las librerías requeridas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import packages\n",
    "import math\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow.keras.utils import Sequence\n",
    "from datetime import timedelta\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import time\n",
    "\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conjunto de datos household power consumption"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hay 2.075.259 mediciones recopiladas en 4 años. Están disponibles diferentes magnitudes eléctricas y algunos valores de submedición. \n",
    "\n",
    "Nos centraremos en tres variables (features):\n",
    "\n",
    "- Fecha: fecha en formato dd / mm / aaaa\n",
    "- Hora: hora en formato hh: mm: ss\n",
    "- Potencia_activa_global: potencia activa promedio por minuto global del hogar (en kilovatios)\n",
    "\n",
    "\n",
    "En esta lección, predeciremos la cantidad de Global_active_power con 10 minutos de anticipación.\n",
    "\n",
    "Hemos bajado los datos y los tenemos localmente."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/alvaro/anaconda3/envs/tf2/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3063: DtypeWarning: Columns (2,3,4,5,6,7) have mixed types.Specify dtype option on import or set low_memory=False.\n",
      "  interactivity=interactivity, compiler=compiler, result=result)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Time</th>\n",
       "      <th>Global_active_power</th>\n",
       "      <th>Global_reactive_power</th>\n",
       "      <th>Voltage</th>\n",
       "      <th>Global_intensity</th>\n",
       "      <th>Sub_metering_1</th>\n",
       "      <th>Sub_metering_2</th>\n",
       "      <th>Sub_metering_3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>16/12/2006</td>\n",
       "      <td>17:24:00</td>\n",
       "      <td>4.216</td>\n",
       "      <td>0.418</td>\n",
       "      <td>234.840</td>\n",
       "      <td>18.400</td>\n",
       "      <td>0.000</td>\n",
       "      <td>1.000</td>\n",
       "      <td>17.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>16/12/2006</td>\n",
       "      <td>17:25:00</td>\n",
       "      <td>5.360</td>\n",
       "      <td>0.436</td>\n",
       "      <td>233.630</td>\n",
       "      <td>23.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>1.000</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>16/12/2006</td>\n",
       "      <td>17:26:00</td>\n",
       "      <td>5.374</td>\n",
       "      <td>0.498</td>\n",
       "      <td>233.290</td>\n",
       "      <td>23.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>2.000</td>\n",
       "      <td>17.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>16/12/2006</td>\n",
       "      <td>17:27:00</td>\n",
       "      <td>5.388</td>\n",
       "      <td>0.502</td>\n",
       "      <td>233.740</td>\n",
       "      <td>23.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>1.000</td>\n",
       "      <td>17.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>16/12/2006</td>\n",
       "      <td>17:28:00</td>\n",
       "      <td>3.666</td>\n",
       "      <td>0.528</td>\n",
       "      <td>235.680</td>\n",
       "      <td>15.800</td>\n",
       "      <td>0.000</td>\n",
       "      <td>1.000</td>\n",
       "      <td>17.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date      Time Global_active_power Global_reactive_power  Voltage  \\\n",
       "0  16/12/2006  17:24:00               4.216                 0.418  234.840   \n",
       "1  16/12/2006  17:25:00               5.360                 0.436  233.630   \n",
       "2  16/12/2006  17:26:00               5.374                 0.498  233.290   \n",
       "3  16/12/2006  17:27:00               5.388                 0.502  233.740   \n",
       "4  16/12/2006  17:28:00               3.666                 0.528  235.680   \n",
       "\n",
       "  Global_intensity Sub_metering_1 Sub_metering_2  Sub_metering_3  \n",
       "0           18.400          0.000          1.000            17.0  \n",
       "1           23.000          0.000          1.000            16.0  \n",
       "2           23.000          0.000          2.000            17.0  \n",
       "3           23.000          0.000          1.000            17.0  \n",
       "4           15.800          0.000          1.000            17.0  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read the dataset into python\n",
    "df = pd.read_csv('../Datos/household_power_consumption.csv', delimiter=';')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocesamiento"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transformamos el conjunto de datos df así:\n",
    "\n",
    "- Creando la función *date_time* en formato *DateTime* combinando Fecha y Hora.\n",
    "- Convertiendo *Global_active_power* a numérico y eliminar los valores faltantes (1,25%).\n",
    "- Ordenando las variables por tiempo en el nuevo conjunto de datos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of rows and columns after removing missing values: (2049280, 2)\n",
      "The time series starts from:  2006-12-16 17:24:00\n",
      "The time series ends on:  2010-12-11 23:59:00\n",
      "CPU times: user 5min 15s, sys: 620 ms, total: 5min 16s\n",
      "Wall time: 5min 17s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "# Código tomado de  https://towardsdatascience.com/time-series-analysis-visualization-forecasting-with-lstm-77a905180eba\n",
    "# con algunos cambios menores\n",
    "#\n",
    "df['date_time'] = pd.to_datetime(df['Date'] + ' ' + df['Time'])\n",
    "df['Global_active_power'] = pd.to_numeric(df['Global_active_power'], errors='coerce')\n",
    "df = df.dropna(subset=['Global_active_power'])\n",
    "\n",
    "df['date_time'] = pd.to_datetime(df['date_time'])\n",
    "\n",
    "df = df.loc[:, ['date_time', 'Global_active_power']]\n",
    "df.sort_values('date_time', inplace=True, ascending=True)\n",
    "df = df.reset_index(drop=True)\n",
    "\n",
    "print('Número de filas y columnas después de eliminar los valores faltantes:', df.shape)\n",
    "print('La serie temporal comienza en: ', df [' date_time '].min ())\n",
    "print ('La serie temporal termina en:', , df['date_time'].max())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 2049280 entries, 0 to 2049279\n",
      "Data columns (total 2 columns):\n",
      " #   Column               Dtype         \n",
      "---  ------               -----         \n",
      " 0   date_time            datetime64[ns]\n",
      " 1   Global_active_power  float32       \n",
      "dtypes: datetime64[ns](1), float32(1)\n",
      "memory usage: 23.5 MB\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_time</th>\n",
       "      <th>Global_active_power</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2006-12-16 17:24:00</td>\n",
       "      <td>4.216</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2006-12-16 17:25:00</td>\n",
       "      <td>5.360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2006-12-16 17:26:00</td>\n",
       "      <td>5.374</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2006-12-16 17:27:00</td>\n",
       "      <td>5.388</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2006-12-16 17:28:00</td>\n",
       "      <td>3.666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2006-12-16 17:29:00</td>\n",
       "      <td>3.520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2006-12-16 17:30:00</td>\n",
       "      <td>3.702</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2006-12-16 17:31:00</td>\n",
       "      <td>3.700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2006-12-16 17:32:00</td>\n",
       "      <td>3.668</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2006-12-16 17:33:00</td>\n",
       "      <td>3.662</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            date_time  Global_active_power\n",
       "0 2006-12-16 17:24:00                4.216\n",
       "1 2006-12-16 17:25:00                5.360\n",
       "2 2006-12-16 17:26:00                5.374\n",
       "3 2006-12-16 17:27:00                5.388\n",
       "4 2006-12-16 17:28:00                3.666\n",
       "5 2006-12-16 17:29:00                3.520\n",
       "6 2006-12-16 17:30:00                3.702\n",
       "7 2006-12-16 17:31:00                3.700\n",
       "8 2006-12-16 17:32:00                3.668\n",
       "9 2006-12-16 17:33:00                3.662"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.info()\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.astype({\"Global_active_power\": 'float32'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 2049280 entries, 0 to 2049279\n",
      "Data columns (total 2 columns):\n",
      " #   Column               Dtype         \n",
      "---  ------               -----         \n",
      " 0   date_time            datetime64[ns]\n",
      " 1   Global_active_power  float32       \n",
      "dtypes: datetime64[ns](1), float32(1)\n",
      "memory usage: 23.5 MB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Datos de entrenamiento, validación y prueba"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A continuación, dividimos el conjunto de datos en conjuntos de datos de *entrenamiento, validación y prueba*.\n",
    "\n",
    "\n",
    "- *df_test* contiene los datos de los últimos 7 días en el conjunto de datos original. \n",
    "- *df_val* tiene datos 14 días antes del conjunto de datos de prueba. \n",
    "- *df_train* tiene el resto de los datos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split into training, validation and test datasets.\n",
    "# Since it's timeseries we should do it by date.\n",
    "test_cutoff_date = df['date_time'].max() - timedelta(days=7)\n",
    "val_cutoff_date = test_cutoff_date - timedelta(days=14)\n",
    "\n",
    "df_test = df[df['date_time'] > test_cutoff_date]\n",
    "df_val = df[(df['date_time'] > val_cutoff_date) & (df['date_time'] <= test_cutoff_date)]\n",
    "df_train = df[df['date_time'] <= val_cutoff_date]\n",
    "\n",
    "#check out the datasets\n",
    "print('Test dates: {} to {}'.format(df_test['date_time'].min(), df_test['date_time'].max()))\n",
    "print('Validation dates: {} to {}'.format(df_val['date_time'].min(), df_val['date_time'].max()))\n",
    "print('Train dates: {} to {}'.format(df_train['date_time'].min(), df_train['date_time'].max()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Transformacion de datos para Tensorflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "División del conjunto de datos en marcos de datos más pequeños\n",
    "\n",
    "Como se mencionó anteriormente, queremos pronosticar el Global_active_power que será de 10 minutos en el futuro.\n",
    "\n",
    "El siguiente gráfico visualiza el problema: usar los datos rezagados de $(t-n($ a $(t-1)$ para predecir el objetivo $(t + 10)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<figure>\n",
    "<center>\n",
    "<img src=\"../Imagenes/image-11.png\" width=\"600\" height=\"600\" align=\"center\"/>\n",
    "</center>\n",
    "</figure>\n",
    "\n",
    "Imagen tomada de: [3 Steps to Time Series Forecasting: LSTM with TensorFlow Keras\n",
    "](https://www.justintodata.com/forecast-time-series-lstm-with-tensorflow-keras/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "No es eficiente recorrer el conjunto de datos mientras se entrena el modelo. Además tensorflow espera los datos de la forma como vamos a organizarlos.\n",
    "\n",
    "Por eso, queremos transformar el conjunto de datos con cada fila que representa los datos históricos y el objetivo."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<figure>\n",
    "<center>\n",
    "<img src=\"../Imagenes/image-10.png\" width=\"400\" height=\"300\" align=\"center\"/>\n",
    "</center>\n",
    "</figure>\n",
    "\n",
    "Imagen tomada de: [3 Steps to Time Series Forecasting: LSTM with TensorFlow Keras\n",
    "](https://www.justintodata.com/forecast-time-series-lstm-with-tensorflow-keras/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "De esta manera, solo necesitamos entrenar el modelo usando cada fila de la matriz anterior.\n",
    "\n",
    "Ahora aquí vienen los desafíos:\n",
    "\n",
    "- ¿Cómo convertimos el conjunto de datos a la nueva estructura?\n",
    "- ¿Cómo manejamos esta nueva estructura de datos más grande cuando la memoria de nuestra computadora es limitada?\n",
    "\n",
    "\n",
    "Como resultado, se define la función *create_ts_files*:\n",
    "\n",
    "- para convertir el conjunto de datos original en el nuevo conjunto de datos anterior.\n",
    "- al mismo tiempo, para dividir el nuevo conjunto de datos en archivos más pequeños, lo que es más fácil de procesar."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clase create_ts_files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dentro de esta función, definimos los siguientes parámetros:\n",
    "\n",
    "- *start_index*: el momento más temprano en ser incluido en todos los datos históricos para la previsión.\n",
    "En esta práctica, queremos incluir el historial desde el principio, por lo que establecemos el valor predeterminado en 0.\n",
    "- end_index: el último tiempo que se incluirá en todos los datos históricos para la previsión.\n",
    "En esta práctica, queremos incluir todo el historial, por lo que establecemos el valor predeterminado en Ninguno.\n",
    "- *history_length*: esto se menciona anteriormente, que es el número de pasos de tiempo para mirar hacia atrás para cada pronóstico.\n",
    "- *step_size*: el paso de la ventana del historial.\n",
    "- *Global_active_power* no cambia rápidamente a lo largo del tiempo. Entonces, para ser más eficientes, podemos dejar step_size = 10. De esta manera, reducimos la muestra para usar cada 10 minutos de datos en el pasado para predecir la cantidad futura. Solo estamos mirando t-1, t-11, t-21 hasta t-n para predecir t + 10.\n",
    "- *target_step*: el número de períodos en el futuro para predecir.\n",
    "Como se mencionó anteriormente, estamos tratando de predecir la potencia_activa global con 10 minutos de anticipación. Entonces esta característica = 10.\n",
    "- *num_rows_per_file*: el número de registros para poner en cada archivo.\n",
    "\n",
    "\n",
    "Esto es necesario para dividir el nuevo conjunto de datos grande en archivos más pequeños.\n",
    "carpeta_datos: la única carpeta que contendrá todos los archivos.\n",
    "\n",
    "\n",
    "Al final, solo se necesita saber que esta función crea una carpeta con archivos y que cada archivo contiene un dataframe de pandas que se parece al nuevo conjunto de datos del gráfico anterior.\n",
    "Cada uno de estos dataframe tiene columnas:\n",
    "\n",
    "- $y$, que es el objetivo a predecir. Este será el valor en *t + target_step* ($t + 10$).\n",
    "- *x_lag {i}*, el valor en el tiempo *t + target_step - i* ($t + 10 - 11$, $t + 10 - 21$, y así sucesivamente), es decir, el valor rezagado en comparación con $y$.\n",
    "\n",
    "Al mismo tiempo, la función también devuelve el número de retrasos (len (col_names) -1) en los dataframes. Este número será necesario para definir la forma de los modelos de TensorFlow más adelante."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Goal of the model:\n",
    "#  Predict Global_active_power at a specified time in the future.\n",
    "#   Eg. We want to predict how much Global_active_power will be ten minutes from now.\n",
    "#       We can use all the values from t-1, t-2, t-3, .... t-history_length to predict t+10\n",
    "\n",
    "\n",
    "def create_ts_files(dataset, \n",
    "                    start_index, \n",
    "                    end_index, \n",
    "                    history_length, \n",
    "                    step_size, \n",
    "                    target_step, \n",
    "                    num_rows_per_file, \n",
    "                    data_folder):\n",
    "    assert step_size > 0\n",
    "    assert start_index >= 0\n",
    "    \n",
    "    if not os.path.exists(data_folder):\n",
    "        os.makedirs(data_folder)\n",
    "    \n",
    "    time_lags = sorted(range(target_step+1, target_step+history_length+1, step_size), reverse=True)\n",
    "    col_names = [f'x_lag{i}' for i in time_lags] + ['y']\n",
    "    start_index = start_index + history_length\n",
    "    if end_index is None:\n",
    "        end_index = len(dataset) - target_step\n",
    "    \n",
    "    rng = range(start_index, end_index)\n",
    "    num_rows = len(rng)\n",
    "    num_files = math.ceil(num_rows/num_rows_per_file)\n",
    "    \n",
    "    # for each file.\n",
    "    print(f'Creating {num_files} files.')\n",
    "    for i in range(num_files):\n",
    "        filename = f'{data_folder}/ts_file{i}.pkl'\n",
    "        \n",
    "        if i % 10 == 0:\n",
    "            print(f'{filename}')\n",
    "            \n",
    "        # get the start and end indices.\n",
    "        ind0 = i*num_rows_per_file\n",
    "        ind1 = min(ind0 + num_rows_per_file, end_index)\n",
    "        data_list = []\n",
    "        \n",
    "        # j in the current timestep. Will need j-n to j-1 for the history. And j + target_step for the target.\n",
    "        for j in range(ind0, ind1):\n",
    "            indices = range(j-1, j-history_length-1, -step_size)\n",
    "            data = dataset[sorted(indices) + [j+target_step]]\n",
    "            \n",
    "            # append data to the list.\n",
    "            data_list.append(data)\n",
    "\n",
    "        df_ts = pd.DataFrame(data=data_list, columns=col_names)\n",
    "        df_ts.to_pickle(filename)\n",
    "            \n",
    "    return len(col_names)-1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Antes de aplicar la función create_ts_files, también necesitamos:\n",
    "\n",
    "- escalar  *global_active_power* para trabajar con la red.\n",
    "- definir *n, history_length*, como 7 días (7 * 24 * 60 minutos).\n",
    "- definir *step_size* dentro de los datos históricos en 10 minutos.\n",
    "- establecer *target_step* en 10, de modo que estemos pronosticando *global_active_power* 10 minutos después de los datos históricos.\n",
    "\n",
    "Después de estos, aplicamos *create_ts_files* para:\n",
    "\n",
    "- Crear 158 archivos (cada uno con un marco de datos de pandas) dentro de la carpeta ts_data.\n",
    "- Devolver *num_timesteps* como el número de retrasos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "global_active_power = df_train['Global_active_power'].values\n",
    "\n",
    "# Scaled to work with Neural networks.\n",
    "scaler = MinMaxScaler(feature_range=(0, 1))\n",
    "global_active_power_scaled = scaler.fit_transform(global_active_power.reshape(-1, 1)).reshape(-1, )\n",
    "\n",
    "history_length = 7*24*60  # The history length in minutes.\n",
    "step_size = 10  # The sampling rate of the history. Eg. If step_size = 1, then values from every minute will be in the history.\n",
    "                #                                       If step size = 10 then values every 10 minutes will be in the history.\n",
    "target_step = 10  # The time step in the future to predict. Eg. If target_step = 0, then predict the next timestep after the end of the history period.\n",
    "                  #                                             If target_step = 10 then predict 10 timesteps the next timestep (11 minutes after the end of history).\n",
    "\n",
    "# The csv creation returns the number of rows and number of features. We need these values below.\n",
    "num_timesteps = create_ts_files(global_active_power_scaled,\n",
    "                                start_index=0,\n",
    "                                end_index=None,\n",
    "                                history_length=history_length,\n",
    "                                step_size=step_size,\n",
    "                                target_step=target_step,\n",
    "                                num_rows_per_file=128*100,\n",
    "                                data_folder='ts_data')\n",
    "\n",
    "# I found that the easiest way to do time series with tensorflow is by creating pandas files with the lagged time steps (eg. x{t-1}, x{t-2}...) and \n",
    "# the value to predict y = x{t+n}. We tried doing it using TFRecords, but that API is not very intuitive and lacks working examples for time series.\n",
    "# The resulting file using these parameters is over 17GB. If history_length is increased, or  step_size is decreased, it could get much bigger.\n",
    "# Hard to fit into laptop memory, so need to use other means to load the data from the hard drive.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La carpeta ts_data tiene alrededor de 16 GB y solo usamos los últimos 7 días de datos para predecir. ¡Ahora puede ver por qué es necesario dividir el conjunto de datos en dataframes más pequeños!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clase TimeSeriesLoader"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "En este procedimiento, creamos una clase *TimeSeriesLoader* para transformar y alimentar los *dataframe* en el modelo.\n",
    "\n",
    "Hay funciones integradas de Keras como Keras Sequence, tf.data API. Pero no son muy eficientes para este propósito.\n",
    "\n",
    "\n",
    "\n",
    "Dentro de esta clase, definimos:\n",
    "\n",
    "- __init__: la configuración inicial del objeto, que incluye:\n",
    "- *ts_folder*, que será *ts_data* que acabamos de crear.\n",
    "- *filename_format*, que es el formato de cadena de los nombres de archivo en *ts_folder*. Por ejemplo, cuando los archivos son *ts_file0.pkl, ts_file1.pkl,…, ts_file100.pkl*, el formato sería \"ts_file{}.pkl\".\n",
    "- *num_chunks*: el número total de archivos (fragmentos).\n",
    "- *get_chunk*: este método toma el dataframe de uno de los archivos y lo procesa para que esté listo para el entrenamiento.\n",
    "- *shuffle_chunks*: este método baraja el orden de los fragmentos que se devuelven en get_chunk. Esta es una buena práctica para modelar.\n",
    "\n",
    "\n",
    "Las definiciones pueden parecer un poco confusas. Pero siga leyendo, verá este objeto en acción en el siguiente paso."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#\n",
    "# So we can handle loading the data in chunks from the hard drive instead of having to load everything into memory.\n",
    "# \n",
    "# The reason we want to do this is so we can do custom processing on the data that we are feeding into the LSTM.\n",
    "# LSTM requires a certain shape and it is tricky to get it right.\n",
    "#\n",
    "class TimeSeriesLoader:\n",
    "    def __init__(self, ts_folder, filename_format):\n",
    "        self.ts_folder = ts_folder\n",
    "        \n",
    "        # find the number of files.\n",
    "        i = 0\n",
    "        file_found = True\n",
    "        while file_found:\n",
    "            filename = self.ts_folder + '/' + filename_format.format(i)\n",
    "            file_found = os.path.exists(filename)\n",
    "            if file_found:\n",
    "                i += 1\n",
    "                \n",
    "        self.num_files = i\n",
    "        self.files_indices = np.arange(self.num_files)\n",
    "        self.shuffle_chunks()\n",
    "        \n",
    "    def num_chunks(self):\n",
    "        return self.num_files\n",
    "    \n",
    "    def get_chunk(self, idx):\n",
    "        assert (idx >= 0) and (idx < self.num_files)\n",
    "        \n",
    "        ind = self.files_indices[idx]\n",
    "        filename = self.ts_folder + '/' + filename_format.format(ind)\n",
    "        df_ts = pd.read_pickle(filename)\n",
    "        num_records = len(df_ts.index)\n",
    "        \n",
    "        features = df_ts.drop('y', axis=1).values\n",
    "        target = df_ts['y'].values\n",
    "        \n",
    "        # reshape for input into LSTM. Batch major format.\n",
    "        features_batchmajor = np.array(features).reshape(num_records, -1, 1)\n",
    "        return features_batchmajor, target\n",
    "    \n",
    "    # this shuffles the order the chunks will be outputted from get_chunk.\n",
    "    def shuffle_chunks(self):\n",
    "        np.random.shuffle(self.files_indices)\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Después de definir, aplicamos este *TimeSeriesLoader* a la carpeta ts_data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts_folder = 'ts_data'\n",
    "filename_format = 'ts_file{}.pkl'\n",
    "tss = TimeSeriesLoader(ts_folder, filename_format)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ahora, con el objeto *tss* apunta a nuestro conjunto de datos, ¡finalmente estamos listos para LSTM!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Crea el modelo LSTM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La memoria a largo y corto plazo (LSTM) es una arquitectura de red neuronal recurrente artificial (RNN) utilizada en el campo del aprendizaje profundo.\n",
    "\n",
    "Las redes LSTM son adecuadas para clasificar, procesar y hacer predicciones basadas en datos de series de tiempo, ya que puede haber retrasos de duración desconocida entre eventos importantes en una serie de tiempo.\n",
    "\n",
    "\n",
    "Vamos a construir un modelo LSTM basado en la API funcional de *tf.keras\n",
    "\n",
    "\n",
    "Los procedimientos son los siguientes.\n",
    "\n",
    "- Definir la forma del dataset de entrada:\n",
    "    - *num_timesteps*, el número de retrasos en los marcos de datos que establecimos en la preparación de los datos.\n",
    "    - el número de series de tiempo como 1. Ya que solo estamos usando una variable de *global_active_power*.\n",
    "- Definir el número de unidades, 4 * unidades * (unidades + 2) es el número de parámetros del LSTM. Cuanto mayor sea el número, más parámetros habrá en el modelo.\n",
    "- Definir el *dropout*, que se utiliza para evitar el sobreajuste.\n",
    "- Especificar la capa de salida para que tenga una función de activación lineal. \n",
    "- Definir el modelo."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define el modelo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the Keras model.\n",
    "# Use hyperparameter optimization if you have the time.\n",
    "\n",
    "ts_inputs = tf.keras.Input(shape=(num_timesteps, 1))\n",
    "\n",
    "# units=10 -> The cell and hidden states will be of dimension 10.\n",
    "#             The number of parameters that need to be trained = 4*units*(units+2)\n",
    "x = layers.LSTM(units=10)(ts_inputs)\n",
    "x = layers.Dropout(0.2)(x)\n",
    "outputs = layers.Dense(1, activation='linear')(x)\n",
    "model = tf.keras.Model(inputs=ts_inputs, outputs=outputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Luego también definimos la función de optimización y la función de pérdida. Nuevamente, ajustar estos hiperparámetros para encontrar la mejor opción sería una mejor práctica."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compila"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specify the training configuration.\n",
    "model.compile(optimizer=tf.keras.optimizers.SGD(learning_rate=0.01),\n",
    "              loss=tf.keras.losses.MeanSquaredError(),\n",
    "              metrics=['mse'])\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Entrenamiento"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Entrenamos cada trozo en lotes y solo corremos durante una época. Idealmente, entrenaríamos para múltiples épocas para redes neuronales."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "\n",
    "# train in batch sizes of 128.\n",
    "BATCH_SIZE = 128\n",
    "NUM_EPOCHS = 1\n",
    "NUM_CHUNKS = tss.num_chunks()\n",
    "\n",
    "for epoch in range(NUM_EPOCHS):\n",
    "    print('epoch #{}'.format(epoch))\n",
    "    for i in range(NUM_CHUNKS):\n",
    "        X, y = tss.get_chunk(i)\n",
    "        \n",
    "        # model.fit does train the model incrementally. ie. Can call multiple times in batches.\n",
    "        # https://github.com/keras-team/keras/issues/4446\n",
    "        model.fit(x=X, y=y, batch_size=BATCH_SIZE)\n",
    "        \n",
    "    # shuffle the chunks so they're not in the same order next time around.\n",
    "    tss.shuffle_chunks()\n",
    "\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluación del modelo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Al igual que el conjunto de datos de entrenamiento, también creamos una carpeta de los datos de validación, que prepara el conjunto de datos de validación para el ajuste del modelo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "  # evaluate the model on the validation set.\n",
    "#\n",
    "# Create the validation CSV like we did before with the training.\n",
    "global_active_power_val = df_val['Global_active_power'].values\n",
    "global_active_power_val_scaled = scaler.transform(global_active_power_val.reshape(-1, 1)).reshape(-1, )\n",
    "\n",
    "history_length = 7*24*60  # The history length in minutes.\n",
    "step_size = 10  # The sampling rate of the history. Eg. If step_size = 1, then values from every minute will be in the history.\n",
    "                #                                       If step size = 10 then values every 10 minutes will be in the history.\n",
    "target_step = 10  # The time step in the future to predict. Eg. If target_step = 0, then predict the next timestep after the end of the history period.\n",
    "                  #                                             If target_step = 10 then predict 10 timesteps the next timestep (11 minutes after the end of history).\n",
    "\n",
    "# The csv creation returns the number of rows and number of features. We need these values below.\n",
    "num_timesteps = create_ts_files(global_active_power_val_scaled,\n",
    "                                start_index=0,\n",
    "                                end_index=None,\n",
    "                                history_length=history_length,\n",
    "                                step_size=step_size,\n",
    "                                target_step=target_step,\n",
    "                                num_rows_per_file=128*100,\n",
    "                                data_folder='ts_val_data')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Además de realizar pruebas con el conjunto de datos de validación, también realizamos pruebas con un modelo de línea de base utilizando solo el punto histórico más reciente (t + 10-11).\n",
    "\n",
    "El código detallado de Python se encuentra a continuación."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If we assume that the validation dataset can fit into memory we can do this.\n",
    "df_val_ts = pd.read_pickle('ts_val_data/ts_file0.pkl')\n",
    "\n",
    "\n",
    "features = df_val_ts.drop('y', axis=1).values\n",
    "features_arr = np.array(features)\n",
    "\n",
    "# reshape for input into LSTM. Batch major format.\n",
    "num_records = len(df_val_ts.index)\n",
    "features_batchmajor = features_arr.reshape(num_records, -1, 1)\n",
    "\n",
    "\n",
    "y_pred = model.predict(features_batchmajor).reshape(-1, )\n",
    "y_pred = scaler.inverse_transform(y_pred.reshape(-1, 1)).reshape(-1 ,)\n",
    "\n",
    "y_act = df_val_ts['y'].values\n",
    "y_act = scaler.inverse_transform(y_act.reshape(-1, 1)).reshape(-1 ,)\n",
    "\n",
    "print('validation mean squared error: {}'.format(mean_squared_error(y_act, y_pred)))\n",
    "\n",
    "#baseline\n",
    "y_pred_baseline = df_val_ts['x_lag11'].values\n",
    "y_pred_baseline = scaler.inverse_transform(y_pred_baseline.reshape(-1, 1)).reshape(-1 ,)\n",
    "print('validation baseline mean squared error: {}'.format(mean_squared_error(y_act, y_pred_baseline)))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El conjunto de datos de validación que utiliza LSTM da un error cuadrático medio (MSE) de 0,418. Mientras que el modelo de línea de base tiene un MSE de 0.428. El LSTM lo hace un poco mejor que la línea de base.\n",
    "\n",
    "Podríamos hacerlo mejor con el ajuste de hiperparámetros y más épocas. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [Regresar al inicio](#Contenido)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}