Optimize UI for Merlion dashboard (#139)

* design changes * Add new assets to setup.py * Have file upload update the file selection. * Clean up dashboard code a bit. * Reset column selections if new file selected. * Add extra log to AutoML. * Update figures. * Use more flexible offsets in models & evaluators. * Update example notebooks. Co-authored-by: Aadyot Bhatnagar <abhatnagar@salesforce.com>
salesforce · Dec 10, 2022 · cc6e9a3 · cc6e9a3
1 parent e79f223
commit cc6e9a3
Show file tree

Hide file tree

Showing 55 changed files with 760 additions and 472 deletions.
diff --git a/README.md b/README.md
@@ -128,9 +128,9 @@ To use it, install Merlion with the optional ``dashboard`` dependency (i.e.
 You can view the dashboard at http://localhost:8050.
 Below, we show some screenshots of the dashboard for both anomaly detection and forecasting.
 
-![anomaly dashboard](https://github.com/salesforce/Merlion/raw/main/docs/source/_static/dashboard_anomaly.png)
+![anomaly dashboard](https://github.com/salesforce/Merlion/raw/main/figures/dashboard_anomaly.png)
 
-![forecast dashboard](https://github.com/salesforce/Merlion/raw/main/docs/source/_static/dashboard_forecast.png)
+![forecast dashboard](https://github.com/salesforce/Merlion/raw/main/figures/dashboard_forecast.png)
 
 To help you get started with using Merlion in your own code, we provide below some minimal examples using Merlion
 default models for both anomaly detection and forecasting.
@@ -187,7 +187,7 @@ print(f"Precision: {p:.4f}, Recall: {r:.4f}, F1: {f1:.4f}\n"
 ```
 ```
 Precision: 0.6667, Recall: 0.6667, F1: 0.6667
-Mean Time To Detect: 1 days 10:30:00
+Mean Time To Detect: 1 days 10:22:30
 ```
 ### Forecasting
 Here, we show the code to replicate the results from the forecasting dashboard above.

diff --git a/docs/source/_static/dashboard_anomaly.png b/docs/source/_static/dashboard_anomaly.png
diff --git a/docs/source/_static/dashboard_file.png b/docs/source/_static/dashboard_file.png
diff --git a/docs/source/_static/dashboard_forecast.png b/docs/source/_static/dashboard_forecast.png
diff --git a/docs/source/_static/figures b/docs/source/_static/figures
@@ -0,0 +1 @@
+../../../figures
diff --git a/docs/source/merlion.dashboard.rst b/docs/source/merlion.dashboard.rst
@@ -14,7 +14,7 @@ The app has three tabs. The first one is called "file manager" in which you can
 (the datasets will be stored in ``~/merlion/data``), check basic statistics of the datasets, visualize
 the time series data, or download a particular trained model:
 
-.. image:: _static/dashboard_file.png
+.. image:: _static/figures/dashboard_file.png
 
 You can click "Drag & Drop" to upload the file to the ``merlion`` folder (our app is designed to support
 docker deployment, so it doesn't allow to open a local file directly). If you use the app on a local
@@ -29,7 +29,7 @@ and click the download button. The model and its configuration file will be comp
 
 The second tab is used to train a time series anomaly detection model:
 
-.. image:: _static/dashboard_anomaly.png
+.. image:: _static/figures/dashboard_anomaly.png
 
 The app provides full support for these models, where you can choose different algorithms and set particular parameters
 according to your needs. To train a model, you need to:
@@ -56,7 +56,7 @@ label column.
 
 The third tab is used to train a time series forecasting model supported in Merlion:
 
-.. image:: _static/dashboard_forecast.png
+.. image:: _static/figures/dashboard_forecast.png
 
 The app provides full support for these models, where you can choose different algorithms and set particular parameters
 according to your needs. To train a model, you need to:

diff --git a/examples/TimeSeries.ipynb b/examples/TimeSeries.ipynb
@@ -174,6 +174,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
+      "time\n",
       "2020-03-02 09:12:00     667.118\n",
       "2020-03-02 09:13:00     611.751\n",
       "2020-03-02 09:14:00     599.456\n",
@@ -254,7 +255,7 @@
       "DatetimeIndex(['2020-03-02 09:12:00', '2020-03-02 09:13:00',\n",
       "               '2020-03-02 09:14:00', '2020-03-02 09:15:00',\n",
       "               '2020-03-02 09:16:00'],\n",
-      "              dtype='datetime64[ns]', freq=None)\n"
+      "              dtype='datetime64[ns]', name='time', freq=None)\n"
      ]
     }
    ],
@@ -326,6 +327,7 @@
       "\n",
       "kpi[1:5] = \n",
       "\n",
+      "time\n",
       "2020-03-02 09:13:00     611.751\n",
       "2020-03-02 09:14:00     599.456\n",
       "2020-03-02 09:15:00     621.446\n",
@@ -498,6 +500,7 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
+      "time\n",
       "2020-03-02 09:12:00     667.118\n",
       "2020-03-02 09:13:00     611.751\n",
       "2020-03-02 09:14:00     599.456\n",
@@ -511,6 +514,7 @@
       "2020-05-01 23:50:00     935.405\n",
       "Name: kpi_renamed, Length: 86807, dtype: float64\n",
       "\n",
+      "time\n",
       "2020-03-02 09:12:00    0.0\n",
       "2020-03-02 09:13:00    0.0\n",
       "2020-03-02 09:14:00    0.0\n",
@@ -544,6 +548,7 @@
      "output_type": "stream",
      "text": [
       "Univariate kpi_renamed\n",
+      "time\n",
       "2020-03-02 09:12:00     667.118\n",
       "2020-03-02 09:13:00     611.751\n",
       "2020-03-02 09:14:00     599.456\n",
@@ -558,6 +563,7 @@
       "Name: kpi_renamed, Length: 86807, dtype: float64\n",
       "\n",
       "Univariate kpi_label\n",
+      "time\n",
       "2020-03-02 09:12:00    0.0\n",
       "2020-03-02 09:13:00    0.0\n",
       "2020-03-02 09:14:00    0.0\n",
@@ -643,7 +649,7 @@
     {
      "data": {
       "text/plain": [
-       "(1583140320.0, (667.118, 0.0))"
+       "(1583140320.0, [667.118, 0.0])"
       ]
      },
      "execution_count": 23,
@@ -668,6 +674,7 @@
       "\n",
       "aligned[1:5] = \n",
       "                          kpi  kpi_label\n",
+      "time                                    \n",
       "2020-03-02 09:13:00   611.751        0.0\n",
       "2020-03-02 09:14:00   599.456        0.0\n",
       "2020-03-02 09:15:00   621.446        0.0\n",
@@ -752,7 +759,7 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "/Users/abhatnagar/Desktop/Merlion_public/merlion/utils/time_series.py:617: UserWarning: The univariates comprising this time series are not aligned (they have different time stamps). The length returned is equal to the length of the _union_ of all time stamps present in any of the univariates.\n",
+      "/Users/abhatnagar/Desktop/Merlion/merlion/utils/time_series.py:672: UserWarning: The univariates comprising this time series are not aligned (they have different time stamps). The length returned is equal to the length of the _union_ of all time stamps present in any of the univariates.\n",
       "  warnings.warn(warning)\n",
       "The univariates comprising this time series are not aligned (they have different time stamps). The length returned is equal to the length of the _union_ of all time stamps present in any of the univariates.\n"
      ]
@@ -842,6 +849,7 @@
      "data": {
       "text/plain": [
        "                          kpi  kpi_label\n",
+       "time                                    \n",
        "2020-03-05 12:00:00  1166.819        0.0\n",
        "2020-03-05 12:01:00  1345.504        0.0\n",
        "2020-03-05 12:02:00  1061.391        0.0\n",
@@ -875,6 +883,7 @@
      "data": {
       "text/plain": [
        "                          kpi  kpi_label\n",
+       "time                                    \n",
        "2020-03-02 09:12:00       NaN        0.0\n",
        "2020-03-02 09:13:00   611.751        0.0\n",
        "2020-03-02 09:14:00   599.456        0.0\n",
@@ -918,6 +927,7 @@
      "text": [
       "Left\n",
       "                          kpi  kpi_label\n",
+      "time                                    \n",
       "2020-03-02 09:12:00   667.118        0.0\n",
       "2020-03-02 09:13:00   611.751        0.0\n",
       "2020-03-02 09:14:00   599.456        0.0\n",
@@ -935,6 +945,7 @@
       "\n",
       "Right\n",
       "                          kpi  kpi_label\n",
+      "time                                    \n",
       "2020-05-01 00:00:00  1381.110        0.0\n",
       "2020-05-01 00:01:00  1807.039        0.0\n",
       "2020-05-01 00:02:00  1833.385        0.0\n",

diff --git a/examples/anomaly/0_AnomalyIntro.ipynb b/examples/anomaly/0_AnomalyIntro.ipynb
diff --git a/examples/anomaly/1_AnomalyFeatures.ipynb b/examples/anomaly/1_AnomalyFeatures.ipynb
diff --git a/examples/anomaly/3_AnomalyNewModel.ipynb b/examples/anomaly/3_AnomalyNewModel.ipynb
diff --git a/examples/forecast/0_ForecastIntro.ipynb b/examples/forecast/0_ForecastIntro.ipynb
diff --git a/examples/forecast/1_ForecastFeatures.ipynb b/examples/forecast/1_ForecastFeatures.ipynb
diff --git a/examples/forecast/3_ForecastExogenous.ipynb b/examples/forecast/3_ForecastExogenous.ipynb
@@ -297,7 +297,84 @@
    "execution_count": 2,
    "id": "f2ea8bed",
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "The earliest univariate starts at 2010-02-05 00:00:00, but the latest univariate starts at 2011-11-11 00:00:00, a difference of 644 days 00:00:00. This is more than 10% of the length of the shortest univariate (350 days 00:00:00). You may want to check that the univariates cover the same window of time.\n",
+      "Stack (most recent call last):\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py\", line 197, in _run_module_as_main\n",
+      "    return _run_code(code, main_globals, None,\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py\", line 87, in _run_code\n",
+      "    exec(code, run_globals)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/ipykernel_launcher.py\", line 16, in <module>\n",
+      "    app.launch_new_instance()\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/traitlets/config/application.py\", line 845, in launch_instance\n",
+      "    app.start()\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/ipykernel/kernelapp.py\", line 612, in start\n",
+      "    self.io_loop.start()\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/platform/asyncio.py\", line 199, in start\n",
+      "    self.asyncio_loop.run_forever()\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py\", line 596, in run_forever\n",
+      "    self._run_once()\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py\", line 1890, in _run_once\n",
+      "    handle._run()\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/events.py\", line 80, in _run\n",
+      "    self._context.run(self._callback, *self._args)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/ioloop.py\", line 688, in <lambda>\n",
+      "    lambda f: self._run_callback(functools.partial(callback, future))\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/ioloop.py\", line 741, in _run_callback\n",
+      "    ret = callback()\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/gen.py\", line 814, in inner\n",
+      "    self.ctx_run(self.run)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/gen.py\", line 775, in run\n",
+      "    yielded = self.gen.send(value)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/ipykernel/kernelbase.py\", line 374, in dispatch_queue\n",
+      "    yield self.process_one()\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/gen.py\", line 250, in wrapper\n",
+      "    runner = Runner(ctx_run, result, future, yielded)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/gen.py\", line 741, in __init__\n",
+      "    self.ctx_run(self.run)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/gen.py\", line 775, in run\n",
+      "    yielded = self.gen.send(value)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/ipykernel/kernelbase.py\", line 358, in process_one\n",
+      "    yield gen.maybe_future(dispatch(*args))\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/gen.py\", line 234, in wrapper\n",
+      "    yielded = ctx_run(next, result)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/ipykernel/kernelbase.py\", line 261, in dispatch_shell\n",
+      "    yield gen.maybe_future(handler(stream, idents, msg))\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/gen.py\", line 234, in wrapper\n",
+      "    yielded = ctx_run(next, result)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/ipykernel/kernelbase.py\", line 536, in execute_request\n",
+      "    self.do_execute(\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tornado/gen.py\", line 234, in wrapper\n",
+      "    yielded = ctx_run(next, result)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/ipykernel/ipkernel.py\", line 302, in do_execute\n",
+      "    res = shell.run_cell(code, store_history=store_history, silent=silent)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/ipykernel/zmqshell.py\", line 539, in run_cell\n",
+      "    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/IPython/core/interactiveshell.py\", line 2898, in run_cell\n",
+      "    result = self._run_cell(\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/IPython/core/interactiveshell.py\", line 2944, in _run_cell\n",
+      "    return runner(coro)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/IPython/core/async_helpers.py\", line 68, in _pseudo_sync_runner\n",
+      "    coro.send(None)\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/IPython/core/interactiveshell.py\", line 3169, in run_cell_async\n",
+      "    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/IPython/core/interactiveshell.py\", line 3361, in run_ast_nodes\n",
+      "    if (await self.run_code(code, result,  async_=asy)):\n",
+      "  File \"/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/IPython/core/interactiveshell.py\", line 3441, in run_code\n",
+      "    exec(code_obj, self.user_global_ns, self.user_ns)\n",
+      "  File \"<ipython-input-2-f4b6cbd5939f>\", line 9, in <module>\n",
+      "    exog = TimeSeries.from_pd(ts[[\"IsHoliday\", \"MarkDown1\", \"MarkDown2\", \"MarkDown3\", \"MarkDown4\", \"MarkDown5\"]])\n",
+      "  File \"/Users/abhatnagar/Desktop/Merlion/merlion/utils/time_series.py\", line 794, in from_pd\n",
+      "    return cls(df=df, freq=freq, check_aligned=check_aligned)\n",
+      "  File \"/Users/abhatnagar/Desktop/Merlion/merlion/utils/time_series.py\", line 493, in __init__\n",
+      "    logger.warning(\n"
+     ]
+    }
+   ],
    "source": [
     "from merlion.utils import TimeSeries\n",
     "\n",
@@ -332,10 +409,10 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "13:37:26 - cmdstanpy - INFO - Chain [1] start processing\n",
-      "13:37:26 - cmdstanpy - INFO - Chain [1] done processing\n",
-      "13:37:27 - cmdstanpy - INFO - Chain [1] start processing\n",
-      "13:37:27 - cmdstanpy - INFO - Chain [1] done processing\n"
+      "17:50:59 - cmdstanpy - INFO - Chain [1] start processing\n",
+      "17:50:59 - cmdstanpy - INFO - Chain [1] done processing\n",
+      "17:50:59 - cmdstanpy - INFO - Chain [1] start processing\n",
+      "17:50:59 - cmdstanpy - INFO - Chain [1] done processing\n"
      ]
     },
     {
@@ -409,7 +486,7 @@
        "      <th>MarkDown5</th>\n",
        "    </tr>\n",
        "    <tr>\n",
-       "      <th>time</th>\n",
+       "      <th>Date</th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
        "      <th></th>\n",
@@ -421,7 +498,7 @@
        "  <tbody>\n",
        "    <tr>\n",
        "      <th>2010-02-05</th>\n",
-       "      <td>0.0</td>\n",
+       "      <td>False</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
@@ -430,7 +507,7 @@
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2010-02-12</th>\n",
-       "      <td>1.0</td>\n",
+       "      <td>True</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
@@ -439,7 +516,7 @@
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2010-02-19</th>\n",
-       "      <td>0.0</td>\n",
+       "      <td>False</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
@@ -448,7 +525,7 @@
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2010-02-26</th>\n",
-       "      <td>0.0</td>\n",
+       "      <td>False</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
@@ -457,7 +534,7 @@
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2010-03-05</th>\n",
-       "      <td>0.0</td>\n",
+       "      <td>False</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
        "      <td>NaN</td>\n",
@@ -475,7 +552,7 @@
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2012-09-28</th>\n",
-       "      <td>0.0</td>\n",
+       "      <td>False</td>\n",
        "      <td>7106.05</td>\n",
        "      <td>1.91</td>\n",
        "      <td>1.65</td>\n",
@@ -484,7 +561,7 @@
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2012-10-05</th>\n",
-       "      <td>0.0</td>\n",
+       "      <td>False</td>\n",
        "      <td>6037.76</td>\n",
        "      <td>NaN</td>\n",
        "      <td>10.04</td>\n",
@@ -493,7 +570,7 @@
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2012-10-12</th>\n",
-       "      <td>0.0</td>\n",
+       "      <td>False</td>\n",
        "      <td>2145.50</td>\n",
        "      <td>NaN</td>\n",
        "      <td>33.31</td>\n",
@@ -502,7 +579,7 @@
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2012-10-19</th>\n",
-       "      <td>0.0</td>\n",
+       "      <td>False</td>\n",
        "      <td>4461.89</td>\n",
        "      <td>NaN</td>\n",
        "      <td>1.14</td>\n",
@@ -511,7 +588,7 @@
        "    </tr>\n",
        "    <tr>\n",
        "      <th>2012-10-26</th>\n",
-       "      <td>0.0</td>\n",
+       "      <td>False</td>\n",
        "      <td>6152.59</td>\n",
        "      <td>129.77</td>\n",
        "      <td>200.00</td>\n",
@@ -525,18 +602,18 @@
       ],
       "text/plain": [
        "            IsHoliday  MarkDown1  MarkDown2  MarkDown3  MarkDown4  MarkDown5\n",
-       "time                                                                        \n",
-       "2010-02-05        0.0        NaN        NaN        NaN        NaN        NaN\n",
-       "2010-02-12        1.0        NaN        NaN        NaN        NaN        NaN\n",
-       "2010-02-19        0.0        NaN        NaN        NaN        NaN        NaN\n",
-       "2010-02-26        0.0        NaN        NaN        NaN        NaN        NaN\n",
-       "2010-03-05        0.0        NaN        NaN        NaN        NaN        NaN\n",
+       "Date                                                                        \n",
+       "2010-02-05      False        NaN        NaN        NaN        NaN        NaN\n",
+       "2010-02-12       True        NaN        NaN        NaN        NaN        NaN\n",
+       "2010-02-19      False        NaN        NaN        NaN        NaN        NaN\n",
+       "2010-02-26      False        NaN        NaN        NaN        NaN        NaN\n",
+       "2010-03-05      False        NaN        NaN        NaN        NaN        NaN\n",
        "...               ...        ...        ...        ...        ...        ...\n",
-       "2012-09-28        0.0    7106.05       1.91       1.65    1549.10    3946.03\n",
-       "2012-10-05        0.0    6037.76        NaN      10.04    3027.37    3853.40\n",
-       "2012-10-12        0.0    2145.50        NaN      33.31     586.83   10421.01\n",
-       "2012-10-19        0.0    4461.89        NaN       1.14    1579.67    2642.29\n",
-       "2012-10-26        0.0    6152.59     129.77     200.00     272.29    2924.15\n",
+       "2012-09-28      False    7106.05       1.91       1.65    1549.10    3946.03\n",
+       "2012-10-05      False    6037.76        NaN      10.04    3027.37    3853.40\n",
+       "2012-10-12      False    2145.50        NaN      33.31     586.83   10421.01\n",
+       "2012-10-19      False    4461.89        NaN       1.14    1579.67    2642.29\n",
+       "2012-10-26      False    6152.59     129.77     200.00     272.29    2924.15\n",
        "\n",
        "[143 rows x 6 columns]"
       ]