Time Series Extrinsic Regression

This repository contains the source code for Time Series Extrinsic Regression (TSER). We aim to learn the relationship between a time series and a scalar value. We note the similarity with a parallel field in the Statistics community known as Scalar-on-Function-Regression (SoFR) and is working on implementing those methods.

Data

The archive containing 19 time series regression datasets can be found at Monash UEA UCR Time Series Extrinsic Regression Archive. We recommend you to read the paper for an overview of the datasets and their sources.

The data folder contains the actual feature definitions for each data set, as well as a sample data set that can be used for demo purposes.

Models

The following models are implemented in this repository:

Classical ML models

1. Support Vector Regression (SVR) - A wrapper function for sklearn SVR
2. Random Forest Regressor (RF) - A wrapper function for sklearn RandomForestRegressor
3. XGBoost (XGB) - A wrapper function for XGBoost package
4. Linear Regression (LR) - A wrapper function for sklearn LinearRegression
5. Ridge Regression (Ridge) - A wrapper function for sklearn RidgeCV

Deep Learning for TSC

1. Fully Convolutional Network (FCN)
2. Residual Network (ResNet)
3. Inception Time (InceptionTime)

TSC models

1. Random Convolutional Kernels Transform (Rocket)

Features Transform

Some simple feature transformation have been implemented.

1. Principal Component Analysis (PCA)
2. Functional Principal Component Analysis (FPCA)
3. FPCA with BSpline Smoothing (FPCA-BSpline)

Code

The code is mainly divided as follows:

• The demo.py file contains demo code for a single experiment run.

Arguments:
-d --data_path      : path to dataset
-p --problem        : dataset name
-r --regressor      : name of the model
-t --transformer    : name of the transformer
-i --iteration      : iteration number
-n --normalisation  : normalisation (none, standard, minmax)

• The run_experiments file contains code to run a set of experiments.
• The models folder contains the models used for regression.
• The utils folder contains helper functions for the program.
• After each run, the results will be saved to the output folder.

Dependencies

All python packages needed are listed in requirements.txt file and can be installed simply using the pip command.

Some of the main packages are:

• keras
• matplotlib
• numba
• numpy
• pandas
• scikit-fda
• sklearn
• scipy
• tqdm
• xgboost

Results

These are the results on the 19 Time series regression datasets from Monash UEA UCR Time Series Regression Archive. The initial benchmark results in the paper showed that a simple linear model such as Rocket performs best for the time series regression task. The full results can be obtained here.

Reference

If you use any part of this work, please cite:

@article{
  Tan2020TSER,
  title={Time Series Extrinsic Regression}, 
  author={Tan, Chang Wei and Bergmeir, Christoph and Petitjean, Francois and Webb, Geoffrey I},
  journal={Data Mining and Knowledge Discovery},
  pages={1--29},
  year={2021},
  publisher={Springer},
  doi={https://doi.org/10.1007/s10618-021-00745-9}
}

Acknowledgement

We appreciate the data donation from all the donors. We would also like to thank Hassan Fawaz for providing the base code for Deep Learning models and Angus Dempster for providing the code for Rocket.