Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New abstractions for rolling-window forecasters. #127

Merged
merged 19 commits into from
Oct 19, 2022

Conversation

Emerald01
Copy link
Collaborator

@Emerald01 Emerald01 commented Oct 14, 2022

This commit introduces new abstractions for sklearn-style models (i.e. they support fit() and predict()) which predict future values based on a fixed historical window.

  1. RollingWindowDataset supports __iter__ and __getitem__ which process data to the required (past, future) format
  2. SKLearnForecaster(ForecasterBase) handles (1) univariate: the hybrid (multioutput + autoregression) train and forecast (2) multivariate with prediction_stride = 1: autoregression train and forecast (3) multivariate with prediction_stride > 1: set prediction_stride = max_forecast_steps, and directly return the model's prediction of the target univariate for the next max_forecast_steps timestamps
  3. minor: remove sample_mode, I feel the stats sampling mode is never used and complicates the interface
  4. Unittests pass for all these modes mentioned, performance-wise and result-wise, same as before,
    for example, python test_boostingtrees yields the following logs:
2022-10-19 14:22:30,957 (resample:144) WARNING: Inferred granularity 0 days 01:00:00
2022-10-19 14:22:31,282 (resample:144) WARNING: Inferred granularity 0 days 01:00:00
2022-10-19 14:22:31,570 (resample:144) WARNING: Inferred granularity 0 days 01:00:00
--------------------------------------------------------------------------------
2022-10-19 14:22:31,736 (test_boostingtrees:83) INFO: test_forecast_multi_autoregression
--------------------------------------------------------------------------------
2022-10-19 14:22:31,853 (sklearn_base:124) INFO: Model is working on a multivariate dataset with prediction_stride = 1, model will be trained to autoregressively predict all univariates.
2022-10-19 14:22:36,791 (test_boostingtrees:89) INFO: Immediate forecast RMSE: 2.95
2022-10-19 14:22:36,836 (test_boostingtrees:97) INFO: Look-ahead RMSE with time_series_prev: 18.95
.--------------------------------------------------------------------------------
2022-10-19 14:22:36,921 (test_boostingtrees:109) INFO: test_forecast_multi_seq2seq
--------------------------------------------------------------------------------
2022-10-19 14:22:37,030 (sklearn_base:130) INFO: Model is working on a multivariate dataset with prediction_stride > 1. Model will directly forecast the target univariate for the next 2 timestamps.
2022-10-19 14:22:40,525 (test_boostingtrees:115) INFO: Immediate forecast RMSE: 3.61
2022-10-19 14:22:40,567 (test_boostingtrees:123) INFO: Look-ahead RMSE with time_series_prev: 19.55
.--------------------------------------------------------------------------------
2022-10-19 14:22:40,627 (test_boostingtrees:135) INFO: test_forecast_uni
--------------------------------------------------------------------------------
2022-10-19 14:22:40,675 (sklearn_base:118) INFO: Model is working on a univariate dataset, hybrid of sequence and autoregression training strategy will be adopted with prediction_stride = 2.
2022-10-19 14:22:43,828 (test_boostingtrees:142) INFO: Immediate forecast RMSE: 1.47
2022-10-19 14:22:43,854 (test_boostingtrees:150) INFO: Look-ahead RMSE with time_series_prev: 17.27

@salesforce-cla
Copy link

Thanks for the contribution! Unfortunately we can't verify the commit author(s): Tian Lan <t***@t***.i***.s***.com>. One possible solution is to add that email to your GitHub account. Alternatively you can change your commits to another email and force push the change. After getting your commits associated with your GitHub account, sign the Salesforce.com Contributor License Agreement and this Pull Request will be revalidated.

@aadyotb aadyotb requested a review from yihaocs October 14, 2022 18:25
@Emerald01 Emerald01 closed this Oct 16, 2022
@Emerald01 Emerald01 reopened this Oct 16, 2022
Copy link
Contributor

@aadyotb aadyotb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the effort Tian! In addition to the changes mentioned inline, can you update the docs/source folder to ensure that the new files are included in the API doc?

merlion/models/forecast/trees.py Outdated Show resolved Hide resolved
tests/forecast/test_boostingtrees.py Outdated Show resolved Hide resolved
merlion/models/utils/rolling_window_dataset.py Outdated Show resolved Hide resolved
merlion/models/utils/rolling_window_dataset.py Outdated Show resolved Hide resolved
merlion/models/utils/rolling_window_dataset.py Outdated Show resolved Hide resolved
merlion/models/forecast/autoregressivebase.py Outdated Show resolved Hide resolved
merlion/models/forecast/autoregressivebase.py Outdated Show resolved Hide resolved
merlion/models/forecast/autoregressivebase.py Outdated Show resolved Hide resolved
merlion/models/forecast/autoregressivebase.py Outdated Show resolved Hide resolved
merlion/models/forecast/trees.py Outdated Show resolved Hide resolved
merlion/models/utils/rolling_window_dataset.py Outdated Show resolved Hide resolved
merlion/models/utils/rolling_window_dataset.py Outdated Show resolved Hide resolved
merlion/models/utils/rolling_window_dataset.py Outdated Show resolved Hide resolved
@aadyotb aadyotb changed the title Tian dev tree autoregressor v2 New abstractions for rolling-window forecasters. Oct 18, 2022
@aadyotb aadyotb force-pushed the tian-dev-tree-autoregressor-v2 branch from cd66e44 to 3dfaa25 Compare October 19, 2022 21:18
@aadyotb aadyotb merged commit 65b273d into main Oct 19, 2022
@aadyotb aadyotb deleted the tian-dev-tree-autoregressor-v2 branch October 19, 2022 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants