fix historical forecasts retraining of TFMs

[dennisbader]

Fixes #1461.

Summary

Currently historical_forecasts trains the same model object multiple times. For TorchForecastingModels this an issue as instead of training a fresh model from scratch for each iteration, we are rather "fine-tuning" the existing model.

• trains a model from scratch when retrain=True

Important things to consider
Below are some changes to the current behavior. IMO these make sense, but let me know if think of better approaches:

• currently, calling historical_forecasts (HF) on an already fitted model with retrain=True will raise an error if input dimensions/covariates/... do not match the input from initial training -> this will not be raised with this PR as we train new model from scratch
• currently, the model instance self gets updated (with fit calls) in each iteration of HF using retrain=True -> with this PR, the instance gets only updated in the first iteration if it has not been fit before. (?) Do we even want to "mess" with self at all?

[codecov-commenter]

Codecov Report

Base: 93.95% // Head: 93.91% // Decreases project coverage by -0.03% ⚠️

Coverage data is based on head (bf370ed) compared to base (ed83ff8).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1465      +/-   ##
==========================================
- Coverage   93.95%   93.91%   -0.04%     
==========================================
  Files         122      122              
  Lines       10728    10730       +2     
==========================================
- Hits        10079    10077       -2     
- Misses        649      653       +4     

Impacted Files	Coverage Δ
darts/models/forecasting/forecasting_model.py	97.06% <100.00%> (+0.01%)	⬆️
darts/utils/statistics.py	88.65% <0.00%> (-1.04%)	⬇️
darts/timeseries.py	91.78% <0.00%> (-0.23%)	⬇️
darts/ad/anomaly_model/filtering_am.py	91.93% <0.00%> (-0.13%)	⬇️
...arts/models/forecasting/torch_forecasting_model.py	89.52% <0.00%> (-0.05%)	⬇️
darts/models/forecasting/block_rnn_model.py	98.24% <0.00%> (-0.04%)	⬇️
darts/models/forecasting/nhits.py	99.27% <0.00%> (-0.01%)	⬇️
darts/datasets/__init__.py	100.00% <0.00%> (ø)
darts/utils/data/tabularization.py	100.00% <0.00%> (ø)
darts/models/forecasting/regression_model.py	97.32% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[hrzn]

Nice, thanks!!
I would say we shouldn't mess at all anymore with self :)
There's an error in the quickstart notebook now, could you have a look?

[hrzn]

I'm thinking we should always call untrained_model() even for the first pass here, WDYT?

[solalatus]

Extremely good that you clarified this, Kudos!
Question: would it not be good also to have a "finetune" behavior?
I am right now trying to do a finetuning style transfer learning approach, and this would come in handy.

[solalatus]

I recon, this would entail some decisions about what to do with the optimizer state, the LR_schedule,...

I am struggling right now to do a reset of these without touching the model weights. I can give some input if someone is willing to try a less hacky way then me.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[dennisbader]

As discussed, leaving the model object unchanged in historical_forecasts() brought some necessary changes with it. See for example the quickstart notebook.

[dennisbader]

Extremely good that you clarified this, Kudos! Question: would it not be good also to have a "finetune" behavior? I am right now trying to do a finetuning style transfer learning approach, and this would come in handy.

Hi @solalatus. I believe the fine tuning would make the historical_forecasts too complex. Also this would only be supported by our TorchForecastingModel (TFM). I would rather opt for making it easier to fine-tune our TFMs in general.

IMO it's best keep this separate, but let me know if I'm missing something

[solalatus]

Absolutely agree. I was thinking about some different function then the standard historical forecast. In fact, as I tried to think it over in this "issue" a reinit feature and the normal fit would suffice.

I hacked together a really ugly solution you can see in this gist, any feedback and thoughts are much appreciated!

[hrzn]

LGTM, thanks @dennisbader