[FEATURE] Include val_series in historical_forecasts if retrain=True

[tuomijal]

Is your feature request related to a current problem? Please describe.
Currently historical_forecasts function's retrain parameter only generates train_series in the cross-validation loop. This is a problem if we want to monitor val_loss as well which is a common scenario.

Describe proposed solution
Include the possiblity to generate val_series in cross-validation loop as well.

[madtoinou]

Hi @tuomijal, thank you for this feature request.

How do you expect the validation series to be created? Extract m timestamps after the end of the training series? How do you expect this to behave when getting closer to the end of the series (the last historical forecasts iterations) ; reducing the size of the validation series, not retraining the model if there is not enough timestamps for it (using the last version) or not trying to forecasts the end of the series?

Would you be willing to implement this feature?

[tuomijal]

Hi @madtoinou, apologies for the delay and thank you for continued support. I have been thinking about this and come to the conclusion that this is not a priority. I can give this more thought and effort if wider demand for the feature appears. We can table this until then.