add iTransformer multivariate forecaster

[kashif]

Implemented the probabilistic iTransformer model from the paper: https://arxiv.org/pdf/2310.06625.pdf

[kashif]

@lostella I realize that the input_size is not really needed and we should not have it... this will allow us to train the same model on different-sized multivariate dim potentially... I use input_size to reshape things but perhaps we can find another way.

[lostella]

@kashif yes, if I read the code correctly you should be able to set input_size = past_target.shape[2]

[lostella]

I think this examples folder we should remove eventually. How about keeping his notebook close to the model? In its subpackage I mean. Pip install should anyway ignore it

[kashif]

this example folder is nice as its the only one with a multivariate model (showing the grouping and evaluation) as well as plotting some of the dims of the multivariate...

[lostella]

@kashif maybe for tests, we introduce a test_multivariate_estimators.py module?

[kashif]

ready!

[lostella]

Thanks @kashif! 🚀

[sirdawar]

Trying to use this sample with my own dataset, now when trying to use estimator I get following error:
GluonTSDataError: Input for field "target" does not have the requireddimension (field: target, ndim observed: 1, expected ndim: 2).

Where does it infer to have target dim of 2 and how to change that?

[kashif]

you cannot change it as iTransformers is an inherently multivariate model... so the dataset has to be of shape: [batch_size, context_length, multivariate_Dim > 1]

[kashif]

recall that the model takes a rep. of variate, makes a set of vectors out of them, and passes them to an unmasked transformer. The transformer as you know is a set-equivariant layer; since there is no ordering of the variates in a multivariate dataset, this layer makes perfect sense. However, iTransformer doesn't make sense in the univariate setting as there would be just a single vector as input to the transformer, which is then just an MLP...

[sirdawar]

Thanks! One more thing, how to define the training set length, I am wondering if dataloader is sampling it somehow, since no matter how long time-series I put training time is same.

[kashif]

right that is because while training a shared (global) deep learning models where the sizes of the time series can vary, the scheme is to randomly sample some fixed context-length window and subsequent prediction-length window within the training dataset... this way over time the model will learn via all the data. Use the context_length argument to the estimator to set this

[sirdawar]

Okay, and how to make it learn more over time?

[kashif]

do more steps of gradient descent and hope it doesn't overfit...

[sirdawar]

Thanks so far! Training works, well but I get error when I want to evaluate it in the step

forecasts = list(forecast_it)
targets = list(ts_it)

IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

I am thinking that the way I create dataset might be wrong. For example my dataset goes like

`

Date | ObjectName | Consumption_Elec | Temperature | Consumption_Heat -- | -- | -- | -- | --

2018-01-01 01:00:00 | Object1 | 17.16 | 0.0 | 0.1155
2018-01-01 01:00:00 | Object1 | 17.16 | 0.0 | 0.2100
...

`

And I am trying to create a dataset by
PandasDataset.from_long_dataframe(train_df, target=["Consumption_Heat","Consumption_Elec"], item_id="ObjectName",feat_dynamic_real='Temperature',timestamp="Date",freq=freq,unchecked=True)