Add Reversible Instance Normalization to Torch Models

[alexcolpitts96]

Link to paper: https://openreview.net/forum?id=cGDAkQo1C0p

I have been implementing TSMixer (#1807) and noted that it was using RevIN (Reversible Instance Normalization).

I went to the original source paper to understand the purpose of it and it seems to obtain fantastic results as essentially a preprocessing step within the models themselves that helps address shifting distributions. The paper shows improved performance on Informer, N-BEATS, and SCINet (#1860).

Using my implementation and plugging it into N-HiTS, I was able to improve performance by a good marging using the Energy dataset.

with:
######################################
MSE: 0.2514766273597132
MAE: 0.334557913679719

without:
######################################
MSE: 0.30861193252719554
MAE: 0.41421373603271244

My main questions are around implementation. Can we add a wrapper around the forward method in the Torch-based models? Is there some other way to add this feature without having to edit a ton of the torch models?

[madtoinou]

Hi @alexcolpitts96,

Thanks for again bringing up state-of-the-art methods here, I really like the Table 3.

It would probably be possible to modify the forward() method in the PLForecastingModule class that would act as a wrapper to call RevIn in both "norm" and "denorm" modes on the input and output of the model.

RevIn seems be compatible with probabilistic forecasts out-of-the-box but some sanity checks should be performed.