Merge branch 'master' into issue-1329

CHANGELOG.md

@@ -10,21 +10,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `DeseasonalityTransform` ([#1307](https://github.com/tinkoff-ai/etna/pull/1307))
 - 
 - Add extension with models from `statsforecast`: `StatsForecastARIMAModel`, `StatsForecastAutoARIMAModel`, `StatsForecastAutoCESModel`, `StatsForecastAutoETSModel`, `StatsForecastAutoThetaModel` ([#1295](https://github.com/tinkoff-ai/etna/pull/1297))
+- Notebook `feature_selection` ([#875](https://github.com/tinkoff-ai/etna/pull/875))
 - 
-- 
+- Implementation of PatchTS model ([#1277](https://github.com/tinkoff-ai/etna/pull/1277))
 
 ### Changed
 - 
 - 
-- 
-- 
+- Unify errors, warnings and checks in models ([#1312](https://github.com/tinkoff-ai/etna/pull/1312))
+- Remove upper limitation on version of numba ([#1321](https://github.com/tinkoff-ai/etna/pull/1321))
 
 ### Fixed
 - Pipeline ensembles fail in `etna forecast` CLI ([#1331](https://github.com/tinkoff-ai/etna/pull/1331))
 - 
-- 
+- Fix performance of `DeepARModel` and `TFTModel` ([#1322](https://github.com/tinkoff-ai/etna/pull/1322))
 - `mrmr` feature selection working with categoricals ([#1311](https://github.com/tinkoff-ai/etna/pull/1311))
-- 
+- Fix version of `statsforecast` to 1.4 to avoid dependency conflicts during installation ([#1313](https://github.com/tinkoff-ai/etna/pull/1313))
+- Add inverse transformation into `predict` method of pipelines ([#1314](https://github.com/tinkoff-ai/etna/pull/1314))
 
 ### Removed
 - Building docker images with cuda 10.2 ([#1306](https://github.com/tinkoff-ai/etna/pull/1306))

README.md

@@ -175,15 +175,19 @@ We have also prepared a set of tutorials for an easy introduction:
 | [Get started](https://github.com/tinkoff-ai/etna/tree/master/examples/get_started.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/get_started.ipynb) |
 | [Backtest](https://github.com/tinkoff-ai/etna/tree/master/examples/backtest.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/backtest.ipynb) |
 | [EDA](https://github.com/tinkoff-ai/etna/tree/master/examples/EDA.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/EDA.ipynb) |
-| [Outliers](https://github.com/tinkoff-ai/etna/tree/master/examples/outliers.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/outliers.ipynb) |
-| [Clustering](https://github.com/tinkoff-ai/etna/tree/master/examples/clustering.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/clustering.ipynb) |
+| [Regressors and exogenous data](https://github.com/tinkoff-ai/etna/tree/master/examples/exogenous_data.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/exogenous_data.ipynb) |
+| [Custom model and transform](https://github.com/tinkoff-ai/etna/tree/master/examples/custom_transform_and_model.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/custom_transform_and_model.ipynb) |
 | [Deep learning models](https://github.com/tinkoff-ai/etna/tree/master/examples/NN_examples.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/NN_examples.ipynb) |
 | [Ensembles](https://github.com/tinkoff-ai/etna/tree/master/examples/ensembles.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/ensembles.ipynb) |
-| [Custom Transform and Model](https://github.com/tinkoff-ai/etna/tree/master/examples/custom_transform_and_model.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/custom_transform_and_model.ipynb) |
-| [Exogenous data](https://github.com/tinkoff-ai/etna/tree/master/examples/exogenous_data.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/exogenous_data.ipynb) |
-| [Forecasting strategies](https://github.com/tinkoff-ai/etna/blob/master/examples/forecasting_strategies.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/forecasting_strategies.ipynb) |
-| [Classification](https://github.com/tinkoff-ai/etna/blob/master/examples/classification.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/classification.ipynb) |
+| [Outliers](https://github.com/tinkoff-ai/etna/tree/master/examples/outliers.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/outliers.ipynb) |
+| [Forecasting strategies](https://github.com/tinkoff-ai/etna/tree/master/examples/forecasting_strategies.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/forecasting_strategies.ipynb) |
+| [Forecast interpretation](https://github.com/tinkoff-ai/etna/tree/master/examples/forecast_interpretation.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/forecast_interpretation.ipynb) |
+| [Clustering](https://github.com/tinkoff-ai/etna/tree/master/examples/clustering.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/clustering.ipynb) |
+| [AutoML](https://github.com/tinkoff-ai/etna/tree/master/examples/automl.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/automl.ipynb) |
+| [Inference: using saved pipeline on a new data](https://github.com/tinkoff-ai/etna/tree/master/examples/inference.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/inference.ipynb) |
 | [Hierarchical time series](https://github.com/tinkoff-ai/etna/blob/master/examples/hierarchical_pipeline.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/hierarchical_pipeline.ipynb) |
+| [Classification](https://github.com/tinkoff-ai/etna/blob/master/examples/classification.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/classification.ipynb) |
+| [Feature selection](https://github.com/tinkoff-ai/etna/blob/master/examples/feature_selection.ipynb) | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tinkoff-ai/etna/master?filepath=examples/feature_selection.ipynb) |
 
 ## Documentation
 

etna/analysis/feature_relevance/relevance_table.py

@@ -36,6 +36,7 @@ def _prepare_df(df: pd.DataFrame, df_exog: pd.DataFrame, segment: str, regressor
 
 def get_statistics_relevance_table(df: pd.DataFrame, df_exog: pd.DataFrame) -> pd.DataFrame:
  """Calculate relevance table with p-values from tsfresh.
+
  Parameters
  ----------
  df:

etna/models/deadline_ma.py

@@ -91,6 +91,15 @@ def get_model(self) -> "DeadlineMovingAverageModel":
  """
  return self
 
+ def _check_not_used_columns(self, ts: TSDataset):
+ columns = set(ts.columns.get_level_values("feature"))
+ columns_not_used = columns.difference({"target"})
+ if columns_not_used:
+ warnings.warn(
+ message=f"This model doesn't work with exogenous features. "
+ f"Columns {columns_not_used} won't be used."
+ )
+
  def fit(self, ts: TSDataset) -> "DeadlineMovingAverageModel":
  """Fit model.
 
@@ -109,14 +118,8 @@ def fit(self, ts: TSDataset) -> "DeadlineMovingAverageModel":
  if freq not in self._freqs_available:
  raise ValueError(f"Freq {freq} is not supported! Use daily or hourly frequency!")
 
+ self._check_not_used_columns(ts)
  self._freq = freq
-
- columns = set(ts.columns.get_level_values("feature"))
- if columns != {"target"}:
- warnings.warn(
- message=f"{type(self).__name__} does not work with any exogenous series or features. "
- f"It uses only target series for predict/\n "
- )
  return self
 
  @staticmethod

etna/models/holt_winters.py

@@ -201,6 +201,15 @@ def __init__(
  self._last_train_timestamp: Optional[pd.Timestamp] = None
  self._train_freq: Optional[str] = None
 
+ def _check_not_used_columns(self, df: pd.DataFrame):
+ columns = df.columns
+ columns_not_used = set(columns).difference({"target", "timestamp"})
+ if columns_not_used:
+ warnings.warn(
+ message=f"This model doesn't work with exogenous features. "
+ f"Columns {columns_not_used} won't be used."
+ )
+
  def fit(self, df: pd.DataFrame, regressors: List[str]) -> "_HoltWintersAdapter":
  """
  Fit Holt-Winters' model.
@@ -217,8 +226,7 @@ def fit(self, df: pd.DataFrame, regressors: List[str]) -> "_HoltWintersAdapter":
  Fitted model
  """
  self._train_freq = determine_freq(timestamps=df["timestamp"])
-
- self._check_df(df)
+ self._check_not_used_columns(df)
 
  targets = df["target"]
  targets.index = df["timestamp"]
@@ -268,21 +276,11 @@ def predict(self, df: pd.DataFrame) -> np.ndarray:
  """
  if self._result is None or self._model is None:
  raise ValueError("This model is not fitted! Fit the model before calling predict method!")
- self._check_df(df)
 
  forecast = self._result.predict(start=df["timestamp"].min(), end=df["timestamp"].max())
  y_pred = forecast.values
  return y_pred
 
- def _check_df(self, df: pd.DataFrame):
- columns = df.columns
- columns_not_used = set(columns).difference({"target", "timestamp"})
- if columns_not_used:
- warnings.warn(
- message=f"This model does not work with exogenous features and regressors.\n "
- f"{columns_not_used} will be dropped"
- )
-
  def get_model(self) -> HoltWintersResultsWrapper:
  """Get :py:class:`statsmodels.tsa.holtwinters.results.HoltWintersResultsWrapper` model that was fitted inside etna class.
 
@@ -303,7 +301,7 @@ def _check_mul_components(self):
  if (model.trend is not None and model.trend == "mul") or (
  model.seasonal is not None and model.seasonal == "mul"
  ):
- raise ValueError("Forecast decomposition is only supported for additive components!")
+ raise NotImplementedError("Forecast decomposition is only supported for additive components!")
 
  def _rescale_components(self, components: pd.DataFrame) -> pd.DataFrame:
  """Rescale components when Box-Cox transform used."""
@@ -335,15 +333,17 @@ def forecast_components(self, df: pd.DataFrame) -> pd.DataFrame:
  raise ValueError("This model is not fitted!")
 
  if df["timestamp"].min() <= self._last_train_timestamp:
- raise ValueError("To estimate in-sample prediction decomposition use `predict` method.")
+ raise NotImplementedError(
+ "This model can't make forecast decomposition on history data! "
+ "Use method predict for in-sample prediction decomposition."
+ )
 
  horizon = determine_num_steps(
  start_timestamp=self._last_train_timestamp, end_timestamp=df["timestamp"].max(), freq=self._train_freq
  )
  horizon_steps = np.arange(1, horizon + 1)
 
  self._check_mul_components()
- self._check_df(df)
 
  level = fit_result.level.values
  trend = fit_result.trend.values
@@ -404,10 +404,12 @@ def predict_components(self, df: pd.DataFrame) -> pd.DataFrame:
  raise ValueError("This model is not fitted!")
 
  if df["timestamp"].min() < self._first_train_timestamp or df["timestamp"].max() > self._last_train_timestamp:
- raise ValueError("To estimate out-of-sample prediction decomposition use `forecast` method.")
+ raise NotImplementedError(
+ "This model can't make prediction decomposition on future out-of-sample data! "
+ "Use method forecast for future out-of-sample prediction decomposition."
+ )
 
  self._check_mul_components()
- self._check_df(df)
 
  level = fit_result.level.values
  trend = fit_result.trend.values

etna/models/nn/__init__.py

@@ -6,6 +6,7 @@
  from etna.models.nn.mlp import MLPModel
  from etna.models.nn.nbeats import NBeatsGenericModel
  from etna.models.nn.nbeats import NBeatsInterpretableModel
+ from etna.models.nn.patchts import PatchTSModel
  from etna.models.nn.rnn import RNNModel
  from etna.models.nn.tft import TFTModel
  from etna.models.nn.utils import PytorchForecastingDatasetBuilder