sklearn.ensemble.BaggingRegressor()

[zhizhuaa]

NeuralNetRegressor () and sklearn. Ensemble. BaggingRegressor () can not run. But NeuralNetClassifier () and sklearn. Ensemble. BaggingClassifier combination can run.

[BenjaminBossan]

Could you give more details? Is it because you get the following error message?

ValueError: The target data shouldn't be 1-dimensional but instead have 2 dimensions ...

[zhizhuaa]

The code is as follows.

from sklearn.datasets import make_regression
import numpy as np
import torch.nn as nn
import torch.nn.functional as F
from skorch import NeuralNetRegressor
from sklearn.ensemble import BaggingRegressor

# This is a toy dataset for regression, 1000 data points with 20 features each
X_regr, y_regr = make_regression(1000, 20, n_informative=10, random_state=0)
X_regr = X_regr.astype(np.float32)
y_regr = y_regr.astype(np.float32) / 100
y_regr = y_regr.reshape(-1, 1)

class RegressorModule(nn.Module):
    def __init__(
            self,
            num_units=10,
            nonlin=F.relu,
    ):
        super(RegressorModule, self).__init__()
        self.num_units = num_units
        self.nonlin = nonlin

        self.dense0 = nn.Linear(20, num_units)
        self.nonlin = nonlin
        self.dense1 = nn.Linear(num_units, 10)
        self.output = nn.Linear(10, 1)

    def forward(self, X, **kwargs):
        X = self.nonlin(self.dense0(X))
        X = F.relu(self.dense1(X))
        X = self.output(X)
        return X

net_regr = NeuralNetRegressor(
    RegressorModule,
    max_epochs=20,
    lr=0.1,
    device='cuda',  # uncomment this to train with CUDA
)

bagging = BaggingRegressor(estimator=net_regr, n_estimators=10, random_state=42)

bagging.fit(X_regr, y_regr)

# Making prediction for first 5 data points of X
y_pred = bagging.predict(X_regr[:5])

The brief run error is as follows.

raise ValueError(msg)
ValueError: The target data shouldn't be 1-dimensional but instead have 2 dimensions, with the second dimension having the same size as the number of regression targets (usually 1). Please reshape your target data to be 2-dimensional (e.g. y = y.reshape(-1, 1).

[BenjaminBossan]

Yes, I'm sorry about this annoyance, we should fix it in skorch.

Until then, here is a small hack that should resolve the issue:

class MyNeuralNetRegressor(NeuralNetRegressor):
    def fit(self, X, y, **fit_params):
        if y.ndim == 1:
            y = y.reshape(-1, 1)
        return super().fit(X, y, **fit_params)

Replace NeuralNetRegressor in your code with MyNeuralNetRegressor and it should work. If it works, still leave this issue open as a reminder to fix the issue properly.

[zhizhuaa]

Thank you very much. It worked.

[BenjaminBossan]

With the latest skorch release, this workaround should no longer be necessary, i.e. NeuralNetRegressor works with 1-dim y. Just make sure that the y_pred that is returned from the module is also 1-dim.