skOpt having issues with backtesting.Optimize (skOpt)

[kiann00]

Expected Behavior

backtesting.optimize(model = 'skopt',....) should just work fine, when I test it in my local desktop

Actual Behavior

Bug error message below

Traceback (most recent call last):
  File "/root/environments/kiann_SPX_Backtesting_Module_Production_v1a.py", line 357, in <module>
    stats_skopt_2, heatmap, optimize_result = bt.optimize(
  File "/root/environments/my_env/lib/python3.10/site-packages/backtesting/backtesting.py", line                       1490, in optimize
    output = _optimize_skopt()
  File "/root/environments/my_env/lib/python3.10/site-packages/backtesting/backtesting.py", line                       1456, in _optimize_skopt
    res = forest_minimize(
  File "/root/environments/my_env/lib/python3.10/site-packages/skopt/optimizer/forest.py", line 1                      86, in forest_minimize
    return base_minimize(func, dimensions, base_estimator,
  File "/root/environments/my_env/lib/python3.10/site-packages/skopt/optimizer/base.py", line 300                      , in base_minimize
    result = optimizer.tell(next_x, next_y)
  File "/root/environments/my_env/lib/python3.10/site-packages/skopt/optimizer/optimizer.py", lin                      e 493, in tell
    return self._tell(x, y, fit=fit)
  File "/root/environments/my_env/lib/python3.10/site-packages/skopt/optimizer/optimizer.py", lin                      e 536, in _tell
    est.fit(self.space.transform(self.Xi), self.yi)
  File "/root/environments/my_env/lib/python3.10/site-packages/sklearn/ensemble/_forest.py", line                       341, in fit
    self._validate_params()
  File "/root/environments/my_env/lib/python3.10/site-packages/sklearn/base.py", line 570, in _va                      lidate_params
    validate_parameter_constraints(
  File "/root/environments/my_env/lib/python3.10/site-packages/sklearn/utils/_param_validation.py                      ", line 97, in validate_parameter_constraints
    raise InvalidParameterError(
sklearn.utils._param_validation.InvalidParameterError: The 'criterion' parameter of ExtraTreesReg                      ressor must be a str among {'friedman_mse', 'absolute_error', 'squared_error', 'poisson'}. Got 'm                      se' instead.

Steps to Reproduce

# we need to make sure the scikit learn modules are saved down
data_test = data_[-130000:] # truncate the data

s_time = time.time()
bt = Backtest(data_test, Signal_Strategy_v2, cash=10_000, commission=.002)
stats_skopt_2 = bt.run()
# bt.plot()
print('time for the initial strategy test is', time.time() - s_time)

def maximize_func(stats):
    # return stats['Equity Final [$]'] / stats['Max. Drawdown [%]'] * stats['Expectancy [%]']
    if stats['Max. Drawdown [%]'] == 0:
        return stats['Equity Final [$]'] * stats['Expectancy [%]']
    else:
        return stats['Equity Final [$]'] * stats['Expectancy [%]'] / stats['Max. Drawdown [%]']

s_time = time.time()

# Note : for method = 'skopt', we only need interval end-points
# We can input constraints per this line if needed : constraint=lambda p: p.rsi_lower < p.rsi_upper,

stats_skopt_2, heatmap, optimize_result = bt.optimize(
    rsi_upper = [50,90],
    rsi_lower = [10,40],
    rsi_window = [10,25],
    rsi_close = [30, 70],
    maximize= maximize_func,
    constraint = lambda p: (p.rsi_lower < p.rsi_upper) & ( p.rsi_lower < p.rsi_close < p.rsi_upper) &\
        (p.macd_upper > p.macd_close > p.macd_lower),
    method='skopt',
    max_tries=300,
    random_state=0,
    return_heatmap=True,
    return_optimization=True)

display(stats_skopt_2['_strategy'])
print('time for the backtesting optimization strategy tests is', time.time() - s_time)

stats_skopt_2.to_frame().to_csv('machine_spx_stats.csv')
pd.DataFrame(heatmap).reset_index().to_csv('machine_spx_heatmap.csv')

### Additional info


- Backtesting version: 0.3.3   <!-- From backtesting.__version__ -->
- `bokeh.__version__`:
- OS: linux
- numpy 1.23.4
- scikit optimize 0.9.0

[bigbenbeer]

I've got the same problem. Very simply optimization and I get the same The 'criterion' parameter of ExtraTreesRegressor must be a str among {'poisson', 'squared_error', 'absolute_error', 'friedman_mse'}. Got 'mse' instead. Error.

In my case, I tried changing the backtest.optimize function to provide the criterion as input and that also did not seem to work.

[eervin123]

Getting this error as well. I fixed the np.int vs int that was discussed here, but now I'm getting the above error and I can't find for the life of me where mse is being passed ;)

[rambam1120]

I read a bit online and it seemed like the reason for this error is because scikit-learn changed the default values or something for the function ExtraTreesRegressor as it got updated.

I'm quite new when it comes to all this coding stuff but the way I fixed it was byt uninstalling scikit-learn and reinstalling an older version of it, where 'mse' still used to work (I used 1.1.3 follwing the discussion here freqtrade/freqtrade#7897):

pip uninstall scikit-learn

pip install scikit-learn==1.1.3

This solved the issue for me, hope this helps

[efeint01]

Worked. Thanks

[MrDenfish]

worked for me as well, thanks!!... I did need to correct a small issue in transformers.py line 275 if self.is_int: return np.round(X_orig).astype(int) return X_orig was originally np.int

[PilotGFX]

pip install numpy 1.22
pip install scikit-optimize==0.9.0
pip install scikit-learn==1.1.3

[Quantbrinx]

This di

Worked. Thanks

still working for you guys? didn't work me

[JanHomann]

Worked. Thanks!

[kernc]

No longer an issue after switching to SAMBO in 9184a0b. Fix released.