Warning shown with verbosity=-1

[memeplex]

According to the documentation < 0: Fatal, = 0: Error (Warning), = 1: Info, > 1: Debug but I'm unable to get rid of the 2^max_depth > num_leaves warning.

How you are using LightGBM?

LightGBM component: Python package

Environment info

Operating System: MacOS 11.0.1 (Big Sur)

CPU/GPU model: 2,3 GHz Dual-Core Intel Core i5 / Intel Iris Plus Graphics 640 1536 MB

Python version: 3.8.6

Error message and / or logs

[LightGBM] [Warning] Accuracy may be bad since you didn't explicitly set num_leaves OR 2^max_depth > num_leaves. (num_leaves=31).

Reproducible example(s)

import lightgbm as lgb
import pandas as pd
import numpy as np

X = pd.DataFrame({
    "x1": np.arange(100),
    "x2": np.arange(100),
})
y = X.x1 + X.x2
lgb.train({"max_depth": 10, "verbose": -1}, lgb.Dataset(X, y))

[memeplex]

Besides, the UserWarning mentioned in #3640 and #3379 is also shown when verbosity = -1. This is a python warning and as such probably works at a different level, but I believe it would be better to treat all kind of warnings in a consistent way.

[memeplex]

Some additional info:

1. It only happens the first time, I've to restart the kernel in order to reproduce it one more time. As in another recent report of mine, some global state seems to be persisted between invocations (probably config, since it's global).

2. Looking at the Booster class initializer the problem seems to happen here:

            train_set.construct()                                         <--- shows the warning
            # copy the parameters from train_set
            params.update(train_set.get_params())
            params_str = param_dict_to_str(params)
            self.handle = ctypes.c_void_p()
            _safe_call(_LIB.LGBM_BoosterCreate(          <--- but config is initialized here (too late?).

[memeplex]

Oh, no, I think it's much simpler! It's all in config.cpp:

 // check for conflicts
  CheckParamConflict();  <---- shows the warning

  if (verbosity == 1) {.  <---- sets log level
    LightGBM::Log::ResetLogLevel(LightGBM::LogLevel::Info);
  } else if (verbosity == 0) {
    LightGBM::Log::ResetLogLevel(LightGBM::LogLevel::Warning);
  } else if (verbosity >= 2) {
    LightGBM::Log::ResetLogLevel(LightGBM::LogLevel::Debug);
  } else {
    LightGBM::Log::ResetLogLevel(LightGBM::LogLevel::Fatal);
  }

[StrikerRUS]

Kindly ask attention from @guolinke @chivee @shiyu1994 @btrotta to this issue. I'm afraid this issue makes debugging process in #3450 harder.

[h-vetinari]

I'm being hit by the same issue, opened #3742 to shift CheckParamConflict(); after the LogLevel-processing.

[memeplex]

I'm also getting:

[LightGBM] [Warning] min_sum_hessian_in_leaf is set=116036.20329539999, min_child_weight=-1 will be ignored. Current value: min_sum_hessian_in_leaf=116036.20329539999

in 3.1.1 with verbosity=-1, would this one also be fixed by #3742 or should I open a new report?

[StrikerRUS]

@memeplex
I think #3742 should fix this warning as well. Could you please try nightly build?

[joshi-abhishek]

Well #3742 did not fix it. Any update on this?

[StrikerRUS]

@joshi-abhishek Please provide a reproducible example that doesn't silence LightGBM after setting verbose=-1.
I don't see any logs for 3.2.1 version and example from #3641 (comment).

import lightgbm as lgb
import pandas as pd
import numpy as np

print(lgb.__version__)

X = pd.DataFrame({
    "x1": np.arange(100),
    "x2": np.arange(100),
})
y = X.x1 + X.x2
lgb.train({"max_depth": 10, "verbose": -1}, lgb.Dataset(X, y))

>>> 3.2.1

When I remove verbose param I expectedly get a lot of logs:

3.2.1
[LightGBM] [Warning] Accuracy may be bad since you didn't explicitly set num_leaves OR 2^max_depth > num_leaves. (num_leaves=31).
[LightGBM] [Warning] Accuracy may be bad since you didn't explicitly set num_leaves OR 2^max_depth > num_leaves. (num_leaves=31).
[LightGBM] [Warning] Auto-choosing col-wise multi-threading, the overhead of testing was 0.000045 seconds.
You can set `force_col_wise=true` to remove the overhead.
[LightGBM] [Info] Total Bins 68
[LightGBM] [Info] Number of data points in the train set: 100, number of used features: 2
[LightGBM] [Info] Start training from score 99.000000
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf
[LightGBM] [Warning] No further splits with positive gain, best gain: -inf

...

[memeplex]

@StrikerRUS I think @joshi-abhishek is right, there still are very annoying warnings showing all the time, specially when running inside a grid search.

In version 3.2.1:

import warnings
import numpy as np
import lightgbm as lgb

from sklearn.model_selection import GridSearchCV

X = np.random.random((1000, 20))
y = np.random.random(1000)

gs = GridSearchCV(
    lgb.LGBMRegressor(verbose=-1),
    dict(min_sum_hessian_in_leaf=[10, 100, 1000]),
)
with warnings.catch_warnings():
    warnings.filterwarnings("ignore", category=UserWarning)
    gs.fit(X, y, verbose=False)

The output is:

[LightGBM] [Warning] min_sum_hessian_in_leaf is set=10, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=10
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=10, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=10
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=10, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=10
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=10, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=10
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=100, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=100
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=100, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=100
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=100, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=100
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=100, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=100
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=100, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=100
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=1000, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=1000
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=1000, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=1000
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=1000, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=1000
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=1000, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=1000
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=1000, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=1000
[LightGBM] [Warning] min_sum_hessian_in_leaf is set=1000, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=1000

This happens with many parameters, some aliases will throw the warning while others not.

Notice that I'm already going to great lengths in order to suppress the noise:

1. verbose=-1 to initializer.
2. verbose=False to fit.
3. Have to silence python specific warnings since the python wrapper doesn't honour the verbose arguments.

This by itself is undesirable, but after all that boilerplate it still doesn't work as expected.

[StrikerRUS]

@memeplex This is a different warning comparing to the original one in this issue.

[LightGBM] [Warning] min_sum_hessian_in_leaf is set=10, min_child_weight=0.001 will be ignored. Current value: min_sum_hessian_in_leaf=10

says that you are passing param alias that has a direct equivalent in constructor arguments.
You can avoid this warning by using min_child_weight instead of min_sum_hessian_in_leaf (#3436 (comment)).

#3742 didn't fix alias warning because alias checking happens during params parsing earlier than setting logging level based on parsed verbose param.

LightGBM/src/io/config.cpp

Lines 24 to 35 in 926526c

	if (key.size() > 0) {
	auto value_search = params->find(key);
	if (value_search == params->end()) { // not set
	params->emplace(key, value);
	} else {
	Log::Warning("%s is set=%s, %s=%s will be ignored. Current value: %s=%s",
	key.c_str(), value_search->second.c_str(), key.c_str(), value.c_str(),
	key.c_str(), value_search->second.c_str());
	}
	}
	} else {
	Log::Warning("Unknown parameter %s", kv);

Please open a new issue for this.

[memeplex]

Please open a new issue for this.

Done here #4518

[IavTavares]

def objective_reg(trial):
    
    param = {
        'objective': 'regression', # we're in a regression problem
        'metric': 'l2',
        #"verbosity": -1,
        'lambda_l1': trial.suggest_loguniform('lambda_l1', 1e-8, 10.0),
        'lambda_l2': trial.suggest_loguniform('lambda_l2', 1e-8, 10.0),
        'num_leaves': trial.suggest_int('num_leaves', 2, 502,10),
        'max_depth': trial.suggest_categorical('max_depth', [10,20,100]),
        'feature_fraction': trial.suggest_uniform('feature_fraction', 0.4, 1.0),
        'bagging_fraction': trial.suggest_uniform('bagging_fraction', 0.4, 1.0),
        'bagging_freq': trial.suggest_int('bagging_freq', 1, 10,3),
        'min_child_samples': trial.suggest_int('min_child_samples', 5, 100,5)
    }
    
    # Add a callback for pruning = early stopping.
    pruning_callback = optuna.integration.LightGBMPruningCallback(trial, "l2")
    
    #lgbmc = lgb.train(param, dtrain)
    
    #predictions=lgbmc.predict(valid_x) # returns predictions for IE 
    
    lgbm_reg=LGBMRegressor(verbosity = -1,**param)
    scores = cross_val_score(
        lgbm_reg, X_train, np.ravel(Y_train), 
        cv = cv_generator,
        scoring ='neg_mean_absolute_error'
    )
    
    return scores.mean()

study = optuna.create_study(pruner=optuna.pruners.MedianPruner(n_warmup_steps=10),
                            direction='maximize')
optuna.logging.set_verbosity(optuna.logging.WARNING)

study.optimize(objective_reg, n_trials=5)

When I run the above in a Optuna, I get the following warnings:

[LightGBM] [Warning] feature_fraction is set=0.4187936548052027, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.4187936548052027
[LightGBM] [Warning] lambda_l1 is set=1.2934822202413716e-05, reg_alpha=0.0 will be ignored. Current value: lambda_l1=1.2934822202413716e-05
[LightGBM] [Warning] bagging_fraction is set=0.9243403549897538, subsample=1.0 will be ignored. Current value: bagging_fraction=0.9243403549897538
[LightGBM] [Warning] lambda_l2 is set=1.306943140036112e-08, reg_lambda=0.0 will be ignored. Current value: lambda_l2=1.306943140036112e-08
[LightGBM] [Warning] bagging_freq is set=7, subsample_freq=0 will be ignored. Current value: bagging_freq=7
(...)

[eromoe]

Why this is closed? I got same issue as @IavTavares , did you solve it ?

[StrikerRUS]

@eromoe

Why this is closed?

Because this is a different issue and it's being tracked in #4518.

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.