Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use double precision in threaded calculation of linear tree coefficients (fixes #5226) #5368

Merged
merged 9 commits into from
Jul 29, 2022
Merged

Use double precision in threaded calculation of linear tree coefficients (fixes #5226) #5368

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a1099e1
Use double for model coefficient matrices
btrotta Jul 12, 2022
4f4990f
Don't parallelize Eigen calls
btrotta Jul 12, 2022
97d211d
Merge branch 'master' into linear-threading
btrotta Jul 12, 2022
edd6864
Remove n_jobs workaround
btrotta Jul 12, 2022
49f15c3
Update preprocessor directives for R config
btrotta Jul 13, 2022
27a3066
Use random data in test
btrotta Jul 13, 2022
60335e0
Add comment
btrotta Jul 13, 2022
32f564c
Set random seed
btrotta Jul 13, 2022
1536569
Update R config
btrotta Jul 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ include_directories(${EIGEN_DIR})

# See https://gitlab.com/libeigen/eigen/-/blob/master/COPYING.README
add_definitions(-DEIGEN_MPL2_ONLY)
add_definitions(-DEIGEN_DONT_PARALLELIZE)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add this to the flags used by the R package as well?

LightGBM/R-package/configure.ac

Line 38 in 44fe591

LGB_CPPFLAGS="${LGB_CPPFLAGS} -DEIGEN_MPL2_ONLY"

LightGBM/R-package/configure.win

Line 22 in 44fe591

LGB_CPPFLAGS="${LGB_CPPFLAGS} -DEIGEN_MPL2_ONLY"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change seems to be causing CI to fail; I think the problem is that I need to regenerate the configure file in R-package. Is there a way to do this on Windows?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For non-Windows, we have a comment-triggered CI job that will update configure based on changes to configure.ac. I'll trigger that job right now.

For configure.win, nothing needs to be regenerated. configure.win is executed directly, instead of being used as a template.

These things are documented at https://github.com/microsoft/LightGBM/tree/master/R-package#changing-the-cran-package but that README is fairly large so it's easy to miss.


if(__BUILD_FOR_R)
find_package(LibR REQUIRED)
Expand Down
2 changes: 1 addition & 1 deletion R-package/configure
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1713,7 +1713,7 @@ LGB_CPPFLAGS=""
# Eigen #
#########

LGB_CPPFLAGS="${LGB_CPPFLAGS} -DEIGEN_MPL2_ONLY"
LGB_CPPFLAGS="${LGB_CPPFLAGS} -DEIGEN_MPL2_ONLY -DEIGEN_DONT_PARALLELIZE"

###############
# MM_PREFETCH #
Expand Down
2 changes: 1 addition & 1 deletion R-package/configure.ac
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ LGB_CPPFLAGS=""
# Eigen #
#########

LGB_CPPFLAGS="${LGB_CPPFLAGS} -DEIGEN_MPL2_ONLY"
LGB_CPPFLAGS="${LGB_CPPFLAGS} -DEIGEN_MPL2_ONLY -DEIGEN_DONT_PARALLELIZE"

###############
# MM_PREFETCH #
Expand Down
2 changes: 1 addition & 1 deletion R-package/configure.win
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ LGB_CPPFLAGS=""
# Eigen #
#########

LGB_CPPFLAGS="${LGB_CPPFLAGS} -DEIGEN_MPL2_ONLY"
LGB_CPPFLAGS="${LGB_CPPFLAGS} -DEIGEN_MPL2_ONLY -DEIGEN_DONT_PARALLELIZE"

###############
# MM_PREFETCH #
Expand Down
4 changes: 2 additions & 2 deletions src/treelearner/linear_tree_learner.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ void LinearTreeLearner::InitLinear(const Dataset* train_data, const int max_leav
// store only upper triangular half of matrix as an array, in row-major order
// this requires (max_num_feat + 1) * (max_num_feat + 2) / 2 entries (including the constant terms of the regression)
// we add another 8 to ensure cache lines are not shared among processors
XTHX_.push_back(std::vector<float>((max_num_feat + 1) * (max_num_feat + 2) / 2 + 8, 0));
XTg_.push_back(std::vector<float>(max_num_feat + 9, 0.0));
XTHX_.push_back(std::vector<double>((max_num_feat + 1) * (max_num_feat + 2) / 2 + 8, 0));
XTg_.push_back(std::vector<double>(max_num_feat + 9, 0.0));
}
XTHX_by_thread_.clear();
XTg_by_thread_.clear();
Expand Down
8 changes: 4 additions & 4 deletions src/treelearner/linear_tree_learner.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,10 +118,10 @@ class LinearTreeLearner: public SerialTreeLearner {
/*! \brief map dataset to leaves */
mutable std::vector<int> leaf_map_;
/*! \brief temporary storage for calculating linear model coefficients */
mutable std::vector<std::vector<float>> XTHX_;
mutable std::vector<std::vector<float>> XTg_;
mutable std::vector<std::vector<std::vector<float>>> XTHX_by_thread_;
mutable std::vector<std::vector<std::vector<float>>> XTg_by_thread_;
mutable std::vector<std::vector<double>> XTHX_;
mutable std::vector<std::vector<double>> XTg_;
mutable std::vector<std::vector<std::vector<double>>> XTHX_by_thread_;
mutable std::vector<std::vector<std::vector<double>>> XTg_by_thread_;
};

} // namespace LightGBM
Expand Down
2 changes: 1 addition & 1 deletion tests/python_package_test/test_consistency.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ def test_binary_linear():
X_test, _, X_test_fn = fd.load_dataset('.test')
weight_train = fd.load_field('.train.weight')
lgb_train = lgb.Dataset(X_train, y_train, params=fd.params, weight=weight_train)
gbm = lgb.LGBMClassifier(**fd.params, n_jobs=0)
gbm = lgb.LGBMClassifier(**fd.params)
gbm.fit(X_train, y_train, sample_weight=weight_train)
sk_pred = gbm.predict_proba(X_test)[:, 1]
fd.train_predict_check(lgb_train, X_test, X_test_fn, sk_pred)
Expand Down
20 changes: 20 additions & 0 deletions tests/python_package_test/test_engine.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3046,6 +3046,26 @@ def test_interaction_constraints():
train_data, num_boost_round=10)


def test_linear_trees_num_threads():
# check that number of threads does not affect result
np.random.seed(0)
x = np.arange(0, 1000, 0.1)
y = 2 * x + np.random.normal(0, 0.1, len(x))
x = x[:, np.newaxis]
lgb_train = lgb.Dataset(x, label=y)
params = {'verbose': -1,
'objective': 'regression',
'seed': 0,
'linear_tree': True,
'num_threads': 2}
est = lgb.train(params, lgb_train, num_boost_round=100)
pred1 = est.predict(x)
params["num_threads"] = 4
est = lgb.train(params, lgb_train, num_boost_round=100)
pred2 = est.predict(x)
np.testing.assert_allclose(pred1, pred2)


def test_linear_trees(tmp_path):
# check that setting linear_tree=True fits better than ordinary trees when data has linear relationship
np.random.seed(0)
Expand Down
2 changes: 1 addition & 1 deletion windows/LightGBM.vcxproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@
</PropertyGroup>
<ItemDefinitionGroup>
<ClCompile>
<PreprocessorDefinitions>EIGEN_MPL2_ONLY</PreprocessorDefinitions>
<PreprocessorDefinitions>EIGEN_MPL2_ONLY;EIGEN_DONT_PARALLELIZE</PreprocessorDefinitions>
</ClCompile>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug_mpi|x64'">
Expand Down