[DO NOT MERGE] Merge rabit #6001
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Conflicts: tracker/rabit_tracker.py
* Fix compilation failure on Windows * Fix lint
* throw dmlc::Error handled by xgboost jni
* add unittests * Expose RabitAllGatherRing and RabitGetRingPrevRank * Enabled TCP_NODELAY to decrease latency
…#128) * allow timeout to 0 to eanble immediate exit * disable duplicated signature check, overwrite results with same key
* fix missing allgether rabit declaration * fix allgather signature mismatch * fix type conversion * fix GetRingPrevRank
* fix hanging connections * remove logging
* Add RABIT_DLL tag to definitions of rabit APIs. * Fix Travis tests.
…dmlc#136) * De-duplicate macro _CRT_SECURE_NO_WARNINGS / _CRT_SECURE_NO_DEPRECATE * Move all macros to base.h * Fix CI
* set a minimal reducer msg size. Receive the same data size from parent each time. * When parent read from a child, check it receive minimal reduce size. fix bug. Rewrite the minimal reducer size check, make sure it's 1~N times of minimal reduce size Assume the minimal reduce size is X, the logic here is 1: each child upload total_size of message 2: each parent receive X message at least, up to total_size 3: parent reduce X or NxX or total_size message 4: parent sends X or NxX or total_size message to its parent 4: parent's parent receive X message at least, up to total_size. Then reduce X or NxX or total_size message 6: parent's parent sends X or NxX or total_size message to its children 7: parent receives X or NxX or total_size message, sends to its children 8: child receive X or NxN or total_size message. During the whole process, each transfer is (1~N)xX Byte message or up to total_size. if X is larger than total_size, then allreduce allways reduce the whole messages and pass down. * Follow style check rule * fix the cpplint check * fix allreduce_base header seq Co-authored-by: Philip Hyunsu Cho <chohyu01@cs.washington.edu>
|
@dmlc/xgboost-committer I plan to create a fast forward merge using command line and push to master branch directly. Please feel free to share your suggestion. |
7 tasks
Codecov Report
@@ Coverage Diff @@
## dmlc-merge-rabit #6001 +/- ##
=================================================
Coverage 78.52% 78.52%
=================================================
Files 12 12
Lines 3013 3013
=================================================
Hits 2366 2366
Misses 647 647 Continue to review full report at Codecov.
|
|
@trivialfis Let us do a merge commit instead. We can temporarily enable option to merge pull request via a merge commit. |
hcho3
approved these changes
Aug 14, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See #5995 for detail. This PR handles merging RABIT with subtree merge. I'm not entirely sure how will github handle it so right now I created a branch call
dmlc-merge-rabiton dmlc/xgboost.This PR is for testing, for the real merge I plan to do it locally on my machine with subtree merge and push to master branch directly. This way we can merge it smoothly without creating a merge node in git. Also as the PR setting is squashing only so I can't use PR to merge a submodule.