Add function to merge samplers outputs #252
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kgajdamo
requested review from
rusty1s,
ZhengHongming888,
JakubPietrakIntel and
a team
This was referenced Sep 5, 2023
kgajdamo
force-pushed
the
dist-sampler-merge-outputs
branch
from
60c2ce6
to
30a400c
Compare
Codecov Report
@@ Coverage Diff @@
## master #252 +/- ##
==========================================
+ Coverage 83.50% 84.30% +0.80%
==========================================
Files 28 30 +2
Lines 970 1026 +56
==========================================
+ Hits 810 865 +55
- Misses 160 161 +1
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
kgajdamo
force-pushed
the
dist-sampler-merge-outputs
branch
from
90ce94d
to
7bb03aa
Compare
kgajdamo
force-pushed
the
dist-sampler-merge-outputs
branch
from
37c6afa
to
ecd8e9b
Compare
rusty1s
approved these changes
Sep 15, 2023
rusty1s
added a commit
that referenced
this pull request
Sep 15, 2023
#254) This code belongs to the part of the whole distributed training for PyG. This PR is complementary to the [#246](#246). ##Descrption Perform global to local mappings using mapper and create (row, col) based on a sampled_nodes_with_duplicates and sampled_nbrs_per_node. **Other distributed PRs:** pytorch_geometric DistLoader: [#7869](pyg-team/pytorch_geometric#7869) pytorch_geometric DistSampler: [#7974](pyg-team/pytorch_geometric#7974) pyg-lib [MERGED]: [#246](#246) pyg-lib: [#252](#252) pyg-lib: [#253](#253) --------- Co-authored-by: Matthias Fey <matthias.fey@tu-dortmund.de>
rusty1s
added a commit
to pyg-team/pytorch_geometric
that referenced
this pull request
Oct 9, 2023
This code belongs to the part of the whole distributed training for PyG. `DistNeighborSampler` leverages the `NeighborSampler` class from `pytorch_geometric` and the `neighbor_sample` function from `pyg-lib`. However, due to the fact that in case of distributed training it is required to synchronise the results between machines after each layer, the part of the code responsible for sampling was implemented in python. Added suport for the following sampling methods: - node, edge, negative, disjoint, temporal **TODOs:** - [x] finish hetero part - [x] subgraph sampling **This PR should be merged together with other distributed PRs:** pyg-lib: [#246](pyg-team/pyg-lib#246), [#252](pyg-team/pyg-lib#252) GraphStore\FeatureStore: #8083 DistLoaders: 1. #8079 2. #8080 3. #8085 --------- Co-authored-by: JakubPietrakIntel <jakub.pietrak@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: ZhengHongming888 <hongming.zheng@intel.com> Co-authored-by: Jakub Pietrak <97102979+JakubPietrakIntel@users.noreply.github.com> Co-authored-by: Matthias Fey <matthias.fey@tu-dortmund.de>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This code belongs to the part of the whole distributed training for PyG.
Description
Distributed training requires after each layer to merge results between machines. For later algorithms, it is required that the results be sorted according to the sampling order. This PR introduces a function which purpose is to handle merge and sort operations in parallel.
Other distributed PRs:
pytorch_geometric DistLoader: #7869
pytorch_geometric DistSampler: #7974
pyg-lib: #246