Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve MG PageRank scalability #2038

Merged
merged 85 commits into from
Mar 14, 2022
Merged

Improve MG PageRank scalability #2038

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
092cf49
enable multi-stream execution and overlapping communication with comp…
seunghwak Jan 26, 2022
6102cd1
update group_by_and_count to not use reduce_by_key (which is expensiv…
seunghwak Jan 27, 2022
5146b19
add time measurements (should be undone)
seunghwak Jan 27, 2022
9d31282
Merge branch 'upstream_pr2044' into enh_mg_pagerank2
seunghwak Jan 27, 2022
965f0cd
cosmetic updates
seunghwak Jan 31, 2022
5073304
Merge branch 'branch-22.04' of github.com:rapidsai/cugraph into enh_g…
seunghwak Feb 1, 2022
7c02fcf
improve weak scaling behavior of renumber
seunghwak Feb 1, 2022
0744160
move is_first_in_run_t to graph_utils.cuh
seunghwak Feb 1, 2022
077008b
avoid using device lambdas
seunghwak Feb 1, 2022
e0608bb
Merge branch 'upstream_pr2044' into enh_mg_pagerank2
seunghwak Feb 1, 2022
6a0dfa1
fix compile errors
seunghwak Feb 1, 2022
f9a9635
Merge branch 'upstream_pr2044' into enh_mg_pagerank2
seunghwak Feb 1, 2022
41645aa
code cleanup
seunghwak Feb 2, 2022
9bc0284
Merge branch 'upstream_pr2044' into enh_mg_pagerank2
seunghwak Feb 2, 2022
6b4b682
update copy_v_transform_reduce_in_out_nbr to process multiple edge pa…
seunghwak Feb 2, 2022
a0b009e
fix overflow bug with 2^31 or more vertices
seunghwak Feb 2, 2022
c889302
Merge branch 'upstream_pr2044' into enh_mg_pagerank2
seunghwak Feb 2, 2022
b5cb9c4
Merge branch 'branch-22.04' of github.com:rapidsai/cugraph into enh_g…
seunghwak Feb 3, 2022
9a70472
delete temporary performance measurement code
seunghwak Feb 3, 2022
214ada9
delete additional temporary performance measurement code
seunghwak Feb 3, 2022
3a605b5
remove temporary performance measurement code
seunghwak Feb 3, 2022
c537e2d
Merge branch 'branch-22.04' of github.com:rapidsai/cugraph into enh_g…
seunghwak Feb 3, 2022
4852ce4
clang-format & copyright year
seunghwak Feb 3, 2022
96e9693
add temporary performance measurement code
seunghwak Feb 4, 2022
a31050a
Merge branch 'upstream_pr2044' into enh_mg_pagerank2
seunghwak Feb 4, 2022
e8769d3
add temporary performance measurement code to PageRank implementation
seunghwak Feb 4, 2022
dc6acef
add more performance measurements to MG PageRank test
seunghwak Feb 4, 2022
8136b77
add more experimental code (should be cleaned-up before merge)
seunghwak Feb 4, 2022
1e74ddb
Merge branch 'branch-22.04' of github.com:rapidsai/cugraph into enh_m…
seunghwak Feb 8, 2022
3b426be
resolve merge conflicts
seunghwak Feb 9, 2022
f28ccfa
remove some temporary code
seunghwak Feb 10, 2022
1839a61
undo some temporary fix
seunghwak Feb 10, 2022
940ead8
Merge branch 'branch-22.04' of github.com:rapidsai/cugraph into enh_m…
seunghwak Feb 10, 2022
7cb1f03
remove host_barrier (no longer used)
seunghwak Feb 10, 2022
ff5e315
Merge branch 'branch-22.04' of github.com:rapidsai/cugraph into enh_m…
seunghwak Feb 11, 2022
eefe729
reduce temporary memory requirement in R-mat edge list generation
seunghwak Feb 12, 2022
167b5ab
input parameter renaming for clarity
seunghwak Feb 12, 2022
f06f32b
add a heuristic to cut peak memory footprint
seunghwak Feb 13, 2022
6dceeab
improve inconsistencies in naming
seunghwak Feb 13, 2022
febd72d
split groupby_and_shuffle_edgelist to groupby_and_count and shuffle_e…
seunghwak Feb 15, 2022
48fa255
use temporary host buffer to concatenate edge list in edge generation
seunghwak Feb 16, 2022
234a119
update groupby to take mem_frugal_threshold instead of bool mem_frugal
seunghwak Feb 16, 2022
95f2295
move counting unique local edge majors/mionrs inside the graph constr…
seunghwak Feb 16, 2022
15aeb4c
clang-format
seunghwak Feb 16, 2022
e7102e0
copyright year
seunghwak Feb 16, 2022
d8c5178
Merge branch 'branch-22.04' of github.com:rapidsai/cugraph into enh_m…
seunghwak Feb 16, 2022
7b544a6
clang-format
seunghwak Feb 16, 2022
6b40187
cosmetic updates
seunghwak Feb 17, 2022
85f39fb
update louvain tests to take --perf option
seunghwak Feb 17, 2022
48009a6
fix error in comments
seunghwak Feb 17, 2022
59d16a3
add is_first_in_run_pair_t to graph_utils.cuh
seunghwak Feb 17, 2022
b7d9570
add R-mat symmetric cases to coarsen_graph tests
seunghwak Feb 17, 2022
0a0c6c1
if the input grpah is symmetric, update coarsen_grpah to work on the …
seunghwak Feb 17, 2022
86f1d11
clang-format
seunghwak Feb 17, 2022
75d6f81
fix comments
seunghwak Feb 17, 2022
cf48403
Merge branch 'branch-22.04' of github.com:rapidsai/cugraph into bug_s…
seunghwak Feb 17, 2022
ef13c32
Merge branch 'upstream_pr2070' into enh_mg_pagerank2
seunghwak Feb 17, 2022
29e7b84
fix to make merge with enh_mg_louvain PR easier
seunghwak Feb 17, 2022
57a0843
copyright
seunghwak Feb 17, 2022
28b21ef
resolve merge conflicts
seunghwak Feb 17, 2022
6cc6a03
Merge branch 'upstream_pr2070' into enh_mem_footprint_part2
seunghwak Feb 17, 2022
63397c8
resolve merge conflicts
seunghwak Feb 17, 2022
14c47d4
remove one more memory footprint bottleneck in graph object creation
seunghwak Feb 18, 2022
550576b
add graph constructors taking edge list as R-value (so they can be de…
seunghwak Feb 18, 2022
09f3072
cosmetic fix
seunghwak Feb 18, 2022
2b9d37d
update coarsen_graph return type
seunghwak Feb 18, 2022
35fc8d1
refactor graph constructors and implement graph constructors taking e…
seunghwak Feb 18, 2022
dc4a213
update create_grpah_from_edgelist to call graph_t taking R-value edge…
seunghwak Feb 18, 2022
5177e86
update coarsen_graph_impl to call create_graph_from_edgelist instead …
seunghwak Feb 18, 2022
3bdb95e
additional (potential) parallelism vs peak memory trade-off in edge s…
seunghwak Feb 19, 2022
edd7cdd
bug fix
seunghwak Feb 19, 2022
ea3736c
Merge branch 'upstream_pr2070' into enh_mem_footprint_part2
seunghwak Feb 19, 2022
ff12636
fix for a possible hang
seunghwak Feb 20, 2022
56764c4
Merge branch 'branch-22.04' of github.com:rapidsai/cugraph into enh_m…
seunghwak Feb 22, 2022
c369431
Merge branch 'upstream_pr2081' into enh_mg_pagerank2
seunghwak Feb 22, 2022
1692a46
resolve merge conflicts
seunghwak Feb 23, 2022
a34ad17
added temporary code to experiment performance
seunghwak Feb 24, 2022
b1e56e7
additional cut in peak memory and maximum single allocation size (to …
seunghwak Feb 24, 2022
885f28a
Merge branch 'upstream_pr2081' into enh_mg_pagerank2
seunghwak Feb 24, 2022
b4bbb9d
resolve merge conflicts
seunghwak Mar 9, 2022
d716f6c
remove temporary experimental code
seunghwak Mar 9, 2022
99401f3
remove temporary experimental code
seunghwak Mar 9, 2022
0664216
fix formatting error
seunghwak Mar 9, 2022
c136d3a
undo copyright update with the file with no change
seunghwak Mar 9, 2022
a4f6528
clang-format & copyright year
seunghwak Mar 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion cpp/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,6 @@ add_library(cugraph SHARED
src/structure/create_graph_from_edgelist_mg.cu
src/structure/symmetrize_edgelist_sg.cu
src/structure/symmetrize_edgelist_mg.cu
src/utilities/host_barrier.cpp
src/visitors/graph_envelope.cpp
src/visitors/visitors_factory.cpp
src/visitors/bfs_visitor.cpp
Expand Down
6 changes: 3 additions & 3 deletions cpp/include/cugraph/prims/copy_to_adj_matrix_row_col.cuh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2020-2021, NVIDIA CORPORATION.
* Copyright (c) 2020-2022, NVIDIA CORPORATION.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -170,7 +170,7 @@ void copy_to_matrix_major(raft::handle_t const& handle,
matrix_partition_device_view_t<vertex_t, edge_t, weight_t, GraphViewType::is_multi_gpu>(
graph_view.get_matrix_partition_view(i));

if (col_comm_rank == i) {
if (i == col_comm_rank) {
auto vertex_partition =
vertex_partition_device_view_t<vertex_t, GraphViewType::is_multi_gpu>(
graph_view.get_vertex_partition_view());
Expand Down Expand Up @@ -365,7 +365,7 @@ void copy_to_matrix_minor(raft::handle_t const& handle,
matrix_partition_device_view_t<vertex_t, edge_t, weight_t, GraphViewType::is_multi_gpu>(
graph_view.get_matrix_partition_view(size_t{0}));
for (int i = 0; i < row_comm_size; ++i) {
if (row_comm_rank == i) {
if (i == row_comm_rank) {
auto vertex_partition =
vertex_partition_device_view_t<vertex_t, GraphViewType::is_multi_gpu>(
graph_view.get_vertex_partition_view());
Expand Down
Loading