Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update C++ SG and MG Louvain tests to support Rmat and benchmark tests #2054

Merged
merged 4 commits into from
Feb 2, 2022
Merged

Update C++ SG and MG Louvain tests to support Rmat and benchmark tests #2054

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e72822d
update SG and MG Louvain tests to support Rmat and benchmark tests
ChuckHastings Feb 1, 2022
117b85a
fix clang-format issues
ChuckHastings Feb 1, 2022
4b10e93
update copyright
ChuckHastings Feb 1, 2022
99f0f33
add check for weighted graph
ChuckHastings Feb 2, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion cpp/src/community/legacy/louvain.cu
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2021, NVIDIA CORPORATION.
* Copyright (c) 2021-2022, NVIDIA CORPORATION.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -95,6 +95,7 @@ std::pair<size_t, typename graph_view_t::weight_type> louvain(
using vertex_t = typename graph_view_t::vertex_type;
using weight_t = typename graph_view_t::weight_type;

CUGRAPH_EXPECTS(graph_view.has_data(), "Graph must be weighted");
detail::check_clustering(graph_view, clustering);

std::unique_ptr<Dendrogram<vertex_t>> dendrogram;
Expand Down
3 changes: 2 additions & 1 deletion cpp/src/community/louvain_impl.cuh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2020-2021, NVIDIA CORPORATION.
* Copyright (c) 2020-2022, NVIDIA CORPORATION.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -103,6 +103,7 @@ std::pair<size_t, typename graph_view_t::weight_type> louvain(
using vertex_t = typename graph_view_t::vertex_type;
using weight_t = typename graph_view_t::weight_type;

CUGRAPH_EXPECTS(graph_view.is_weighted(), "Graph must be weighted");
detail::check_clustering(graph_view, clustering);

std::unique_ptr<Dendrogram<vertex_t>> dendrogram;
Expand Down
109 changes: 87 additions & 22 deletions cpp/tests/community/louvain_test.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@
#include <vector>

struct Louvain_Usecase {
bool test_weighted_{false};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, are we always testing for weighted graphs only?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Louvain requires a weighted graph. All of the computations operate on the edge weights.

I'll add a CUGRAPH_EXPECTS in the algorithm implementation to check this, since we don't currently check that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we implicitly assume edge weights of 1 if the input graph is unweighted? Or no real use case for this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume finding communities in unweighted graphs to be something practical.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The python layer will add a weight of 1 for edges if the input graph is unweighted.

I think it would make the C++ code unwieldy to allow the input graph to be unweighted. Once we coarsen the graph it must be weighted, so we'd end up having to have different code paths for the original graph than from the coarsened graphs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha, just wanted to point out that there is memory requirement vs implementation complexity trade-off.

size_t max_level_{100};
double resolution_{1};
bool check_correctness_{false};
int expected_level_{0};
float expected_modularity_{0};
Expand All @@ -47,8 +48,10 @@ class Tests_Louvain
virtual void TearDown() {}

template <typename vertex_t, typename edge_t, typename weight_t, typename result_t>
void run_legacy_test(Louvain_Usecase const& louvain_usecase, input_usecase_t const& input_usecase)
void run_legacy_test(std::tuple<Louvain_Usecase const&, input_usecase_t const&> const& param)
{
auto [louvain_usecase, input_usecase] = param;

raft::handle_t handle{};

bool directed{false};
Expand Down Expand Up @@ -82,9 +85,10 @@ class Tests_Louvain
}

template <typename vertex_t, typename edge_t, typename weight_t, typename result_t>
void run_current_test(Louvain_Usecase const& louvain_usecase,
input_usecase_t const& input_usecase)
void run_current_test(std::tuple<Louvain_Usecase const&, input_usecase_t const&> const& param)
{
auto [louvain_usecase, input_usecase] = param;

raft::handle_t handle{};

// Can't currently check correctness if we renumber
Expand All @@ -93,7 +97,7 @@ class Tests_Louvain

auto [graph, d_renumber_map_labels] =
cugraph::test::construct_graph<vertex_t, edge_t, weight_t, false, false>(
handle, input_usecase, louvain_usecase.test_weighted_, renumber);
handle, input_usecase, true, renumber);

auto graph_view = graph.view();

Expand Down Expand Up @@ -296,65 +300,126 @@ TEST(louvain_legacy_renumbered, success)
}
}

using Tests_Louvain_File = Tests_Louvain<cugraph::test::File_Usecase>;
using Tests_Louvain_Rmat = Tests_Louvain<cugraph::test::Rmat_Usecase>;
using Tests_Louvain_File = Tests_Louvain<cugraph::test::File_Usecase>;
using Tests_Louvain_File32 = Tests_Louvain<cugraph::test::File_Usecase>;
using Tests_Louvain_File64 = Tests_Louvain<cugraph::test::File_Usecase>;
using Tests_Louvain_Rmat = Tests_Louvain<cugraph::test::Rmat_Usecase>;
using Tests_Louvain_Rmat32 = Tests_Louvain<cugraph::test::Rmat_Usecase>;
using Tests_Louvain_Rmat64 = Tests_Louvain<cugraph::test::Rmat_Usecase>;

TEST_P(Tests_Louvain_File, CheckInt32Int32FloatFloatLegacy)
{
auto param = GetParam();
run_legacy_test<int32_t, int32_t, float, float>(std::get<0>(param), std::get<1>(param));
run_legacy_test<int32_t, int32_t, float, float>(
override_File_Usecase_with_cmd_line_arguments(GetParam()));
}

TEST_P(Tests_Louvain_File, CheckInt32Int32FloatFloat)
{
auto param = GetParam();
run_current_test<int32_t, int32_t, float, float>(std::get<0>(param), std::get<1>(param));
run_current_test<int32_t, int32_t, float, float>(
override_File_Usecase_with_cmd_line_arguments(GetParam()));
}

TEST_P(Tests_Louvain_File, CheckInt64Int64FloatFloat)
{
auto param = GetParam();
run_current_test<int64_t, int64_t, float, float>(std::get<0>(param), std::get<1>(param));
run_current_test<int64_t, int64_t, float, float>(
override_File_Usecase_with_cmd_line_arguments(GetParam()));
}

TEST_P(Tests_Louvain_File32, CheckInt32Int32FloatFloat)
{
run_current_test<int32_t, int32_t, float, float>(
override_File_Usecase_with_cmd_line_arguments(GetParam()));
}

TEST_P(Tests_Louvain_File64, CheckInt64Int64FloatFloat)
{
run_current_test<int64_t, int64_t, float, float>(
override_File_Usecase_with_cmd_line_arguments(GetParam()));
}

TEST_P(Tests_Louvain_Rmat, CheckInt32Int32FloatFloatLegacy)
{
auto param = GetParam();
run_legacy_test<int32_t, int32_t, float, float>(
std::get<0>(param), override_Rmat_Usecase_with_cmd_line_arguments(std::get<1>(param)));
override_Rmat_Usecase_with_cmd_line_arguments(GetParam()));
}

TEST_P(Tests_Louvain_Rmat, CheckInt32Int32FloatFloat)
{
auto param = GetParam();
run_current_test<int32_t, int32_t, float, float>(
std::get<0>(param), override_Rmat_Usecase_with_cmd_line_arguments(std::get<1>(param)));
override_Rmat_Usecase_with_cmd_line_arguments(GetParam()));
}

TEST_P(Tests_Louvain_Rmat, CheckInt64Int64FloatFloat)
{
auto param = GetParam();
run_current_test<int64_t, int64_t, float, float>(
std::get<0>(param), override_Rmat_Usecase_with_cmd_line_arguments(std::get<1>(param)));
override_Rmat_Usecase_with_cmd_line_arguments(GetParam()));
}

TEST_P(Tests_Louvain_Rmat32, CheckInt32Int32FloatFloat)
{
run_current_test<int32_t, int32_t, float, float>(
override_Rmat_Usecase_with_cmd_line_arguments(GetParam()));
}

TEST_P(Tests_Louvain_Rmat64, CheckInt64Int64FloatFloat)
{
run_current_test<int64_t, int64_t, float, float>(
override_Rmat_Usecase_with_cmd_line_arguments(GetParam()));
}

// FIXME: Expand testing once we evaluate RMM memory use
INSTANTIATE_TEST_SUITE_P(
simple_test,
Tests_Louvain_File,
::testing::Combine(::testing::Values(Louvain_Usecase{true, true, 3, 0.408695}),
::testing::Combine(::testing::Values(Louvain_Usecase{100, 1, true, 3, 0.408695}),
::testing::Values(cugraph::test::File_Usecase("test/datasets/karate.mtx"))));

INSTANTIATE_TEST_SUITE_P(
file_benchmark_test, /* note that the test filename can be overridden in benchmarking (with
--gtest_filter to select only the file_benchmark_test with a specific
vertex & edge type combination) by command line arguments and do not
include more than one File_Usecase that differ only in filename
(to avoid running same benchmarks more than once) */
Tests_Louvain_File32,
::testing::Combine(
// disable correctness checks for large graphs
::testing::Values(Louvain_Usecase{}),
::testing::Values(cugraph::test::File_Usecase("test/datasets/karate.mtx"))));

INSTANTIATE_TEST_SUITE_P(
file64_benchmark_test, /* note that the test filename can be overridden in benchmarking (with
--gtest_filter to select only the file_benchmark_test with a specific
vertex & edge type combination) by command line arguments and do not
include more than one File_Usecase that differ only in filename
(to avoid running same benchmarks more than once) */
Tests_Louvain_File64,
::testing::Combine(
// disable correctness checks for large graphs
::testing::Values(Louvain_Usecase{}),
::testing::Values(cugraph::test::File_Usecase("test/datasets/karate.mtx"))));

INSTANTIATE_TEST_SUITE_P(
rmat_benchmark_test, /* note that scale & edge factor can be overridden in benchmarking (with
--gtest_filter to select only the rmat_benchmark_test with a specific
vertex & edge type combination) by command line arguments and do not
include more than one Rmat_Usecase that differ only in scale or edge
factor (to avoid running same benchmarks more than once) */
Tests_Louvain_Rmat,
Tests_Louvain_Rmat32,
::testing::Combine(
// disable correctness checks for large graphs
::testing::Values(Louvain_Usecase{}),
::testing::Values(cugraph::test::Rmat_Usecase(20, 32, 0.57, 0.19, 0.19, 0, true, false))));

INSTANTIATE_TEST_SUITE_P(
rmat64_benchmark_test, /* note that scale & edge factor can be overridden in benchmarking (with
--gtest_filter to select only the rmat_benchmark_test with a specific
vertex & edge type combination) by command line arguments and do not
include more than one Rmat_Usecase that differ only in scale or edge
factor (to avoid running same benchmarks more than once) */
Tests_Louvain_Rmat64,
::testing::Combine(
// disable correctness checks for large graphs
::testing::Values(Louvain_Usecase{true, false}),
::testing::Values(Louvain_Usecase{}),
::testing::Values(cugraph::test::Rmat_Usecase(20, 32, 0.57, 0.19, 0.19, 0, true, false))));

CUGRAPH_TEST_PROGRAM_MAIN()
17 changes: 14 additions & 3 deletions cpp/tests/community/mg_louvain_helper.cu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,14 +264,25 @@ coarsen_graph(

template void single_gpu_renumber_edgelist_given_number_map(
raft::handle_t const& handle,
rmm::device_uvector<int>& d_edgelist_rows,
rmm::device_uvector<int>& d_edgelist_cols,
rmm::device_uvector<int>& d_renumber_map_gathered_v);
rmm::device_uvector<int32_t>& d_edgelist_rows,
rmm::device_uvector<int32_t>& d_edgelist_cols,
rmm::device_uvector<int32_t>& d_renumber_map_gathered_v);

template void single_gpu_renumber_edgelist_given_number_map(
raft::handle_t const& handle,
rmm::device_uvector<int64_t>& d_edgelist_rows,
rmm::device_uvector<int64_t>& d_edgelist_cols,
rmm::device_uvector<int64_t>& d_renumber_map_gathered_v);

template std::unique_ptr<cugraph::graph_t<int32_t, int32_t, float, false, false>> coarsen_graph(
raft::handle_t const& handle,
cugraph::graph_view_t<int32_t, int32_t, float, false, false> const& graph_view,
int32_t const* labels);

template std::unique_ptr<cugraph::graph_t<int64_t, int64_t, float, false, false>> coarsen_graph(
raft::handle_t const& handle,
cugraph::graph_view_t<int64_t, int64_t, float, false, false> const& graph_view,
int64_t const* labels);

} // namespace test
} // namespace cugraph
Loading