Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor all path #5409

Merged
merged 21 commits into from
Mar 23, 2023
Merged

Refactor all path #5409

merged 21 commits into from
Mar 23, 2023

Conversation

nevermore3
Copy link
Contributor

@nevermore3 nevermore3 commented Mar 17, 2023

What type of PR is this?

  • bug
  • feature
  • enhancement

What problem(s) does this PR solve?

Issue(s) number:

close #5400
close

Description:

 Using the two-way BFS algorithm, a heuristic algorithm is used in the expansion process
 when the number of vid to be expanded on the left and right
 exceeds the threshold(FLAGS_path_threshold_size)

 if size(leftVids) / size(rightVids) >= FLAGS_path_threshold_ratio(default 2)
    expandFromRight
 else if size(rightVids) / size(leftVids) >= FLAGS_path_threshold_ratio(default 2)
    expandFromLeft
 else
    expandFromLeft
    expandFromRight

 this way can avoid uneven calculation distribution due to data skew
 finally the path is constructed using an asynchronous process in the adjacency list

 adjList is an adjacency list structure
 which saves the vids and all adjacent edges that expand one step
 when expanding, if the vid has already been visited, do not visit again
 leftAdjList_ save result of forward expansion
 rightAdjList_ save result of backward expansion
BEFORE:
(root@nebula) [sf100]> find all path from 30786325647364 to 6597070124812 OVER * YIELD path as p |yield count(*)
+----------+
| count(*) |
+----------+
| 206      |
+----------+
Got 1 rows (time spent 17914996/17907209 us)

NOW:
(root@nebula) [sf100]> find all path from 30786325647364 to 6597070124812 OVER * YIELD path as p |yield count(*)
+----------+
| count(*) |
+----------+
| 206      |
+----------+
Got 1 rows (time spent 976543/976590 us)

How do you solve it?

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Checklist:

Tests:

  • Unit test(positive and negative cases)
  • Function test
  • Performance test
  • N/A

Affects:

  • Documentation affected (Please add the label if documentation needs to be modified.)
  • Incompatibility (If it breaks the compatibility, please describe it and add the label.)
  • If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
  • Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:

ex. Fixed the bug .....

@nevermore3 nevermore3 force-pushed the refactor_all_path branch 5 times, most recently from c46dd50 to ae8d23a Compare March 19, 2023 15:56
@nevermore3 nevermore3 added ready-for-testing PR: ready for the CI test ready for review labels Mar 20, 2023
@nevermore3 nevermore3 marked this pull request as ready for review March 20, 2023 03:39
@nevermore3 nevermore3 force-pushed the refactor_all_path branch 5 times, most recently from 26a1c3d to 5875f89 Compare March 20, 2023 06:56
src/graph/executor/PathBaseExecutor.h Outdated Show resolved Hide resolved
src/graph/executor/algo/AllPathsExecutor.cpp Outdated Show resolved Hide resolved
src/graph/executor/algo/AllPathsExecutor.cpp Show resolved Hide resolved
@nevermore3 nevermore3 force-pushed the refactor_all_path branch 5 times, most recently from 7dc2b72 to 2266bec Compare March 21, 2023 07:16
yixinglu
yixinglu previously approved these changes Mar 22, 2023
@Sophie-Xie Sophie-Xie merged commit af4ac39 into vesoft-inc:master Mar 23, 2023
@nevermore3 nevermore3 deleted the refactor_all_path branch March 23, 2023 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

enhance the performance of allpath
4 participants