Skip to content

[WIP][GRAPHX] Allow to turn off the normalization applied in the end of static PageRank utilities #32484

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

ebonnal
Copy link
Contributor

@ebonnal ebonnal commented May 9, 2021

What changes were proposed in this pull request?

Overload methods PageRank.runWithOptions and PageRank.runWithOptionsWithPreviousPageRank (not to break any user-facing signature) with a normalized parameter that describes "whether or not to normalize the rank sum".

Why are the changes needed?

https://issues.apache.org/jira/browse/SPARK-

When dealing with a non negligible proportion of sinks in a graph, algorithm based on incremental update of ranks can get a precision gain for free if they are allowed to manipulate non normalized ranks.

Does this PR introduce any user-facing change?

No

How was this patch tested?

By adding a unit test that verifies that (even when dealing with a graph containing a sink) we end up with the same result for both these scenarios:
a)

  • Run 6 iterations of pagerank in a row using PageRank.runWithOptions with normalization enabled

b)

  • Run 2 iterations using PageRank.runWithOptions with normalization disabled
  • Resume from the preRankGraph1 and run 2 more iterations using PageRank.runWithOptionsWithPreviousPageRank with normalization disabled
  • Finally resume from the preRankGraph2 and run 2 more iterations using PageRank.runWithOptionsWithPreviousPageRank with normalization enabled

@github-actions github-actions bot added the GRAPHX label May 9, 2021
@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@ebonnal ebonnal force-pushed the enh/opt-pagerank-normalization branch from e9430da to 0150eda Compare May 9, 2021 13:51
…nk with a 'normalized' parameter to trigger or not the normalization
@ebonnal ebonnal force-pushed the enh/opt-pagerank-normalization branch from 0150eda to e407e88 Compare May 9, 2021 14:01
@ebonnal
Copy link
Contributor Author

ebonnal commented May 9, 2021

replaced by #32485

@ebonnal ebonnal closed this May 9, 2021
@ebonnal ebonnal deleted the enh/opt-pagerank-normalization branch May 9, 2021 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants