[SPARK-6943][SPARK-6944] DAG visualization on SparkUI#5729

Closed

andrewor14 wants to merge 47 commits intoapache:masterfrom

andrewor14:viz2

Commits on Apr 17, 2015

Scope all RDD methods
Andrew Or
committed
Add a few missing scopes to certain RDD methods
Andrew Or
committed
Expose the necessary information in RDDInfo
Andrew Or
committed
Translate RDD information to dot file
Andrew Or
committed
First working implementation of visualization with vis.js
Andrew Or
committed

Commits on Apr 22, 2015

Re-implement scopes through annotations instead
Andrew Or
committed
Revert a few unintended style changes
Andrew Or
committed
Move RDD scope util methods and logic to its own file
Andrew Or
committed
Merge branch 'master' of github.com:apache/spark into viz
Andrew Or
committed

Commits on Apr 23, 2015

Reimplement rendering with dagre-d3 instead of viz.js
Andrew Or
committed

Commits on Apr 27, 2015

Merge branch 'master' of github.com:apache/spark into viz
Andrew Or
committed
Fill in documentation + miscellaneous minor changes
Andrew Or
committed
Embed the viz in the UI in a toggleable manner
Andrew Or
committed
Add ID to node label (minor)
Andrew Or
committed

Commits on Apr 28, 2015

Rat excludes
Andrew Or
committed
Re-implement scopes using closures instead of annotations
Andrew Or
committed
Ensure that HadoopRDD is actually serializable
Andrew Or
committed
Merge branch 'master' of github.com:apache/spark into viz2
Andrew Or
committed
Fix scala style
Andrew Or
committed
Merge branch 'master' of github.com:apache/spark into viz2
Andrew Or
committed

Commits on Apr 29, 2015

Remove all return statements in `withScope`
Andrew Or
committed
Add parent IDs to StageInfo
Andrew Or
committed
Fix line too long
Andrew Or
committed
Remove more return statements from scopes
Andrew Or
committed

Commits on May 1, 2015

Introduce visualization to the Job Page
Andrew Or
committed
Connect RDDs across stages + update style
Andrew Or
committed

Commits on May 3, 2015

Translate stage boxes taking into account the width
Andrew Or
committed
Add link from jobs to stages
Andrew Or
committed
Merge branch 'master' of github.com:apache/spark into viz2
Andrew Or
committed
Refactor + clean up + document JS visualization code
Andrew Or
committed
Separate HTML generating code from listener
Andrew Or
committed

Commits on May 4, 2015

Rename references to viz scopes to viz clusters
Andrew Or
committed
Implement a programmatic representation of operator scopes
Andrew Or
committed
Extract visualization logic from listener
Andrew Or
committed
Rename OperatorScope -> RDDOperationScope
Andrew Or
committed
Display cached RDDs on the viz
Andrew Or
committed
Merge branch 'master' of github.com:apache/spark into viz2
Andrew Or
committed
Fix tests
Andrew Or
committed
Fix python tests
Andrew Or
committed
Merge branch 'master' of github.com:apache/spark into viz2
Andrew Or
committed
Fix python style
Andrew Or
committed
children -> child (minor)
Andrew Or
committed
Restore "null" as default for RDD name
Andrew Or
committed
Wrap a few more RDD functions in an operation scope
Andrew Or
committed
Add tests for RDDOperationScope
Andrew Or
committed
Change RDD cache color to red (minor)
Andrew Or
committed
Round corners of RDD boxes on stage page (minor)
Andrew Or
committed