[SPARK-7296][WebUI] Timeline view for Stage page

[sarutak]

This PR is related to #2342, adding timeline view for Stage page.

Following image is a capture of the timeline.

With this timeline, we can understand following things of a Stage.

• When/where each task ran
• Total duration of each task
• Proportion of the time each task spends

Also, this timeline view can scrollable and zoomable.

[kayousterhout]

Can you write a method in here:

def toProportion(time: Long) = (time.toDouble / totalExecutionTime * 100).toLong

and then use it on line 542/546/550 etc. instead of copying that code?

[kayousterhout]

I had a few style comments, but this is mostly looking good! Thanks for all of your work on this @sarutak; this will be an awesome feature to have!

[kayousterhout]

Also, just to cleanup the PR a bit, would you mind adding a current screenshot of what this looks like to the PR description? I think that just makes it easier for folks who, later, are trying to understand what this PR did.

[sarutak]

Thanks @kayousterhout !
I've fixed things you mentioned and update the description, adding a capture.

[sarutak]

I'll update the capture in the description because this implementation will take the most recent tasks, not first tasks.

[sarutak]

I've updated, thanks.

[SparkQA]

Test build #32726 has finished for PR 5843 at commit b769c65.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #32723 has finished for PR 5843 at commit 1899097.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #32725 has finished for PR 5843 at commit 16cb2e6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[kayousterhout]

nit: I think this looks better with no border, because I think the border draws the eye too much towards the legend rather than to the visualization (although happy to be overruled if others have strong feelings otherwise)

[kayousterhout]

Ok I took one last look at this and played around with it again...LGTM except for the two small issues I pointed out!

[SparkQA]

Test build #32728 has finished for PR 5843 at commit 2e5d605.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class Pipeline(override val uid: String) extends Estimator[PipelineModel]
  • final class DecisionTreeClassifier(override val uid: String)
  • final class GBTClassifier(override val uid: String)
  • class LogisticRegression(override val uid: String)
  • final class OneVsRest(override val uid: String)
  • final class RandomForestClassifier(override val uid: String)
  • class BinaryClassificationEvaluator(override val uid: String)
  • final class Binarizer(override val uid: String)
  • final class Bucketizer(override val uid: String)
  • class ElementwiseProduct(override val uid: String)
  • class HashingTF(override val uid: String) extends UnaryTransformer[Iterable[_], Vector, HashingTF]
  • final class IDF(override val uid: String) extends Estimator[IDFModel] with IDFBase
  • class Normalizer(override val uid: String) extends UnaryTransformer[Vector, Vector, Normalizer]
  • class OneHotEncoder(override val uid: String)
  • class PolynomialExpansion(override val uid: String)
  • class StandardScaler(override val uid: String) extends Estimator[StandardScalerModel]
  • class StringIndexer(override val uid: String) extends Estimator[StringIndexerModel]
  • class Tokenizer(override val uid: String) extends UnaryTransformer[String, Seq[String], Tokenizer]
  • class RegexTokenizer(override val uid: String)
  • class VectorAssembler(override val uid: String)
  • class VectorIndexer(override val uid: String) extends Estimator[VectorIndexerModel]
  • final class Word2Vec(override val uid: String) extends Estimator[Word2VecModel] with Word2VecBase
  • class Param[T](val parent: String, val name: String, val doc: String, val isValid: T => Boolean)
  • class DoubleParam(parent: String, name: String, doc: String, isValid: Double => Boolean)
  • class IntParam(parent: String, name: String, doc: String, isValid: Int => Boolean)
  • class FloatParam(parent: String, name: String, doc: String, isValid: Float => Boolean)
  • class LongParam(parent: String, name: String, doc: String, isValid: Long => Boolean)
  • class BooleanParam(parent: String, name: String, doc: String) // No need for isValid
  • class ALS(override val uid: String) extends Estimator[ALSModel] with ALSParams
  • final class DecisionTreeRegressor(override val uid: String)
  • final class GBTRegressor(override val uid: String)
  • class LinearRegression(override val uid: String)
  • final class RandomForestRegressor(override val uid: String)
  • class CrossValidator(override val uid: String) extends Estimator[CrossValidatorModel]
  • trait Identifiable

[sarutak]

Thanks for your comment @kayousterhout .
I've fixed issues you pointed out and updated the capture.

[SparkQA]

Test build #32738 has finished for PR 5843 at commit 4ba9604.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[kayousterhout]

I've merged this into master and 1.4! Thanks @sarutak, and also @shroffpradyumn for the alternate implementation that led to the much improved formatting here!!

There are a few small remaining issues that I'll fix in a follow-on commit shortly.