[WIP] Measurement for SPARK-16929.

[jinxing64]

What changes were proposed in this pull request?

This pr doesn't target for merging. It's a measurement for #16867, in which store successful taskIds in successfulTaskIdsSet in TreeSet, thus the time complexity is O(n/2) when get median duration in checkSpeculatableTasks.

[SparkQA]

Test build #73654 has finished for PR 17112 at commit 6825bd7.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jinxing64]

The unit test "Measurement for SPARK-16929." added is the measurement.
In TaskSetManagerSuite.scala line 1049, if newAlgorithm=true, successfulTaskIdsSet will be used to get the median duration. If newAlgorithm=false, old algorithm(Arrays.sort) will be used.

I calculate the time used for getting median duration in TaskSetManager.scala line 957.
If tasksNum=1000(TaskSetManagerSuite.scala line 1043), I did this test multiple times, results are as below:

newAlgorithm	time cost
false	5ms, 3ms, 4ms, 3ms, 3ms
true	2ms, 4ms, 2ms, 2ms, 3ms

if tasksNum=100000:

newAlgorithm	time cost
false	107ms, 109ms, 103ms, 100ms, 107ms
true	17ms, 14ms, 14ms, 13ms, 14ms

if tasksNum=150000:

newAlgorithm	time cost
false	133ms, 146ms, 127ms, 163ms, 114ms
true	14ms, 13ms, 15ms, 16ms, 14ms

As we can see, new algorithm(TreeSet) has better performance than old algorithm(Arrays.sort). When tasksNum=100000, Arrays.sort costs over 100ms every time, while in new algorithm all below 20ms.

[srowen]

Put [WIP] in the title for clarit

[SparkQA]

Test build #73889 has finished for PR 17112 at commit 61b96ff.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #74765 has finished for PR 17112 at commit cfc7e33.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.