[WIP][SPARK-19937] Collect metrics of block sizes when shuffle.#17276

Closed

jinxing64 wants to merge 28 commits intoapache:masterfrom

jinxing64:SPARK-19937

Commits on Mar 15, 2017

Collect metrics of block sizes from executor.
jinxing
committed

Commits on Mar 16, 2017

change in JsonProtocolSuite
jinxing
committed
fix JsonProtocolSuite.
jinxing
committed

Commits on Mar 20, 2017

Remove MAX_BLOCK_SIZE, add UNDERESTIMATED_BLOCKS_NUM and UNDERESTIMATED_BLOCKS_SIZE.
jinxing
committed
wip
jinxing
committed
fix taskMetricsFromJson and taskMetricsToJson
jinxing
committed
fix
jinxing
committed
Fix json.
jinxing
committed

Commits on Mar 21, 2017

remove maxBlockSize and fix remoteBytesReadToMem.
jinxing
committed
small fix
jinxing
committed
Remove the new metrics from event log
jinxing
committed
small fix
jinxing
committed
add log for shuffle read.
jinxing
committed
fix code style.
jinxing
committed

Commits on Mar 22, 2017

Commits on Mar 23, 2017

add remoteBytesReadToMem and underestimatedBlocksSize to api
jinxing
committed
fix log
jinxing
committed

Commits on Mar 24, 2017

show distribution with probabilities.
jinxing
committed
fix
jinxing
committed
change MimaExcludes.
jinxing
committed

Commits on Mar 25, 2017

fix json expectations.
jinxing
committed

Commits on Mar 26, 2017

small fix
jinxing
committed
change size of underestimated blocks and remove the duplicate code.
jinxing
committed
remote unused import
jinxing
committed