[Improvement] There should be a bounded and blocked queue when sendShuffleData in client side

[xianjingfeng]

In RssShuffleManager，the workQueue of threadPoolExecutoris unbounded now. If sendShuffleData not fast enough, it will cost a lot of memory

[roryqi]

Will it bring any other problems?

[xianjingfeng]

Will it bring any other problems?

It may cause OOM

[roryqi]

We have a class WriteBufferManager. The class will control the memory that we can use. It is strange to cause OOM. I doubt whether it is caused by the issue that pr #67 want to fix.

[xianjingfeng]

May be it is, i am not sure. I once found out an application failed due to requestExecutorMemory fail. Should we limit memory usage when sending data.

[roryqi]

May be it is, i am not sure. I once found out an application failed due to requestExecutorMemory fail. Should we limit memory usage when sending data.

I think we don't need, because we have WriteBufferManager. If our WriteBufferManager works well, OOM shouldn't occur. If our WriteBufferManager has bug, we should fix it.