[NSE-417] Sort spill support framework

[zhouyuan]

What changes were proposed in this pull request?

This patch add spill support for sort.

• The Spill() API of Sort is connected to the TaskMemoryManager of Spark. If under lower memory, Spark will try toissue Spill on Sort.
• On Spill() is called, executor will try to pick one partiton(one task) then try to write the sort content(keys + payloads) to disks, then release the memory for these consents. For following outputing, sort will first load the spill contents into memory then continue to output the results.
• Please note the Spill will be only called from the following operators of Sort, like SortMergeJoin - this makes spill happens only on sort results outputting phase.
• Currently the spill files will be written into each yarn container, this should be improved to use Spark's blockmanager.

How was this patch tested?

locally verified

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/native-sql-engine/issues

Then could you also rename commit message and pull request title in the following format?

[NSE-${ISSUES_ID}] ${detailed message}

See also:

• Other pull requests

[github-actions]

#417

[zhouyuan]

should refine this logic to allow repeated spill