[SPARK-24525][SS] Provide an option to limit number of rows in a MemorySink#21559

Closed

mukulmurthy wants to merge 16 commits intoapache:masterfrom

mukulmurthy:SPARK-24525

Commits on Jun 12, 2018

Add in logic to determine the max rows a sink can have
Mukul Murthy
committed
Make MemorySink and MemorySinkV2 respect row and byte limits
Mukul Murthy
committed
Make tests compile
Mukul Murthy
committed
Make microbatch memory writer work with limits
Mukul Murthy
committed
Test MemorySinkV2 with limits
Mukul Murthy
committed
Add MemorySink test with limit
Mukul Murthy
committed
rename method
Mukul Murthy
committed

Commits on Jun 13, 2018

Don't use byte limit, and log if we truncate rows
Mukul Murthy
committed
Update tests
Mukul Murthy
committed
minor refactor
Mukul Murthy
committed
fixed indenting a bit
Mukul Murthy
committed
make helper methods private
Mukul Murthy
committed
Add additional safeguard that we don't call take() on a negative number
Mukul Murthy
committed

Commits on Jun 14, 2018

Move truncate method to parent class
Mukul Murthy
committed
Address Burak's comments
Mukul Murthy
committed

Commits on Jun 15, 2018

fix documentation
Mukul Murthy
committed