[SPARK-7884] Move block deserialization from BlockStoreShuffleFetcher to ShuffleReader#6423

Closed

massie wants to merge 20 commits intoapache:masterfrom

massie:shuffle-api-cleanup

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SPARK-7884] Move block deserialization from BlockStoreShuffleFetcher to ShuffleReader#6423

[SPARK-7884] Move block deserialization from BlockStoreShuffleFetcher to ShuffleReader#6423
massie wants to merge 20 commits intoapache:masterfrom
massie:shuffle-api-cleanup

Commits on Jun 2, 2015

[SPARK-7884] Allow Spark shuffle APIs to be more customizable

Make BlockStoreShuffleFetcher visible to shuffle package

Commits on Jun 8, 2015

Small code style changes

Commits on Jun 9, 2015

Close Block InputStream immediately after all records are read

Explicitly cast iterator in branches for type clarity

Minor Scala style fixes

Update shuffle read metrics in ShuffleReader instead of BlockStoreShuffleFetcher.

Commits on Jun 10, 2015

Small import nit

Small Scala import cleanup

Return visibility of BlockStoreShuffleFetcher to private[hash]

Consolidate metric code. Make it clear why InterrubtibleIterator is needed.

Commits on Jun 11, 2015

Remove unnecessary map() on return Iterator

Commits on Jun 12, 2015

Update tests to check that BufferReleasingStream is closing delegate InputStream

Small code cleanup for readability

Commits on Jun 18, 2015

Use PrivateMethodTester on check that delegate stream is closed

Commits on Jun 22, 2015

Add test to ensure HashShuffleReader is freeing resources

Commits on Jun 23, 2015

Revert "Add test to ensure HashShuffleReader is freeing resources"

Added test for HashShuffleReader.read()

Merge pull request #1 from kayousterhout/massie_shuffle-api-cleanup

Minor Scala style fixes

[SPARK-7884] Move block deserialization from BlockStoreShuffleFetcher to ShuffleReader#6423massie wants to merge 20 commits intoapache:masterapache/spark:masterfrom massie:shuffle-api-cleanupmassie/spark:shuffle-api-cleanupCopy head branch name to clipboard

Commits

Commits on Jun 2, 2015

Commits on Jun 8, 2015

Commits on Jun 9, 2015

Commits on Jun 10, 2015

Commits on Jun 11, 2015

Commits on Jun 12, 2015

Commits on Jun 18, 2015

Commits on Jun 22, 2015

Commits on Jun 23, 2015

[SPARK-7884] Move block deserialization from BlockStoreShuffleFetcher to ShuffleReader#6423
massie wants to merge 20 commits intoapache:masterfrom
massie:shuffle-api-cleanup