Skip to content

Conversation

@rxin
Copy link
Contributor

@rxin rxin commented Dec 3, 2014

MappedRDD, MappedValuesRDD, FlatMappedValuesRDD, FilteredRDD, GlommedRDD, FlatMappedRDD are not necessary. They can be implemented trivially using MapPartitionsRDD.

…PartitionsRDD.

MappedRDD, MappedValuesRDD, FlatMappedValuesRDD, FilteredRDD, GlommedRDD, FlatMappedRDD are not necessary. They can be implemented trivially using MapPartitionsRDD.
@SparkQA
Copy link

SparkQA commented Dec 3, 2014

Test build #24086 has started for PR 3578 at commit eb1a89b.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Dec 3, 2014

Test build #24086 has finished for PR 3578 at commit eb1a89b.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24086/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Dec 3, 2014

Test build #24112 has started for PR 3578 at commit eed9853.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Dec 3, 2014

Test build #24112 has finished for PR 3578 at commit eed9853.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24112/
Test PASSed.

@pwendell
Copy link
Contributor

pwendell commented Dec 4, 2014

LGTM - the main value of these classes is that when you are making a slide to describe RDD transformations it's nice to put "MappedRDD" "FilteredRDD" etc. While useful pedagogically, that's hardly a reason to keep them around :P

@rxin
Copy link
Contributor Author

rxin commented Dec 4, 2014

Yea - I was thinking maybe we can change the toString implementation for MapPartitionedRDD and add a field similar to the old one. We can do that later if there is demand.

@rxin
Copy link
Contributor Author

rxin commented Dec 4, 2014

Merging in master.

@asfgit asfgit closed this in c3ad486 Dec 4, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants