-
Notifications
You must be signed in to change notification settings - Fork 29k
SPARK-6045 RecordWriter should be checked against null in PairRDDFunctio... #4794
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…tions#saveAsNewAPIHadoopDataset
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why check here? if it's null then this fails already in the loop. Just assert it's not null with require after it's obtained, if this is supposed to die in this case anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review, updated as suggested.
|
Can one of the admins verify this patch? |
…tions#saveAsNewAPIHadoopDataset
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's no string interpolation here, and there is no actual NewRecordWriter class (it's an alias). Don't worry I can edit that before merge. I will have Jenkins test just for completeness
|
OK to test |
|
Jenkins, test this please |
|
Test build #28021 has started for PR 4794 at commit
|
|
Test build #28021 has finished for PR 4794 at commit
|
|
Test PASSed. |
|
OK this looks like no-risk small fix. I'll adjust it and merge. |
...ns#saveAsNewAPIHadoopDataset