[SPARK-3935][Core] log the number of records that has been written

[jackylk]

There is a unused variable(count) in saveAsHadoopDataset in PairRDDFunctions.scala. The initial idea of this variable seems to count the number of records, so I am adding a log statement to log the number of records that has been written to the writer.

[AmplabJenkins]

Can one of the admins verify this patch?

[andrewor14]

ok to test. Hey @jackylk could you be a little more specific in your log message? (What records, logged to where?)

[SparkQA]

QA tests have started for PR 2791 at commit 3bf43c7.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2791 at commit 3bf43c7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21833/
Test PASSed.

[jackylk]

how about this?

        logInfo(s"$count records of RDD_${self.id} are written to ${hadoopConf.get("mapred.output.dir")}")

[pwendell]

What about just removing this instead of logging it? If it's un-used, we can just remove it.

[jackylk]

ok

[SparkQA]

QA tests have started for PR 2791 at commit a874047.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2791 at commit a874047.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21853/
Test PASSed.

[andrewor14]

Ok thanks I merge

[andrewor14]

Hey @jackylk do you have a JIRA account?

[jackylk]

For this PR, I used wangfei's account in JIRA. I will create my account next time.

[andrewor14]

Ok, I'll put you under wangfei