[SPARK-14237][SQL] De-duplicate partition value appending logic in various buildReader() implementations

[liancheng]

What changes were proposed in this pull request?

Various data sources share approximately the same code for partition value appending. This PR adds a utility method to eliminate this duplication.

How was this patch tested?

Existing tests should do the work.

[SparkQA]

Test build #54431 has finished for PR 12033 at commit 01fbf17.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • // check the type in next() and we get a class cast exception. If we make that function

[SparkQA]

Test build #54436 has finished for PR 12033 at commit 60a0ef0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[cloud-fan]

We will send the reader function to executors, and run it many times for all the files in that RDD partition. However, the projection we used to append partition values can be only initialized once.

Should we extend the reader function to make it have an initialize method? e.g. we can create a trait ReaderFunction and define the initialize and execute API.

[liancheng]

Makes sense. We may also add a appendPartitionValues method in buildReader, so that partition value appending can be abstracted away using a default implementation. Only special data sources like Parquet need to override and take care of it.

[yhuai]

want to quickly update this pr?

[liancheng]

Closing this one in favor of #12866. @cloud-fan Your comment is also addressed there.