[SPARK-6910] [SQL] Support for pushing predicates down to metastore for partition pruning

[liancheng]

This PR forks PR #7421 authored by @piaozhexiu and adds a workaround for fixing the occasional test failures occurred in PR #7421. Please refer to these two comments for details.

(To any committers who might merge this PR: please don't forget to set the "author" field to @piaozhexiu.)

[liancheng]

Gonna fire a bunch of Jenkins builds to see whether the workaround works...

[SparkQA]

Test build #1103 has started for PR 7492 at commit 1bb9b52.

[liancheng]

OK, I guess 10 builds should be enough :)

[SparkQA]

Test build #1100 has finished for PR 7492 at commit 1bb9b52.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1102 has finished for PR 7492 at commit 1bb9b52.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1101 has finished for PR 7492 at commit 1bb9b52.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #37712 has finished for PR 7492 at commit 1bb9b52.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

Um, the last wave of Jenkins builds failed due to an embarrassing typo... Updated the PR, but Jenkins is being shut down now. Will try to fire another build wave later.

[SparkQA]

Test build #37713 has finished for PR 7492 at commit 53b46b6.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #37714 has finished for PR 7492 at commit 536ac41.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1104 has finished for PR 7492 at commit 536ac41.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1108 has finished for PR 7492 at commit 536ac41.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1106 has finished for PR 7492 at commit 536ac41.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1105 has finished for PR 7492 at commit 536ac41.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1107 has finished for PR 7492 at commit 536ac41.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

@marmbrus @piaozhexiu The last five builds all passed. Seems that this workaround is valid. Just fired another 3 builds to double check.

[piaozhexiu]

@liancheng Thank you so much for the fix! Look forward to merging the patch again.

[SparkQA]

Test build #1110 has finished for PR 7492 at commit 536ac41.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1111 has finished for PR 7492 at commit 536ac41.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #1112 has finished for PR 7492 at commit 536ac41.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

@marmbrus I think it's probably OK to merge this one first. But I still haven't got any clue about the root cause mentioned in #7421 (comment) yet.

[marmbrus]

Great, thanks guys!