[SPARK-16134][SQL] optimizer rules for typed filter

[cloud-fan]

What changes were proposed in this pull request?

This PR adds 3 optimizer rules for typed filter:

1. push typed filter down through SerializeFromObject and eliminate the deserialization in filter condition.
2. pull typed filter up through SerializeFromObject and eliminate the deserialization in filter condition.
3. combine adjacent typed filters and share the deserialized object among all the condition expressions.

This PR also adds TypedFilter logical plan, to separate it from normal filter, so that the concept is more clear and it's easier to write optimizer rules.

How was this patch tested?

TypedFilterOptimizationSuite

[cloud-fan]

cc @yhuai @liancheng @clockfly

[SparkQA]

Test build #61035 has finished for PR 13846 at commit 6200460.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class TypedFilter(

[cloud-fan]

retest this please

[SparkQA]

Test build #61083 has finished for PR 13846 at commit 6200460.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class TypedFilter(

[ueshin]

I think we can also push down if TypedFilter( Filter( SerializeFromObject(child) ) ) into Filter( SerializeFromObject( TypedFilter(child) ) ).
e.g. ds.map(...).filter(byExpr).filter(byFunc).
What do you think?

[cloud-fan]

Well, it's true, and Filter can be any other unary operators whose output is derived from its child, e.g. Sort.

However, I don't think ds.map(...).filter(byExpr).filter(byFunc) is a common case, i.e. mixing typed and untyped operations interlaced. If there is an easy and general way to optimize it, I'm happy to have it, or I'd like to leave it.

what do you think?

[ueshin]

Hmm, I don't think mixing typed and untyped is not a common case, but I don't have any idea to optimize easy and general way so I think we can leave it for now.

[liancheng]

Shall we prepend rather than append found filters here? Otherwise filter predicates will be evaluated in reverse order after being combined. Also would be nice to comment about this.

[SparkQA]

Test build #61463 has finished for PR 13846 at commit c1e7d9f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #61466 has finished for PR 13846 at commit 1a2241f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

LGTM, merged to master.

(Also merged to branch-2.0 by mistake, will revert it ASAP. Sorry for the trouble.)

[liancheng]

Reverted the commit on branch-2.0.