Skip to content

Conversation

@zeocio
Copy link

@zeocio zeocio commented Nov 12, 2015

…ishable

Propagate pushed filters to PhyicalRDD in DataSourceStrategy.apply

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets call it extraInformation in both places.

@marmbrus
Copy link
Contributor

Can you fix the title / description so it doesn't truncate. Maybe add a test too so this doesn't get lost in refactoring.

@zeocio
Copy link
Author

zeocio commented Nov 13, 2015

@marmbrus i was thinking about test cases, which sql/xxxSuite should this test case be added to?

@marmbrus
Copy link
Contributor

maybe sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala ?

@zeocio zeocio changed the title [SPARK-11390] [SQL] Query plan with/without filterPushdown indistingu… [SPARK-11390] [SQL] Query plan with/without filterPushdown indistinguishable Nov 13, 2015
@zeocio
Copy link
Author

zeocio commented Nov 13, 2015

Changes to string formatting of Filter expr can break the test case. Did this recently change from "(key#27 = 15)" to "EqualTo(key,15)" ?

@zeocio
Copy link
Author

zeocio commented Nov 13, 2015

@marmbrus is the PR good to go?

@yhuai
Copy link
Contributor

yhuai commented Nov 13, 2015

ok to test

@SparkQA
Copy link

SparkQA commented Nov 13, 2015

Test build #45891 has finished for PR 9679 at commit e8a40cc.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 13, 2015

Test build #45900 has finished for PR 9679 at commit 96684bc.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 14, 2015

Test build #45903 has finished for PR 9679 at commit c697837.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Nov 16, 2015

Test build #45960 has finished for PR 9679 at commit 9597b39.

  • This patch passes all tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

…ishable

Propagate pushed filters to PhyicalRDD in DataSourceStrategy.apply

Add test case with parquet data source.

Style fix.

style fix

Revert PhysicalRDD string formatting.
@SparkQA
Copy link

SparkQA commented Nov 16, 2015

Test build #46011 has finished for PR 9679 at commit 7e47f43.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * case class JSONOptions(\n

@zeocio
Copy link
Author

zeocio commented Nov 16, 2015

Please merge.

@marmbrus
Copy link
Contributor

Thanks, merging to master and 1.6.

asfgit pushed a commit that referenced this pull request Nov 16, 2015
…shable

…ishable

Propagate pushed filters to PhyicalRDD in DataSourceStrategy.apply

Author: Zee Chen <zeechen@us.ibm.com>

Closes #9679 from zeocio/spark-11390.

(cherry picked from commit 985b38d)
Signed-off-by: Michael Armbrust <michael@databricks.com>
@asfgit asfgit closed this in 985b38d Nov 16, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants