-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-13321][SQL] Support nested UNION in parser #11204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| | | ||
| selectStatement[topLevel] | ||
| | | ||
| (LPAREN selectStatement[true]) => nestedSetOpSelectStatement[topLevel] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Passing true or false here doesn't really matter. This topLevel argument works only in setOpSelectStatement rule. We will pass in the real topLevel argument into the rule later.
|
Test build #51295 has finished for PR 11204 at commit
|
| } | ||
|
|
||
| test("nesting UNION") { | ||
| parser.parsePlan("SELECT `u_1`.`id` FROM (((SELECT `t0`.`id` FROM `default`.`t0`) " + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you test the actual plan?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. I will update this.
|
Test build #51310 has finished for PR 11204 at commit
|
|
Test build #51314 has finished for PR 11204 at commit
|
|
|
||
| comparePlans(parsedSame, expected) | ||
|
|
||
| val parsed2 = parser.parsePlan( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Recursively nested UNION.
Conflicts: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/CatalystQlSuite.scala
|
@hvanhovell Do you have other comments on this PR? |
|
Test build #51630 has finished for PR 11204 at commit
|
|
LGTM |
|
cc @rxin |
|
Thanks - merging in master. |
|
Note: I reverted this patch because HiveCompatibilitySuite.union16 was hanging from this. Somehow we weren't running test cases in HiveCompatibilitySuite anymore for pull requests. |
|
@rxin Do you need me to submit a PR to disable the following test case? This test case requires the nested Union support. Thanks! spark/sql/hive/src/test/scala/org/apache/spark/sql/hive/LogicalPlanToSQLSuite.scala Lines 133 to 140 in 8725058
|
|
Please do. |
|
@viirya when you fixed this issue, please make sure you add a unit test aside from Hive compat test. |
|
@rxin ok, I will look into this today. |
JIRA: https://issues.apache.org/jira/browse/SPARK-13321
The following SQL can not be parsed with current parser:
We should fix it.