-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-17068][SQL] Make view-usage visible during analysis #14657
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Test build #63811 has finished for PR 14657 at commit
|
|
Test build #63841 has finished for PR 14657 at commit
|
| // Skip projects and subquery aliases added by the Analyzer and the SQLBuilder. | ||
| def cleanQuery(p: LogicalPlan): LogicalPlan = p match { | ||
| case SubqueryAlias(_, child) => cleanQuery(child) | ||
| case SubqueryAlias(_, child, _) => cleanQuery(child) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not that big of a deal, but wouldn't it be easier and more robust if we just do
case s: SubqueryAlias => cleanQuery(s.child)
case p: Project => cleanQuery(p.child)
case child => child
|
Can you update the documentation of relevant classes to make sure we document this new contract clearly? |
|
Test build #63868 has finished for PR 14657 at commit
|
|
LGTM - merging in master. |
|
(I didn't merge this in 2.0) |
| val relationAlias = alias.getOrElse(table) | ||
| if (name.database.isDefined || !tempTables.contains(table)) { | ||
| val metadata = externalCatalog.getTable(db, table) | ||
| val view = Option(metadata.tableType).collect { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we are too conservative here, CatalogTable.tableType should never be null, and we use this assumption in a lot of places.
What changes were proposed in this pull request?
This PR adds a field to subquery alias in order to make the usage of views in a resolved
LogicalPlanmore visible (and more understandable).For example, the following view and query:
...now yields the following analyzed plan:
How was this patch tested?
Added tests for the two code paths in
SessionCatalogSuite(sql/core) andHiveMetastoreCatalogSuite(sql/hive)