[SPARK-17234] [SQL] Table Existence Checking when Index Table with the Same Name Exists #14801
Conversation
|
Can we avoid introducing new exception types? It is super annoying to match those in Python. |
|
Test build #64404 has finished for PR 14801 at commit
|
|
Sure, will revert it back and use the existing |
|
Test build #64455 has finished for PR 14801 at commit
|
|
Test build #64456 has finished for PR 14801 at commit
|
|
retest this please |
|
Test build #64545 has finished for PR 14801 at commit
|
|
cc @rxin @cloud-fan |
| try { | ||
| client.getTableOption(db, table).isDefined | ||
| } catch { | ||
| case e: AnalysisException if e.message.contains("Hive index table is not supported") => true |
There was a problem hiding this comment.
this looks hacky. Actually why we use getTableOption to implement the semantic of tableExists? I'd like to make HiveClient provide the tableExists API directly.
There was a problem hiding this comment.
Yeah, that is a good idea. Let me add it.
|
Test build #64599 has finished for PR 14801 at commit
|
| } | ||
|
|
||
| test(s"$version: tableExists") { | ||
| // No exception should be thrown |
There was a problem hiding this comment.
hive's tableExists may throw exception?
There was a problem hiding this comment.
We can let Hive return an exception. However, we set the flag to false.
|
LGTM, pending jenkins. |
|
Test build #64618 has finished for PR 14801 at commit
|
|
retest this please |
|
Test build #64619 has finished for PR 14801 at commit
|
|
thanks, merging to master! |
|
|
||
| if (tableExists(db, table) && !ignoreIfExists) { | ||
| throw new TableAlreadyExistsException(db = db, table = table) | ||
| } |
There was a problem hiding this comment.
After a few months, I found the above code looks weird. We should follow the same logics in InMemoryCatalog.scala. Will improve it.
4 participants
What changes were proposed in this pull request?
Hive Index tables are not supported by Spark SQL. Thus, we issue an exception when users try to access Hive Index tables. When the internal function
tableExiststries to access Hive Index tables, it always gets the same error message:Hive index table is not supported. This message could be confusing to users, since their SQL operations could be completely unrelated to Hive Index tables. For example, when users try to alter a table to a new name and there exists an index table with the same name, the expected exception should be aTableAlreadyExistsException.This PR made the following changes:
AnalysisExceptiontype:SQLFeatureNotSupportedException. When users try to access anIndex Table, we will issue aSQLFeatureNotSupportedException.tableExistsreturnstruewhen hitting aSQLFeatureNotSupportedExceptionand the feature isHive index table.requireTableNotExistsforSessionCatalog'screateTableAPI; otherwise, the current implementation relies on the Hive's internal checking.How was this patch tested?
Added a test case