[SPARK-20680][SQL][FOLLOW-UP] Add HiveVoidType in HiveClientImpl #29423
Conversation
| val columnType = getSparkSQLDataType(hc) | ||
| val metadata = if (hc.getType != columnType.catalogString) { | ||
| val metadata = if (hc.getType != columnType.catalogString && | ||
| hc.getType != HiveVoidType.catalogString) { |
There was a problem hiding this comment.
Avoid void type add to Metadata.
Without this, Hive void type will be put in Metadata when we load hive column.
There was a problem hiding this comment.
what's the drawback of it?
BTW this check only works for top-level void columns, not inner fields.
There was a problem hiding this comment.
No sure it will be, just think we do not need to use void type in other place.
| } | ||
| } | ||
|
|
||
| test("Add HiveVoidType to compatible with Hive void type") { |
There was a problem hiding this comment.
let's add the JIRA ticket ID in the beginning.
| override def simpleString: String = "void" | ||
| } | ||
|
|
||
| case object HiveVoidType extends HiveVoidType { |
There was a problem hiding this comment.
can we use one single object instead of one class + one object?
|
Test build #127399 has finished for PR 29423 at commit
|
|
|
||
| test("SPARK-20680: Add HiveVoidType to compatible with Hive void type") { | ||
| withView("v1") { | ||
| sql("drop view if exists v1") |
There was a problem hiding this comment.
can you check which test creates view v1 but forget to clear it?
|
Test build #127408 has finished for PR 29423 at commit
|
| sql("CREATE DATABASE IF NOT EXISTS db2") | ||
| sql("USE db2") | ||
| checkAnswer(spark.table("default.v1"), Row(1)) | ||
| sql("DROP VIEW v1") |
There was a problem hiding this comment.
Just a patch, the view v1 create in default db, but current db is db2.
There was a problem hiding this comment.
can we use withView("default.v1")
|
Test build #127414 has finished for PR 29423 at commit
|
|
Test build #127427 has finished for PR 29423 at commit
|
|
thanks, merging to master! |
|
thanks for merging! |
3 participants
What changes were proposed in this pull request?
Discussion with comment.
Add
HiveVoidTypeclass inHiveClientImplthen we can replaceNullTypetoHiveVoidTypebefore we call hive client.Why are the changes needed?
Better compatible with hive.
More details in #29244.
Does this PR introduce any user-facing change?
Yes, user can create view with null type in Hive.
How was this patch tested?
New test.