HIVE-27224: Enhance drop table/partition command #5851
Conversation
asf-ci-hive
added
tests pending
tests failed
and removed
tests pending
tests failed
labels
asf-ci-hive
added
tests unstable
tests pending
tests failed
and removed
tests pending
tests unstable
tests failed
labels
dengzhhu653
changed the title
asf-ci-hive
added
tests passed
tests pending
and removed
tests pending
tests passed
labels
|
dengzhhu653
force-pushed
the
HIVE-27224
branch
from
e603abc to
e54f5a9
Compare
asf-ci-hive
added
tests failed
tests pending
tests unstable
and removed
tests pending
tests failed
labels
asf-ci-hive
added
tests pending
tests passed
and removed
tests unstable
tests pending
labels
| if (ugiTransport.getClientUGI() == null) { | ||
| ugiTransport.setClientUGI(clientUgi); | ||
| } | ||
| clientUgi = ugiTransport.getClientUGI(); |
There was a problem hiding this comment.
Is this line unnecessary? clientUgi is already initialized.
There was a problem hiding this comment.
The ugi is identical: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L483-L491,
reuse the ugi cached in ugiTransport if possible so the connection will get the same FileSystem instance from cache in the whole lifetime
...ne-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java
Outdated
Show resolved
Hide resolved
| if (request.isNeedResult()) { | ||
| AddPartitionsHandler addPartsOp = AbstractOperationHandler.offer(this, request); | ||
| if (addPartsOp.success() && request.isNeedResult()) { | ||
| AddPartitionsHandler.AddPartitionsResult addPartsResult = addPartsOp.getResult(); |
There was a problem hiding this comment.
Can we store the partition list in the AddPartitionsResult and return it directly here?
There was a problem hiding this comment.
no enough, the addPartsOp.success() need to check on the state(success or not) of addPartsOp.getResult()
| if (async) { | ||
| OPID_CLEANER.schedule(() -> OPID_TO_HANDLER.remove(id), 1, TimeUnit.HOURS); | ||
| } | ||
| afterExecute(resultV); |
There was a problem hiding this comment.
If afterExecute() is needed only when the execute() is success, we can check the result here
| afterExecute(resultV); | |
| if (resultV != null && resultV.success()) { | |
| afterExecute(resultV); | |
| }``` |
There was a problem hiding this comment.
the afterExecute is also called in case of failure to free up some resources the handler might hold
asf-ci-hive
added
tests pending
tests unstable
and removed
tests passed
tests pending
tests unstable
labels
|
wecharyu
left a comment
wecharyu
left a comment
There was a problem hiding this comment.
LGTM(non-binding), it's a nice enhance to such heavy operations and a good start to split the huge HMSHandler class.
If we have some cycles, we can separate the create table/database, truncate table, and set/update stats from the |
|
Very appreciate for the review, @wecharyu and @saihemanth-cloudera! |
Filed HIVE-29402 to track the future work |
5 participants
What changes were proposed in this pull request?
Why are the changes needed?
Does this PR introduce any user-facing change?
How was this patch tested?