-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HBASE-28064:Implement truncate_region command #5462
HBASE-28064:Implement truncate_region command #5462
Conversation
💔 -1 overall
This message was automatically generated. |
0d2c228
to
3bc82c7
Compare
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
3bc82c7
to
2a6ae93
Compare
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
recheck |
retest |
The failed tests is not looking related to my changes. Not sure if its a transient issue. |
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
Outdated
Show resolved
Hide resolved
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java
Outdated
Show resolved
Hide resolved
hbase-protocol-shaded/src/main/protobuf/server/master/MasterProcedure.proto
Outdated
Show resolved
Hide resolved
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Outdated
Show resolved
Hide resolved
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
Show resolved
Hide resolved
final MasterFileSystem mfs = env.getMasterServices().getMasterFileSystem(); | ||
final Path tableDir = CommonFSUtils.getTableDir(mfs.getRootDir(), getTableName()); | ||
final Configuration conf = env.getMasterConfiguration(); | ||
HRegionFileSystem.deleteRegionFromFileSystem(conf, mfs.getFileSystem(), tableDir, region); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would delete the region dir entirely. I don't see where we recreate the region dir, later. Isn't this a problem?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would delete the region dir entirely. I don't see where we recreate the region dir, later. Isn't this a problem?
While developing feature, I observed that subsequent task "TRUNCATE_REGION_MAKE_ONLINE" creates directory for region. So not added logic to create the directory explicitely.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing that. Yeah, that should be ok. Just one extra comment on the rollback: I see you are considering it only if we are in TRUNCATE_REGION_PRE_OPERATION. Couldn't we rollback from TRUNCATE_REGION_MAKE_OFFLINE too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing that. Yeah, that should be ok. Just one extra comment on the rollback: I see you are considering it only if we are in TRUNCATE_REGION_PRE_OPERATION. Couldn't we rollback from TRUNCATE_REGION_MAKE_OFFLINE too?
Thanks @wchevreuil. This is valid concern. I this its already handled here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing it out!
...e-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateRegionProcedure.java
Outdated
Show resolved
Hide resolved
...e-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateRegionProcedure.java
Outdated
Show resolved
Hide resolved
...e-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateRegionProcedure.java
Outdated
Show resolved
Hide resolved
Thanks for submitting this PR, looks like a great idea. I have a few questions/comments after an initial review. Please reply on those so that we can get this progressing. Thanks, Wellington. |
2a6ae93
to
523e4f9
Compare
Thanks @wchevreuil for reviewing this. I have incorporated most of the review comments and answered the questions inline. Can you please check? |
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, +1. The UT failures looks unrelated.
...e-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateRegionProcedure.java
Outdated
Show resolved
Hide resolved
...e-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateRegionProcedure.java
Outdated
Show resolved
Hide resolved
...e-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateRegionProcedure.java
Outdated
Show resolved
Hide resolved
...e-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateRegionProcedure.java
Outdated
Show resolved
Hide resolved
...e-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateRegionProcedure.java
Outdated
Show resolved
Hide resolved
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi2.java
Outdated
Show resolved
Hide resolved
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAsyncRegionAdminApi2.java
Show resolved
Hide resolved
Thanks @vaijosh for working on this. Sorry for late review. I have added a few comments/suggestions. Please check. |
|
523e4f9
to
3366b2a
Compare
…ctly from FS Implemented truncate_region command.
3366b2a
to
07c871e
Compare
💔 -1 overall
This message was automatically generated. |
...e-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateRegionProcedure.java
Show resolved
Hide resolved
Skimmed latest push and updated comments. Looks fine high level. Also one suggestion, please avoid force push and keep just 1 commit. It's very hard to review what all changed. We anyways squash and merge, so should be fine to have several commits. |
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
Thanks @NihalJain for the suggestion. But if this is not convenient, next time onwards I can do separate commits, and we can squash commits in the end. |
Yes it is a little inconvenient to compare across previous commits. And thanks for considering this. I have approved the PR. Looks good to me, please check once on users field comment before commit. |
Hi @NihalJain
Yes this was valid concern. But I have simulated and tested crash scenario and checked that we don't get NPE. |
Awesome thanks for the due diligence and again thanks for working on this one. :) |
…ctly from FS Implemented review comment.
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Nihal Jain <nihaljain@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Nihal Jain <nihaljain@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Nihal Jain <nihaljain@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Nihal Jain <nihaljain@apache.org>
Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Nihal Jain <nihaljain@apache.org>
HBASE-28064:Implement truncate_region command to truncate region directly from FS
Implemented truncate_region command and added unit/system tests.