-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HBASE-26999 HStore should try write WAL compaction marker before repl… #4407
Conversation
…acing compacted files in StoreEngine
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I said on the jira, please hold on merging this 'simple' PR.
We need to discuss more here.
Thanks~
🎊 +1 overall
This message was automatically generated. |
Thanks for your comments, Duo. Just pasting the jira discussion here:
Makes sense. I guess both ways are problematic. But maybe we are more likely to fail at wal marker writing time?
Agree it would be the ideal. It's very challenging, though, to guarantee that all threads changing file system state would be interrupted upon a detection that RS is aborting. The store may be closing separately to the compaction. There's also the compacted files discharger chore running in the background. As an alternative for this immediate problem, we may keep the current order: update SFT, update SFM, write wal marker, but rollback SFM update in case of errors on write wal marker? |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the current approach can solve the problem, so +1.
Will open follow on issues to do some refactoring, for example, move the write compaction method completely to StoreEngine, which may make the code a bit cleaner...
Thanks @wchevreuil for the quick actions.
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
#4407) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Andrew Purtell <apurtell@apache.org>
Please do not ignore the spotless error before merging? |
apache#4407) Change-Id: I3374bd718055de0922f4793683a730ec00e894c9 Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Andrew Purtell <apurtell@apache.org>
…acing compacted files in StoreEngine