HBASE-24380 : Provide WAL splitting journal logging #1860
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
apurtell
requested changes
Jun 11, 2020
|
|
||
| abstract List<Path> close() throws IOException; | ||
| abstract List<Path> close(MonitoredTask status) throws IOException; |
| @@ -52,15 +53,16 @@ public RecoveredEditsOutputSink(WALSplitter walSplitter, | |||
| } | |||
|
|
|||
| @Override | |||
| public void append(EntryBuffers.RegionEntryBuffer buffer) throws IOException { | |||
| public void append(EntryBuffers.RegionEntryBuffer buffer, MonitoredTask status) | |||
| @@ -86,26 +88,27 @@ private RecoveredEditsWriter getRecoveredEditsWriter(TableName tableName, byte[] | |||
| } | |||
|
|
|||
| @Override | |||
| public List<Path> close() throws IOException { | |||
| public List<Path> close(MonitoredTask status) throws IOException { | |||
| * @return true when there is no error. | ||
| */ | ||
| private boolean closeWriters() throws IOException { | ||
| private boolean closeWriters(MonitoredTask status) throws IOException { |
| @@ -83,25 +85,26 @@ public void append(EntryBuffers.RegionEntryBuffer buffer) throws IOException { | |||
| } | |||
|
|
|||
| @Override | |||
| public List<Path> close() throws IOException { | |||
| public List<Path> close(MonitoredTask status) throws IOException { | |||
There was a problem hiding this comment.
Does this context have access to the OutputSink's MonitoredTask field? Can we do that instead, so we don't have to change these method signatures?
| @@ -57,7 +58,8 @@ public BoundedRecoveredEditsOutputSink(WALSplitter walSplitter, | |||
| } | |||
|
|
|||
| @Override | |||
| public void append(EntryBuffers.RegionEntryBuffer buffer) throws IOException { | |||
| public void append(EntryBuffers.RegionEntryBuffer buffer, MonitoredTask status) | |||
There was a problem hiding this comment.
Does this context have access to the OutputSink's MonitoredTask field? Can we do that instead, so we don't have to change these method signatures?
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
virajjasani
commented
Jun 16, 2020
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredEditsOutputSink.java
Outdated
Show resolved
Hide resolved
hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java
Outdated
Show resolved
Hide resolved
apurtell
requested changes
Jun 16, 2020
| @@ -233,7 +251,10 @@ void writeRegionEntries(List<WAL.Entry> entries) throws IOException { | |||
| incrementNanoTime(System.nanoTime() - startTime); | |||
| } catch (IOException e) { | |||
| e = e instanceof RemoteException ? ((RemoteException) e).unwrapRemoteException() : e; | |||
| LOG.error(HBaseMarkers.FATAL, "Got while writing log entry to log", e); | |||
| final String errorMsg = | |||
| "Failed to write log entry " + currentWalEntry.toString() + " to log"; | |||
There was a problem hiding this comment.
IOException is thrown by writer.append(logEntry) and when we reach there, we are quite certain that currentWalEntry will be assigned some value for sure.
There was a problem hiding this comment.
But this brings a good point, Exception catch should rather be at a place where it is needed, entire block doesn't need to be within try/catch if only single line of code is throwing specific Exception. Let me take care of this.
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
Picking back... |
asfgit
pushed a commit
that referenced
this pull request
Jun 19, 2020
Signed-off-by: Andrew Purtell <apurtell@apache.org> Conflicts: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredEditsOutputSink.java hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RecoveredEditsOutputSink.java
asfgit
pushed a commit
that referenced
this pull request
Jun 19, 2020
Signed-off-by: Andrew Purtell <apurtell@apache.org> Conflicts: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredEditsOutputSink.java hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RecoveredEditsOutputSink.java
virajjasani
added a commit
to virajjasani/hbase
that referenced
this pull request
Jun 20, 2020
Signed-off-by: Andrew Purtell <apurtell@apache.org>
virajjasani
added a commit
to virajjasani/hbase
that referenced
this pull request
Jun 20, 2020
Signed-off-by: Andrew Purtell <apurtell@apache.org>
virajjasani
added a commit
that referenced
this pull request
Jun 22, 2020
Signed-off-by: Andrew Purtell <apurtell@apache.org>
clarax
pushed a commit
to clarax/hbase
that referenced
this pull request
Nov 15, 2020
Signed-off-by: Andrew Purtell <apurtell@apache.org> Conflicts: hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredEditsOutputSink.java hbase-server/src/main/java/org/apache/hadoop/hbase/wal/BoundedRecoveredHFilesOutputSink.java hbase-server/src/main/java/org/apache/hadoop/hbase/wal/RecoveredEditsOutputSink.java
wchevreuil
pushed a commit
to wchevreuil/hbase
that referenced
this pull request
May 24, 2021
…ging) (apache#1860) (apache#1937) Signed-off-by: Andrew Purtell <apurtell@apache.org> Signed-off-by: Guanghao Zhang <zghao@apache.org> (cherry picked from commit d154c79) Change-Id: I4c1c975d3408005a76072ab971c0a5c1217f47cf
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Sample output: