Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-22810 Initialize an separate ThreadPoolExecutor for taking/restoring snapshot #486

Merged
merged 1 commit into from
Aug 15, 2019

Conversation

openinx
Copy link
Member

@openinx openinx commented Aug 13, 2019

…oring snapshot

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 38 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 hbaseanti 0 Patch does not have any anti-patterns.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+1 mvninstall 337 master passed
+1 compile 54 master passed
+1 checkstyle 80 master passed
+1 shadedjars 274 branch has no errors when building our shaded downstream artifacts.
+1 javadoc 37 master passed
0 spotbugs 254 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 250 master passed
_ Patch Compile Tests _
+1 mvninstall 304 the patch passed
+1 compile 56 the patch passed
+1 javac 56 the patch passed
-1 checkstyle 77 hbase-server: The patch generated 4 new + 167 unchanged - 2 fixed = 171 total (was 169)
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedjars 274 patch has no errors when building our shaded downstream artifacts.
+1 hadoopcheck 946 Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 javadoc 33 the patch passed
+1 findbugs 255 the patch passed
_ Other Tests _
+1 unit 6636 hbase-server in the patch passed.
+1 asflicense 27 The patch does not generate ASF License warnings.
10077
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/1/artifact/out/Dockerfile
GITHUB PR #486
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 4d977315dc04 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-486/out/precommit/personality/provided.sh
git revision master / 8c1edb3
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/1/artifact/out/diff-checkstyle-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/1/testReport/
Max. process+thread count 5094 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/1/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 46 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 hbaseanti 0 Patch does not have any anti-patterns.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+1 mvninstall 331 master passed
+1 compile 52 master passed
+1 checkstyle 75 master passed
+1 shadedjars 275 branch has no errors when building our shaded downstream artifacts.
+1 javadoc 36 master passed
0 spotbugs 256 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 253 master passed
_ Patch Compile Tests _
+1 mvninstall 301 the patch passed
+1 compile 53 the patch passed
+1 javac 53 the patch passed
-1 checkstyle 75 hbase-server: The patch generated 4 new + 167 unchanged - 2 fixed = 171 total (was 169)
+1 whitespace 1 The patch has no whitespace issues.
+1 shadedjars 265 patch has no errors when building our shaded downstream artifacts.
+1 hadoopcheck 970 Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 javadoc 33 the patch passed
+1 findbugs 249 the patch passed
_ Other Tests _
-1 unit 6854 hbase-server in the patch failed.
+1 asflicense 32 The patch does not generate ASF License warnings.
10271
Reason Tests
Failed junit tests hadoop.hbase.master.assignment.TestOpenRegionProcedureHang
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/2/artifact/out/Dockerfile
GITHUB PR #486
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 499a55e83057 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-486/out/precommit/personality/provided.sh
git revision master / e69af5a
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/2/artifact/out/diff-checkstyle-hbase-server.txt
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/2/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/2/testReport/
Max. process+thread count 4524 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/2/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 37 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 hbaseanti 0 Patch does not have any anti-patterns.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ master Compile Tests _
+1 mvninstall 323 master passed
+1 compile 53 master passed
+1 checkstyle 74 master passed
+1 shadedjars 261 branch has no errors when building our shaded downstream artifacts.
+1 javadoc 38 master passed
0 spotbugs 207 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 206 master passed
_ Patch Compile Tests _
+1 mvninstall 297 the patch passed
+1 compile 52 the patch passed
+1 javac 52 the patch passed
-1 checkstyle 72 hbase-server: The patch generated 4 new + 167 unchanged - 2 fixed = 171 total (was 169)
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedjars 263 patch has no errors when building our shaded downstream artifacts.
+1 hadoopcheck 905 Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 javadoc 32 the patch passed
+1 findbugs 243 the patch passed
_ Other Tests _
+1 unit 6706 hbase-server in the patch passed.
+1 asflicense 24 The patch does not generate ASF License warnings.
9937
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/3/artifact/out/Dockerfile
GITHUB PR #486
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux aec1cc3557ba 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-486/out/precommit/personality/provided.sh
git revision master / 27ed2ac
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/3/artifact/out/diff-checkstyle-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/3/testReport/
Max. process+thread count 4995 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/3/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@@ -1431,6 +1431,8 @@ private void startServiceThreads() throws IOException{
conf.getInt("hbase.master.executor.meta.serverops.threads", 5));
this.executorService.startExecutorService(ExecutorType.M_LOG_REPLAY_OPS,
conf.getInt("hbase.master.executor.logreplayops.threads", 10));
this.executorService.startExecutorService(ExecutorType.MASTER_SNAPSHOT_OPERATIONS,
conf.getInt("hbase.master.executor.snapshot.threads", 3));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better make a static field for this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 84 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 hbaseanti 0 Patch does not have any anti-patterns.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ master Compile Tests _
0 mvndep 40 Maven dependency ordering for branch
+1 mvninstall 455 master passed
+1 compile 96 master passed
+1 checkstyle 136 master passed
+1 shadedjars 376 branch has no errors when building our shaded downstream artifacts.
+1 javadoc 64 master passed
0 spotbugs 308 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 366 master passed
_ Patch Compile Tests _
0 mvndep 17 Maven dependency ordering for patch
+1 mvninstall 413 the patch passed
+1 compile 100 the patch passed
+1 javac 100 the patch passed
-1 checkstyle 104 hbase-server: The patch generated 3 new + 162 unchanged - 7 fixed = 165 total (was 169)
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedjars 371 patch has no errors when building our shaded downstream artifacts.
+1 hadoopcheck 1282 Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 javadoc 65 the patch passed
+1 findbugs 393 the patch passed
_ Other Tests _
+1 unit 204 hbase-common in the patch passed.
-1 unit 13817 hbase-server in the patch failed.
+1 asflicense 66 The patch does not generate ASF License warnings.
18990
Reason Tests
Failed junit tests hadoop.hbase.util.TestFromClientSide3WoUnsafe
hadoop.hbase.client.TestFromClientSide3
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/4/artifact/out/Dockerfile
GITHUB PR #486
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux cd6837ae2514 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-486/out/precommit/personality/provided.sh
git revision master / 27ed2ac
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/4/artifact/out/diff-checkstyle-hbase-server.txt
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/4/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/4/testReport/
Max. process+thread count 4725 (vs. ulimit of 10000)
modules C: hbase-common hbase-server U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/4/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 38 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 hbaseanti 0 Patch does not have any anti-patterns.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ master Compile Tests _
0 mvndep 39 Maven dependency ordering for branch
+1 mvninstall 340 master passed
+1 compile 74 master passed
+1 checkstyle 93 master passed
+1 shadedjars 263 branch has no errors when building our shaded downstream artifacts.
+1 javadoc 54 master passed
0 spotbugs 257 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 305 master passed
_ Patch Compile Tests _
0 mvndep 14 Maven dependency ordering for patch
+1 mvninstall 286 the patch passed
+1 compile 71 the patch passed
+1 javac 71 the patch passed
-1 checkstyle 69 hbase-server: The patch generated 3 new + 162 unchanged - 7 fixed = 165 total (was 169)
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedjars 264 patch has no errors when building our shaded downstream artifacts.
+1 hadoopcheck 893 Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 javadoc 55 the patch passed
+1 findbugs 310 the patch passed
_ Other Tests _
+1 unit 174 hbase-common in the patch passed.
-1 unit 6870 hbase-server in the patch failed.
+1 asflicense 38 The patch does not generate ASF License warnings.
10643
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/5/artifact/out/Dockerfile
GITHUB PR #486
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 2b37abec7345 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-486/out/precommit/personality/provided.sh
git revision master / 53db390
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/5/artifact/out/diff-checkstyle-hbase-server.txt
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/5/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/5/testReport/
Max. process+thread count 4656 (vs. ulimit of 10000)
modules C: hbase-common hbase-server U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/5/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@Apache9 Apache9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1.

@Apache9
Copy link
Contributor

Apache9 commented Aug 15, 2019

But please fix the checkstyle issues before merging...

@openinx openinx merged commit 8ffc45a into apache:master Aug 15, 2019
@openinx
Copy link
Member Author

openinx commented Aug 15, 2019

The checkstyle is not an issue I think, the '(' is preceded by a whitespace because we want to make the code align as before :

  /**
   * Messages originating from Client to Master.<br>
   * C_M_CREATE_TABLE<br>
   * Client asking Master to create a table.
   */
  C_M_CREATE_TABLE          (47, ExecutorType.MASTER_TABLE_OPERATIONS),
  /**
   * Messages originating from Client to Master.<br>
   * C_M_SNAPSHOT_TABLE<br>
   * Client asking Master to snapshot an offline table.
   */
  C_M_SNAPSHOT_TABLE        (48, ExecutorType.MASTER_SNAPSHOT_OPERATIONS),
  /**
   * Messages originating from Client to Master.<br>
   * C_M_RESTORE_SNAPSHOT<br>
   * Client asking Master to restore a snapshot.
   */
  C_M_RESTORE_SNAPSHOT      (49, ExecutorType.MASTER_SNAPSHOT_OPERATIONS),

asfgit pushed a commit that referenced this pull request Aug 15, 2019
asfgit pushed a commit that referenced this pull request Aug 15, 2019
asfgit pushed a commit that referenced this pull request Aug 15, 2019
asfgit pushed a commit that referenced this pull request Aug 15, 2019
asfgit pushed a commit that referenced this pull request Aug 15, 2019
asfgit pushed a commit that referenced this pull request Aug 15, 2019
asfgit pushed a commit that referenced this pull request Aug 15, 2019
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 58 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 hbaseanti 0 Patch does not have any anti-patterns.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ master Compile Tests _
0 mvndep 39 Maven dependency ordering for branch
+1 mvninstall 426 master passed
+1 compile 100 master passed
+1 checkstyle 139 master passed
+1 shadedjars 343 branch has no errors when building our shaded downstream artifacts.
+1 javadoc 70 master passed
0 spotbugs 327 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 390 master passed
_ Patch Compile Tests _
0 mvndep 18 Maven dependency ordering for patch
+1 mvninstall 405 the patch passed
+1 compile 107 the patch passed
+1 javac 107 the patch passed
-1 checkstyle 104 hbase-server: The patch generated 3 new + 162 unchanged - 7 fixed = 165 total (was 169)
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedjars 359 patch has no errors when building our shaded downstream artifacts.
+1 hadoopcheck 1294 Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 javadoc 70 the patch passed
+1 findbugs 432 the patch passed
_ Other Tests _
+1 unit 217 hbase-common in the patch passed.
-1 unit 19484 hbase-server in the patch failed.
+1 asflicense 58 The patch does not generate ASF License warnings.
24660
Reason Tests
Failed junit tests hadoop.hbase.client.TestFromClientSide
hadoop.hbase.replication.TestReplicationDisableInactivePeer
hadoop.hbase.snapshot.TestFlushSnapshotFromClient
hadoop.hbase.client.TestFromClientSide3
hadoop.hbase.replication.TestReplicationKillSlaveRSWithSeparateOldWALs
hadoop.hbase.replication.TestReplicationSmallTests
hadoop.hbase.master.TestAssignmentManagerMetrics
hadoop.hbase.replication.TestReplicationSmallTestsSync
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/6/artifact/out/Dockerfile
GITHUB PR #486
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 8e8f7bc61373 4.4.0-154-generic #181-Ubuntu SMP Tue Jun 25 05:29:03 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-486/out/precommit/personality/provided.sh
git revision master / d9d5f69
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/6/artifact/out/diff-checkstyle-hbase-server.txt
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/6/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/6/testReport/
Max. process+thread count 4829 (vs. ulimit of 10000)
modules C: hbase-common hbase-server U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-486/6/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

infraio pushed a commit to infraio/hbase that referenced this pull request Aug 17, 2020
symat pushed a commit to symat/hbase that referenced this pull request Feb 17, 2021
…oring snapshot (apache#486)

(cherry picked from commit b4734e7)

Change-Id: Id5c952da1949f7039d6caa5329c6599ca54fbba4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants