Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-26681 Introduce a little RAMBuffer for bucketcache to reduce gc and improve throughput #4043

Merged
merged 3 commits into from
Aug 2, 2022

Conversation

YutSean
Copy link
Contributor

@YutSean YutSean commented Jan 18, 2022

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 7m 59s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚 mvninstall 14m 28s branch-1 passed
+1 💚 compile 0m 46s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 compile 0m 51s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 checkstyle 1m 57s branch-1 passed
+1 💚 shadedjars 3m 38s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 49s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 0m 43s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗 spotbugs 3m 20s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 3m 17s branch-1 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 19s the patch passed
+1 💚 compile 0m 47s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javac 0m 47s the patch passed
+1 💚 compile 0m 52s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 javac 0m 52s the patch passed
-1 ❌ checkstyle 1m 44s hbase-server: The patch generated 1 new + 16 unchanged - 0 fixed = 17 total (was 16)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 3m 20s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 8m 2s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 33s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 0m 43s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
-1 ❌ findbugs 3m 16s hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
_ Other Tests _
-1 ❌ unit 157m 15s hbase-server in the patch failed.
+1 💚 asflicense 0m 34s The patch does not generate ASF License warnings.
217m 57s
Reason Tests
FindBugs module:hbase-server
Integral division result cast to double or float in new org.apache.hadoop.hbase.io.hfile.bucket.BufferedBucketCache(String, long, int, int[], int, int, String, int, Configuration) At BufferedBucketCache.java:double or float in new org.apache.hadoop.hbase.io.hfile.bucket.BufferedBucketCache(String, long, int, int[], int, int, String, int, Configuration) At BufferedBucketCache.java:[line 68]
Failed junit tests hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint
hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/artifact/out/Dockerfile
GITHUB PR #4043
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 3c8b22b2c05c 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4043/out/precommit/personality/provided.sh
git revision branch-1 / a3e7d36
Default Java Azul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/artifact/out/diff-checkstyle-hbase-server.txt
findbugs https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/artifact/out/new-findbugs-hbase-server.html
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/artifact/out/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/testReport/
Max. process+thread count 4708 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/console
versions git=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 6s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚 mvninstall 12m 48s branch-1 passed
+1 💚 compile 0m 45s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 compile 0m 51s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 checkstyle 1m 55s branch-1 passed
+1 💚 shadedjars 3m 40s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 51s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 0m 44s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗 spotbugs 3m 22s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 3m 18s branch-1 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 20s the patch passed
+1 💚 compile 0m 45s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javac 0m 45s the patch passed
+1 💚 compile 0m 51s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 javac 0m 51s the patch passed
-1 ❌ checkstyle 1m 44s hbase-server: The patch generated 1 new + 16 unchanged - 0 fixed = 17 total (was 16)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 3m 23s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 6m 58s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 32s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 0m 43s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 findbugs 3m 11s the patch passed
_ Other Tests _
-1 ❌ unit 164m 13s hbase-server in the patch failed.
+1 💚 asflicense 0m 38s The patch does not generate ASF License warnings.
215m 12s
Reason Tests
Failed junit tests hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint
hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
hadoop.hbase.TestPartialResultsFromClientSide
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/2/artifact/out/Dockerfile
GITHUB PR #4043
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 2e27082b906b 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4043/out/precommit/personality/provided.sh
git revision branch-1 / a3e7d36
Default Java Azul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/2/artifact/out/diff-checkstyle-hbase-server.txt
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/2/artifact/out/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/2/testReport/
Max. process+thread count 4509 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/2/console
versions git=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚 mvninstall 13m 58s branch-1 passed
+1 💚 compile 0m 59s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 compile 1m 3s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 checkstyle 2m 23s branch-1 passed
+1 💚 shadedjars 4m 16s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 1m 0s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 0m 57s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗 spotbugs 3m 49s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 3m 44s branch-1 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 54s the patch passed
+1 💚 compile 0m 52s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javac 0m 52s the patch passed
+1 💚 compile 0m 58s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 javac 0m 58s the patch passed
+1 💚 checkstyle 2m 0s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 4m 19s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 8m 48s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 38s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 0m 51s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 findbugs 3m 59s the patch passed
_ Other Tests _
-1 ❌ unit 189m 59s hbase-server in the patch failed.
+1 💚 asflicense 0m 37s The patch does not generate ASF License warnings.
250m 11s
Reason Tests
Failed junit tests hadoop.hbase.replication.TestReplicationSource
hadoop.hbase.namequeues.TestNamedQueueRecorder
hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles
hadoop.hbase.snapshot.TestExportSnapshot
hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/3/artifact/out/Dockerfile
GITHUB PR #4043
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 93328c06d304 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4043/out/precommit/personality/provided.sh
git revision branch-1 / a3e7d36
Default Java Azul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/3/artifact/out/patch-unit-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/3/testReport/
Max. process+thread count 4578 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/3/console
versions git=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@YutSean
Copy link
Contributor Author

YutSean commented Jan 19, 2022

The failed UTs passed locally though.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 4m 54s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚 mvninstall 10m 16s branch-1 passed
+1 💚 compile 0m 43s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 compile 0m 50s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 checkstyle 1m 46s branch-1 passed
+1 💚 shadedjars 3m 21s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 48s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 0m 42s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗 spotbugs 3m 9s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 3m 6s branch-1 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 10s the patch passed
+1 💚 compile 0m 46s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javac 0m 46s the patch passed
+1 💚 compile 0m 49s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 javac 0m 49s the patch passed
+1 💚 checkstyle 1m 31s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 3m 2s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 5m 17s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 32s the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 0m 41s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 findbugs 2m 59s the patch passed
_ Other Tests _
+1 💚 unit 130m 27s hbase-server in the patch passed.
+1 💚 asflicense 0m 41s The patch does not generate ASF License warnings.
179m 0s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/artifact/out/Dockerfile
GITHUB PR #4043
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 59e24a57f1fc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-4043/out/precommit/personality/provided.sh
git revision branch-1 / a2406f3
Default Java Azul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/testReport/
Max. process+thread count 4549 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4043/1/console
versions git=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9 Apache9 merged commit 0a0a474 into apache:branch-1 Aug 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants