HBASE-28821: Optimise the memory-utilisation during persistence of bucketcache. #6240

jhungund · 2024-09-12T07:37:31Z

During the serialisation of backing map of bucketcache persistence, we create a new BackingMapEntry.Builder object for every entry in the bucket map. This can be optimized by creating a single builder object and reusing it for every entry object.

Change-Id: I6f6c05dfa256ce964cd1d66e5050d9424db4be46

Apache-HBase · 2024-09-12T08:29:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 41s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 22s		branch-2 passed
+1 💚	compile	2m 59s		branch-2 passed
+1 💚	checkstyle	0m 40s		branch-2 passed
+1 💚	spotbugs	1m 37s		branch-2 passed
+1 💚	spotless	0m 47s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 14s		the patch passed
+1 💚	compile	2m 57s		the patch passed
+1 💚	javac	2m 57s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 37s		the patch passed
+1 💚	spotbugs	1m 42s		the patch passed
+1 💚	hadoopcheck	17m 19s		Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚	spotless	0m 49s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 11s		The patch does not generate ASF License warnings.
		38m 44s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux e14d9ce6226e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `cb5854b`
Default Java	Eclipse Adoptium-11.0.23+9
Max. process+thread count	77 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-12T11:44:17Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 41s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 15s		branch-2 passed
+1 💚	compile	0m 57s		branch-2 passed
+1 💚	javadoc	0m 27s		branch-2 passed
+1 💚	shadedjars	5m 59s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 5s		the patch passed
+1 💚	compile	0m 57s		the patch passed
+1 💚	javac	0m 57s		the patch passed
+1 💚	javadoc	0m 26s		the patch passed
+1 💚	shadedjars	5m 54s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	207m 18s		hbase-server in the patch passed.
		233m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 59f80d6869e2 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `cb5854b`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/testReport/
Max. process+thread count	4474 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-12T11:59:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 44s		Docker mode activated.
-0 ⚠️	yetus	0m 6s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 27s		branch-2 passed
+1 💚	compile	0m 52s		branch-2 passed
+1 💚	javadoc	0m 27s		branch-2 passed
+1 💚	shadedjars	6m 16s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 12s		the patch passed
+1 💚	compile	0m 52s		the patch passed
+1 💚	javac	0m 52s		the patch passed
+1 💚	javadoc	0m 26s		the patch passed
+1 💚	shadedjars	6m 10s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	222m 33s		hbase-server in the patch passed.
		249m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 643383337f81 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `cb5854b`
Default Java	Eclipse Adoptium-11.0.23+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/testReport/
Max. process+thread count	4382 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-12T12:08:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 47s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 35s		branch-2 passed
+1 💚	compile	0m 44s		branch-2 passed
+1 💚	javadoc	0m 28s		branch-2 passed
+1 💚	shadedjars	4m 57s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 29s		the patch passed
+1 💚	compile	0m 44s		the patch passed
+1 💚	javac	0m 44s		the patch passed
+1 💚	javadoc	0m 26s		the patch passed
+1 💚	shadedjars	4m 56s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	234m 47s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		257m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux da6ebe479e17 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `cb5854b`
Default Java	Temurin-1.8.0_412-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/testReport/
Max. process+thread count	4086 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

rmdmattingly

out of curiosity, have you done any performance testing of this change?

pankaj72981

LGTM

…cketcache. During the serialisation of backing map of bucketcache persistence, we create a new BackingMapEntry.Builder object for every entry in the bucket map. This can be optimized by creating a single builder object and reusing it for every entry object. Change-Id: I6f6c05dfa256ce964cd1d66e5050d9424db4be46

jhungund · 2024-09-16T13:30:06Z

out of curiosity, have you done any performance testing of this change?

The persister thread runs asynchronous and does not impact any of the user operations. Roughtly, for a 2GB backing-map data, the performance of this thread reduced slightly from 30-35sec to 40sec of execution time.

Apache-HBase · 2024-09-16T14:20:16Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 1s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ branch-2 Compile Tests _
+1 💚	mvninstall	4m 33s		branch-2 passed
+1 💚	compile	3m 36s		branch-2 passed
+1 💚	checkstyle	0m 45s		branch-2 passed
+1 💚	spotbugs	2m 5s		branch-2 passed
+1 💚	spotless	1m 0s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	4m 17s		the patch passed
+1 💚	compile	3m 16s		the patch passed
+1 💚	javac	3m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 40s		the patch passed
+1 💚	spotbugs	2m 15s		the patch passed
+1 💚	hadoopcheck	17m 49s		Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚	spotless	0m 53s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 10s		The patch does not generate ASF License warnings.
		44m 27s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux ea031925f5a7 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Eclipse Adoptium-11.0.23+9
Max. process+thread count	77 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-16T17:35:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
-0 ⚠️	yetus	0m 6s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 38s		branch-2 passed
+1 💚	compile	0m 43s		branch-2 passed
+1 💚	javadoc	0m 26s		branch-2 passed
+1 💚	shadedjars	5m 28s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 35s		the patch passed
+1 💚	compile	0m 43s		the patch passed
+1 💚	javac	0m 43s		the patch passed
+1 💚	javadoc	0m 25s		the patch passed
+1 💚	shadedjars	5m 26s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	215m 37s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		239m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 326d9195b3df 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Temurin-1.8.0_412-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/testReport/
Max. process+thread count	4316 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-16T17:42:51Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 44s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 18s		branch-2 passed
+1 💚	compile	0m 51s		branch-2 passed
+1 💚	javadoc	0m 26s		branch-2 passed
+1 💚	shadedjars	6m 7s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 14s		the patch passed
+1 💚	compile	0m 52s		the patch passed
+1 💚	javac	0m 52s		the patch passed
+1 💚	javadoc	0m 26s		the patch passed
+1 💚	shadedjars	6m 13s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	220m 40s		hbase-server in the patch passed.
		247m 6s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 776b792f2f09 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Eclipse Adoptium-11.0.23+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/testReport/
Max. process+thread count	4415 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-16T18:27:47Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 53s		Docker mode activated.
-0 ⚠️	yetus	0m 6s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	4m 10s		branch-2 passed
+1 💚	compile	1m 20s		branch-2 passed
+1 💚	javadoc	0m 36s		branch-2 passed
+1 💚	shadedjars	6m 28s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 6s		the patch passed
+1 💚	compile	1m 0s		the patch passed
+1 💚	javac	1m 0s		the patch passed
+1 💚	javadoc	0m 33s		the patch passed
+1 💚	shadedjars	7m 19s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	217m 21s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		247m 0s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux ce7c92627fac 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/testReport/
Max. process+thread count	4401 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/2/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-18T21:59:45Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 1s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ branch-2 Compile Tests _
+1 💚	mvninstall	5m 33s		branch-2 passed
+1 💚	compile	4m 0s		branch-2 passed
+1 💚	checkstyle	0m 57s		branch-2 passed
+1 💚	spotbugs	2m 5s		branch-2 passed
+1 💚	spotless	1m 0s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	5m 1s		the patch passed
+1 💚	compile	4m 17s		the patch passed
+1 💚	javac	4m 17s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 48s		the patch passed
+1 💚	spotbugs	2m 19s		the patch passed
+1 💚	hadoopcheck	22m 38s		Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚	spotless	0m 48s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 12s		The patch does not generate ASF License warnings.
		52m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 15d146649e31 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Eclipse Adoptium-11.0.23+9
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-19T01:01:10Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
-0 ⚠️	yetus	0m 6s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 18s		branch-2 passed
+1 💚	compile	1m 0s		branch-2 passed
+1 💚	javadoc	0m 29s		branch-2 passed
+1 💚	shadedjars	6m 0s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 7s		the patch passed
+1 💚	compile	0m 57s		the patch passed
+1 💚	javac	0m 57s		the patch passed
+1 💚	javadoc	0m 27s		the patch passed
+1 💚	shadedjars	5m 58s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	207m 45s		hbase-server in the patch passed.
		234m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 282e343d5a09 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/testReport/
Max. process+thread count	4449 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-19T02:34:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 59s		Docker mode activated.
-0 ⚠️	yetus	0m 6s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 37s		branch-2 passed
+1 💚	compile	1m 14s		branch-2 passed
+1 💚	javadoc	0m 41s		branch-2 passed
+1 💚	shadedjars	7m 13s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 33s		the patch passed
+1 💚	compile	1m 5s		the patch passed
+1 💚	javac	1m 5s		the patch passed
+1 💚	javadoc	0m 38s		the patch passed
+1 💚	shadedjars	6m 56s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	296m 49s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		327m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 7224598d9767 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Temurin-1.8.0_412-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/testReport/
Max. process+thread count	4203 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-19T02:37:29Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 0s		Docker mode activated.
-0 ⚠️	yetus	0m 7s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	5m 27s		branch-2 passed
+1 💚	compile	1m 40s		branch-2 passed
+1 💚	javadoc	0m 50s		branch-2 passed
+1 💚	shadedjars	9m 43s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	4m 59s		the patch passed
+1 💚	compile	1m 12s		the patch passed
+1 💚	javac	1m 12s		the patch passed
+1 💚	javadoc	0m 35s		the patch passed
+1 💚	shadedjars	8m 6s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	292m 25s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		330m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux d4e676834fda 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Eclipse Adoptium-11.0.23+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/testReport/
Max. process+thread count	4074 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/3/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-19T03:17:42Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 47s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 17s		branch-2 passed
+1 💚	compile	3m 1s		branch-2 passed
+1 💚	checkstyle	0m 38s		branch-2 passed
+1 💚	spotbugs	1m 44s		branch-2 passed
+1 💚	spotless	0m 47s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 3s		the patch passed
+1 💚	compile	2m 59s		the patch passed
+1 💚	javac	2m 59s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 35s		the patch passed
+1 💚	spotbugs	1m 42s		the patch passed
+1 💚	hadoopcheck	16m 58s		Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚	spotless	0m 46s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 10s		The patch does not generate ASF License warnings.
		38m 10s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 412617a8f8e9 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Eclipse Adoptium-11.0.23+9
Max. process+thread count	78 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-19T06:34:29Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
-0 ⚠️	yetus	0m 4s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 2s		branch-2 passed
+1 💚	compile	0m 52s		branch-2 passed
+1 💚	javadoc	0m 27s		branch-2 passed
+1 💚	shadedjars	5m 35s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 51s		the patch passed
+1 💚	compile	0m 53s		the patch passed
+1 💚	javac	0m 53s		the patch passed
+1 💚	javadoc	0m 25s		the patch passed
+1 💚	shadedjars	5m 31s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	210m 49s		hbase-server in the patch passed.
		234m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 81d14c5987fa 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/testReport/
Max. process+thread count	4440 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-19T06:53:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 44s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 15s		branch-2 passed
+1 💚	compile	0m 53s		branch-2 passed
+1 💚	javadoc	0m 27s		branch-2 passed
+1 💚	shadedjars	6m 13s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 14s		the patch passed
+1 💚	compile	0m 53s		the patch passed
+1 💚	javac	0m 53s		the patch passed
+1 💚	javadoc	0m 26s		the patch passed
+1 💚	shadedjars	6m 8s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	227m 0s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		253m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux d0fe2d913014 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Eclipse Adoptium-11.0.23+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/testReport/
Max. process+thread count	4412 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-19T06:55:49Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 47s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 25s		branch-2 passed
+1 💚	compile	0m 45s		branch-2 passed
+1 💚	javadoc	0m 26s		branch-2 passed
+1 💚	shadedjars	4m 52s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 32s		the patch passed
+1 💚	compile	0m 45s		the patch passed
+1 💚	javac	0m 45s		the patch passed
+1 💚	javadoc	0m 26s		the patch passed
+1 💚	shadedjars	4m 53s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	234m 7s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		256m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#6240
JIRA Issue	HBASE-28821
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux fc35e529bd5c 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `bec5151`
Default Java	Temurin-1.8.0_412-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/testReport/
Max. process+thread count	4210 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6240/4/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

jhungund · 2024-09-19T07:12:06Z

Hi @wchevreuil, The test failures seem unrelated to the change. The tests passed on top of this change, when executed locally.

[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.hbase.TestAcidGuaranteesWithAdaptivePolicy
[INFO] Running org.apache.hadoop.hbase.replication.TestReplicationMetricsforUI
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.apache.hadoop.hbase.SystemExitRule (file:/Users/janardhan.hungund/hbase-down/hbase/hbase-common/target/hbase-common-2.7.0-SNAPSHOT-tests.jar)
WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.hbase.SystemExitRule
WARNING: System::setSecurityManager will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.apache.hadoop.hbase.SystemExitRule (file:/Users/janardhan.hungund/hbase-down/hbase/hbase-common/target/hbase-common-2.7.0-SNAPSHOT-tests.jar)
WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.hbase.SystemExitRule
WARNING: System::setSecurityManager will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.apache.hadoop.hbase.SystemExitRule$1 (file:/Users/janardhan.hungund/hbase-down/hbase/hbase-common/target/hbase-common-2.7.0-SNAPSHOT-tests.jar)
WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.hbase.SystemExitRule$1
WARNING: System::setSecurityManager will be removed in a future release
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.766 s - in org.apache.hadoop.hbase.replication.TestReplicationMetricsforUI
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.apache.hadoop.hbase.SystemExitRule$1 (file:/Users/janardhan.hungund/hbase-down/hbase/hbase-common/target/hbase-common-2.7.0-SNAPSHOT-tests.jar)
WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.hbase.SystemExitRule$1
WARNING: System::setSecurityManager will be removed in a future release
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 193.738 s - in org.apache.hadoop.hbase.TestAcidGuaranteesWithAdaptivePolicy
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0

(base) janardhan.hungund@MacBook-Pro-4 hbase % git log
commit bec5151 (HEAD -> branch-2-for-HBASE-28821, upstream-fork/branch-2-for-HBASE-28821)
Author: Janardhan Hungund janardhan.hungund@cloudera.com
Date: Fri Sep 6 19:34:15 2024 +0530

HBASE-28821: Optimise the memory-utilisation during persistence of bucketcache.

During the serialisation of backing map of bucketcache persistence,
we create a new BackingMapEntry.Builder object for every entry in the
bucket map. This can be optimized by creating a single builder object
and reusing it for every entry object.

Change-Id: I6f6c05dfa256ce964cd1d66e5050d9424db4be46

Thanks,
Janardhan

wchevreuil approved these changes Sep 12, 2024

View reviewed changes

rmdmattingly reviewed Sep 12, 2024

View reviewed changes

pankaj72981 approved these changes Sep 12, 2024

View reviewed changes

jhungund force-pushed the branch-2-for-HBASE-28821 branch from cb5854b to bec5151 Compare September 16, 2024 13:27

wchevreuil merged commit 3c17f61 into apache:branch-2 Sep 19, 2024
1 check failed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-28821: Optimise the memory-utilisation during persistence of bucketcache. #6240

HBASE-28821: Optimise the memory-utilisation during persistence of bucketcache. #6240

jhungund commented Sep 12, 2024

Apache-HBase commented Sep 12, 2024

Apache-HBase commented Sep 12, 2024

Apache-HBase commented Sep 12, 2024

Apache-HBase commented Sep 12, 2024

rmdmattingly left a comment

pankaj72981 left a comment

jhungund commented Sep 16, 2024

Apache-HBase commented Sep 16, 2024

Apache-HBase commented Sep 16, 2024

Apache-HBase commented Sep 16, 2024

Apache-HBase commented Sep 16, 2024

Apache-HBase commented Sep 18, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

jhungund commented Sep 19, 2024

HBASE-28821: Optimise the memory-utilisation during persistence of bucketcache. #6240

HBASE-28821: Optimise the memory-utilisation during persistence of bucketcache. #6240

Conversation

jhungund commented Sep 12, 2024

Apache-HBase commented Sep 12, 2024

Apache-HBase commented Sep 12, 2024

Apache-HBase commented Sep 12, 2024

Apache-HBase commented Sep 12, 2024

rmdmattingly left a comment

Choose a reason for hiding this comment

pankaj72981 left a comment

Choose a reason for hiding this comment

jhungund commented Sep 16, 2024

Apache-HBase commented Sep 16, 2024

Apache-HBase commented Sep 16, 2024

Apache-HBase commented Sep 16, 2024

Apache-HBase commented Sep 16, 2024

Apache-HBase commented Sep 18, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

Apache-HBase commented Sep 19, 2024

jhungund commented Sep 19, 2024