HBASE-28186 Rebase CacheAwareBalance related commits into master branch #5500

wchevreuil · 2023-11-02T23:44:50Z

This PR is for rebasing all the CacheAwareBalancer commits into master. Rahul has shared the design doc in HBASE-27999.

Documentation on how this works and how it can be enabled has been provided in the HBASE-28097 PR

This is relevant for cloud storage deployments, where the cost of moving a large, fully cached region to a new RS is high, requiring reads to go all the way to the cloud storage whilst the region is being "re-cached" on the new RS.

@ragarkar can share more details about his tests results.

Apache-HBase · 2023-11-03T01:00:12Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 16s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 55s	master passed
+1 💚	compile	4m 45s	master passed
+1 💚	checkstyle	1m 13s	master passed
+0 🆗	refguide	2m 39s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotless	0m 42s	branch has no errors when running spotless:check.
+1 💚	spotbugs	11m 35s	master passed
-0 ⚠️	patch	1m 53s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 38s	the patch passed
+1 💚	compile	4m 46s	the patch passed
+1 💚	cc	4m 46s	the patch passed
-0 ⚠️	javac	4m 46s	root generated 2 new + 910 unchanged - 0 fixed = 912 total (was 910)
-0 ⚠️	checkstyle	1m 6s	root: The patch generated 2 new + 13 unchanged - 0 fixed = 15 total (was 13)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+0 🆗	refguide	1m 54s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	9m 47s	Patch does not cause any errors with Hadoop 3.2.4 3.3.6.
+1 💚	hbaseprotoc	3m 46s	the patch passed
+1 💚	spotless	0m 39s	patch has no errors when running spotless:check.
+1 💚	spotbugs	12m 35s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 46s	The patch does not generate ASF License warnings.
		69m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5500
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool refguide
uname	Linux 044d63166233 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `fa4c896`
Default Java	Eclipse Adoptium-11.0.17+8
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5500/1/yetus-general-check/output/branch-site/book.html
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/artifact/yetus-general-check/output/diff-compile-javac-root.txt
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/artifact/yetus-general-check/output/diff-checkstyle-root.txt
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5500/1/yetus-general-check/output/patch-site/book.html
Max. process+thread count	175 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-03T01:01:49Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 21s	master passed
+1 💚	compile	2m 13s	master passed
+1 💚	shadedjars	6m 22s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	4m 12s	master passed
-0 ⚠️	patch	11m 15s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 32s	the patch passed
+1 💚	compile	2m 30s	the patch passed
+1 💚	javac	2m 30s	the patch passed
+1 💚	shadedjars	6m 55s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	4m 11s	the patch passed
		_ Other Tests _
-1 ❌	unit	34m 9s	root in the patch failed.
		71m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5500
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 7fb1ff50b06e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `fa4c896`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/testReport/
Max. process+thread count	2460 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-03T05:35:02Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 12s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 28s	master passed
+1 💚	compile	1m 41s	master passed
+1 💚	shadedjars	4m 50s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 33s	master passed
-0 ⚠️	patch	7m 59s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 17s	the patch passed
+1 💚	compile	1m 41s	the patch passed
+1 💚	javac	1m 41s	the patch passed
+1 💚	shadedjars	4m 52s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 34s	the patch passed
		_ Other Tests _
-1 ❌	unit	315m 25s	root in the patch failed.
		344m 31s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5500
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux e4f08803bf80 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `fa4c896`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/testReport/
Max. process+thread count	5472 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

ragarkar · 2023-11-03T08:37:42Z

@ragarkar can share more details about his tests results.
Added test results showing that the regions are retained by the region server beyond server restart and the impact on cache has been attached to HBASE-27999.

Apache-HBase · 2023-11-03T10:55:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 12s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 51s	master passed
+1 💚	compile	4m 49s	master passed
+1 💚	checkstyle	1m 3s	master passed
+0 🆗	refguide	3m 15s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotless	0m 53s	branch has no errors when running spotless:check.
+1 💚	spotbugs	14m 49s	master passed
-0 ⚠️	patch	2m 4s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 39s	the patch passed
+1 💚	compile	4m 54s	the patch passed
+1 💚	cc	4m 54s	the patch passed
-0 ⚠️	javac	4m 54s	root generated 2 new + 910 unchanged - 0 fixed = 912 total (was 910)
-0 ⚠️	checkstyle	1m 7s	root: The patch generated 2 new + 13 unchanged - 0 fixed = 15 total (was 13)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+0 🆗	refguide	2m 14s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	hadoopcheck	9m 20s	Patch does not cause any errors with Hadoop 3.2.4 3.3.6.
+1 💚	hbaseprotoc	3m 58s	the patch passed
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	14m 5s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 57s	The patch does not generate ASF License warnings.
		75m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5500
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool refguide
uname	Linux c1f097f48cbf 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `fa4c896`
Default Java	Eclipse Adoptium-11.0.17+8
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5500/2/yetus-general-check/output/branch-site/book.html
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/artifact/yetus-general-check/output/diff-compile-javac-root.txt
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/artifact/yetus-general-check/output/diff-checkstyle-root.txt
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5500/2/yetus-general-check/output/patch-site/book.html
Max. process+thread count	177 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-03T14:41:10Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 24s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 34s	master passed
+1 💚	compile	1m 50s	master passed
+1 💚	shadedjars	5m 12s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 46s	master passed
-0 ⚠️	patch	8m 33s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 36s	the patch passed
+1 💚	compile	1m 47s	the patch passed
+1 💚	javac	1m 47s	the patch passed
+1 💚	shadedjars	5m 12s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 50s	the patch passed
		_ Other Tests _
+1 💚	unit	269m 41s	root in the patch passed.
		301m 38s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5500
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 393a8ee3566e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `fa4c896`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/testReport/
Max. process+thread count	8226 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-03T15:57:13Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 31s	master passed
+1 💚	compile	1m 34s	master passed
+1 💚	shadedjars	5m 21s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 24s	master passed
-0 ⚠️	patch	8m 16s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 14s	the patch passed
+1 💚	compile	1m 36s	the patch passed
+1 💚	javac	1m 36s	the patch passed
+1 💚	shadedjars	5m 15s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 24s	the patch passed
		_ Other Tests _
-1 ❌	unit	348m 36s	root in the patch failed.
		377m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5500
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 24d61e652182 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `fa4c896`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/testReport/
Max. process+thread count	5154 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

taklwu

I reviewed all patches and they are the same as the feature branch.

taklwu

for this commit 895d865 , can we remove the message as part of the commit body ?

Change-Id: I1fc241eaa334b438adbf0e245a1f6a56be605f8d

also I'm wondered how these patches will be landing in master, will it be a single commit ? or will we keep the history of those verified commits , and those rebasing commit how would you handle it ? 2469aca and 917bfce

I was reviewing the first commit but I didn't know i cannot review and submit approve per each commit

wchevreuil · 2023-11-10T09:43:17Z

also I'm wondered how these patches will be landing in master, will it be a single commit ? or will we keep the history of those verified commits , and those rebasing commit how would you handle it ? 2469aca and 917bfce

No, idea is to preserve the original commits. Let me squash the rebasing commits in, so that we only have the 4 original commits from Rahul.

#5339) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Reviewew-by: Kota-SH <shanmukhaharipriya@gmail.com>

#5342) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

#5495) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

wchevreuil · 2023-11-10T12:40:16Z

@taklwu , I had now squashed the rebase commits with the original ones and solved the conflicts. I believe this would allow us to merge only the original 4 commits from @ragarkar into master branch.

Apache-HBase · 2023-11-10T13:11:00Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 12s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 15s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 20s	master passed
+1 💚	compile	1m 41s	master passed
+1 💚	shadedjars	4m 50s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 35s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
-1 ❌	mvninstall	0m 26s	root in the patch failed.
-1 ❌	compile	0m 9s	root in the patch failed.
-0 ⚠️	javac	0m 9s	root in the patch failed.
-1 ❌	shadedjars	1m 22s	patch has 76 errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 7s	hbase-protocol-shaded in the patch failed.
-0 ⚠️	javadoc	0m 22s	hbase-server generated 3 new + 22 unchanged - 0 fixed = 25 total (was 22)
-0 ⚠️	javadoc	0m 9s	root in the patch failed.
		_ Other Tests _
-1 ❌	unit	3m 25s	root in the patch failed.
		20m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5500
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 411022a6aa8d 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7f3921a`
Default Java	Temurin-1.8.0_352-b08
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt
compile	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-root.txt
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-root.txt
shadedjars	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk8-hadoop3-check/output/patch-javadoc-hbase-protocol-shaded.txt
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk8-hadoop3-check/output/patch-javadoc-root.txt
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/testReport/
Max. process+thread count	651 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-10T13:13:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 26s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 33s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 55s	master passed
+1 💚	compile	1m 49s	master passed
+1 💚	shadedjars	5m 16s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	2m 56s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
-1 ❌	mvninstall	0m 26s	root in the patch failed.
-1 ❌	compile	0m 9s	root in the patch failed.
-0 ⚠️	javac	0m 9s	root in the patch failed.
-1 ❌	shadedjars	1m 22s	patch has 76 errors when building our shaded downstream artifacts.
-0 ⚠️	javadoc	0m 6s	hbase-protocol-shaded in the patch failed.
-0 ⚠️	javadoc	0m 21s	hbase-server generated 3 new + 94 unchanged - 1 fixed = 97 total (was 95)
-0 ⚠️	javadoc	0m 9s	root in the patch failed.
		_ Other Tests _
-1 ❌	unit	3m 31s	root in the patch failed.
		22m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5500
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 57e126674fe3 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7f3921a`
Default Java	Eclipse Adoptium-11.0.17+8
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt
compile	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-root.txt
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-root.txt
shadedjars	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-protocol-shaded.txt
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-hbase-server.txt
javadoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/testReport/
Max. process+thread count	641 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-11-10T13:25:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	prototool	0m 0s	prototool was not available.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 53s	master passed
+1 💚	compile	4m 50s	master passed
+1 💚	checkstyle	1m 10s	master passed
+0 🆗	refguide	2m 24s	branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotless	0m 41s	branch has no errors when running spotless:check.
+1 💚	spotbugs	11m 38s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
-1 ❌	mvninstall	0m 26s	root in the patch failed.
-1 ❌	compile	0m 10s	root in the patch failed.
-0 ⚠️	cc	0m 10s	root in the patch failed.
-0 ⚠️	javac	0m 10s	root in the patch failed.
-0 ⚠️	checkstyle	1m 3s	root: The patch generated 2 new + 13 unchanged - 0 fixed = 15 total (was 13)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+0 🆗	refguide	1m 58s	patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
-1 ❌	hadoopcheck	0m 7s	The patch causes 76 errors with Hadoop v3.2.4.
-1 ❌	hadoopcheck	0m 14s	The patch causes 76 errors with Hadoop v3.3.6.
-1 ❌	hbaseprotoc	0m 5s	hbase-protocol-shaded in the patch failed.
-1 ❌	hbaseprotoc	0m 15s	hbase-client in the patch failed.
-1 ❌	hbaseprotoc	0m 11s	hbase-balancer in the patch failed.
-1 ❌	hbaseprotoc	0m 26s	hbase-server in the patch failed.
-1 ❌	hbaseprotoc	0m 9s	root in the patch failed.
+1 💚	spotless	0m 37s	patch has no errors when running spotless:check.
-1 ❌	spotbugs	0m 6s	hbase-protocol-shaded in the patch failed.
-1 ❌	spotbugs	0m 13s	hbase-client in the patch failed.
-1 ❌	spotbugs	0m 10s	hbase-balancer in the patch failed.
-1 ❌	spotbugs	0m 26s	hbase-server in the patch failed.
-1 ❌	spotbugs	0m 12s	root in the patch failed.
		_ Other Tests _
+1 💚	asflicense	0m 39s	The patch does not generate ASF License warnings.
		34m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5500
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile cc hbaseprotoc prototool refguide
uname	Linux eff3038966fe 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7f3921a`
Default Java	Eclipse Adoptium-11.0.17+8
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5500/3/yetus-general-check/output/branch-site/book.html
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-mvninstall-root.txt
compile	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-compile-root.txt
cc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-compile-root.txt
javac	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-compile-root.txt
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/diff-checkstyle-root.txt
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-5500/3/yetus-general-check/output/patch-site/book.html
hadoopcheck	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-javac-3.2.4.txt
hadoopcheck	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-javac-3.3.6.txt
hbaseprotoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-hbaseprotoc-hbase-protocol-shaded.txt
hbaseprotoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-hbaseprotoc-hbase-client.txt
hbaseprotoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-hbaseprotoc-hbase-balancer.txt
hbaseprotoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-hbaseprotoc-hbase-server.txt
hbaseprotoc	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-hbaseprotoc-root.txt
spotbugs	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-spotbugs-hbase-protocol-shaded.txt
spotbugs	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-spotbugs-hbase-client.txt
spotbugs	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-spotbugs-hbase-balancer.txt
spotbugs	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt
spotbugs	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/artifact/yetus-general-check/output/patch-spotbugs-root.txt
Max. process+thread count	175 (vs. ulimit of 30000)
modules	C: hbase-protocol-shaded hbase-common hbase-client hbase-balancer hbase-server . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5500/3/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil requested review from apurtell, bbeaudreault, ankitsinghal, petersomogyi, taklwu and Apache9 November 2, 2023 23:44

taklwu previously approved these changes Nov 9, 2023

View reviewed changes

taklwu self-requested a review November 9, 2023 21:49

taklwu reviewed Nov 9, 2023

View reviewed changes

ragarkar added 4 commits November 10, 2023 12:35

HBASE-27997 Enhance prefetch executor to record region prefetch infor… (

4da1aca

#5339) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Reviewew-by: Kota-SH <shanmukhaharipriya@gmail.com>

HBASE-27998 Enhance region metrics to include prefetch ratio for each… (

373b6ce

#5342) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

HBASE-27999 Implement cache prefetch aware load balancer (#5376)

07a44b2

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

HBASE-28097 Add documentation section for the Cache Aware balancer fu… (

d3f8d16

#5495) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>

wchevreuil force-pushed the HBASE-27389-master-rebase branch from 484030f to d3f8d16 Compare November 10, 2023 12:37

wchevreuil requested a review from taklwu November 10, 2023 12:40

wchevreuil closed this Nov 13, 2023

wchevreuil deleted the HBASE-27389-master-rebase branch November 13, 2023 14:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-28186 Rebase CacheAwareBalance related commits into master branch #5500

HBASE-28186 Rebase CacheAwareBalance related commits into master branch #5500

wchevreuil commented Nov 2, 2023

Apache-HBase commented Nov 3, 2023

Apache-HBase commented Nov 3, 2023

Apache-HBase commented Nov 3, 2023

ragarkar commented Nov 3, 2023 •

edited

Loading

Apache-HBase commented Nov 3, 2023

Apache-HBase commented Nov 3, 2023

Apache-HBase commented Nov 3, 2023

taklwu left a comment •

edited

Loading

taklwu left a comment •

edited

Loading

wchevreuil commented Nov 10, 2023

wchevreuil commented Nov 10, 2023

Apache-HBase commented Nov 10, 2023

Apache-HBase commented Nov 10, 2023

Apache-HBase commented Nov 10, 2023

HBASE-28186 Rebase CacheAwareBalance related commits into master branch #5500

HBASE-28186 Rebase CacheAwareBalance related commits into master branch #5500

Conversation

wchevreuil commented Nov 2, 2023

Apache-HBase commented Nov 3, 2023

Apache-HBase commented Nov 3, 2023

Apache-HBase commented Nov 3, 2023

ragarkar commented Nov 3, 2023 • edited Loading

Apache-HBase commented Nov 3, 2023

Apache-HBase commented Nov 3, 2023

Apache-HBase commented Nov 3, 2023

taklwu left a comment • edited Loading

Choose a reason for hiding this comment

taklwu left a comment • edited Loading

Choose a reason for hiding this comment

wchevreuil commented Nov 10, 2023

wchevreuil commented Nov 10, 2023

Apache-HBase commented Nov 10, 2023

Apache-HBase commented Nov 10, 2023

Apache-HBase commented Nov 10, 2023

ragarkar commented Nov 3, 2023 •

edited

Loading

taklwu left a comment •

edited

Loading

taklwu left a comment •

edited

Loading