HBASE-26988 dynamic configuration of loadbalance.bytable #4384

d-c-manning · 2022-04-29T05:15:33Z

Some tests call conf.unset(HConstants.HBASE_MASTER_LOADBALANCE_BYTABLE); but that only works if dynamic reconfiguration will reset the value. Without this fix (or test fixes), the tests only pass if some ordering constraints are guaranteed.

Apache-HBase · 2022-04-29T05:49:23Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 9s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 24s	master passed
+1 💚	compile	0m 22s	master passed
+1 💚	checkstyle	0m 12s	master passed
+1 💚	spotbugs	0m 25s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 11s	the patch passed
+1 💚	compile	0m 21s	the patch passed
+1 💚	javac	0m 21s	the patch passed
+1 💚	checkstyle	0m 10s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 37s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	0m 27s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 8s	The patch does not generate ASF License warnings.
		24m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4384
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 60c7663ef5d7 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f4eb907`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	64 (vs. ulimit of 30000)
modules	C: hbase-balancer U: hbase-balancer
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-04-29T05:52:26Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 19s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 37s	master passed
+1 💚	compile	0m 18s	master passed
+1 💚	shadedjars	3m 40s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 14s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 35s	the patch passed
+1 💚	compile	0m 15s	the patch passed
+1 💚	javac	0m 15s	the patch passed
+1 💚	shadedjars	3m 50s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 13s	the patch passed
		_ Other Tests _
+1 💚	unit	11m 28s	hbase-balancer in the patch passed.
		27m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4384
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b97bde627e8f 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f4eb907`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/testReport/
Max. process+thread count	193 (vs. ulimit of 30000)
modules	C: hbase-balancer U: hbase-balancer
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-04-29T05:56:45Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 54s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 30s	master passed
+1 💚	compile	0m 21s	master passed
+1 💚	shadedjars	4m 53s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 13s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 46s	the patch passed
+1 💚	compile	0m 14s	the patch passed
+1 💚	javac	0m 14s	the patch passed
+1 💚	shadedjars	4m 39s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 11s	the patch passed
		_ Other Tests _
+1 💚	unit	11m 44s	hbase-balancer in the patch passed.
		31m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4384
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 4d00d1238faa 5.4.0-1068-aws #72~18.04.1-Ubuntu SMP Thu Mar 3 08:49:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f4eb907`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/testReport/
Max. process+thread count	165 (vs. ulimit of 30000)
modules	C: hbase-balancer U: hbase-balancer
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-04-29T06:21:31Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 14s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 10s	master passed
+1 💚	compile	0m 21s	master passed
+1 💚	checkstyle	0m 10s	master passed
+1 💚	spotbugs	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 13s	the patch passed
+1 💚	compile	0m 21s	the patch passed
+1 💚	javac	0m 21s	the patch passed
+1 💚	checkstyle	0m 9s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 28s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	0m 28s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		23m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4384
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 5f7e93bcf1e2 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f4eb907`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	64 (vs. ulimit of 30000)
modules	C: hbase-balancer U: hbase-balancer
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-04-29T06:24:58Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 14s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 42s	master passed
+1 💚	compile	0m 15s	master passed
+1 💚	shadedjars	3m 40s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 14s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 41s	the patch passed
+1 💚	compile	0m 15s	the patch passed
+1 💚	javac	0m 15s	the patch passed
+1 💚	shadedjars	3m 42s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 12s	the patch passed
		_ Other Tests _
+1 💚	unit	11m 28s	hbase-balancer in the patch passed.
		27m 10s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4384
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b2b38c852b13 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f4eb907`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/testReport/
Max. process+thread count	190 (vs. ulimit of 30000)
modules	C: hbase-balancer U: hbase-balancer
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-04-29T06:25:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 47s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 45s	master passed
+1 💚	compile	0m 15s	master passed
+1 💚	shadedjars	4m 43s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 11s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 18s	the patch passed
+1 💚	compile	0m 11s	the patch passed
+1 💚	javac	0m 11s	the patch passed
+1 💚	shadedjars	3m 53s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 9s	the patch passed
		_ Other Tests _
+1 💚	unit	11m 6s	hbase-balancer in the patch passed.
		27m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4384
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f212f8bb9ce0 5.4.0-1068-aws #72~18.04.1-Ubuntu SMP Thu Mar 3 08:49:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f4eb907`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/testReport/
Max. process+thread count	163 (vs. ulimit of 30000)
modules	C: hbase-balancer U: hbase-balancer
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4384/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Signed-off-by: Andrew Purtell <apurtell@apache.org> Conflicts: hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java

d-c-manning added 2 commits April 28, 2022 22:13

HBASE-26988 dynamic configuration of loadbalance.bytable

4cdf322

HBASE-26988 dynamic configuration of StochasticLoadBalancer

a89b7cd

apurtell approved these changes Apr 29, 2022

View reviewed changes

apurtell merged commit 4a33ed1 into apache:master Apr 29, 2022

d-c-manning deleted the HBASE-26988 branch April 29, 2022 21:43

d-c-manning mentioned this pull request Apr 29, 2022

HBASE-26989 TestStochasticLoadBalancer fixes for performance and consistency #4385

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-26988 dynamic configuration of loadbalance.bytable #4384

HBASE-26988 dynamic configuration of loadbalance.bytable #4384

d-c-manning commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

HBASE-26988 dynamic configuration of loadbalance.bytable #4384

HBASE-26988 dynamic configuration of loadbalance.bytable #4384

Conversation

d-c-manning commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022

Apache-HBase commented Apr 29, 2022