Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-28774: Protect hbase:meta from any hotspotting regions by balancing them to different region servers #6173

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

ranganathg
Copy link

@ranganathg ranganathg commented Aug 22, 2024

HBASE-28774
The idea is to provide a configuration where by one can configure mutually exclusive tables. For example: hbase:meta,SYSTEM.CATALOG can be mutually exclusive tables. Anytime these 2 tables come together the new plan will be costly and would need a rebalancing.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@ranganathg ranganathg marked this pull request as ready for review August 22, 2024 14:57

Map<ServerName, List<RegionInfo>> clusterState = this.cluster.clusterState;

for (Map.Entry<ServerName, List<RegionInfo>> entry : clusterState.entrySet()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let us add comment about what we are doing and how it would impact the balancer decision with an example.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

* on same RS
*/
@InterfaceAudience.Private public class MutuallyExclusiveTablesCostFunction extends CostFunction {
public static final String MUTUALLY_EXCLUSIVE_TABLES_KEY =
Copy link
Contributor

@Umeshkumar9414 Umeshkumar9414 Aug 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: good to add comments for config.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 47s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 3m 11s master passed
+1 💚 compile 0m 18s master passed
+1 💚 javadoc 0m 15s master passed
+1 💚 shadedjars 5m 21s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 4s the patch passed
+1 💚 compile 0m 16s the patch passed
+1 💚 javac 0m 16s the patch passed
+1 💚 javadoc 0m 12s the patch passed
+1 💚 shadedjars 5m 19s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
-1 ❌ unit 10m 20s /patch-unit-hbase-balancer.txt hbase-balancer in the patch failed.
30m 29s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #6173
JIRA Issue HBASE-28774
Optional Tests javac javadoc unit compile shadedjars
uname Linux d6f9069d05cc 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / be1b2fd
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/testReport/
Max. process+thread count 262 (vs. ulimit of 30000)
modules C: hbase-balancer U: hbase-balancer
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 50s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ master Compile Tests _
+1 💚 mvninstall 3m 50s master passed
+1 💚 compile 0m 23s master passed
+1 💚 checkstyle 0m 9s master passed
+1 💚 spotbugs 0m 29s master passed
+1 💚 spotless 0m 51s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 27s the patch passed
+1 💚 compile 0m 27s the patch passed
+1 💚 javac 0m 27s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 10s /results-checkstyle-hbase-balancer.txt hbase-balancer: The patch generated 15 new + 1 unchanged - 0 fixed = 16 total (was 1)
+1 💚 spotbugs 0m 40s the patch passed
+1 💚 hadoopcheck 11m 56s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 ❌ spotless 0m 19s patch has 70 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 💚 asflicense 0m 8s The patch does not generate ASF License warnings.
30m 31s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #6173
JIRA Issue HBASE-28774
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname Linux 3881e6c1ab0c 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / be1b2fd
Default Java Eclipse Adoptium-17.0.11+9
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 81 (vs. ulimit of 30000)
modules C: hbase-balancer U: hbase-balancer
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6173/6/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants