HBASE-28696 Partition BackupSystemTable queries #6067

rmdmattingly · 2024-07-09T19:39:15Z

When successfully taking an incremental backup, one of our final steps is to delete bulk load metadata from the system table for the bulk loads that needed to be captured in the given backup. This means that we will basically truncate the entire bulk loads system table in a single batch of the deletes after successfully taking an incremental backup. Depending on your usage, one may run tons of bulk loads between backups, so this design is needlessly fragile. We should partition these deletes so that we never erroneously fail a backup due to this; there are a few other cases where we generated unbounded multi requests in the BackupSystemTable that this PR addresses too.

A few questions here:

Do we want to make the batch size configurable? Seems like yet another config in hbase that I'd like to avoid since it will plausibly never be customized
It is okay to just use Table#batch like this, correct?
I've tested this code in our QA environment, and unit testing feels like we're just testing the Lists class. That said, would people prefer that I add a unit test?

@charlesconnell @ndimiduk

hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupSystemTable.java

ndimiduk

Yes, BufferedMutator wraps up this use-case nicely. Please clean this up a bit more and it looks good for merge.

ndimiduk · 2024-09-02T08:06:22Z

hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupSystemTable.java

@@ -1885,6 +1887,13 @@ private String cellKeyToBackupSetName(Cell current) {
    return Bytes.toString(data).substring(SET_KEY_PREFIX.length());
  }

+  private void executeBufferedMutations(Table table, List<? extends Mutation> mutations)


This helper function is now kinda weird. I think that you can replace all the above try (Table table = ...) { ... } with try (BufferedMutator bm = ...) { ... }. On quick glance, you never use the table instance other than to get the name in order to create the BufferedMutator instance.

Apache-HBase · 2024-09-06T15:49:22Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 44s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	3m 5s		master passed
+1 💚	compile	0m 32s		master passed
+1 💚	checkstyle	0m 12s		master passed
+1 💚	spotbugs	0m 33s		master passed
+1 💚	spotless	0m 45s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 55s		the patch passed
+1 💚	compile	0m 30s		the patch passed
+1 💚	javac	0m 30s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 11s		the patch passed
+1 💚	spotbugs	0m 38s		the patch passed
+1 💚	hadoopcheck	10m 37s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚	spotless	0m 44s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 12s		The patch does not generate ASF License warnings.
		28m 9s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6067/6/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6067
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 240abe9a05e0 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `6c5020a`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	84 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6067/6/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-09-06T16:04:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 40s		Docker mode activated.
-0 ⚠️	yetus	0m 4s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	4m 34s		master passed
+1 💚	compile	0m 25s		master passed
+1 💚	javadoc	0m 19s		master passed
+1 💚	shadedjars	8m 13s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	4m 45s		the patch passed
+1 💚	compile	0m 32s		the patch passed
+1 💚	javac	0m 32s		the patch passed
+1 💚	javadoc	0m 28s		the patch passed
+1 💚	shadedjars	7m 49s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	13m 57s		hbase-backup in the patch passed.
		42m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6067/6/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6067
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 742f99fc7a5a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `6c5020a`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6067/6/testReport/
Max. process+thread count	3792 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6067/6/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

ndimiduk

Nice one @rmdmattingly

Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>