HBASE-29451 Add Docs section describing BucketCache Time based priority by wchevreuil · Pull Request #7289 · apache/hbase

wchevreuil · 2025-09-09T16:02:10Z

No description provided.

Change-Id: I3de6b19030d242787ecb715590fbe5c2a1f7fe0c

Copilot

Pull Request Overview

This PR adds comprehensive documentation for the Time Based Priority feature in BucketCache, which was introduced in HBASE-28463. The documentation covers configuration options, usage strategies, and implementation details for this caching optimization feature.

Added detailed explanation of Time Based Priority for BucketCache including its purpose and benefits
Documented three different strategies for defining data age: cell timestamps, custom cell qualifiers, and custom value provider
Provided configuration examples and implementation guidance for each strategy

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

src/main/asciidoc/_chapters/architecture.adoc

taklwu

LGTM, just few minor comment.

src/main/asciidoc/_chapters/architecture.adoc

taklwu · 2025-09-09T18:18:03Z

src/main/asciidoc/_chapters/architecture.adoc

+are recommended.
+
+Alternatively, consider to adjust the initial window size to the same as the hot age threshold, and
+two windows only per tier:


nit hbase.hstore.compaction.date.tiered.windows.per.tier = 2 can you explain why we should have two windows per tier for the initial setup ?

It's how date tiered compaction works, we cannot have less then two windows. Unfortunately, this is not explained in the "Date Tiered Compaction" section of the ref guide, however, going through the implementation details of another feature is out of scope of this doc jira.

taklwu · 2025-09-09T18:19:55Z

src/main/asciidoc/_chapters/architecture.adoc

+Finally, more intricate schemas may incorporate domain-specific logic for defining the age of
+each record. The custom value provider facilitates the integration of custom code to implement
+the appropriate parsing of the date value that should be used for the priority comparison.


nit: should we have a link here to the section of Using a Custom value provider for Time Based Priority

sure, let me add it.

kgeisz

LGTM overall. I just have a few very minor nits

src/main/asciidoc/_chapters/architecture.adoc

Co-authored-by: Kevin Geiszler <kgeiszler@cloudera.com>

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Co-authored-by: Kevin Geiszler <kgeiszler@cloudera.com>

Co-authored-by: Tak Lon (Stephen) Wu <taklwu@apache.org>

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Co-authored-by: Kevin Geiszler <kgeiszler@cloudera.com>

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…y type Change-Id: I270bf39f31687f9589cc2c62274af47db2d6b856

Apache-HBase · 2025-09-10T16:17:39Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
			_ Patch Compile Tests _
			_ Other Tests _
		1m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7289/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7289
Optional Tests
uname	Linux e9e55a73f94a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c767924`
Max. process+thread count	39 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7289/4/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2025-09-10T16:31:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 30s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
			_ master Compile Tests _
+1 💚	mvninstall	3m 22s		master passed
+0 🆗	refguide	2m 23s		branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotless	0m 50s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 8s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+0 🆗	refguide	2m 10s		patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚	spotless	0m 47s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 12s		The patch does not generate ASF License warnings.
		14m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7289/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#7289
Optional Tests	dupname asflicense codespell detsecrets refguide spotless
uname	Linux 3186184b5841 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `c767924`
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-7289/4/yetus-general-check/output/branch-site/book.html
refguide	https://nightlies.apache.org/hbase/HBase-PreCommit-GitHub-PR/PR-7289/4/yetus-general-check/output/patch-site/book.html
Max. process+thread count	61 (vs. ulimit of 30000)
modules	C: . U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7289/4/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Copilot

Pull Request Overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-11T14:44:36Z

src/main/asciidoc/_chapters/architecture.adoc

+[source,java]
+----
+public class RowKeyPortionTieringValueProvider implements CustomTieredCompactor.TieringValueProvider {
+   private SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMddHHmmss");


SimpleDateFormat is not thread-safe. Consider using ThreadLocal or DateTimeFormatter from java.time package for thread safety.

Copilot · 2025-09-11T14:44:36Z

src/main/asciidoc/_chapters/architecture.adoc

+       try {
+           return sdf.parse(datePortion).getTime();
+       } catch (ParseException e) {
+           //handle error


The error handling comment is too generic. Consider providing specific guidance on how errors should be handled, such as logging the error or returning a default value.

Suggested change

//handle error

e.printStackTrace(); // Log the error for debugging

wchevreuil · 2025-09-12T11:14:26Z

Any further comments, @taklwu ?

PDavid

Many thanks for documenting this. 👍

@tag

…ch (#7706) * HBASE-29573: Fully load QuotaCache instead of reading individual rows on demand (#7282) Signed-off by: Ray Mattingly <rmattingly@apache.org> * HBASE-26974 Introduce a LogRollProcedure (#5408) Co-authored-by: huiruan <huiruan@tencent.com> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-27355 Separate meta read requests from master and client (#7261) Co-authored-by: huiruan <huiruan@tencent.com> Signed-off-by: Duo Zhang <zhangduo@apache.org> Reviewed-by: Aman Poonia <aman.poonia.29@gmail.com> * HBASE-27157 Potential race condition in WorkerAssigner (#4577) Close #7299 Co-authored-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Lijin Bin <binlijin@apache.org> * HBASE-29451 Add Docs section describing BucketCache Time based priority (#7289) Signed-off-by: Dávid Paksy <paksyd@apache.org> Reviewed-by: Kevin Geiszler <kevin.j.geiszler@gmail.com> Reviewed-by: Tak Lon (Stephen) Wu <taklwu@apache.org> * HBASE-29577 Fix NPE from RegionServerRpcQuotaManager when reloading configuration (#7285) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Charles Connell <cconnell@apache.org> * HBASE-29590 Use hadoop 3.4.2 as default hadooop3 dependency (#7301) Signed-off-by: Nihal Jain <nihaljain@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> * Modern backup failures can cause backup system to lock up (#7288) Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com> Signed-off-by: Charles Connell <cconnell@apache.org> Signed-off-by: Ray Mattingly <rmattingly@apache.org> * Revert "Modern backup failures can cause backup system to lock up (#7288)" (#7307) This reverts commit c6a0c3b. * HBASE-29448 Modern backup failures can cause backup system to lock up (#7308) Co-authored-by: Hernan Romer <nanug33@gmail.com> Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com> Signed-off-by: Charles Connell <cconnell@apache.org> Signed-off-by: Ray Mattingly <rmattingly@apache.org> * HBASE-29548 Update ApacheDS to 2.0.0.AM27 and ldap-api to 2.1.7 (#7305) Signed-off-by: Nihal Jain <nihaljain@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29602 Add -Djava.security.manager=allow to JDK18+ surefire JVM flags (#7315) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Balazs Meszaros <meszibalu@apache.org> * HBASE-29601 Handle Junit 5 tests in TestCheckTestClasses (#7311) Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29592 Add hadoop 3.4.2 in client integration tests (#7306) Signed-off-by: Nihal Jain <nihaljain@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29587 Set Test category for TestSnapshotProcedureEarlyExpiration (#7292) Signed-off-by: Dávid Paksy <paksyd@apache.org> * HBASE-29610 Add and use String constants for Junit 5 @tag annotations (#7322) Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29591 Add hadoop 3.4.2 in hadoop check (#7320) Signed-off-by: Istvan Toth <stoty@apache.org> * HBASE-29609 Upgrade checkstyle and Maven checkstyle plugin (#7321) Signed-off-by: Istvan Toth <stoty@apache.org> * HBASE-29608 Add test to make sure we do not have copy paste errors in the TAG value (#7324) Signed-off-by: Istvan Toth <stoty@apache.org> * HBASE-29608 Addendum remove jdk9+ only API calls * Revert "HBASE-29609 Upgrade checkstyle and Maven checkstyle plugin (#7321)" (#7332) This reverts commit 04d48ee. * HBASE-29612 Remove HBaseTestingUtil.forceChangeTaskLogDir (#7326) Co-authored-by: Daniel Roudnitsky <droudnitsky1@bloomberg.net> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29576 Replicate HBaseClassTestRule functionality for Junit 5 (#7331) Signed-off-by: Istvan Toth <stoty@apache.org> * HBASE-29576 Addendum fix typo Jupitor -> Jupiter * HBASE-29619 Don't use Java 14+ style case statements in RestoreBackupSystemTableProcedure (#7336) Signed-off-by: Dávid Paksy <paksyd@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29550 Reflection error in TestRSGroupsKillRS with Java 21 (#7327) Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29615 Update Small tests description wrt reuseForks in docs (#7335) Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-28440 Add support for using mapreduce sort in HFileOutputFormat2 (#7294) Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com> Signed-off-by: Ray Mattingly <rmattingly@apache.org> * HBASE-29623 Blocks for CFs with BlockCache disabled may still get cached on write or compaction (#7339) Signed-off-by: Peter Somogyi <psomogyi@apache.org> * HBASE-29627 Handle any block cache fetching errors when reading a block in HFileReaderImpl (#7341) Signed-off-by: Peter Somogyi <psomogyi@apache.org> * HBASE-29614 Remove static final field modification in tests around Unsafe (#7337) Signed-off-by: Peng Lu <lupeng@apache.org> * HBASE-29504 [DOC] Document Namespace Auto-Creation During Restore (#7199) * HBASE-29629 Record the quota user name value on metrics for RpcThrottlingExceptions (#7345) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> * HBASE-29497 Mention HFiles for incremental backups (#7216) * HBASE-29497 Mention HFiles for incremental backups * enhance the documention change * HBASE-29505 [DOC] Document Enhanced Options for Backup Delete Command (#7200) * HBASE-29505 [DOC] Document Enhanced Options for Backup Delete Command * update the doc with cautions * HBASE-29631 Fix race condition in IncrementalTableBackupClient when HFiles are archived during backup (#7346) Co-authored-by: Hernan Romer <nanug33@gmail.com> Co-authored-by: skhillon <skhillon@hubspot.com> Signed-off-by: Ray Mattingly <rmattingly@apache.org> * HBASE-29626: Refactor server side scan metrics for Coproc hooks (#7340) Signed-off-by: Viraj Jasani <vjasani@apache.org> * HBASE-29152 Replace site skin with Reflow2 Maven skin (#7355) - Replaced the Maven Fluido skin with the newer [Reflow2 Maven skin](https://devacfr.github.io/reflow-maven-skin/doc/reflow-documentation.html#doc-get-started) (Apache Phoenix project uses this). This brings newer Bootstrap (before we used 2.3.2, after 4.x - still not ideal because 5.x is the latest major version but it is an improvement). - The new skin also brings new more modern look. - Made sure only local resources are used by the website and the book.html - so no CDN is used - as before. We cannot load remote content as it is banned by central ASF Content Security Policy. - Fixed our site text customization was not working in project-info-reports.properties file (fixed filename, fixed keys) Signed-off-by: Istvan Toth <stoty@apache.org> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> * HBASE-29636 Implement TimedOutTestsListener for junit 5 (#7352) Signed-off by: Chandra Sekhar K <chandrasekhar188k@gmail.com> * HBASE-29223 Migrate Master Status Jamon page back to JSP (#6875) The JSP code is equivalent to the Jamon code, just changed the syntax back to JSP. Request attributes are used to transfer data between JSP pages. Tried to preserve the code as much as possible but did some changes: Sub-templates were usually extracted to separate JSP file (and included with `<jsp:include`), in some case it was extracted as Java method. Extracted some sections from master page to separate JSP pages: - Software Attributes - Warnings Extracted the long JavaScript from the master page which executes on page load to separate JS file. Extracted some frequently used static methods to a new util class: `MasterStatusUtil`. Also added unit tests for the static methods in `MasterStatusUtil`. Changed the Master Status page back to `/master.jsp` again. Now made sure that `/master-status` redirects to `/master.jsp`. Signed-off-by: Istvan Toth <stoty@apache.org> * HBASE-29647 Restore preWALRestore and postWALRestore coprocessor hooks (#7368) Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29637 Implement ResourceCheckerJUnitListener for junit 5 (#7366) Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Istvan Toth <stoty@apache.org> * HBASE-29604 BackupHFileCleaner uses flawed time based check (#7360) Adds javadoc mentioning the concurrent usage and thread-safety need of FileCleanerDelegate#getDeletableFiles. Fixes a potential thread-safety issue in BackupHFileCleaner: this class tracks timestamps to block the deletion of recently loaded HFiles that might be needed for backup purposes. The timestamps were being registered from inside the concurrent method, which could result in recently added files getting deleted. Moved the timestamp registration to the postClean method, which is called only a single time per cleaner run, so recently loaded HFiles are in fact protected from deletion. Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> * HBASE-29650 Upgrade tomcat-jasper to 9.0.110 (#7372) Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29653 Upgrade os-maven-plugin to 1.7.1 for RISC-V riscv64 support (#7376) Signed-off-by: Istvan Toth <stoty@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29659 Replace reflow-default-webdeps to fix site build failure (#7386) Replace reflow-default-webdeps with separate webjar dependencies because reflow-default-webdeps causes a Maven ERROR in the build log and Yetus build considers the site build to be broken. Turned off these features in the site skin which we don't need and would require to add more JavaScript: - highlight.js, - lightbox.js, - smooth scrolling. Improved code blocks style to look good without highlight.js. Also extracted MathJax.js and fontawesome (needed for RefGuide) versions to Maven properties. Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29531 Migrate RegionServer Status Jamon page back to JSP (#7371) This is the 2/3 step of the Jamon to JSP migration: the Region Server Status page. Did the migration the same way as for the Master Status page: #6875 Migrated the Jamon code to JSP as close as possible. Extracted the duplicated `formatZKString` method to new java class: ZKStringFormatter and added unit tests. Changed the Region Server Status page back to `/regionserver.jsp`. Made sure that `/rs-status` redirects to `/regionserver.jsp`. Extracted the BlockCache inline CSS styles to `hbase.css` file. Also extracted the large BlockCache Hit Ratio periods paging JavaScript code to separate .js file. Introduced a `src/main/resources/hbase-webapps/common` directory where we can place common JSP files which are used by both Master and RegionServer JSP pages. This required to adjust the JSP compiler Maven Antrun plugin a bit. Extracted the inline tablesorter initialization JavaScript code to separate file. Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29663 TimeBasedLimiters should support dynamic configuration refresh (#7387) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Signed-off-by: Charles Connell <cconnell@apache.org> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> * HBASE-29609 Upgrade checkstyle and Maven checkstyle plugin to support Java 14+ syntax Signed-off-by: Istvan Toth <stoty@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29680 release-util.sh should not hardcode JAVA_HOME for spotless (#7404) Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29677: Thread safety in QuotaRefresherChore (#7401) Signed-off by: Ray Mattingly <rmattingly@apache.org> * HBASE-29351 Quotas: adaptive wait intervals (#7396) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Signed-off-by: Charles Connell <cconnell@apache.org> * HBASE-29679: Suppress stack trace in RpcThrottlingException (#7403) Signed-off by: Ray Mattingly <rmattingly@apache.org> * HBASE-29461 Alphabetize the list of variables that can be dynamically configured (#7165) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Signed-off-by: Istvan Toth <stoty@apache.org> Reviewed by: Kota-SH <shanmukhaharipriya@gmail.com> * HBASE-29690 Correct typo in TableReplicationQueueStorage.removeAllQueues exception message (#7420) Co-authored-by: Daniel Roudnitsky <droudnitsky1@bloomberg.net> Signed-off-by: Nihal Jain <nihaljain@apache.org> Signed-off-by: Pankaj Kumar <pankajkumar@apache.org> * HBASE-29651 Bump jruby to 9.4.14.0 to fix multiple CVEs (#7405) This change fixes the following list of CVEs: - **CVE-2025-43857**: Fixed in JRuby 9.4.13.0 - **CVE-2025-27219**: Fixed in JRuby 9.4.14.0 - **CVE-2025-27220**: Fixed in JRuby 9.4.14.0 Signed-off-by: Nihal Jain <nihaljain@apache.org> Signed-off-by: Pankaj Kumar <pankajkumar@apache.org> * HBASE-27126 Support multi-threads cleaner for MOB files (#5833) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Pankaj Kumar <pankajkumar@apache.org> * HBASE-29662 - Avoid regionDir/tableDir creation as part of .regioninfo file creation in HRegion initialize (#7406) Signed-off-by: Andrew Purtell <apurtell@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org> * HBASE-29686 Compatible issue of HFileOutputFormat2#configureRemoteCluster (#7415) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Junegunn Choi <junegunn@apache.org> Signed-off-by: Pankaj Kumar <pankajkumar@apache.org> Reviewed-by: chaijunjie0101 <1340011734@qq.com> * HBASE-29667 Correct block priority to SINGLE on the first write to the bucket cache (#7399) Reviewed by: Kota-SH <shanmukhaharipriya@gmail.com> Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> * [ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil (#7416) TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build Signed-off-by: Nihal Jain <nihaljain@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29700 Always close RPC servers in AbstractTestIPC (#7434) Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29703 Remove duplicate calls to withNextBlockOnDiskSize (#7440) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-29702 Remove shade plugin from hbase-protocol-shaded (#7438) Signed-off-by: Nihal Jain <nihaljain@apache.org> Signed-off-by: Duo Zhang <zhangduo@apache.org> * HBASE-28996: Implement Custom ReplicationEndpoint to Enable WAL Backup to External Storage (#6633) * HBASE-28996: Implement Custom ReplicationEndpoint to Enable WAL Backup to External Storage * fix spotless error * HBASE-29025: Enhance the full backup command to support Continuous Backup (#6710) * HBASE-29025: Enhance the full backup command to support continuous backup * add new check for full backup command regards to continuous backup flag * minor fixes * HBASE-29210: Introduce Validation for PITR-Critical Backup Deletion (#6848) Signed-off-by: Andor Molnár <andor@apache.org> Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> * HBASE-29261: Investigate flaw in backup deletion validation of PITR-critical backups and propose correct approach (#6922) * improve the logic of backup deletion validation of PITR-critical backups * add new tests * HBASE-29133: Implement "pitr" Command for Point-in-Time Restore (#6717) Signed-off-by: Andor Molnar <andor@apache.org> Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> * HBASE-29255: Integrate backup WAL cleanup logic with the delete command (#7007) * Store bulkload files in daywise bucket as well * Integrate backup WAL cleanup logic with the delete command * address the review comments * address the review comments * address the review comments * add more unit tests to cover all cases * address the review comments * HBASE-28990 Modify Incremental Backup for Continuous Backup (#6788) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Signed-off-by: Andor Molnár andor@apache.org Reviewed by: Kota-SH <shanmukhaharipriya@gmail.com> Reviewed by: Vinayak Hegde <vinayakph123@gmail.com> Reviewed by: Kevin Geiszler <kevin.j.geiszler@gmail.com> * HBASE-29350: Ensure Cleanup of Continuous Backup WALs After Last Backup is Force Deleted (#7090) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Reviewed by: Kevin Geiszler <kevin.j.geiszler@gmail.com> * HBASE-29219 Ignore Empty WAL Files While Consuming Backed-Up WAL Files (#7106) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Reviewed by: Kota-SH <shanmukhaharipriya@gmail.com> Reviewed by: Kevin Geiszler <kevin.j.geiszler@gmail.com> * HBASE-29406: Skip Copying Bulkloaded Files to Backup Location in Continuous Backup (#7119) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Reviewed by: Kevin Geiszler <kevin.j.geiszler@gmail.com> * HBASE-29449 Update backup describe command for continuous backup (#7045) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Reviewed by: Kevin Geiszler <kevin.j.geiszler@gmail.com> * HBASE-29445 Add Option to Specify Custom Backup Location in PITR (#7153) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> * HBASE-29441 ReplicationSourceShipper should delegate the empty wal entries handling to ReplicationEndpoint (#7145) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> * HBASE-29459 Capture bulkload files only till IncrCommittedWalTs during Incremental Backup (#7166) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Reviewed by: Kevin Geiszler <kevin.j.geiszler@gmail.com> * HBASE-29310 Handle Bulk Load Operations in Continuous Backup (#7150) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Reviewed by: Kevin Geiszler <kevin.j.geiszler@gmail.com> * HBASE-28957 spotless apply after rebase * HBASE-29375 Add Unit Tests for BackupAdminImpl and Improve Test Granularity (#7171) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Reviewed by: Kevin Geiszler <kevin.j.geiszler@gmail.com> * HBASE-29519 Copy Bulkloaded Files in Continuous Backup (#7222) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Signed-off-by: Andor Molnár <andor@apache.org> * HBASE-29524 Handle bulk-loaded HFiles in delete and cleanup process (#7239) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Reviewed by: Kota-SH <shanmukhaharipriya@gmail.com> * [HBASE-29520] Utilize Backed-up Bulkloaded Files in Incremental Backup (#7246) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> * Revert "HBASE-29310 Handle Bulk Load Operations in Continuous Backup (#7150)" (#7290) This reverts commit 5ac2a73. * HBASE-29521: Update Restore Command to Handle Bulkloaded Files (#7300) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Signed-off-by: Andor Molnár andor@apache.org Reviewed by: Kevin Geiszler <kevin.j.geiszler@gmail.com> Reviewed by: Kota-SH <shanmukhaharipriya@gmail.com> * HBASE-29656 Scan WALs to identify bulkload operations for incremental backup (#7400) * Scan WALs to identify bulkload operations for incremental backup * Update unit test * Info log * Minor test fix * Address review comments * Spotless apply * Addressed review comment * spotless * Remove log * Retrigger CI --------- Co-authored-by: Ankit Solomon <asolomon@cloudera.com> * HBASE-28957. Build + spotless fix * HBASE-29826: Backup merge is failing because .backup.manifest cannot be found (#7664) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> * HBASE-29825: Incremental backup is failing due to incorrect timezone (#7683) Change-Id: I8702eca4adc81bad2c18ea4990d09556c9506a34 * HBASE-29687: Extend IntegrationTestBackupRestore to handle continuous backups (#7417) * Extend IntegrationTestBackup restore into a base class with continuous and non-continuous subclasses Change-Id: I0c70c417b86c7732b58642a51c75897c35b16cb6 * Add more test cases to runTestSingle for testContinuousBackupRestore Change-Id: Id043400bf85c7b696bb94bef7cb17ed9dad13334 * Add more test cases for full continuous backup; Change while loop to a for loop Change-Id: I5ba3276919e6bbdf343c134fa287c69f3854a8a2 * Add delete test case Change-Id: I25fe484e9c227b7a31cb3768def3c12f66d617ac * Start adding changes after looking at WIP PR in GitHub Change-Id: Ie9aece8a3ec55739d618ebf2d2f173a41a116eb6 * Continue adding changes after looking at WIP PR in GitHub Change-Id: Ie345e623089979f028b13aed13e5ec93e025eff8 * Run mvn spotless:apply Change-Id: I98eb019dd93dfc8e21b6c730e0e2e60314102724 * Add documentation for runTestMulti and runTestSingle Change-Id: I4de6fc485aa1ff6e0d8d837e081f8dde20bb3f67 * Update documentation Change-Id: I911180a8f263f801a5c299d43d0215fe444f22d3 * Enhance delete test case Change-Id: I78fe59f800cde7c89b11760a49d774c5173a862c * Update method name to verifyBackupExistenceAfterMerge Change-Id: Ia150d21f48bb160d9e8bcf922799dc18c0b7c77c * Address review comments Change-Id: I9d5b55e36b44367ac8ace08a5859c42b796fefd4 * Add wait for region servers in replication checkpoint to catch up with latest Put timestamp Change-Id: Ic438ca292bc01827d46725e006bfa0c21bc95f01 * Handle command line arg parsing and conf setup in base class Change-Id: I9d52e774e84dc389d42aa63315529a2590c40cb8 * Fix spotless error Change-Id: I27eec25091842376ee7a059a9688c6f5ab385ac7 * Fix checkstyle errors for IntegrationTestBackupRestore.java Change-Id: I18ab629df4af4e93b42ec1b0d576fd411279c775 * Remove initializeConfFromCommandLine() Change-Id: Ibc96fd712e384cc3ca5a2c4575e47e65e62c60fa * Change info log message to debug Change-Id: Ie8e94ce978836b1314525138726a13641360aae6 * Run mvn spotless:apply Change-Id: Ibeea379a65e801b60ec5124938b7aa17087025f0 * HBASE-29815: Fix issue where backup integration tests are not running in IntelliJ (#7625) Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> --------- Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Lijin Bin <binlijin@apache.org> Signed-off-by: Dávid Paksy <paksyd@apache.org> Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Charles Connell <cconnell@apache.org> Signed-off-by: Nihal Jain <nihaljain@apache.org> Signed-off-by: Ray Mattingly <rmattingly@apache.org> Signed-off-by: Balazs Meszaros <meszibalu@apache.org> Signed-off-by: Istvan Toth <stoty@apache.org> Signed-off-by: Peter Somogyi <psomogyi@apache.org> Signed-off-by: Peng Lu <lupeng@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Signed-off-by: Tak Lon (Stephen) Wu <taklwu@apache.org> Signed-off-by: Pankaj Kumar <pankajkumar@apache.org> Signed-off-by: Andrew Purtell <apurtell@apache.org> Signed-off-by: Junegunn Choi <junegunn@apache.org> Signed-off-by: Andor Molnár <andor@apache.org> Signed-off-by: Andor Molnar <andor@apache.org> Signed-off-by: Andor Molnár andor@apache.org Co-authored-by: Charles Connell <cconnell@apache.org> Co-authored-by: Ruanhui <32773751+frostruan@users.noreply.github.com> Co-authored-by: huiruan <huiruan@tencent.com> Co-authored-by: Duo Zhang <zhangduo@apache.org> Co-authored-by: Wellington Ramos Chevreuil <wchevreuil@apache.org> Co-authored-by: Junegunn Choi <junegunn@apache.org> Co-authored-by: Istvan Toth <stoty@apache.org> Co-authored-by: Hernan Romer <nanug33@gmail.com> Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com> Co-authored-by: Ray Mattingly <rmattingly@apache.org> Co-authored-by: Sreenivasulu <sreenivasulured2y@gmail.com> Co-authored-by: Dávid Paksy <paksyd@apache.org> Co-authored-by: Daniel Roudnitsky <droudnitsky1@bloomberg.net> Co-authored-by: vinayak hegde <vinayakph123@gmail.com> Co-authored-by: Siddharth Khillon <sidkhillon24@gmail.com> Co-authored-by: skhillon <skhillon@hubspot.com> Co-authored-by: sanjeet006py <36011005+sanjeet006py@users.noreply.github.com> Co-authored-by: DieterDP <90392398+DieterDP-ng@users.noreply.github.com> Co-authored-by: gong-flying <106514313+gong-flying@users.noreply.github.com> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Co-authored-by: Andrew Purtell <apurtell@apache.org> Co-authored-by: droudnitsky <168442446+droudnitsky@users.noreply.github.com> Co-authored-by: xavifeds8 <65709700+xavifeds8@users.noreply.github.com> Co-authored-by: Chandra Sekhar K <chandra@apache.org> Co-authored-by: gvprathyusha6 <70918688+gvprathyusha6@users.noreply.github.com> Co-authored-by: mokai <mokai87@126.com> Co-authored-by: Huginn <63332600+Huginn-kio@users.noreply.github.com> Co-authored-by: Liu Xiao <42756849+liuxiaocs7@users.noreply.github.com> Co-authored-by: asolomon <ankitsolomon@gmail.com> Co-authored-by: Andor Molnár <andor@apache.org> Co-authored-by: Ankit Solomon <asolomon@cloudera.com> Co-authored-by: Andor Molnar <andor@cloudera.com>

HBASE-29451 Add Docs section describing BucketCache Time based priority

da24a7c

Change-Id: I3de6b19030d242787ecb715590fbe5c2a1f7fe0c

This comment has been minimized.

Sign in to view

taklwu requested a review from Copilot September 9, 2025 16:29

Copilot AI reviewed Sep 9, 2025

View reviewed changes

src/main/asciidoc/_chapters/architecture.adoc Outdated Show resolved Hide resolved

src/main/asciidoc/_chapters/architecture.adoc Outdated Show resolved Hide resolved

src/main/asciidoc/_chapters/architecture.adoc Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

taklwu reviewed Sep 9, 2025

View reviewed changes

kgeisz approved these changes Sep 9, 2025

View reviewed changes

wchevreuil and others added 7 commits September 10, 2025 14:14

Update src/main/asciidoc/_chapters/architecture.adoc

a82409d

Co-authored-by: Kevin Geiszler <kgeiszler@cloudera.com>

Update src/main/asciidoc/_chapters/architecture.adoc

326e3db

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/main/asciidoc/_chapters/architecture.adoc

ad2034b

Co-authored-by: Kevin Geiszler <kgeiszler@cloudera.com>

Update src/main/asciidoc/_chapters/architecture.adoc

3eed1d0

Co-authored-by: Kevin Geiszler <kgeiszler@cloudera.com>

Update src/main/asciidoc/_chapters/architecture.adoc

5dba831

Co-authored-by: Tak Lon (Stephen) Wu <taklwu@apache.org>

Update src/main/asciidoc/_chapters/architecture.adoc

033e4b3

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/main/asciidoc/_chapters/architecture.adoc

0659a35

Co-authored-by: Kevin Geiszler <kgeiszler@cloudera.com>

This comment has been minimized.

Sign in to view

Update src/main/asciidoc/_chapters/architecture.adoc

2bb7688

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

This comment has been minimized.

Sign in to view

Adding links to the configuration sections of each time based priorit…

c767924

…y type Change-Id: I270bf39f31687f9589cc2c62274af47db2d6b856

wchevreuil requested review from Copilot and taklwu September 11, 2025 14:43

Copilot AI reviewed Sep 11, 2025

View reviewed changes

PDavid approved these changes Sep 12, 2025

View reviewed changes

wchevreuil merged commit d592404 into apache:master Sep 15, 2025
1 check passed

	//handle error
	e.printStackTrace(); // Log the error for debugging

Conversation

wchevreuil commented Sep 9, 2025

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

taklwu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

taklwu Sep 9, 2025

Choose a reason for hiding this comment

Uh oh!

wchevreuil Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

taklwu Sep 9, 2025

Choose a reason for hiding this comment

Uh oh!

wchevreuil Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

wchevreuil Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

kgeisz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Apache-HBase commented Sep 10, 2025

Uh oh!

Apache-HBase commented Sep 10, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

wchevreuil commented Sep 12, 2025

Uh oh!

PDavid left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants