Skip to content

branch-4.0: [feat][iceberg] Support Iceberg Meta Procedure implementations #56257#56732

Merged
yiguolei merged 1 commit intobranch-4.0from
auto-pick-56257-branch-4.0
Oct 11, 2025
Merged

branch-4.0: [feat][iceberg] Support Iceberg Meta Procedure implementations #56257#56732
yiguolei merged 1 commit intobranch-4.0from
auto-pick-56257-branch-4.0

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Oct 9, 2025

Cherry-picked from #56257

@vinlee19
[feat][iceberg] Support Iceberg Meta Procedure implementations (#56257)
fc635b9 
### What problem does this PR solve?

This PR extends the OPTIMIZE TABLE framework introduced in #55679 by
implementing additional Iceberg meta procedure actions. Building upon
the foundation established for Iceberg
table optimization, this enhancement adds critical snapshot management
operations that enable more sophisticated Iceberg table maintenance
workflows.


#### New Iceberg Actions Implemented

This PR introduces **5 new Iceberg meta procedure actions**:

1. **`cherrypick_snapshot`** - Cherry-picks changes from a specific
snapshot
2. **`fast_forward`** - Fast-forwards one branch to match another
branch's latest snapshot
3. **`rollback_to_snapshot`** - Rolls back table to a specific snapshot
4. **`rollback_to_timestamp`** - Rolls back table to a specific
timestamp
  5. **`set_current_snapshot`** - Sets a specific snapshot as current

  #### Example Usage
  ```sql
  -- Cherry-pick changes from a snapshot
  OPTIMIZE TABLE iceberg_catalog.db.table
PROPERTIES("action" = "cherrypick_snapshot", "snapshot_id" =
"123456789");
```
```
  -- Fast-forward branch to match another branch
  OPTIMIZE TABLE iceberg_catalog.db.table
PROPERTIES("action" = "fast_forward", "branch" = "feature", "to" =
"main");
```
```
  -- Rollback to specific snapshot
  OPTIMIZE TABLE iceberg_catalog.db.table
PROPERTIES("action" = "rollback_to_snapshot", "snapshot_id" =
"987654321");
```

The regression testing strategy utilizes internal Iceberg catalog operations for table creation, data insertion, and branch/tag management, ensuring test stability and eliminating
  dependencies on external tools like Spark SQL for test data preparation.
@github-actions github-actions bot requested a review from yiguolei as a code owner October 9, 2025 03:15
@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 9, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Oct 9, 2025
@dataroaring dataroaring reopened this Oct 9, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 9, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 9, 2025

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

doris-robot commented Oct 9, 2025

ClickBench: Total hot run time: 30.03 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fc635b9e9d8c0e257d6374f7117371d7a2ec6ed2, data reload: true

query1	0.04	0.03	0.03
query2	0.12	0.04	0.04
query3	0.28	0.08	0.08
query4	1.72	0.11	0.11
query5	0.27	0.24	0.25
query6	1.24	0.64	0.64
query7	0.03	0.02	0.02
query8	0.06	0.04	0.03
query9	0.62	0.55	0.52
query10	0.58	0.58	0.59
query11	0.19	0.11	0.11
query12	0.16	0.12	0.12
query13	0.63	0.62	0.62
query14	0.81	0.85	0.82
query15	0.89	0.88	0.87
query16	0.41	0.39	0.41
query17	1.06	1.08	1.10
query18	0.20	0.20	0.19
query19	1.92	1.87	1.92
query20	0.02	0.02	0.01
query21	15.45	0.94	0.57
query22	0.80	0.98	0.70
query23	15.25	1.37	0.57
query24	16.29	0.47	0.17
query25	0.21	0.05	0.04
query26	0.35	0.16	0.12
query27	0.05	0.05	0.05
query28	10.74	0.96	0.92
query29	12.75	3.92	3.25
query30	0.31	0.13	0.11
query31	3.00	0.62	0.39
query32	3.31	0.56	0.52
query33	3.27	3.29	3.34
query34	17.35	5.74	5.01
query35	5.06	5.08	5.10
query36	0.73	0.55	0.53
query37	0.10	0.06	0.06
query38	0.09	0.03	0.04
query39	0.04	0.03	0.04
query40	0.19	0.16	0.14
query41	0.09	0.03	0.02
query42	0.04	0.02	0.03
query43	0.03	0.04	0.04
Total cold run time: 116.75 s
Total hot run time: 30.03 s

@vinlee19
Copy link
Contributor

vinlee19 commented Oct 11, 2025

run p0

@github-actions
Copy link
Contributor Author

github-actions bot commented Oct 11, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Oct 11, 2025
@github-actions
Copy link
Contributor Author

github-actions bot commented Oct 11, 2025

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit 325d32c into branch-4.0 Oct 11, 2025
24 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hello-stephen @doris-robot @vinlee19 @yiguolei @dataroaring