-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[opt](cloud) optimize load performance for mow table when pack small files #58839
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
TPC-H: Total hot run time: 35920 ms |
TPC-DS: Total hot run time: 181221 ms |
ClickBench: Total hot run time: 27.51 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
|
run buildall |
TPC-DS: Total hot run time: 182820 ms |
ClickBench: Total hot run time: 27.33 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
|
PR approved by anyone and no changes requested. |
|
PR approved by at least one committer and no changes requested. |
…files (apache#58839) Related PR: apache#57770 Problem Summary: The main change is to offload the entire delete bitmap calculation process—including file closing, rowset building, segment loading, and bitmap calculation—to a background thread pool. This prevents blocking the memtable flush thread, enhancing performance and concurrency.
…files (apache#58839) Related PR: apache#57770 Problem Summary: The main change is to offload the entire delete bitmap calculation process—including file closing, rowset building, segment loading, and bitmap calculation—to a background thread pool. This prevents blocking the memtable flush thread, enhancing performance and concurrency.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #57770
Problem Summary:
The main change is to offload the entire delete bitmap calculation process—including file closing, rowset building, segment loading, and bitmap calculation—to a background thread pool. This prevents blocking the memtable flush thread, enhancing performance and concurrency.
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)