[improve](cloud-mow) batch get tablet stats when get_delete_bitmap_update_lock #47281

mymeiyi · 2025-01-21T11:26:38Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

as the title,
before:

after:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-01-21T11:26:43Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

cosven · 2025-01-21T11:53:20Z

cloud/src/meta-service/meta_service.cpp

+    } else {
+        int32_t batch_size = config::max_mow_tablet_stat_num_per_batch;


How about adding some tests for this branch?

+1， #45206 added many unit tests, this PR should update these tests as well

enable this feature by default, so the original test works for it

cloud/src/meta-service/meta_service.cpp

mymeiyi · 2025-01-23T07:26:31Z

run buildall

bobhan1 · 2025-01-24T12:50:18Z

cloud/src/meta-service/meta_service.cpp

+        stats_values.reserve(batch_size);
+        size_t num_acquired = request->tablet_indexes_size();
+
+        for (size_t i = response->base_compaction_cnts_size(); i < num_acquired; i += batch_size) {


Transaction::batch_get handled batch splitting inside(BatchGetOptions::concurrency), so don't need to do this in caller place?

mymeiyi · 2025-05-26T07:30:15Z

run buildall

doris-robot · 2025-05-26T07:46:52Z

Cloud UT Coverage Report

Increment line coverage 66.48% (117/176) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	83.26% (1114/1338)
Line Coverage	66.37% (18743/28240)
Region Coverage	66.03% (9301/14086)
Branch Coverage	55.80% (5005/8970)

zhannngchen

LGTM

github-actions · 2025-05-27T13:56:02Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-05-27T13:56:05Z

PR approved by anyone and no changes requested.

dataroaring

LGTM

…date_lock (apache#47281)

…te_bitmap_update_lock #47281 (#52151) Cherry-pick from #47281

…date_lock (apache#47281)

…te_bitmap_update_lock (#47281) (#52225) pick #47281

mymeiyi requested review from dataroaring, gavinchou and w41ter as code owners January 21, 2025 11:26

mymeiyi force-pushed the batch-get-2 branch from 918dfb8 to 253ea98 Compare January 21, 2025 11:28

cosven reviewed Jan 21, 2025

View reviewed changes

zhannngchen reviewed Jan 22, 2025

View reviewed changes

cloud/src/meta-service/meta_service.cpp Show resolved Hide resolved

mymeiyi force-pushed the batch-get-2 branch 3 times, most recently from f3495f5 to 535fc73 Compare January 23, 2025 07:05

bobhan1 reviewed Jan 24, 2025

View reviewed changes

dataroaring added the dev/3.0.x label Jan 24, 2025

mymeiyi force-pushed the batch-get-2 branch from 535fc73 to 3e38faa Compare May 26, 2025 07:23

batch get tablet stats and meta

bdb2c26

mymeiyi force-pushed the batch-get-2 branch from 3e38faa to bdb2c26 Compare May 26, 2025 07:26

zhannngchen approved these changes May 27, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 27, 2025

github-actions bot added the reviewed label May 27, 2025

dataroaring approved these changes May 29, 2025

View reviewed changes

dataroaring merged commit ca84354 into apache:master May 29, 2025
25 of 26 checks passed

github-actions bot added the dev/3.0.x-conflict label May 29, 2025

koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025

[improve](cloud-mow) batch get tablet stats when get_delete_bitmap_up…

75d6617

…date_lock (apache#47281)

mymeiyi added a commit to mymeiyi/doris that referenced this pull request Jun 19, 2025

[improve](cloud-mow) batch get tablet stats when get_delete_bitmap_up…

15c7a91

…date_lock (apache#47281)

mymeiyi added a commit to mymeiyi/doris that referenced this pull request Jun 23, 2025

[improve](cloud-mow) batch get tablet stats when get_delete_bitmap_up…

9853024

…date_lock (apache#47281)

mymeiyi added a commit to mymeiyi/doris that referenced this pull request Jun 23, 2025

[improve](cloud-mow) batch get tablet stats when get_delete_bitmap_up…

56365c0

…date_lock (apache#47281)

mymeiyi mentioned this pull request Jun 23, 2025

branch-3.1: [improve](cloud-mow) batch get tablet stats when get_delete_bitmap_update_lock (#47281) #52151

Merged

morrySnow pushed a commit that referenced this pull request Jun 24, 2025

branch-3.1: [improve](cloud-mow) batch get tablet stats when get_dele…

997903b

…te_bitmap_update_lock #47281 (#52151) Cherry-pick from #47281

morrySnow added the dev/3.1.0-merged label Jun 24, 2025

mymeiyi added a commit to mymeiyi/doris that referenced this pull request Jun 24, 2025

[improve](cloud-mow) batch get tablet stats when get_delete_bitmap_up…

a3b960a

…date_lock (apache#47281)

mymeiyi mentioned this pull request Jun 24, 2025

branch-3.0: [improve](cloud-mow) batch get tablet stats when get_delete_bitmap_update_lock (#47281) #52225

Merged

dataroaring pushed a commit that referenced this pull request Jun 25, 2025

branch-3.0: [improve](cloud-mow) batch get tablet stats when get_dele…

4a8260e

…te_bitmap_update_lock (#47281) (#52225) pick #47281

dataroaring added dev/3.0.7-merged and removed dev/3.0.x dev/3.0.x-conflict labels Jun 25, 2025

gavinchou mentioned this pull request Aug 15, 2025

Release Note 3.0.7 #54868

Open

mymeiyi deleted the batch-get-2 branch September 10, 2025 06:39

		} else {
		int32_t batch_size = config::max_mow_tablet_stat_num_per_batch;

[improve](cloud-mow) batch get tablet stats when get_delete_bitmap_update_lock #47281

[improve](cloud-mow) batch get tablet stats when get_delete_bitmap_update_lock #47281

Uh oh!

Conversation

mymeiyi commented Jan 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jan 21, 2025

Uh oh!

cosven Jan 21, 2025

Choose a reason for hiding this comment

Uh oh!

zhannngchen Jan 22, 2025

Choose a reason for hiding this comment

Uh oh!

mymeiyi Mar 31, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mymeiyi commented Jan 23, 2025

Uh oh!

bobhan1 Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mymeiyi commented May 26, 2025

Uh oh!

doris-robot commented May 26, 2025

Cloud UT Coverage Report

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented May 27, 2025

Uh oh!

github-actions bot commented May 27, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

mymeiyi commented Jan 21, 2025 •

edited

Loading

bobhan1 Jan 24, 2025 •

edited

Loading