[Fix](cloud-mow) Fix wrong tablet_id in delete bitmap #55771

bobhan1 · 2025-09-08T07:20:06Z

What problem does this PR solve?

fix wrong tablet_id in DeleteBitmap

Status CloudMetaMgr::get_tablet_meta(int64_t tablet_id, TabletMetaSharedPtr* tablet_meta) {
    // ...
    *tablet_meta = std::make_shared<TabletMeta>();
    (*tablet_meta)
            ->init_from_pb(cloud_tablet_meta_to_doris(std::move(*resp.mutable_tablet_meta())));
    VLOG_DEBUG << "get tablet meta, tablet_id: " << (*tablet_meta)->tablet_id();
    // tablet_meta._delete_bitmap._tablet_id is always 0
    return Status::OK();
}

TabletMeta::TabletMeta()
        : _tablet_uid(0, 0),
          _schema(new TabletSchema),
          _delete_bitmap(new DeleteBitmap(_tablet_id)) {}

fix [Opt](mow) Opt mow load performance and CPU usage #55073, [Fix](case) Comment some case to avoid p0 stuck #55733 case stuck due to not sending reply

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-09-08T07:20:12Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

bobhan1 · 2025-09-08T07:24:22Z

run buildall

doris-robot · 2025-09-08T07:44:02Z

TPC-H: Total hot run time: 34926 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9387a611e68fa0d31d88e3ddfffc6fec6e836cb8, data reload: false

------ Round 1 ----------------------------------
q1	17645	5316	5119	5119
q2	2000	328	248	248
q3	10220	1307	731	731
q4	10219	1042	528	528
q5	7520	2442	2391	2391
q6	190	172	138	138
q7	955	769	652	652
q8	9343	1384	1136	1136
q9	7071	5168	5168	5168
q10	6913	2392	1961	1961
q11	507	302	275	275
q12	379	364	226	226
q13	17776	3690	3035	3035
q14	246	251	217	217
q15	582	508	504	504
q16	1008	1003	938	938
q17	605	881	362	362
q18	7542	7198	7092	7092
q19	1225	1000	587	587
q20	342	341	227	227
q21	3736	3260	2413	2413
q22	1095	1039	978	978
Total cold run time: 107119 ms
Total hot run time: 34926 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5210	5164	5143	5143
q2	245	334	224	224
q3	2198	2714	2363	2363
q4	1342	1813	1350	1350
q5	4314	4641	4672	4641
q6	221	178	133	133
q7	2057	1971	1832	1832
q8	2699	2689	2631	2631
q9	7333	7422	7234	7234
q10	3137	3404	2941	2941
q11	601	535	541	535
q12	706	847	646	646
q13	3549	3976	3319	3319
q14	310	321	282	282
q15	518	487	493	487
q16	1073	1112	1074	1074
q17	1174	1612	1461	1461
q18	7915	7828	7776	7776
q19	851	881	891	881
q20	2052	2056	1892	1892
q21	4848	4439	4291	4291
q22	1094	1059	1017	1017
Total cold run time: 53447 ms
Total hot run time: 52153 ms

dataroaring

LGTM

github-actions · 2025-09-08T08:19:13Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-09-08T08:19:16Z

PR approved by anyone and no changes requested.

bobhan1 · 2025-09-12T10:39:11Z

run buildall

doris-robot · 2025-09-12T11:22:29Z

TPC-H: Total hot run time: 35736 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 434cf588e22e5adef8e5d8266679bfd02ff07f89, data reload: false

------ Round 1 ----------------------------------
q1	17587	5234	5110	5110
q2	2000	337	217	217
q3	10249	1305	746	746
q4	10213	1039	534	534
q5	7564	2408	2418	2408
q6	187	171	140	140
q7	954	767	652	652
q8	9367	1357	1137	1137
q9	7058	5136	5163	5136
q10	6913	2398	1975	1975
q11	483	309	301	301
q12	353	378	239	239
q13	17767	3708	3036	3036
q14	242	251	222	222
q15	551	499	503	499
q16	1009	999	948	948
q17	625	890	368	368
q18	7521	7327	7113	7113
q19	1233	967	616	616
q20	387	362	260	260
q21	3871	3235	3045	3045
q22	1081	1050	1034	1034
Total cold run time: 107215 ms
Total hot run time: 35736 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5237	5147	5147	5147
q2	254	347	228	228
q3	2236	2731	2342	2342
q4	1357	1814	1377	1377
q5	4439	4680	4600	4600
q6	222	181	133	133
q7	2070	1970	1819	1819
q8	2659	2648	2661	2648
q9	7491	7282	7315	7282
q10	3127	3362	2864	2864
q11	592	524	508	508
q12	716	788	636	636
q13	3911	3833	3374	3374
q14	311	327	283	283
q15	521	515	525	515
q16	1193	1136	1079	1079
q17	1206	1579	1430	1430
q18	8036	7798	7610	7610
q19	822	867	1001	867
q20	2053	2103	1942	1942
q21	5202	4391	4377	4377
q22	1105	1066	1015	1015
Total cold run time: 54760 ms
Total hot run time: 52076 ms

doris-robot · 2025-09-12T11:34:14Z

TPC-DS: Total hot run time: 187949 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 434cf588e22e5adef8e5d8266679bfd02ff07f89, data reload: false

query1	1045	434	398	398
query2	6579	1735	1764	1735
query3	6754	226	226	226
query4	26499	23653	23208	23208
query5	4430	640	481	481
query6	340	258	231	231
query7	4666	526	302	302
query8	319	260	252	252
query9	8684	2642	2641	2641
query10	498	339	283	283
query11	15742	15043	14818	14818
query12	183	118	118	118
query13	1689	571	469	469
query14	12704	9211	9213	9211
query15	249	192	186	186
query16	7817	671	521	521
query17	1673	767	622	622
query18	2050	428	329	329
query19	265	202	184	184
query20	127	122	122	122
query21	236	129	122	122
query22	4291	4117	3998	3998
query23	33971	33359	32771	32771
query24	8127	2366	2402	2366
query25	524	512	425	425
query26	1226	297	159	159
query27	2672	510	363	363
query28	4340	2245	2219	2219
query29	761	611	477	477
query30	293	226	197	197
query31	941	818	734	734
query32	84	73	70	70
query33	584	380	333	333
query34	800	863	520	520
query35	811	832	736	736
query36	964	1032	908	908
query37	119	109	97	97
query38	3447	3458	3498	3458
query39	1554	1439	1426	1426
query40	216	125	120	120
query41	63	64	75	64
query42	128	112	114	112
query43	532	549	469	469
query44	1360	863	839	839
query45	189	179	167	167
query46	865	1008	642	642
query47	1762	1829	1742	1742
query48	392	415	331	331
query49	798	529	398	398
query50	661	698	411	411
query51	3961	3924	3922	3922
query52	113	106	102	102
query53	241	261	190	190
query54	598	585	536	536
query55	89	85	83	83
query56	314	322	288	288
query57	1189	1211	1135	1135
query58	274	265	272	265
query59	2478	2664	2596	2596
query60	343	328	311	311
query61	168	164	160	160
query62	801	725	653	653
query63	228	188	189	188
query64	4429	1177	835	835
query65	4028	3962	3958	3958
query66	1073	458	336	336
query67	15441	15231	15202	15202
query68	8219	926	586	586
query69	489	317	282	282
query70	1393	1381	1287	1287
query71	604	347	308	308
query72	5807	4894	5082	4894
query73	723	619	361	361
query74	9211	8965	8758	8758
query75	4143	3324	2773	2773
query76	3686	1200	747	747
query77	816	405	311	311
query78	9581	9802	8925	8925
query79	2349	804	595	595
query80	674	564	498	498
query81	476	256	228	228
query82	444	170	125	125
query83	296	263	260	260
query84	308	110	95	95
query85	887	457	419	419
query86	353	308	288	288
query87	3770	3711	3601	3601
query88	3342	2207	2216	2207
query89	399	341	290	290
query90	1925	215	210	210
query91	167	175	133	133
query92	86	66	65	65
query93	1679	1042	627	627
query94	702	449	329	329
query95	403	311	304	304
query96	485	567	275	275
query97	2945	3016	2861	2861
query98	237	219	217	217
query99	1447	1403	1319	1319
Total cold run time: 278838 ms
Total hot run time: 187949 ms

doris-robot · 2025-09-12T11:39:42Z

ClickBench: Total hot run time: 29.8 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 434cf588e22e5adef8e5d8266679bfd02ff07f89, data reload: false

query1	0.05	0.05	0.06
query2	0.10	0.06	0.06
query3	0.26	0.09	0.08
query4	1.61	0.12	0.12
query5	0.27	0.26	0.26
query6	1.20	0.66	0.64
query7	0.04	0.03	0.02
query8	0.06	0.04	0.04
query9	0.62	0.54	0.52
query10	0.57	0.59	0.57
query11	0.16	0.11	0.11
query12	0.16	0.13	0.12
query13	0.67	0.62	0.64
query14	1.02	1.03	1.04
query15	0.86	0.84	0.87
query16	0.40	0.38	0.38
query17	1.09	1.09	1.06
query18	0.21	0.20	0.20
query19	1.89	1.87	1.85
query20	0.02	0.01	0.01
query21	15.41	0.94	0.58
query22	0.77	1.21	0.65
query23	14.88	1.42	0.65
query24	7.09	0.98	0.71
query25	0.45	0.09	0.12
query26	0.54	0.16	0.13
query27	0.07	0.05	0.06
query28	9.46	0.92	0.43
query29	12.58	3.91	3.28
query30	0.29	0.13	0.12
query31	2.83	0.62	0.39
query32	3.26	0.56	0.46
query33	3.11	3.11	3.14
query34	16.17	5.49	4.81
query35	4.91	4.88	4.92
query36	0.74	0.51	0.49
query37	0.10	0.07	0.07
query38	0.07	0.05	0.04
query39	0.04	0.03	0.04
query40	0.18	0.16	0.15
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 104.39 s
Total hot run time: 29.8 s

bobhan1 · 2025-09-13T00:05:05Z

run beut

bobhan1 · 2025-09-13T00:05:10Z

run p0

hello-stephen · 2025-09-13T03:30:29Z

BE UT Coverage Report

Increment line coverage 33.33% (4/12) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.21% (17432/33389)
Line Coverage	37.43% (158395/423180)
Region Coverage	32.01% (120823/377480)
Branch Coverage	33.37% (53001/158828)

hello-stephen · 2025-09-13T03:36:12Z

BE Regression && UT Coverage Report

Increment line coverage 91.67% (11/12) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	70.81% (23118/32647)
Line Coverage	57.25% (241659/422147)
Region Coverage	52.62% (201247/382459)
Branch Coverage	54.32% (86659/159521)

…55733 #55771 (#55767) pick #55073 #55733 #55771

bobhan1 · 2025-09-14T12:59:40Z

run p0

hello-stephen · 2025-09-14T15:19:50Z

BE Regression && UT Coverage Report

Increment line coverage 91.67% (11/12) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	70.85% (23129/32647)
Line Coverage	57.28% (241810/422147)
Region Coverage	52.62% (201246/382459)
Branch Coverage	54.36% (86715/159521)

dataroaring

LGTM

github-actions · 2025-09-15T02:01:45Z

PR approved by at least one committer and no changes requested.

dataroaring

LGTM

bobhan1 mentioned this pull request Sep 8, 2025

branch-3.1: [Opt](mow) Opt mow load performance and CPU usage #55073 #55767

Merged

dataroaring previously approved these changes Sep 8, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 8, 2025

github-actions bot added the reviewed label Sep 8, 2025

dataroaring added the dev/3.1.x label Sep 8, 2025

fix

246ddd9

bobhan1 changed the title ~~[Fix](cloud-mow) Fix case stuck due to not sending reply~~ [Fix](cloud-mow) Fix wrong tablet_id in delete bitmap Sep 12, 2025

bobhan1 dismissed dataroaring’s stale review via d61cb59 September 12, 2025 09:16

bobhan1 force-pushed the fix-55073-stuck branch from 9387a61 to d61cb59 Compare September 12, 2025 09:16

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Sep 12, 2025

bobhan1 force-pushed the fix-55073-stuck branch from d61cb59 to d8d2b05 Compare September 12, 2025 10:32

tmp

434cf58

bobhan1 force-pushed the fix-55073-stuck branch from d8d2b05 to 434cf58 Compare September 12, 2025 10:37

morrySnow pushed a commit that referenced this pull request Sep 13, 2025

branch-3.1: [Opt](mow) Opt mow load performance and CPU usage #55073 #…

4c30059

…55733 #55771 (#55767) pick #55073 #55733 #55771

morrySnow added dev/3.1.1-merged and removed dev/3.1.x labels Sep 13, 2025

bobhan1 requested a review from dataroaring September 15, 2025 01:56

dataroaring approved these changes Sep 15, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 15, 2025

mymeiyi approved these changes Sep 15, 2025

View reviewed changes

dataroaring approved these changes Sep 15, 2025

View reviewed changes

dataroaring merged commit c31b1e9 into apache:master Sep 15, 2025
30 of 33 checks passed

morrySnow mentioned this pull request Sep 22, 2025

Release Note 3.1.1 #56289

Closed

[Fix](cloud-mow) Fix wrong tablet_id in delete bitmap #55771

[Fix](cloud-mow) Fix wrong tablet_id in delete bitmap #55771

Uh oh!

Conversation

bobhan1 commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Sep 8, 2025

Uh oh!

bobhan1 commented Sep 8, 2025

Uh oh!

doris-robot commented Sep 8, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Sep 8, 2025

Uh oh!

github-actions bot commented Sep 8, 2025

Uh oh!

bobhan1 commented Sep 12, 2025

Uh oh!

doris-robot commented Sep 12, 2025

Uh oh!

doris-robot commented Sep 12, 2025

Uh oh!

doris-robot commented Sep 12, 2025

Uh oh!

bobhan1 commented Sep 13, 2025

Uh oh!

bobhan1 commented Sep 13, 2025

Uh oh!

hello-stephen commented Sep 13, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Sep 13, 2025

BE Regression && UT Coverage Report

Uh oh!

bobhan1 commented Sep 14, 2025

Uh oh!

hello-stephen commented Sep 14, 2025

BE Regression && UT Coverage Report

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Sep 15, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

bobhan1 commented Sep 8, 2025 •

edited

Loading