[fix](meta) replace partition should not get table version in cloud mode #50888

zddr · 2025-05-14T04:41:58Z

What problem does this PR solve?

In cloud mode, the internal partition deletion logic will update the table version, so should not get version when replace

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
replace partition should not get table version in cloud mode

Release note

replace partition should not get table version in cloud mode

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-05-14T04:42:03Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

zddr · 2025-05-14T04:42:13Z

run buildall

doris-robot · 2025-05-14T05:15:27Z

TPC-H: Total hot run time: 33756 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e20356e61b065a83650da2d9b8db3cf8ede6ae5, data reload: false

------ Round 1 ----------------------------------
q1	26164	5064	5004	5004
q2	2069	272	183	183
q3	10520	1233	688	688
q4	10244	995	514	514
q5	8203	2444	2312	2312
q6	184	163	133	133
q7	920	739	617	617
q8	9324	1282	1112	1112
q9	6750	5019	5047	5019
q10	6874	2303	1910	1910
q11	492	292	268	268
q12	347	365	219	219
q13	17808	3640	3139	3139
q14	221	239	207	207
q15	532	485	497	485
q16	423	432	377	377
q17	581	853	360	360
q18	7638	7235	7086	7086
q19	1586	960	563	563
q20	346	315	210	210
q21	4033	3332	2391	2391
q22	1024	1016	959	959
Total cold run time: 116283 ms
Total hot run time: 33756 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5171	5044	5075	5044
q2	238	329	232	232
q3	2171	2679	2292	2292
q4	1341	1822	1385	1385
q5	4468	4443	4403	4403
q6	216	167	124	124
q7	2010	1973	1768	1768
q8	2586	2470	2467	2467
q9	7180	7139	7044	7044
q10	3055	3204	2748	2748
q11	583	507	489	489
q12	713	793	653	653
q13	3557	3966	3349	3349
q14	289	313	300	300
q15	546	492	496	492
q16	435	480	462	462
q17	1162	1547	1388	1388
q18	7691	7701	7436	7436
q19	840	812	856	812
q20	1987	1967	1821	1821
q21	5227	4787	4690	4690
q22	1107	1096	1003	1003
Total cold run time: 52573 ms
Total hot run time: 50402 ms

doris-robot · 2025-05-14T05:27:42Z

TPC-DS: Total hot run time: 193870 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7e20356e61b065a83650da2d9b8db3cf8ede6ae5, data reload: false

query1	1426	1082	1040	1040
query2	6192	1854	1894	1854
query3	10997	4494	4562	4494
query4	53595	25604	23207	23207
query5	5231	525	510	510
query6	376	219	201	201
query7	4973	529	295	295
query8	343	260	245	245
query9	6105	2695	2688	2688
query10	470	331	276	276
query11	15015	15090	15291	15090
query12	155	121	111	111
query13	1070	531	417	417
query14	10164	6416	6385	6385
query15	211	206	181	181
query16	7075	645	461	461
query17	1083	741	601	601
query18	1557	396	325	325
query19	200	191	207	191
query20	135	130	118	118
query21	215	133	114	114
query22	4377	4468	4207	4207
query23	34406	33503	33598	33503
query24	6641	2475	2492	2475
query25	488	479	416	416
query26	673	285	158	158
query27	2264	512	351	351
query28	3098	2162	2188	2162
query29	585	557	445	445
query30	281	214	191	191
query31	882	870	780	780
query32	77	62	64	62
query33	454	355	314	314
query34	778	878	558	558
query35	805	852	761	761
query36	959	1007	912	912
query37	121	104	79	79
query38	4312	4190	4204	4190
query39	1540	1458	1462	1458
query40	212	125	104	104
query41	61	56	52	52
query42	130	116	112	112
query43	526	525	509	509
query44	1372	839	834	834
query45	180	178	175	175
query46	883	1048	675	675
query47	1848	1885	1815	1815
query48	397	433	328	328
query49	690	525	470	470
query50	694	691	413	413
query51	4275	4262	4250	4250
query52	108	114	100	100
query53	232	273	198	198
query54	604	635	531	531
query55	92	89	88	88
query56	317	306	336	306
query57	1176	1166	1136	1136
query58	289	273	267	267
query59	2771	2834	2727	2727
query60	350	350	308	308
query61	129	130	144	130
query62	720	728	672	672
query63	273	199	195	195
query64	1457	1095	681	681
query65	4455	4286	4267	4267
query66	716	397	306	306
query67	15837	15776	15467	15467
query68	4841	901	525	525
query69	521	318	273	273
query70	1208	1078	1046	1046
query71	445	341	300	300
query72	5997	5009	5252	5009
query73	885	721	353	353
query74	9237	9225	8688	8688
query75	3491	3260	2774	2774
query76	3561	1208	777	777
query77	546	371	298	298
query78	10095	10329	9206	9206
query79	2642	846	601	601
query80	816	543	454	454
query81	508	253	218	218
query82	421	127	94	94
query83	269	251	237	237
query84	287	111	92	92
query85	787	349	312	312
query86	414	321	269	269
query87	4470	4476	4329	4329
query88	3548	2312	2327	2312
query89	412	314	290	290
query90	1834	215	224	215
query91	147	144	109	109
query92	70	68	60	60
query93	2760	930	570	570
query94	678	423	313	313
query95	361	304	286	286
query96	497	587	283	283
query97	3163	3248	3142	3142
query98	235	218	203	203
query99	1319	1389	1290	1290
Total cold run time: 296588 ms
Total hot run time: 193870 ms

doris-robot · 2025-05-14T05:32:53Z

ClickBench: Total hot run time: 28.88 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e20356e61b065a83650da2d9b8db3cf8ede6ae5, data reload: false

query1	0.04	0.04	0.04
query2	0.12	0.12	0.10
query3	0.25	0.20	0.19
query4	1.60	0.19	0.19
query5	0.58	0.58	0.60
query6	1.21	0.71	0.74
query7	0.03	0.02	0.01
query8	0.04	0.04	0.03
query9	0.59	0.51	0.52
query10	0.55	0.58	0.55
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.59	0.61
query14	0.78	0.81	0.82
query15	0.88	0.85	0.86
query16	0.38	0.38	0.40
query17	1.04	1.01	1.04
query18	0.22	0.21	0.21
query19	1.91	1.97	1.83
query20	0.02	0.01	0.01
query21	15.39	0.93	0.56
query22	0.75	1.22	0.67
query23	14.91	1.40	0.60
query24	7.11	2.34	0.40
query25	0.43	0.16	0.13
query26	0.60	0.17	0.14
query27	0.04	0.05	0.05
query28	9.85	0.86	0.43
query29	12.57	4.06	3.34
query30	0.26	0.09	0.07
query31	2.81	0.58	0.38
query32	3.23	0.56	0.47
query33	3.06	3.03	3.07
query34	15.78	5.07	4.46
query35	4.50	4.54	4.54
query36	0.67	0.50	0.47
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.18	0.13	0.12
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 103.59 s
Total hot run time: 28.88 s

fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java

…replace_partition_version

zddr · 2025-05-14T06:39:01Z

run buildall

doris-robot · 2025-05-14T07:10:43Z

TPC-H: Total hot run time: 33695 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b4c24adb114867a3cebac5114380e9cce360f7b4, data reload: false

------ Round 1 ----------------------------------
q1	25907	4978	4904	4904
q2	2067	297	180	180
q3	10395	1221	695	695
q4	10225	988	509	509
q5	7540	2304	2321	2304
q6	173	163	135	135
q7	919	753	606	606
q8	9320	1249	1084	1084
q9	6895	5115	5176	5115
q10	6860	2300	1878	1878
q11	479	279	259	259
q12	347	346	214	214
q13	17777	3688	3074	3074
q14	227	226	219	219
q15	540	488	482	482
q16	419	439	367	367
q17	578	833	356	356
q18	7667	7159	7196	7159
q19	1521	944	534	534
q20	334	343	227	227
q21	3955	3289	2408	2408
q22	1056	1019	986	986
Total cold run time: 115201 ms
Total hot run time: 33695 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5097	5006	4979	4979
q2	230	317	228	228
q3	2143	2661	2302	2302
q4	1333	1786	1353	1353
q5	4351	4390	4407	4390
q6	211	167	127	127
q7	2044	1924	1765	1765
q8	2576	2637	2504	2504
q9	7249	7307	7045	7045
q10	2978	3204	2773	2773
q11	583	509	489	489
q12	672	753	637	637
q13	3452	3815	3251	3251
q14	288	319	287	287
q15	527	484	470	470
q16	431	480	442	442
q17	1137	1509	1376	1376
q18	7712	7730	7443	7443
q19	794	830	1013	830
q20	2013	1998	1833	1833
q21	5080	4830	4656	4656
q22	1149	1052	1022	1022
Total cold run time: 52050 ms
Total hot run time: 50202 ms

doris-robot · 2025-05-14T07:23:03Z

TPC-DS: Total hot run time: 193116 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b4c24adb114867a3cebac5114380e9cce360f7b4, data reload: false

query1	1419	1076	1086	1076
query2	6151	1845	1857	1845
query3	11093	4604	4639	4604
query4	55453	25615	23084	23084
query5	5255	516	455	455
query6	415	218	197	197
query7	5399	493	283	283
query8	349	257	246	246
query9	7570	2670	2649	2649
query10	465	333	274	274
query11	15111	15077	14878	14878
query12	159	109	109	109
query13	1339	526	416	416
query14	9979	6218	7414	6218
query15	200	194	183	183
query16	7011	701	505	505
query17	1105	747	599	599
query18	1529	415	331	331
query19	219	200	187	187
query20	127	123	121	121
query21	208	122	107	107
query22	4375	4416	4381	4381
query23	34170	33570	33795	33570
query24	5593	2452	2431	2431
query25	464	495	415	415
query26	722	270	157	157
query27	1751	476	339	339
query28	2877	2129	2133	2129
query29	592	563	427	427
query30	275	222	198	198
query31	855	869	763	763
query32	76	64	60	60
query33	442	377	311	311
query34	775	838	533	533
query35	778	827	761	761
query36	948	985	875	875
query37	108	103	74	74
query38	4230	4350	4225	4225
query39	1522	1422	1499	1422
query40	222	120	121	120
query41	55	56	53	53
query42	128	118	111	111
query43	512	538	504	504
query44	1303	874	844	844
query45	179	175	172	172
query46	854	1037	659	659
query47	1861	1863	1780	1780
query48	404	441	334	334
query49	705	514	449	449
query50	638	706	406	406
query51	4211	4197	4211	4197
query52	112	107	100	100
query53	227	266	198	198
query54	597	594	531	531
query55	89	80	79	79
query56	324	297	303	297
query57	1147	1208	1124	1124
query58	276	268	260	260
query59	2797	2839	2703	2703
query60	339	331	311	311
query61	130	134	120	120
query62	704	749	659	659
query63	228	191	194	191
query64	1646	1089	714	714
query65	4363	4238	4191	4191
query66	719	411	314	314
query67	16004	15646	15270	15270
query68	6713	876	510	510
query69	534	306	268	268
query70	1239	1123	1099	1099
query71	502	315	305	305
query72	5788	4813	4814	4813
query73	1082	612	354	354
query74	8945	9182	8949	8949
query75	3827	3236	2715	2715
query76	4382	1192	756	756
query77	642	361	291	291
query78	10030	10058	9266	9266
query79	4906	805	576	576
query80	737	515	428	428
query81	488	252	221	221
query82	713	128	95	95
query83	371	252	234	234
query84	293	113	86	86
query85	784	349	320	320
query86	387	312	283	283
query87	4471	4504	4422	4422
query88	3239	2302	2255	2255
query89	471	304	285	285
query90	1918	213	214	213
query91	143	149	114	114
query92	70	63	56	56
query93	2823	905	564	564
query94	705	400	270	270
query95	364	304	289	289
query96	483	574	285	285
query97	3162	3244	3177	3177
query98	236	206	208	206
query99	1453	1480	1282	1282
Total cold run time: 303784 ms
Total hot run time: 193116 ms

doris-robot · 2025-05-14T07:28:13Z

ClickBench: Total hot run time: 28.87 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b4c24adb114867a3cebac5114380e9cce360f7b4, data reload: false

query1	0.03	0.03	0.03
query2	0.12	0.11	0.11
query3	0.26	0.19	0.20
query4	1.59	0.20	0.19
query5	0.62	0.58	0.60
query6	1.17	0.72	0.74
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.57	0.51	0.51
query10	0.56	0.57	0.57
query11	0.16	0.10	0.10
query12	0.15	0.12	0.12
query13	0.62	0.60	0.60
query14	0.82	0.79	0.80
query15	0.87	0.86	0.88
query16	0.40	0.39	0.37
query17	1.04	1.07	1.04
query18	0.24	0.22	0.22
query19	1.98	1.86	1.84
query20	0.01	0.01	0.01
query21	15.39	0.91	0.55
query22	0.75	1.11	0.64
query23	15.06	1.39	0.57
query24	6.93	2.01	0.36
query25	0.31	0.31	0.07
query26	0.62	0.15	0.13
query27	0.05	0.05	0.04
query28	8.83	0.89	0.45
query29	12.53	4.08	3.36
query30	0.25	0.09	0.06
query31	2.81	0.59	0.38
query32	3.23	0.56	0.47
query33	3.04	3.13	3.05
query34	15.84	5.15	4.53
query35	4.55	4.56	4.50
query36	0.67	0.50	0.49
query37	0.09	0.07	0.06
query38	0.04	0.04	0.04
query39	0.04	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.03
query43	0.03	0.04	0.03
Total cold run time: 102.61 s
Total hot run time: 28.87 s

deardeng

LGTM

github-actions · 2025-05-15T03:17:38Z

PR approved by anyone and no changes requested.

dataroaring

LGTM

github-actions · 2025-05-16T11:29:47Z

PR approved by at least one committer and no changes requested.

…ode (#50888) ### What problem does this PR solve? In cloud mode, the internal partition deletion logic will update the table version, so should not get version when replace

…n in cloud mode #50888 (#51027) Cherry-picked from #50888 Co-authored-by: zhangdong <zhangdong@selectdb.com>

…ode (apache#50888) ### What problem does this PR solve? In cloud mode, the internal partition deletion logic will update the table version, so should not get version when replace

zddr added 3 commits May 14, 2025 12:31

1

bec2933

1

b74ee25

1

7e20356

zddr requested review from CalvinKirs, dataroaring and morningman as code owners May 14, 2025 04:41

morrySnow reviewed May 14, 2025

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java Show resolved Hide resolved

morrySnow added the dev/3.0.x label May 14, 2025

zddr added 2 commits May 14, 2025 14:38

add comment

80e6f59

Merge remote-tracking branch 'origin/replace_partition_version' into …

b4c24ad

…replace_partition_version

zddr requested a review from morrySnow May 14, 2025 06:39

deardeng approved these changes May 15, 2025

View reviewed changes

github-actions bot added the reviewed label May 15, 2025

dataroaring approved these changes May 16, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 16, 2025

morrySnow merged commit 84d9ab0 into apache:master May 19, 2025
26 checks passed

morrySnow changed the title ~~[fix](mtmv)replace partition should not get table version in cloud mode~~ [fix](meta) replace partition should not get table version in cloud mode May 19, 2025

github-actions bot mentioned this pull request May 19, 2025

branch-3.0: [fix](meta) replace partition should not get table version in cloud mode #50888 #51027

Merged

dataroaring pushed a commit that referenced this pull request May 20, 2025

branch-3.0: [fix](meta) replace partition should not get table versio…

59b6c9e

…n in cloud mode #50888 (#51027) Cherry-picked from #50888 Co-authored-by: zhangdong <zhangdong@selectdb.com>

dataroaring added dev/3.0.6-merged and removed dev/3.0.x labels May 20, 2025

[fix](meta) replace partition should not get table version in cloud mode #50888

[fix](meta) replace partition should not get table version in cloud mode #50888

Uh oh!

Conversation

zddr commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented May 14, 2025

Uh oh!

zddr commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

Uh oh!

zddr commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

deardeng left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented May 15, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented May 16, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

zddr commented May 14, 2025 •

edited

Loading