[enhance](mtmv)Only restrict MTMV to not allow concurrent insert overwrite execution #48673

zddr · 2025-03-05T05:07:30Z

What problem does this PR solve?

Previously, on #40558, concurrent execution of INSERT OVERWRITE was restricted for all OLAP tables to address potential issues caused by concurrent execution.

In this current PR, the restriction has been modified to only apply to Materialized Views (MTMV), as it appears that users can tolerate the uncertainty in execution results caused by concurrency for OLAP tables.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
Only restrict MTMV to not allow concurrent insert overwrite execution

Release note

Only restrict MTMV to not allow concurrent insert overwrite execution

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-03-05T05:07:34Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

zddr · 2025-03-05T05:08:08Z

run buildall

doris-robot · 2025-03-05T05:28:19Z

TPC-H: Total hot run time: 32734 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 34d4120f4028f62c0a5a087c8ba6b40e554264e8, data reload: false

------ Round 1 ----------------------------------
q1	17613	5194	5021	5021
q2	2049	306	169	169
q3	10403	1341	711	711
q4	10216	1017	562	562
q5	7518	2375	2377	2375
q6	187	169	134	134
q7	919	747	597	597
q8	9317	1243	1038	1038
q9	4962	4898	4888	4888
q10	6809	2335	1918	1918
q11	486	273	272	272
q12	354	360	222	222
q13	17763	3691	3069	3069
q14	218	224	207	207
q15	540	515	475	475
q16	650	606	586	586
q17	560	858	346	346
q18	6919	6448	6469	6448
q19	1214	952	545	545
q20	312	314	189	189
q21	2740	2133	1965	1965
q22	1092	1003	997	997
Total cold run time: 102841 ms
Total hot run time: 32734 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5088	5063	5055	5055
q2	234	324	230	230
q3	2166	2679	2293	2293
q4	1432	1808	1356	1356
q5	4250	4120	4172	4120
q6	205	159	122	122
q7	1883	1915	1814	1814
q8	2565	2652	2616	2616
q9	7214	7180	7227	7180
q10	2999	3214	2840	2840
q11	605	523	494	494
q12	685	787	638	638
q13	3513	3842	3769	3769
q14	272	284	271	271
q15	512	484	482	482
q16	648	686	651	651
q17	1129	1590	1332	1332
q18	7841	7641	7547	7547
q19	793	823	866	823
q20	1938	2061	1845	1845
q21	5402	5002	4865	4865
q22	1098	1038	1021	1021
Total cold run time: 52472 ms
Total hot run time: 51364 ms

doris-robot · 2025-03-05T05:39:48Z

TPC-DS: Total hot run time: 184525 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 34d4120f4028f62c0a5a087c8ba6b40e554264e8, data reload: false

query1	979	381	386	381
query2	6553	1985	1882	1882
query3	6805	211	216	211
query4	26019	23797	23313	23313
query5	4324	677	533	533
query6	284	188	189	188
query7	4596	491	287	287
query8	283	246	218	218
query9	8610	2517	2533	2517
query10	454	304	254	254
query11	15241	15355	14926	14926
query12	165	109	105	105
query13	1685	531	408	408
query14	8735	6283	6274	6274
query15	227	185	170	170
query16	7259	627	479	479
query17	1206	677	537	537
query18	1925	397	292	292
query19	191	175	151	151
query20	124	115	112	112
query21	208	118	104	104
query22	4503	4572	4497	4497
query23	33996	32968	33114	32968
query24	8236	2364	2424	2364
query25	523	459	397	397
query26	1229	270	154	154
query27	2762	502	327	327
query28	4357	2445	2400	2400
query29	746	577	418	418
query30	288	223	188	188
query31	930	836	743	743
query32	71	65	64	64
query33	550	370	310	310
query34	786	862	498	498
query35	802	816	737	737
query36	941	975	886	886
query37	123	95	71	71
query38	4174	4202	4000	4000
query39	1440	1434	1429	1429
query40	213	111	100	100
query41	54	50	56	50
query42	130	109	107	107
query43	502	497	500	497
query44	1308	781	777	777
query45	179	171	164	164
query46	874	1047	614	614
query47	1780	1856	1729	1729
query48	369	398	293	293
query49	789	492	446	446
query50	730	742	401	401
query51	4131	4178	4142	4142
query52	112	109	94	94
query53	230	262	183	183
query54	492	473	408	408
query55	83	78	81	78
query56	268	273	250	250
query57	1152	1143	1083	1083
query58	238	242	266	242
query59	2698	2693	2583	2583
query60	275	264	258	258
query61	124	122	116	116
query62	821	730	678	678
query63	241	195	197	195
query64	4365	1030	667	667
query65	4376	4271	4308	4271
query66	1126	406	301	301
query67	15712	15558	15332	15332
query68	7857	895	509	509
query69	512	296	266	266
query70	1237	1126	1095	1095
query71	397	292	271	271
query72	5280	2707	3732	2707
query73	763	792	351	351
query74	9143	9068	8736	8736
query75	3210	3162	2695	2695
query76	3275	1199	757	757
query77	522	386	281	281
query78	9952	10122	9338	9338
query79	1407	865	585	585
query80	618	527	473	473
query81	498	267	225	225
query82	202	134	95	95
query83	165	181	159	159
query84	245	103	79	79
query85	747	362	310	310
query86	370	318	282	282
query87	4560	4509	4573	4509
query88	2808	2180	2171	2171
query89	399	308	282	282
query90	2000	194	189	189
query91	134	141	108	108
query92	73	63	56	56
query93	1718	1069	575	575
query94	665	431	296	296
query95	354	264	247	247
query96	484	588	267	267
query97	3303	3352	3303	3303
query98	225	209	202	202
query99	1305	1388	1254	1254
Total cold run time: 270130 ms
Total hot run time: 184525 ms

doris-robot · 2025-03-05T05:45:05Z

ClickBench: Total hot run time: 30.35 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 34d4120f4028f62c0a5a087c8ba6b40e554264e8, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.04
query3	0.24	0.07	0.07
query4	1.61	0.10	0.10
query5	0.58	0.54	0.56
query6	1.21	0.72	0.73
query7	0.03	0.01	0.02
query8	0.04	0.04	0.04
query9	0.57	0.55	0.50
query10	0.56	0.61	0.58
query11	0.16	0.10	0.11
query12	0.14	0.11	0.11
query13	0.61	0.61	0.61
query14	2.78	2.79	2.81
query15	0.92	0.84	0.84
query16	0.37	0.37	0.36
query17	1.00	1.07	1.04
query18	0.21	0.20	0.19
query19	1.85	1.82	1.95
query20	0.01	0.01	0.01
query21	15.36	0.85	0.57
query22	0.75	1.20	0.59
query23	15.04	1.34	0.61
query24	6.94	2.11	0.41
query25	0.38	0.36	0.13
query26	0.56	0.17	0.14
query27	0.05	0.05	0.04
query28	9.01	0.88	0.42
query29	12.55	3.90	3.27
query30	0.24	0.09	0.06
query31	2.83	0.60	0.38
query32	3.23	0.55	0.46
query33	2.95	3.01	2.99
query34	15.82	5.17	4.47
query35	4.59	4.57	4.52
query36	0.67	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 104.44 s
Total hot run time: 30.35 s

zddr · 2025-03-05T10:01:47Z

run feut

github-actions · 2025-03-10T07:09:40Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-03-10T07:09:43Z

PR approved by anyone and no changes requested.

morrySnow · 2025-03-27T09:57:02Z

run buildall

zclllyybb

please take revert of https://github.com/apache/doris/pull/41125/files together. others LGTM

doris-robot · 2025-03-27T10:28:38Z

TPC-H: Total hot run time: 34545 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 142ffc36ba677d0287c4eb48928e16e023279bed, data reload: false

------ Round 1 ----------------------------------
q1	26164	5143	5062	5062
q2	2073	285	173	173
q3	10403	1262	698	698
q4	10214	994	548	548
q5	7520	2463	2460	2460
q6	199	161	131	131
q7	923	740	616	616
q8	9308	1332	1209	1209
q9	6900	5267	5121	5121
q10	6828	2346	1885	1885
q11	506	278	269	269
q12	352	369	225	225
q13	17769	3707	3133	3133
q14	235	230	213	213
q15	540	506	503	503
q16	631	629	582	582
q17	586	864	351	351
q18	7826	7173	7259	7173
q19	1213	945	594	594
q20	337	343	199	199
q21	4223	2704	2411	2411
q22	1101	1054	989	989
Total cold run time: 115851 ms
Total hot run time: 34545 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5126	5109	5188	5109
q2	244	331	235	235
q3	2199	2726	2307	2307
q4	1425	1813	1532	1532
q5	4600	4439	4379	4379
q6	214	167	126	126
q7	2044	1954	1767	1767
q8	2665	2554	2520	2520
q9	7307	7173	7107	7107
q10	2992	3189	2791	2791
q11	575	536	487	487
q12	698	780	645	645
q13	3588	4009	3338	3338
q14	282	300	269	269
q15	539	508	485	485
q16	670	693	651	651
q17	1158	1605	1372	1372
q18	7817	7663	7359	7359
q19	889	919	1044	919
q20	1978	2055	1864	1864
q21	5307	4956	4847	4847
q22	1135	1065	1034	1034
Total cold run time: 53452 ms
Total hot run time: 51143 ms

zddr · 2025-03-27T10:29:21Z

run buildall

doris-robot · 2025-03-27T10:49:04Z

TPC-H: Total hot run time: 34207 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fa31c0641191b3f5483baa2870c72b914163a52f, data reload: false

------ Round 1 ----------------------------------
q1	26185	5225	5063	5063
q2	2077	286	167	167
q3	10403	1247	703	703
q4	10231	1023	538	538
q5	7504	2407	2349	2349
q6	183	163	133	133
q7	920	752	610	610
q8	9311	1285	1086	1086
q9	6898	5233	5102	5102
q10	6809	2323	1920	1920
q11	480	287	258	258
q12	346	362	228	228
q13	17777	3746	3052	3052
q14	224	244	209	209
q15	555	495	510	495
q16	645	620	596	596
q17	581	875	354	354
q18	7677	7297	7202	7202
q19	1227	972	570	570
q20	326	354	200	200
q21	4155	2659	2380	2380
q22	1092	1046	992	992
Total cold run time: 115606 ms
Total hot run time: 34207 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5183	5150	5160	5150
q2	246	328	232	232
q3	2188	2702	2292	2292
q4	1405	1802	1367	1367
q5	4301	4170	4177	4170
q6	202	164	127	127
q7	1902	1826	1647	1647
q8	2510	2479	2422	2422
q9	6837	6877	6889	6877
q10	2891	3098	2673	2673
q11	565	491	476	476
q12	642	729	579	579
q13	3396	3736	3105	3105
q14	266	282	263	263
q15	528	494	480	480
q16	650	673	633	633
q17	1107	1466	1373	1373
q18	7315	7278	7214	7214
q19	763	835	970	835
q20	1901	1975	1817	1817
q21	5207	4700	4639	4639
q22	1103	1043	976	976
Total cold run time: 51108 ms
Total hot run time: 49347 ms

doris-robot · 2025-03-27T11:00:44Z

TPC-DS: Total hot run time: 187775 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fa31c0641191b3f5483baa2870c72b914163a52f, data reload: false

query1	995	514	469	469
query2	6550	1990	1960	1960
query3	6623	223	215	215
query4	26504	23850	23406	23406
query5	5362	710	494	494
query6	280	194	186	186
query7	4595	514	280	280
query8	308	254	256	254
query9	8401	2554	2564	2554
query10	500	309	277	277
query11	15560	15222	14916	14916
query12	173	116	112	112
query13	1623	546	411	411
query14	9857	6388	6520	6388
query15	210	193	182	182
query16	7645	668	505	505
query17	1406	721	578	578
query18	2027	411	316	316
query19	196	199	178	178
query20	128	126	129	126
query21	218	125	107	107
query22	4105	4307	4132	4132
query23	34410	33061	33146	33061
query24	7816	2424	2425	2424
query25	508	479	401	401
query26	1229	264	163	163
query27	2634	505	332	332
query28	4304	2422	2401	2401
query29	737	579	439	439
query30	284	217	197	197
query31	946	853	807	807
query32	74	71	63	63
query33	568	374	318	318
query34	814	841	504	504
query35	795	836	770	770
query36	972	1062	911	911
query37	132	102	78	78
query38	4133	4064	4239	4064
query39	1485	1412	1406	1406
query40	222	130	108	108
query41	56	54	55	54
query42	122	108	104	104
query43	514	524	496	496
query44	1360	820	813	813
query45	178	179	169	169
query46	911	1054	654	654
query47	1757	1782	1743	1743
query48	375	419	302	302
query49	763	502	438	438
query50	746	786	437	437
query51	4199	4231	4178	4178
query52	111	111	105	105
query53	228	261	178	178
query54	498	494	415	415
query55	85	85	79	79
query56	276	264	283	264
query57	1136	1137	1092	1092
query58	249	258	246	246
query59	2755	2863	2711	2711
query60	286	289	256	256
query61	128	129	164	129
query62	786	732	687	687
query63	230	186	194	186
query64	4176	1006	659	659
query65	4398	4314	4281	4281
query66	1050	416	309	309
query67	16116	15665	15365	15365
query68	8543	904	510	510
query69	500	306	277	277
query70	1271	1126	1136	1126
query71	496	314	279	279
query72	5526	4731	4651	4651
query73	717	605	346	346
query74	9266	9148	8929	8929
query75	4172	3242	2789	2789
query76	3730	1222	797	797
query77	817	402	303	303
query78	10082	10311	9374	9374
query79	1249	900	571	571
query80	732	555	452	452
query81	465	257	222	222
query82	446	142	101	101
query83	190	185	161	161
query84	258	97	78	78
query85	766	356	309	309
query86	331	313	312	312
query87	4593	4613	4304	4304
query88	2953	2255	2271	2255
query89	401	316	292	292
query90	1973	219	223	219
query91	143	154	116	116
query92	142	60	60	60
query93	1196	1049	607	607
query94	687	419	314	314
query95	352	275	268	268
query96	489	577	274	274
query97	3224	3297	3134	3134
query98	217	216	208	208
query99	1482	1428	1294	1294
Total cold run time: 276227 ms
Total hot run time: 187775 ms

doris-robot · 2025-03-27T11:06:06Z

ClickBench: Total hot run time: 31.84 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fa31c0641191b3f5483baa2870c72b914163a52f, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.10	0.10
query3	0.24	0.20	0.19
query4	1.59	0.19	0.19
query5	0.58	0.55	0.56
query6	1.22	0.71	0.73
query7	0.02	0.02	0.02
query8	0.05	0.04	0.03
query9	0.57	0.53	0.51
query10	0.58	0.60	0.57
query11	0.16	0.10	0.11
query12	0.15	0.11	0.11
query13	0.62	0.60	0.61
query14	2.68	2.70	2.69
query15	0.93	0.85	0.84
query16	0.40	0.40	0.39
query17	1.03	1.01	1.03
query18	0.21	0.20	0.20
query19	2.07	1.89	1.89
query20	0.01	0.01	0.01
query21	15.36	0.94	0.56
query22	0.77	1.18	0.73
query23	14.85	1.37	0.63
query24	6.88	1.37	1.25
query25	0.49	0.16	0.08
query26	0.58	0.15	0.13
query27	0.05	0.04	0.04
query28	9.78	0.88	0.44
query29	12.55	4.03	3.39
query30	0.25	0.11	0.06
query31	2.80	0.57	0.38
query32	3.25	0.55	0.49
query33	3.07	3.05	3.09
query34	15.79	5.21	4.52
query35	4.56	4.55	4.59
query36	0.66	0.50	0.49
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 105.44 s
Total hot run time: 31.84 s

zclllyybb

LGTM

zddr · 2025-04-03T12:17:58Z

run buildall

doris-robot · 2025-04-03T12:44:53Z

TPC-H: Total hot run time: 34929 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ec83049b7a5ec554c111e0a05c04cba64e15fa91, data reload: false

------ Round 1 ----------------------------------
q1	25928	5166	5094	5094
q2	2069	310	194	194
q3	10739	1242	720	720
q4	10256	1000	511	511
q5	8405	2416	2339	2339
q6	187	161	130	130
q7	914	749	601	601
q8	9310	1335	1157	1157
q9	6717	5052	5052	5052
q10	6826	2323	1898	1898
q11	491	289	263	263
q12	357	358	228	228
q13	17768	3649	3128	3128
q14	232	235	208	208
q15	539	491	492	491
q16	636	603	581	581
q17	599	868	348	348
q18	7669	7216	7059	7059
q19	1407	968	584	584
q20	333	345	224	224
q21	4107	3554	3173	3173
q22	1044	1026	946	946
Total cold run time: 116533 ms
Total hot run time: 34929 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5158	5103	5063	5063
q2	245	329	231	231
q3	2180	2682	2330	2330
q4	1444	1929	1478	1478
q5	4525	4472	4479	4472
q6	211	163	123	123
q7	1898	1934	1798	1798
q8	2631	2591	2569	2569
q9	7138	7072	7146	7072
q10	3014	3167	2764	2764
q11	572	499	491	491
q12	733	753	613	613
q13	3531	3864	3304	3304
q14	295	286	270	270
q15	536	477	479	477
q16	648	678	645	645
q17	1159	1547	1390	1390
q18	7711	7415	7337	7337
q19	839	841	952	841
q20	1973	2007	1886	1886
q21	5432	4968	4928	4928
q22	1129	1044	963	963
Total cold run time: 53002 ms
Total hot run time: 51045 ms

doris-robot · 2025-04-03T12:56:14Z

TPC-DS: Total hot run time: 168308 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ec83049b7a5ec554c111e0a05c04cba64e15fa91, data reload: false

============================================
query1	839	459	448	448
query2	6260	1942	1948	1942
query3	11034	4423	4303	4303
query4	query5	5990	487	487	487
query6	428	205	187	187
query7	5267	277	281	277
query8	344	229	227	227
query9	8728	2584	2585	2584
query10	504	319	251	251
query11	15294	15173	14792	14792
query12	167	116	104	104
query13	1262	536	392	392
query14	10149	6047	6058	6047
query15	205	190	173	173
query16	7123	635	480	480
query17	1073	703	563	563
query18	1574	392	301	301
query19	319	181	158	158
query20	128	123	126	123
query21	218	124	101	101
query22	4147	4212	4281	4212
query23	33957	33357	33322	33322
query24	6774	2365	2401	2365
query25	504	474	400	400
query26	699	287	156	156
query27	3016	510	337	337
query28	3714	2446	2472	2446
query29	605	580	481	481
query30	277	234	213	213
query31	926	878	753	753
query32	73	68	66	66
query33	473	383	365	365
query34	771	877	521	521
query35	807	862	783	783
query36	967	1030	923	923
query37	121	107	81	81
query38	4288	4239	4177	4177
query39	1481	1498	1554	1498
query40	215	128	118	118
query41	61	68	69	68
query42	127	114	113	113
query43	515	530	493	493
query44	1361	838	829	829
query45	186	183	170	170
query46	844	1036	655	655
query47	1865	1906	1783	1783
query48	397	448	330	330
query49	756	582	417	417
query50	665	689	402	402
query51	4284	4207	4254	4207
query52	109	104	100	100
query53	219	262	187	187
query54	576	591	537	537
query55	86	85	78	78
query56	315	308	294	294
query57	1204	1198	1170	1170
query58	277	259	272	259
query59	2775	2856	2608	2608
query60	328	334	309	309
query61	148	126	125	125
query62	714	722	690	690
query63	232	183	176	176
query64	1870	1032	685	685
query65	4311	4284	4210	4210
query66	718	408	306	306
query67	15769	15614	15266	15266
query68	7217	885	513	513
query69	585	302	262	262
query70	1129	1126	1103	1103
query71	480	309	290	290
query72	5727	4752	4816	4752
query73	1153	645	347	347
query74	8966	9245	8683	8683
query75	4002	3216	2672	2672
query76	5525	1182	768	768
query77	703	380	288	288
query78	10075	10182	9275	9275
query79	1821	816	556	556
query80	736	517	428	428
query81	504	261	236	236
query82	706	127	97	97
query83	349	249	233	233
query84	297	105	86	86
query85	786	441	322	322
query86	366	312	293	293
query87	4542	4486	4503	4486
query88	3095	2223	2192	2192
query89	401	309	294	294
query90	1842	214	210	210
query91	139	139	112	112
query92	74	61	61	61
query93	2781	948	573	573
query94	656	415	307	307
query95	366	292	335	292
query96	479	563	270	270
query97	3210	3180	3151	3151
query98	223	207	199	199
query99	1454	1415	1333	1333
Total cold run time: 251822 ms
Total hot run time: 168308 ms

doris-robot · 2025-04-03T13:01:41Z

ClickBench: Total hot run time: 31.22 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ec83049b7a5ec554c111e0a05c04cba64e15fa91, data reload: false

query1	0.04	0.04	0.03
query2	0.11	0.10	0.11
query3	0.24	0.19	0.19
query4	1.59	0.20	0.20
query5	0.60	0.60	0.61
query6	1.17	0.72	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.51	0.54
query10	0.58	0.59	0.56
query11	0.15	0.11	0.11
query12	0.15	0.11	0.11
query13	0.61	0.60	0.60
query14	2.72	2.70	2.71
query15	0.93	0.85	0.85
query16	0.38	0.40	0.40
query17	1.03	1.01	1.01
query18	0.20	0.19	0.20
query19	1.82	1.92	1.84
query20	0.01	0.02	0.01
query21	15.34	0.94	0.55
query22	0.76	1.18	0.63
query23	15.18	1.36	0.57
query24	6.86	1.52	0.94
query25	0.51	0.20	0.13
query26	0.66	0.16	0.15
query27	0.04	0.05	0.05
query28	9.44	0.92	0.42
query29	12.53	3.94	3.29
query30	0.26	0.09	0.08
query31	2.81	0.59	0.39
query32	3.25	0.55	0.46
query33	3.09	3.06	3.14
query34	16.07	5.14	4.47
query35	4.52	4.52	4.47
query36	0.67	0.49	0.47
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.02
query40	0.18	0.14	0.12
query41	0.09	0.03	0.03
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.47 s
Total hot run time: 31.22 s

zddr · 2025-04-07T08:23:52Z

run buildall

doris-robot · 2025-04-07T09:02:37Z

TPC-H: Total hot run time: 34093 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ec83049b7a5ec554c111e0a05c04cba64e15fa91, data reload: false

------ Round 1 ----------------------------------
q1	26219	5004	4989	4989
q2	2054	271	197	197
q3	10393	1220	682	682
q4	10244	1022	533	533
q5	7533	2369	2361	2361
q6	188	165	135	135
q7	919	765	614	614
q8	9327	1263	1120	1120
q9	6842	5070	5139	5070
q10	6804	2281	1899	1899
q11	476	282	261	261
q12	347	369	224	224
q13	17797	3739	3098	3098
q14	228	228	210	210
q15	534	483	477	477
q16	618	630	579	579
q17	586	832	352	352
q18	7513	7227	7185	7185
q19	1843	947	541	541
q20	324	328	213	213
q21	3823	2652	2375	2375
q22	1022	1016	978	978
Total cold run time: 115634 ms
Total hot run time: 34093 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5139	5083	5077	5077
q2	242	332	227	227
q3	2150	2629	2267	2267
q4	1444	1766	1385	1385
q5	4469	4341	4348	4341
q6	210	174	129	129
q7	1982	1930	1756	1756
q8	2578	2410	2458	2410
q9	7262	7226	7133	7133
q10	2998	3193	2757	2757
q11	568	508	476	476
q12	680	761	595	595
q13	3519	3907	3339	3339
q14	315	304	280	280
q15	511	483	465	465
q16	638	684	657	657
q17	1149	1527	1373	1373
q18	7695	7605	7444	7444
q19	801	801	793	793
q20	1912	1985	1810	1810
q21	5299	4858	4785	4785
q22	1087	1045	1017	1017
Total cold run time: 52648 ms
Total hot run time: 50516 ms

doris-robot · 2025-04-07T09:14:55Z

TPC-DS: Total hot run time: 193516 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ec83049b7a5ec554c111e0a05c04cba64e15fa91, data reload: false

query1	1398	1083	1053	1053
query2	6187	1895	1876	1876
query3	11032	4561	4473	4473
query4	53666	24881	23287	23287
query5	5262	506	496	496
query6	365	200	195	195
query7	5007	491	298	298
query8	313	229	228	228
query9	6155	2569	2592	2569
query10	443	338	276	276
query11	15131	15006	14928	14928
query12	167	114	107	107
query13	1109	517	401	401
query14	10244	6276	6390	6276
query15	217	211	190	190
query16	7010	673	501	501
query17	1113	763	604	604
query18	1556	421	325	325
query19	205	202	174	174
query20	139	122	119	119
query21	214	130	120	120
query22	4581	4438	4469	4438
query23	33901	33628	33223	33223
query24	6615	2446	2463	2446
query25	509	457	400	400
query26	737	281	154	154
query27	2500	506	336	336
query28	3384	2496	2451	2451
query29	579	574	461	461
query30	270	250	187	187
query31	855	853	790	790
query32	76	67	62	62
query33	462	365	299	299
query34	775	847	531	531
query35	811	835	766	766
query36	956	1018	918	918
query37	143	103	72	72
query38	4135	4418	4191	4191
query39	1525	1440	1433	1433
query40	220	123	113	113
query41	54	56	51	51
query42	119	103	108	103
query43	502	519	472	472
query44	1303	816	823	816
query45	178	178	172	172
query46	851	1036	641	641
query47	1841	1890	1802	1802
query48	395	425	305	305
query49	710	511	425	425
query50	649	689	408	408
query51	4256	4243	4237	4237
query52	110	107	101	101
query53	226	261	189	189
query54	588	574	507	507
query55	89	82	90	82
query56	302	320	288	288
query57	1161	1180	1150	1150
query58	288	270	254	254
query59	2678	2807	2833	2807
query60	357	360	312	312
query61	138	133	132	132
query62	755	734	692	692
query63	224	195	192	192
query64	1841	1075	718	718
query65	4384	4355	4300	4300
query66	727	406	299	299
query67	15868	15720	15505	15505
query68	7363	875	517	517
query69	539	300	261	261
query70	1207	1164	1123	1123
query71	511	309	298	298
query72	5933	4916	4901	4901
query73	1325	685	348	348
query74	8901	9165	8753	8753
query75	3768	3226	2731	2731
query76	4215	1199	747	747
query77	684	373	285	285
query78	10065	10111	9311	9311
query79	3475	800	554	554
query80	638	501	428	428
query81	494	258	214	214
query82	487	132	93	93
query83	397	257	234	234
query84	301	114	86	86
query85	805	354	314	314
query86	416	294	289	289
query87	4487	4433	4365	4365
query88	3694	2251	2244	2244
query89	446	309	279	279
query90	1816	217	217	217
query91	142	154	113	113
query92	68	65	53	53
query93	2728	935	580	580
query94	664	401	320	320
query95	375	299	297	297
query96	498	576	281	281
query97	3191	3210	3111	3111
query98	234	221	197	197
query99	1470	1392	1245	1245
Total cold run time: 301898 ms
Total hot run time: 193516 ms

doris-robot · 2025-04-07T09:20:19Z

ClickBench: Total hot run time: 30.52 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ec83049b7a5ec554c111e0a05c04cba64e15fa91, data reload: false

query1	0.03	0.03	0.04
query2	0.13	0.11	0.11
query3	0.26	0.20	0.20
query4	1.59	0.19	0.20
query5	0.61	0.58	0.59
query6	1.19	0.73	0.72
query7	0.03	0.02	0.02
query8	0.04	0.04	0.04
query9	0.58	0.52	0.52
query10	0.56	0.60	0.55
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.61	0.61	0.60
query14	2.65	2.82	2.70
query15	0.93	0.85	0.86
query16	0.41	0.38	0.37
query17	1.01	1.04	1.03
query18	0.21	0.19	0.20
query19	1.91	2.01	1.76
query20	0.01	0.01	0.01
query21	15.37	0.90	0.53
query22	0.76	1.04	0.56
query23	15.15	1.36	0.65
query24	7.94	0.83	0.37
query25	0.53	0.24	0.09
query26	0.63	0.16	0.14
query27	0.05	0.04	0.04
query28	8.68	0.82	0.43
query29	12.64	3.92	3.30
query30	0.25	0.09	0.06
query31	2.83	0.58	0.39
query32	3.23	0.54	0.46
query33	3.02	3.13	3.02
query34	15.71	5.07	4.48
query35	4.53	4.50	4.53
query36	0.67	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 105.53 s
Total hot run time: 30.52 s

zddr · 2025-04-08T02:21:54Z

run cloud_p0

github-actions · 2025-04-09T12:04:29Z

PR approved by at least one committer and no changes requested.

…write execution (#48673) ### What problem does this PR solve? Related PR: #40558 Problem Summary: Previously, on #40558, concurrent execution of `INSERT OVERWRITE` was restricted for all OLAP tables to address potential issues caused by concurrent execution. In this current PR, the restriction has been modified to only apply to Materialized Views (MTMV), as it appears that users can tolerate the uncertainty in execution results caused by concurrency for OLAP tables. ### Release note Only restrict MTMV to not allow concurrent insert overwrite execution

… insert overwrite execution #48673 (#49964) Cherry-picked from #48673 Co-authored-by: zhangdong <zhangdong@selectdb.com>

… insert overwrite execution #48673 (#49965) Cherry-picked from #48673 Co-authored-by: zhangdong <zhangdong@selectdb.com>

…write execution (apache#48673) ### What problem does this PR solve? Related PR: apache#40558 Problem Summary: Previously, on apache#40558, concurrent execution of `INSERT OVERWRITE` was restricted for all OLAP tables to address potential issues caused by concurrent execution. In this current PR, the restriction has been modified to only apply to Materialized Views (MTMV), as it appears that users can tolerate the uncertainty in execution results caused by concurrency for OLAP tables. ### Release note Only restrict MTMV to not allow concurrent insert overwrite execution

…write apache#48673 (apache#3896) pick: apache#48673

…write execution (apache#48673) (apache#3898) pick: apache#48673

1

34d4120

morrySnow added usercase Important user case type label dev/2.1.x dev/3.0.x labels Mar 10, 2025

morrySnow previously approved these changes Mar 10, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 10, 2025

github-actions bot added the reviewed label Mar 10, 2025

1

142ffc3

morrySnow force-pushed the iot_only_mtmv branch from 34d4120 to 142ffc3 Compare March 27, 2025 09:56

morrySnow added the kind/behavior-changed label Mar 27, 2025

zclllyybb reviewed Mar 27, 2025

View reviewed changes

zddr added 2 commits March 27, 2025 18:26

1

d222a58

Merge remote-tracking branch 'origin/iot_only_mtmv' into iot_only_mtmv

fa31c06

zddr dismissed morrySnow’s stale review via fa31c06 March 27, 2025 10:28

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Mar 27, 2025

zclllyybb approved these changes Mar 27, 2025

View reviewed changes

zddr dismissed morrySnow’s stale review via ec83049 April 3, 2025 12:17

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Apr 3, 2025

morrySnow approved these changes Apr 9, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 9, 2025

morrySnow merged commit 9a9b1f7 into apache:master Apr 10, 2025
24 of 25 checks passed

github-actions bot mentioned this pull request Apr 10, 2025

branch-3.0: [enhance](mtmv)Only restrict MTMV to not allow concurrent insert overwrite execution #48673 #49964

Merged

github-actions bot mentioned this pull request Apr 10, 2025

branch-2.1: [enhance](mtmv)Only restrict MTMV to not allow concurrent insert overwrite execution #48673 #49965

Merged

dataroaring pushed a commit that referenced this pull request Apr 25, 2025

branch-3.0: [enhance](mtmv)Only restrict MTMV to not allow concurrent…

c5c1891

… insert overwrite execution #48673 (#49964) Cherry-picked from #48673 Co-authored-by: zhangdong <zhangdong@selectdb.com>

dataroaring added dev/3.0.6-merged and removed dev/3.0.x labels Apr 25, 2025

yiguolei pushed a commit that referenced this pull request May 8, 2025

branch-2.1: [enhance](mtmv)Only restrict MTMV to not allow concurrent…

523681d

… insert overwrite execution #48673 (#49965) Cherry-picked from #48673 Co-authored-by: zhangdong <zhangdong@selectdb.com>

yiguolei added dev/2.1.10-merged and removed dev/2.1.x labels May 8, 2025

yiguolei mentioned this pull request May 13, 2025

2.1.10 Release Note #50853

Closed

suxiaogang223 pushed a commit to suxiaogang223/doris that referenced this pull request Jul 15, 2025

[enhance](mtmv)Only restrict MTMV to not allow concurrent insert over…

63170c4

…write apache#48673 (apache#3896) pick: apache#48673

Hastyshell pushed a commit to Hastyshell/doris that referenced this pull request Jul 21, 2025

[enhance](mtmv)Only restrict MTMV to not allow concurrent insert over…

8f53764

…write execution (apache#48673) (apache#3898) pick: apache#48673

[enhance](mtmv)Only restrict MTMV to not allow concurrent insert overwrite execution #48673

[enhance](mtmv)Only restrict MTMV to not allow concurrent insert overwrite execution #48673

Uh oh!

Conversation

zddr commented Mar 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Mar 5, 2025

Uh oh!

zddr commented Mar 5, 2025

Uh oh!

doris-robot commented Mar 5, 2025

Uh oh!

doris-robot commented Mar 5, 2025

Uh oh!

doris-robot commented Mar 5, 2025

Uh oh!

zddr commented Mar 5, 2025

Uh oh!

github-actions bot commented Mar 10, 2025

Uh oh!

github-actions bot commented Mar 10, 2025

Uh oh!

morrySnow commented Mar 27, 2025

Uh oh!

zclllyybb left a comment

Choose a reason for hiding this comment

Uh oh!

doris-robot commented Mar 27, 2025

Uh oh!

zddr commented Mar 27, 2025

Uh oh!

doris-robot commented Mar 27, 2025

Uh oh!

doris-robot commented Mar 27, 2025

Uh oh!

doris-robot commented Mar 27, 2025

Uh oh!

zclllyybb left a comment

Choose a reason for hiding this comment

Uh oh!

zddr commented Apr 3, 2025

Uh oh!

doris-robot commented Apr 3, 2025

Uh oh!

doris-robot commented Apr 3, 2025

Uh oh!

doris-robot commented Apr 3, 2025

Uh oh!

zddr commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

doris-robot commented Apr 7, 2025

Uh oh!

zddr commented Apr 8, 2025

Uh oh!

github-actions bot commented Apr 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

zddr commented Mar 5, 2025 •

edited

Loading