Skip to content

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Jan 2, 2025

Cherry-picked from #45955

…perties (#45955)

as the auto partition table maybe no any partitions firstly, it's should
same as table's storage medium rather than default storage medium, maybe
user have change different from default value.
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Jan 2, 2025
@dataroaring dataroaring reopened this Jan 2, 2025
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40548 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit da2ee1df90d5a2865c81cc72e032ed0f257acf5a, data reload: false

------ Round 1 ----------------------------------
q1	17903	7377	7196	7196
q2	2069	169	159	159
q3	10635	1102	1178	1102
q4	10235	776	719	719
q5	7730	2854	2696	2696
q6	233	144	141	141
q7	957	610	590	590
q8	9362	1910	1956	1910
q9	6571	6400	6402	6400
q10	7008	2292	2346	2292
q11	466	272	250	250
q12	394	212	208	208
q13	17768	2962	2982	2962
q14	250	208	205	205
q15	578	531	538	531
q16	652	600	596	596
q17	972	554	570	554
q18	7276	6881	6632	6632
q19	1386	1008	1022	1008
q20	457	204	192	192
q21	3976	3254	3209	3209
q22	1073	996	999	996
Total cold run time: 107951 ms
Total hot run time: 40548 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7235	7169	7172	7169
q2	338	241	231	231
q3	2844	2754	2733	2733
q4	1969	1714	1702	1702
q5	5413	5449	5430	5430
q6	217	137	135	135
q7	2111	1707	1728	1707
q8	3207	3348	3387	3348
q9	8498	8487	8534	8487
q10	3475	3426	3413	3413
q11	591	504	496	496
q12	796	564	559	559
q13	10658	2987	3002	2987
q14	295	251	277	251
q15	571	502	525	502
q16	700	667	687	667
q17	1789	1581	1547	1547
q18	7788	7445	7408	7408
q19	1645	1634	1470	1470
q20	2017	1802	1817	1802
q21	5253	5019	5055	5019
q22	1068	1014	991	991
Total cold run time: 68478 ms
Total hot run time: 58054 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190754 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit da2ee1df90d5a2865c81cc72e032ed0f257acf5a, data reload: false

query1	973	386	368	368
query2	6532	2028	2034	2028
query3	6708	209	218	209
query4	33717	23430	23358	23358
query5	4371	463	443	443
query6	282	190	180	180
query7	4642	306	312	306
query8	294	224	229	224
query9	9629	2701	2690	2690
query10	486	259	254	254
query11	18075	15180	15239	15180
query12	160	97	101	97
query13	1646	437	415	415
query14	10744	6475	7236	6475
query15	250	171	178	171
query16	7948	447	470	447
query17	1646	548	528	528
query18	2106	303	302	302
query19	366	153	146	146
query20	116	112	109	109
query21	63	45	44	44
query22	4465	4404	4249	4249
query23	34770	34032	34324	34032
query24	11192	2812	2812	2812
query25	662	406	416	406
query26	1251	170	167	167
query27	2331	304	298	298
query28	7461	2468	2425	2425
query29	858	448	436	436
query30	322	163	165	163
query31	1050	810	821	810
query32	94	60	61	60
query33	779	281	297	281
query34	960	492	501	492
query35	870	707	708	707
query36	1123	936	906	906
query37	129	70	71	70
query38	3954	3833	3891	3833
query39	1478	1431	1406	1406
query40	145	86	83	83
query41	55	56	49	49
query42	112	99	100	99
query43	540	489	480	480
query44	1256	806	814	806
query45	183	186	172	172
query46	1134	714	710	710
query47	1952	1832	1873	1832
query48	453	378	396	378
query49	1165	400	407	400
query50	821	406	423	406
query51	7273	7178	7016	7016
query52	100	95	91	91
query53	254	182	186	182
query54	1404	467	462	462
query55	77	81	77	77
query56	277	280	256	256
query57	1249	1125	1102	1102
query58	245	208	246	208
query59	3185	2994	2786	2786
query60	299	263	268	263
query61	135	133	133	133
query62	903	660	676	660
query63	222	189	187	187
query64	5656	766	702	702
query65	3293	3193	3196	3193
query66	1420	309	319	309
query67	16060	15628	15765	15628
query68	4897	556	571	556
query69	448	269	266	266
query70	1211	1112	1077	1077
query71	326	243	243	243
query72	6503	4092	4067	4067
query73	751	349	361	349
query74	10212	8996	9006	8996
query75	3355	2635	2603	2603
query76	2959	1115	1080	1080
query77	407	294	265	265
query78	10478	9849	9601	9601
query79	2062	627	590	590
query80	1140	430	419	419
query81	567	243	248	243
query82	741	117	123	117
query83	212	154	149	149
query84	242	78	79	78
query85	1281	300	292	292
query86	429	274	300	274
query87	4592	4344	4229	4229
query88	3995	2382	2444	2382
query89	407	293	295	293
query90	2024	185	182	182
query91	178	145	147	145
query92	66	50	51	50
query93	1455	546	537	537
query94	907	298	293	293
query95	353	259	248	248
query96	600	273	290	273
query97	3356	3163	3171	3163
query98	227	212	193	193
query99	1552	1304	1328	1304
Total cold run time: 302304 ms
Total hot run time: 190754 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.69 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit da2ee1df90d5a2865c81cc72e032ed0f257acf5a, data reload: false

query1	0.03	0.02	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.63	0.10	0.11
query5	0.50	0.52	0.50
query6	1.14	0.73	0.74
query7	0.02	0.01	0.01
query8	0.04	0.04	0.03
query9	0.56	0.52	0.51
query10	0.54	0.55	0.55
query11	0.14	0.10	0.11
query12	0.14	0.12	0.11
query13	0.61	0.60	0.59
query14	2.92	2.92	2.91
query15	0.89	0.83	0.83
query16	0.39	0.38	0.38
query17	1.07	1.01	1.03
query18	0.25	0.21	0.21
query19	1.84	1.77	1.98
query20	0.01	0.01	0.01
query21	15.37	0.59	0.58
query22	2.55	2.82	1.41
query23	17.18	0.89	0.86
query24	2.91	1.19	1.95
query25	0.26	0.13	0.09
query26	0.48	0.13	0.13
query27	0.03	0.03	0.04
query28	9.77	1.10	1.07
query29	12.55	3.27	3.26
query30	0.25	0.07	0.06
query31	2.85	0.38	0.37
query32	3.25	0.46	0.47
query33	2.98	3.01	3.02
query34	17.02	4.46	4.43
query35	4.57	4.48	4.50
query36	0.69	0.52	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.07	0.02	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.2 s
Total hot run time: 32.69 s

@zhangstar333
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40809 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit da2ee1df90d5a2865c81cc72e032ed0f257acf5a, data reload: false

------ Round 1 ----------------------------------
q1	17583	7405	7287	7287
q2	2075	165	155	155
q3	10569	1141	1156	1141
q4	10227	777	707	707
q5	8084	2890	2821	2821
q6	232	147	147	147
q7	990	621	604	604
q8	9360	1973	1986	1973
q9	6569	6389	6375	6375
q10	7002	2272	2330	2272
q11	454	266	262	262
q12	395	210	212	210
q13	17775	2990	2978	2978
q14	240	221	214	214
q15	577	529	534	529
q16	668	600	625	600
q17	966	534	539	534
q18	7260	6733	6772	6733
q19	1387	1063	1021	1021
q20	483	204	198	198
q21	4095	3085	3071	3071
q22	1084	989	977	977
Total cold run time: 108075 ms
Total hot run time: 40809 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7245	7269	7238	7238
q2	326	232	232	232
q3	2939	2966	2979	2966
q4	2103	1862	1795	1795
q5	5694	5647	5698	5647
q6	224	135	135	135
q7	2213	1860	1878	1860
q8	3319	3559	3480	3480
q9	8844	8921	8911	8911
q10	3623	3536	3538	3536
q11	599	516	499	499
q12	821	616	641	616
q13	10155	3135	3129	3129
q14	302	267	268	267
q15	563	507	513	507
q16	689	650	641	641
q17	1828	1600	1563	1563
q18	7733	7383	7302	7302
q19	1664	1608	1459	1459
q20	2047	1815	1820	1815
q21	5357	5261	5173	5173
q22	1116	1006	1006	1006
Total cold run time: 69404 ms
Total hot run time: 59777 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192042 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit da2ee1df90d5a2865c81cc72e032ed0f257acf5a, data reload: false

query1	984	370	365	365
query2	6509	2102	2034	2034
query3	6707	213	218	213
query4	33978	23798	23593	23593
query5	4338	445	437	437
query6	274	172	171	171
query7	4639	312	308	308
query8	299	232	234	232
query9	9843	2705	2693	2693
query10	483	280	268	268
query11	18115	15141	15098	15098
query12	152	101	102	101
query13	1638	429	422	422
query14	9855	7540	7535	7535
query15	255	176	182	176
query16	8194	470	498	470
query17	1680	587	559	559
query18	2150	313	320	313
query19	372	156	153	153
query20	113	110	111	110
query21	63	46	52	46
query22	4322	4204	4246	4204
query23	34564	34217	34002	34002
query24	11452	2838	2911	2838
query25	676	418	420	418
query26	1404	175	173	173
query27	2781	307	300	300
query28	7906	2413	2409	2409
query29	887	447	437	437
query30	329	170	172	170
query31	1022	787	782	782
query32	100	61	62	61
query33	791	314	291	291
query34	941	503	519	503
query35	853	708	715	708
query36	1095	965	931	931
query37	139	75	80	75
query38	4033	3974	3885	3885
query39	1469	1437	1429	1429
query40	205	88	86	86
query41	56	52	51	51
query42	113	101	100	100
query43	520	475	481	475
query44	1233	818	813	813
query45	196	170	172	170
query46	1148	715	732	715
query47	1921	1805	1845	1805
query48	460	392	396	392
query49	1172	414	397	397
query50	816	412	415	412
query51	7225	7150	7169	7150
query52	104	89	93	89
query53	257	183	185	183
query54	1235	477	461	461
query55	82	82	81	81
query56	280	254	263	254
query57	1246	1145	1103	1103
query58	242	210	215	210
query59	3260	2962	2928	2928
query60	288	258	256	256
query61	130	122	111	111
query62	844	674	662	662
query63	224	190	188	188
query64	5543	656	650	650
query65	3294	3191	3198	3191
query66	1475	305	316	305
query67	16049	15573	15525	15525
query68	4978	578	579	578
query69	481	273	260	260
query70	1159	1007	1111	1007
query71	423	259	259	259
query72	6513	4038	4011	4011
query73	761	342	352	342
query74	10468	9016	9005	9005
query75	3384	2626	2665	2626
query76	2931	1071	1111	1071
query77	437	290	269	269
query78	10491	9701	9496	9496
query79	1062	585	604	585
query80	645	434	421	421
query81	514	241	239	239
query82	228	122	118	118
query83	243	149	149	149
query84	239	81	75	75
query85	1123	305	291	291
query86	312	289	281	281
query87	4503	4217	4343	4217
query88	3754	2381	2354	2354
query89	382	298	297	297
query90	2164	184	188	184
query91	187	149	171	149
query92	60	51	52	51
query93	1062	551	547	547
query94	843	300	294	294
query95	355	254	258	254
query96	617	284	285	284
query97	3309	3192	3158	3158
query98	214	217	191	191
query99	1514	1338	1303	1303
Total cold run time: 300379 ms
Total hot run time: 192042 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.33 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit da2ee1df90d5a2865c81cc72e032ed0f257acf5a, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.03	0.02
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.52	0.49	0.52
query6	1.14	0.74	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.02
query9	0.56	0.51	0.49
query10	0.56	0.55	0.54
query11	0.13	0.10	0.10
query12	0.15	0.11	0.12
query13	0.63	0.60	0.60
query14	2.89	2.92	2.94
query15	0.89	0.82	0.82
query16	0.38	0.38	0.38
query17	1.05	1.01	1.05
query18	0.24	0.22	0.22
query19	1.98	1.91	1.96
query20	0.02	0.01	0.01
query21	15.36	0.63	0.61
query22	3.09	1.94	2.24
query23	17.15	0.89	0.78
query24	2.96	1.23	2.18
query25	0.24	0.17	0.04
query26	0.52	0.15	0.15
query27	0.03	0.04	0.04
query28	9.61	1.11	1.08
query29	12.52	3.23	3.23
query30	0.24	0.07	0.06
query31	2.86	0.40	0.38
query32	3.24	0.47	0.47
query33	3.01	3.05	3.02
query34	17.00	4.42	4.46
query35	4.54	4.52	4.53
query36	0.70	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.14	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.7 s
Total hot run time: 33.33 s

@zhangstar333
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40750 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a62665787d3019b846bf057b9422040216117dbb, data reload: false

------ Round 1 ----------------------------------
q1	17567	7382	7209	7209
q2	2062	172	187	172
q3	10629	1074	1190	1074
q4	10550	730	730	730
q5	7740	2879	2857	2857
q6	240	146	145	145
q7	979	614	608	608
q8	9587	1933	2028	1933
q9	7749	6807	6389	6389
q10	6949	2312	2303	2303
q11	459	265	272	265
q12	399	214	208	208
q13	17790	3004	3017	3004
q14	240	216	213	213
q15	576	527	522	522
q16	663	593	569	569
q17	996	595	591	591
q18	7205	6680	6549	6549
q19	1389	1130	970	970
q20	494	216	199	199
q21	4179	3367	3238	3238
q22	1070	1002	1002	1002
Total cold run time: 109512 ms
Total hot run time: 40750 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7239	7223	7355	7223
q2	357	238	243	238
q3	3087	2957	3047	2957
q4	2169	1918	1894	1894
q5	5831	5838	5835	5835
q6	225	142	141	141
q7	2256	1858	1826	1826
q8	3499	3586	3510	3510
q9	9044	8866	8814	8814
q10	3579	3641	3553	3553
q11	594	492	497	492
q12	839	622	606	606
q13	9736	3172	3165	3165
q14	311	297	289	289
q15	577	526	531	526
q16	700	657	667	657
q17	1840	1623	1620	1620
q18	8264	7705	7554	7554
q19	1702	1582	1591	1582
q20	2106	1893	1892	1892
q21	5610	5333	5261	5261
q22	1143	1058	1029	1029
Total cold run time: 70708 ms
Total hot run time: 60664 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 199405 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a62665787d3019b846bf057b9422040216117dbb, data reload: false

query1	1290	944	950	944
query2	6244	2129	2177	2129
query3	10840	4141	4354	4141
query4	65891	28283	23861	23861
query5	4968	452	448	448
query6	402	161	179	161
query7	5525	310	316	310
query8	301	212	216	212
query9	8682	2709	2719	2709
query10	450	275	257	257
query11	17231	15445	15901	15445
query12	157	108	104	104
query13	1456	437	469	437
query14	10788	7695	7581	7581
query15	207	188	174	174
query16	7083	461	502	461
query17	1116	567	577	567
query18	1872	316	311	311
query19	208	153	160	153
query20	125	109	107	107
query21	211	110	100	100
query22	4841	4499	4538	4499
query23	34664	34369	34901	34369
query24	6213	2892	2875	2875
query25	519	409	409	409
query26	649	170	169	169
query27	1830	364	371	364
query28	4179	2522	2492	2492
query29	713	477	460	460
query30	240	173	160	160
query31	988	860	834	834
query32	70	59	56	56
query33	401	294	287	287
query34	897	506	517	506
query35	871	738	748	738
query36	1094	970	957	957
query37	125	71	78	71
query38	4147	3977	3998	3977
query39	1539	1474	1497	1474
query40	210	100	100	100
query41	51	50	49	49
query42	118	104	102	102
query43	546	506	501	501
query44	1162	826	832	826
query45	194	173	176	173
query46	1153	741	737	737
query47	2023	1909	1917	1909
query48	482	396	386	386
query49	754	437	393	393
query50	840	441	428	428
query51	7214	7345	7131	7131
query52	106	90	87	87
query53	262	191	183	183
query54	556	456	469	456
query55	77	75	81	75
query56	279	246	257	246
query57	1234	1151	1114	1114
query58	200	203	210	203
query59	3360	3037	3148	3037
query60	273	247	252	247
query61	153	102	107	102
query62	851	706	747	706
query63	222	194	196	194
query64	1345	669	632	632
query65	3283	3221	3200	3200
query66	709	297	304	297
query67	15959	15661	15652	15652
query68	3899	595	578	578
query69	419	266	264	264
query70	1223	1154	1097	1097
query71	355	249	253	249
query72	6106	4020	3993	3993
query73	757	343	349	343
query74	10610	9086	9361	9086
query75	3345	2625	2674	2625
query76	1792	1094	1036	1036
query77	478	266	272	266
query78	10683	9725	9633	9633
query79	1124	618	620	618
query80	825	434	431	431
query81	493	245	237	237
query82	1189	126	118	118
query83	241	147	146	146
query84	283	89	82	82
query85	868	300	289	289
query86	329	261	297	261
query87	4413	4413	4371	4371
query88	3764	2403	2353	2353
query89	414	290	295	290
query90	2023	188	183	183
query91	179	148	150	148
query92	67	51	50	50
query93	1260	565	562	562
query94	829	306	299	299
query95	358	257	252	252
query96	608	283	279	279
query97	3339	3196	3148	3148
query98	214	207	196	196
query99	1725	1460	1447	1447
Total cold run time: 316855 ms
Total hot run time: 199405 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.93 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a62665787d3019b846bf057b9422040216117dbb, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.22	0.06	0.06
query4	1.63	0.11	0.10
query5	0.54	0.51	0.51
query6	1.14	0.73	0.74
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.55	0.51	0.49
query10	0.55	0.54	0.56
query11	0.14	0.11	0.09
query12	0.14	0.11	0.11
query13	0.61	0.59	0.59
query14	2.72	2.72	2.83
query15	0.91	0.85	0.82
query16	0.38	0.39	0.38
query17	1.07	1.08	1.08
query18	0.23	0.22	0.22
query19	1.90	1.83	2.03
query20	0.02	0.01	0.01
query21	15.35	0.57	0.60
query22	2.69	1.90	2.26
query23	16.87	1.03	0.80
query24	3.01	1.16	1.28
query25	0.23	0.12	0.09
query26	0.42	0.13	0.14
query27	0.04	0.05	0.03
query28	10.57	1.13	1.07
query29	12.56	3.21	3.18
query30	0.25	0.06	0.06
query31	2.85	0.39	0.37
query32	3.27	0.47	0.46
query33	3.01	2.94	3.02
query34	17.04	4.51	4.51
query35	4.52	4.49	4.53
query36	0.68	0.49	0.49
query37	0.09	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.02	0.02
query40	0.15	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.73 s
Total hot run time: 32.93 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 61da5be into branch-3.0 Feb 13, 2025
26 of 29 checks passed
@github-actions github-actions bot deleted the auto-pick-45955-branch-3.0 branch February 13, 2025 02:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants