Skip to content

branch-3.0: [fix](merge-on-write) Remove pending delete bitmap check when commit_txn() #47136 #47283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Jan 24, 2025
Merged

branch-3.0: [fix](merge-on-write) Remove pending delete bitmap check when commit_txn() #47136 #47283

dataroaring merged 1 commit into from
Jan 24, 2025

Conversation

@bobhan1
Copy link
Contributor

@bobhan1 bobhan1 commented Jan 21, 2025

pick #47136

@bobhan1 bobhan1 requested a review from dataroaring as a code owner January 21, 2025 11:48
@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 21, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 21, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jan 21, 2025

TPC-H: Total hot run time: 41128 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 021f60e31f32d12a5a83eae1a96ba43eca64dc7f, data reload: false

------ Round 1 ----------------------------------
q1	17716	7769	7270	7270
q2	2061	176	175	175
q3	10786	1093	1216	1093
q4	10456	769	710	710
q5	7746	2882	2898	2882
q6	241	145	142	142
q7	978	601	609	601
q8	9367	2023	2112	2023
q9	6615	6452	6436	6436
q10	7040	2319	2371	2319
q11	477	257	267	257
q12	414	216	212	212
q13	17768	2968	2983	2968
q14	243	216	207	207
q15	563	523	528	523
q16	689	610	622	610
q17	1031	608	535	535
q18	7462	6594	6658	6594
q19	1441	1149	1084	1084
q20	457	209	195	195
q21	4129	3364	3299	3299
q22	1118	993	1002	993
Total cold run time: 108798 ms
Total hot run time: 41128 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7303	7251	7237	7237
q2	326	240	227	227
q3	3010	2941	2957	2941
q4	2094	1849	1809	1809
q5	5705	5828	5779	5779
q6	239	149	142	142
q7	2204	1804	1825	1804
q8	3346	3548	3458	3458
q9	8942	8972	8898	8898
q10	3605	3562	3576	3562
q11	607	509	504	504
q12	794	582	608	582
q13	9467	3197	3195	3195
q14	301	271	283	271
q15	587	526	534	526
q16	715	692	685	685
q17	1864	1615	1635	1615
q18	8254	7845	7522	7522
q19	1802	1727	1532	1532
q20	2099	1890	1893	1890
q21	5520	5416	5472	5416
q22	1147	1100	1036	1036
Total cold run time: 69931 ms
Total hot run time: 60631 ms

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 21, 2025

run cloud_p0

@doris-robot
Copy link

doris-robot commented Jan 21, 2025

TPC-DS: Total hot run time: 198661 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 021f60e31f32d12a5a83eae1a96ba43eca64dc7f, data reload: false

query1	1293	955	927	927
query2	6238	2070	2041	2041
query3	10812	4294	4059	4059
query4	66321	29307	24070	24070
query5	4884	446	452	446
query6	420	171	171	171
query7	5651	309	316	309
query8	313	228	213	213
query9	9192	2695	2682	2682
query10	467	271	258	258
query11	17311	15389	15798	15389
query12	159	103	104	103
query13	1551	448	427	427
query14	11052	7712	7792	7712
query15	211	187	179	179
query16	7142	508	483	483
query17	1102	583	604	583
query18	1929	351	333	333
query19	219	160	158	158
query20	117	117	119	117
query21	203	108	105	105
query22	4966	4770	4445	4445
query23	34769	34435	34539	34435
query24	6181	2925	2945	2925
query25	578	419	420	419
query26	659	173	170	170
query27	1892	362	354	354
query28	4212	2522	2450	2450
query29	710	476	466	466
query30	249	158	162	158
query31	996	828	828	828
query32	69	55	57	55
query33	418	316	276	276
query34	920	512	535	512
query35	877	740	730	730
query36	1093	953	980	953
query37	120	73	72	72
query38	4078	4091	3992	3992
query39	1533	1456	1459	1456
query40	221	99	98	98
query41	48	46	46	46
query42	115	100	99	99
query43	542	492	499	492
query44	1210	827	851	827
query45	185	172	171	171
query46	1162	726	733	726
query47	2059	1930	1949	1930
query48	477	383	387	383
query49	738	390	386	386
query50	868	436	431	431
query51	7438	7361	7019	7019
query52	102	88	88	88
query53	253	180	182	180
query54	564	444	450	444
query55	74	73	82	73
query56	248	240	236	236
query57	1266	1124	1156	1124
query58	209	207	208	207
query59	3143	3056	2905	2905
query60	276	256	250	250
query61	109	104	134	104
query62	794	665	665	665
query63	217	195	205	195
query64	1381	667	637	637
query65	3278	3191	3199	3191
query66	707	300	297	297
query67	15957	15644	15781	15644
query68	3937	586	588	586
query69	425	280	269	269
query70	1196	1096	1117	1096
query71	391	248	274	248
query72	6358	3972	3949	3949
query73	759	357	346	346
query74	10217	9097	8940	8940
query75	3378	2658	2645	2645
query76	2245	936	1192	936
query77	501	264	274	264
query78	10777	9663	9596	9596
query79	1920	590	593	590
query80	1201	443	425	425
query81	536	245	235	235
query82	467	116	111	111
query83	167	143	144	143
query84	285	84	81	81
query85	960	305	292	292
query86	420	294	303	294
query87	4442	4248	4307	4248
query88	3961	2382	2357	2357
query89	419	301	286	286
query90	1971	186	185	185
query91	187	152	146	146
query92	66	51	51	51
query93	2575	548	553	548
query94	909	292	279	279
query95	357	256	265	256
query96	612	281	283	281
query97	3353	3215	3179	3179
query98	218	204	199	199
query99	1566	1310	1291	1291
Total cold run time: 321324 ms
Total hot run time: 198661 ms

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 21, 2025

run p0

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 21, 2025

run performance

1 similar comment
@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 22, 2025

run performance

@doris-robot
Copy link

doris-robot commented Jan 22, 2025

TPC-H: Total hot run time: 40755 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 021f60e31f32d12a5a83eae1a96ba43eca64dc7f, data reload: false

------ Round 1 ----------------------------------
q1	17944	7943	7190	7190
q2	2063	179	168	168
q3	10754	1077	1190	1077
q4	10546	782	731	731
q5	7763	2872	2882	2872
q6	234	149	149	149
q7	980	603	613	603
q8	9362	1976	2057	1976
q9	6610	6383	6354	6354
q10	6999	2318	2310	2310
q11	461	266	267	266
q12	402	209	213	209
q13	17778	2971	2986	2971
q14	250	205	217	205
q15	567	519	533	519
q16	684	599	619	599
q17	975	629	581	581
q18	7275	6588	6718	6588
q19	1389	1141	1014	1014
q20	467	211	193	193
q21	3971	3272	3204	3204
q22	1069	976	981	976
Total cold run time: 108543 ms
Total hot run time: 40755 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7219	7171	7194	7171
q2	324	240	228	228
q3	2945	2930	2961	2930
q4	2044	1862	1797	1797
q5	5717	5763	5747	5747
q6	229	139	147	139
q7	2193	1848	1848	1848
q8	3339	3565	3481	3481
q9	8881	8914	8858	8858
q10	3562	3539	3497	3497
q11	596	498	502	498
q12	800	592	599	592
q13	9827	3100	3142	3100
q14	303	271	287	271
q15	595	540	533	533
q16	728	680	662	662
q17	1842	1614	1622	1614
q18	8215	7856	7669	7669
q19	1647	1508	1613	1508
q20	2080	1884	1907	1884
q21	5561	5287	5385	5287
q22	1177	1001	1006	1001
Total cold run time: 69824 ms
Total hot run time: 60315 ms

@doris-robot
Copy link

doris-robot commented Jan 22, 2025

TPC-DS: Total hot run time: 198072 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 021f60e31f32d12a5a83eae1a96ba43eca64dc7f, data reload: false

query1	1301	958	915	915
query2	6219	2075	1985	1985
query3	10801	4294	4151	4151
query4	66571	28924	23504	23504
query5	5020	475	467	467
query6	412	182	178	178
query7	5597	328	311	311
query8	302	227	227	227
query9	9111	2685	2657	2657
query10	470	286	261	261
query11	17392	15329	15816	15329
query12	171	101	104	101
query13	1597	441	437	437
query14	10811	7632	7571	7571
query15	204	182	192	182
query16	7080	463	467	463
query17	1060	594	571	571
query18	2198	305	306	305
query19	226	148	151	148
query20	109	104	103	103
query21	205	100	102	100
query22	4736	4621	4468	4468
query23	34771	33982	34303	33982
query24	6604	2955	2941	2941
query25	550	400	386	386
query26	653	165	166	165
query27	1888	349	355	349
query28	4392	2471	2449	2449
query29	685	485	444	444
query30	246	167	165	165
query31	1013	832	854	832
query32	68	55	55	55
query33	406	283	288	283
query34	919	509	518	509
query35	849	744	746	744
query36	1105	958	961	958
query37	124	80	68	68
query38	4098	3971	4002	3971
query39	1561	1630	1484	1484
query40	202	97	98	97
query41	51	45	47	45
query42	110	108	104	104
query43	543	491	500	491
query44	1190	837	822	822
query45	195	170	177	170
query46	1166	714	746	714
query47	2014	1908	1915	1908
query48	467	372	375	372
query49	738	409	407	407
query50	859	430	420	420
query51	7365	7234	7282	7234
query52	104	90	93	90
query53	263	186	189	186
query54	585	473	467	467
query55	78	74	78	74
query56	265	260	263	260
query57	1224	1094	1105	1094
query58	228	216	253	216
query59	3253	3004	2932	2932
query60	295	268	276	268
query61	134	133	158	133
query62	770	685	669	669
query63	220	187	192	187
query64	1362	669	634	634
query65	3238	3167	3198	3167
query66	700	298	301	298
query67	16215	15628	15668	15628
query68	3526	589	589	589
query69	434	269	279	269
query70	1213	1038	1133	1038
query71	357	255	261	255
query72	6187	4053	4015	4015
query73	766	341	356	341
query74	10017	8963	9155	8963
query75	3371	2624	2635	2624
query76	1882	1087	1041	1041
query77	534	280	291	280
query78	10434	9605	9616	9605
query79	1143	616	598	598
query80	770	426	429	426
query81	500	251	239	239
query82	1283	123	120	120
query83	171	152	148	148
query84	291	84	79	79
query85	876	362	352	352
query86	328	289	296	289
query87	4468	4416	4400	4400
query88	3578	2393	2371	2371
query89	414	297	298	297
query90	2056	196	190	190
query91	197	162	184	162
query92	62	50	52	50
query93	1252	553	546	546
query94	756	324	293	293
query95	348	254	254	254
query96	607	281	286	281
query97	3325	3131	3149	3131
query98	212	205	198	198
query99	1548	1322	1329	1322
Total cold run time: 318265 ms
Total hot run time: 198072 ms

@bobhan1
[fix](merge-on-write) Remove pending delete bitmap check when `commit…
01cc988 
…_txn()` (apache#47136)

Related PR: apache#46039

Problem Summary:

apache#46039 introduce an defensive check
when `commit_txn()`, but this may influence the commit process. This PR
remove this check totally to eliminate this overhead.
@bobhan1 bobhan1 force-pushed the branch-3.0-pick-47136 branch from 021f60e to 01cc988 Compare January 22, 2025 05:13
@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 22, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jan 22, 2025

TPC-H: Total hot run time: 40732 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 01cc988df0efe8ae64b6c42f06212bb5991df5ae, data reload: false

------ Round 1 ----------------------------------
q1	18032	7716	7291	7291
q2	2058	175	166	166
q3	10698	1084	1210	1084
q4	10426	723	791	723
q5	7754	2853	2818	2818
q6	246	146	143	143
q7	965	618	592	592
q8	9357	1899	2028	1899
q9	6619	6407	6375	6375
q10	7025	2270	2298	2270
q11	463	267	260	260
q12	398	215	207	207
q13	17779	2975	2988	2975
q14	251	209	211	209
q15	569	528	515	515
q16	692	602	606	602
q17	979	554	545	545
q18	7351	6668	6725	6668
q19	1437	1130	982	982
q20	451	203	197	197
q21	3961	3246	3405	3246
q22	1103	1015	965	965
Total cold run time: 108614 ms
Total hot run time: 40732 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7537	7236	7175	7175
q2	325	230	233	230
q3	2953	2911	2910	2910
q4	2075	1851	1793	1793
q5	5724	5761	5707	5707
q6	233	147	143	143
q7	2257	1795	1817	1795
q8	3337	3599	3517	3517
q9	8803	8848	8762	8762
q10	3580	3578	3537	3537
q11	615	499	492	492
q12	795	634	612	612
q13	9299	3155	3161	3155
q14	300	279	292	279
q15	579	524	546	524
q16	712	677	669	669
q17	1854	1607	1600	1600
q18	8310	7744	7723	7723
q19	1676	1557	1521	1521
q20	2139	1890	1881	1881
q21	5613	5290	5392	5290
q22	1178	1086	1046	1046
Total cold run time: 69894 ms
Total hot run time: 60361 ms

@doris-robot
Copy link

doris-robot commented Jan 22, 2025

TPC-DS: Total hot run time: 197903 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 01cc988df0efe8ae64b6c42f06212bb5991df5ae, data reload: false

query1	1318	932	957	932
query2	6266	2042	2089	2042
query3	10953	4337	4340	4337
query4	66639	29193	23468	23468
query5	4998	469	471	469
query6	420	181	186	181
query7	5627	318	306	306
query8	290	205	205	205
query9	9224	2661	2652	2652
query10	458	261	244	244
query11	17436	16911	15766	15766
query12	154	104	101	101
query13	1516	435	424	424
query14	10101	6761	6724	6724
query15	215	184	181	181
query16	7071	495	484	484
query17	1103	584	598	584
query18	1799	319	338	319
query19	194	164	165	164
query20	122	113	111	111
query21	211	108	111	108
query22	4860	4274	4274	4274
query23	34491	34852	34512	34512
query24	6523	2911	2893	2893
query25	543	440	451	440
query26	660	172	180	172
query27	2205	351	352	351
query28	4331	2489	2483	2483
query29	711	487	447	447
query30	267	171	169	169
query31	986	847	850	847
query32	72	59	58	58
query33	461	311	304	304
query34	920	519	540	519
query35	868	777	746	746
query36	1092	993	991	991
query37	123	71	70	70
query38	4093	4065	3958	3958
query39	1517	1493	1461	1461
query40	204	103	101	101
query41	49	48	49	48
query42	114	98	97	97
query43	533	504	496	496
query44	1147	826	823	823
query45	184	168	169	168
query46	1152	743	783	743
query47	2023	1917	1979	1917
query48	459	380	382	380
query49	737	399	376	376
query50	840	424	422	422
query51	7391	7258	7151	7151
query52	98	85	89	85
query53	253	181	181	181
query54	553	462	451	451
query55	78	75	76	75
query56	260	236	240	236
query57	1209	1109	1121	1109
query58	214	208	209	208
query59	3128	3028	2975	2975
query60	280	259	253	253
query61	111	107	102	102
query62	778	671	671	671
query63	212	194	197	194
query64	1360	669	630	630
query65	3348	3221	3219	3219
query66	698	308	305	305
query67	15929	15672	15585	15585
query68	3904	585	567	567
query69	434	268	268	268
query70	1166	1113	1115	1113
query71	344	252	258	252
query72	6165	4014	3994	3994
query73	751	346	345	345
query74	10222	8968	8941	8941
query75	3363	2615	2683	2615
query76	1939	1086	1046	1046
query77	497	279	274	274
query78	10701	9644	9615	9615
query79	1172	600	589	589
query80	830	440	420	420
query81	503	241	235	235
query82	1351	116	119	116
query83	249	147	147	147
query84	286	77	78	77
query85	902	300	296	296
query86	322	306	283	283
query87	4534	4316	4152	4152
query88	3596	2401	2340	2340
query89	413	298	291	291
query90	2042	188	186	186
query91	188	148	150	148
query92	64	52	50	50
query93	1291	551	541	541
query94	786	300	263	263
query95	358	253	256	253
query96	619	286	281	281
query97	3363	3236	3202	3202
query98	212	205	195	195
query99	1562	1312	1310	1310
Total cold run time: 318309 ms
Total hot run time: 197903 ms

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 22, 2025

run performance

@doris-robot
Copy link

doris-robot commented Jan 22, 2025

TPC-H: Total hot run time: 41156 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 01cc988df0efe8ae64b6c42f06212bb5991df5ae, data reload: false

------ Round 1 ----------------------------------
q1	17947	8507	7262	7262
q2	2073	166	167	166
q3	10669	1154	1195	1154
q4	10559	788	794	788
q5	7926	2993	2817	2817
q6	242	150	145	145
q7	988	624	623	623
q8	9695	2011	2065	2011
q9	6797	6549	6493	6493
q10	9346	2348	2318	2318
q11	471	265	266	265
q12	404	213	215	213
q13	17791	2990	3047	2990
q14	234	206	213	206
q15	592	527	530	527
q16	702	618	600	600
q17	988	567	547	547
q18	7379	6738	6584	6584
q19	1405	1055	988	988
q20	488	202	194	194
q21	4012	3316	3242	3242
q22	1099	1027	1023	1023
Total cold run time: 111807 ms
Total hot run time: 41156 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7255	7184	7248	7184
q2	356	250	240	240
q3	3206	2976	2929	2929
q4	2057	1856	1755	1755
q5	5763	5738	5669	5669
q6	218	138	142	138
q7	2199	1805	1837	1805
q8	3351	3501	3518	3501
q9	8905	9037	9073	9037
q10	3677	3613	3610	3610
q11	601	520	525	520
q12	809	660	595	595
q13	5519	3118	3113	3113
q14	290	286	291	286
q15	571	525	521	521
q16	711	669	677	669
q17	1818	1609	1588	1588
q18	8214	7789	7375	7375
q19	1657	1549	1521	1521
q20	2054	1790	1812	1790
q21	5360	5121	5050	5050
q22	1114	1000	1030	1000
Total cold run time: 65705 ms
Total hot run time: 59896 ms

@doris-robot
Copy link

doris-robot commented Jan 22, 2025

TPC-DS: Total hot run time: 191636 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 01cc988df0efe8ae64b6c42f06212bb5991df5ae, data reload: false

query1	999	371	370	370
query2	6509	2092	2096	2092
query3	6710	217	218	217
query4	33782	23480	23449	23449
query5	4331	465	450	450
query6	292	178	173	173
query7	4639	316	329	316
query8	302	228	217	217
query9	9592	2684	2676	2676
query10	500	274	267	267
query11	17993	15272	15145	15145
query12	154	104	106	104
query13	1646	426	414	414
query14	10333	7312	7278	7278
query15	254	174	181	174
query16	8088	445	477	445
query17	1641	585	566	566
query18	2115	310	321	310
query19	260	161	155	155
query20	122	113	107	107
query21	215	106	107	106
query22	4518	4124	4134	4124
query23	34706	33637	34256	33637
query24	11847	2924	2936	2924
query25	685	401	389	389
query26	1805	170	172	170
query27	2858	342	336	336
query28	7612	2391	2446	2391
query29	1060	459	428	428
query30	329	171	168	168
query31	1056	804	799	799
query32	92	61	56	56
query33	777	299	295	295
query34	946	506	546	506
query35	857	713	717	713
query36	1107	961	964	961
query37	195	74	73	73
query38	4019	3939	3862	3862
query39	1467	1443	1433	1433
query40	289	102	101	101
query41	59	54	56	54
query42	115	111	103	103
query43	538	497	477	477
query44	1279	824	807	807
query45	185	164	167	164
query46	1158	712	730	712
query47	1920	1801	1820	1801
query48	468	377	376	376
query49	1292	387	390	387
query50	804	418	433	418
query51	7262	6932	7213	6932
query52	103	95	89	89
query53	264	187	191	187
query54	1210	478	461	461
query55	81	79	82	79
query56	279	250	249	249
query57	1235	1118	1085	1085
query58	237	217	221	217
query59	3289	3118	2995	2995
query60	287	268	259	259
query61	111	113	107	107
query62	866	675	692	675
query63	232	198	197	197
query64	5243	678	644	644
query65	3307	3248	3251	3248
query66	1448	316	317	316
query67	15972	15614	15576	15576
query68	4364	585	583	583
query69	419	283	277	277
query70	1207	1059	1077	1059
query71	332	255	258	255
query72	6381	3945	4235	3945
query73	753	355	359	355
query74	10110	9008	9009	9008
query75	3401	2626	2665	2626
query76	2744	991	1131	991
query77	396	314	296	296
query78	10358	9670	9622	9622
query79	1061	632	604	604
query80	666	471	470	470
query81	507	242	238	238
query82	221	119	114	114
query83	165	173	146	146
query84	241	78	82	78
query85	970	303	298	298
query86	328	291	299	291
query87	4494	4336	4293	4293
query88	3862	2402	2364	2364
query89	390	297	294	294
query90	2013	190	190	190
query91	187	152	150	150
query92	62	55	53	53
query93	1052	564	540	540
query94	838	297	287	287
query95	355	265	260	260
query96	618	286	282	282
query97	3313	3188	3177	3177
query98	216	200	200	200
query99	1496	1330	1278	1278
Total cold run time: 299466 ms
Total hot run time: 191636 ms

@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 23, 2025

run performance

@doris-robot
Copy link

doris-robot commented Jan 23, 2025

TPC-H: Total hot run time: 41530 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 01cc988df0efe8ae64b6c42f06212bb5991df5ae, data reload: false

------ Round 1 ----------------------------------
q1	18009	8440	7327	7327
q2	2051	174	183	174
q3	10664	1183	1250	1183
q4	10510	798	802	798
q5	7852	2989	2934	2934
q6	243	158	164	158
q7	1040	648	649	648
q8	9698	2026	2047	2026
q9	6815	6439	6496	6439
q10	7543	2348	2382	2348
q11	532	281	282	281
q12	415	234	216	216
q13	18925	3050	3013	3013
q14	244	227	221	221
q15	570	522	527	522
q16	693	607	626	607
q17	1012	550	589	550
q18	7336	6717	6614	6614
q19	1442	1083	1031	1031
q20	467	207	199	199
q21	4041	3332	3224	3224
q22	1092	1017	1023	1017
Total cold run time: 111194 ms
Total hot run time: 41530 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7779	7272	7317	7272
q2	352	252	245	245
q3	3200	3044	2948	2948
q4	2065	1903	1821	1821
q5	5657	5719	5700	5700
q6	218	142	144	142
q7	2175	1808	1872	1808
q8	3410	3513	3490	3490
q9	8852	8914	8882	8882
q10	3632	3547	3543	3543
q11	585	516	538	516
q12	810	587	575	575
q13	6033	3200	3098	3098
q14	297	296	272	272
q15	584	524	517	517
q16	704	691	682	682
q17	1820	1636	1609	1609
q18	8159	7772	7393	7393
q19	1678	1591	1516	1516
q20	2054	1824	1832	1824
q21	5294	5190	5034	5034
q22	1098	1025	1025	1025
Total cold run time: 66456 ms
Total hot run time: 59912 ms

@doris-robot
Copy link

doris-robot commented Jan 23, 2025

TPC-DS: Total hot run time: 192038 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 01cc988df0efe8ae64b6c42f06212bb5991df5ae, data reload: false

query1	974	375	371	371
query2	6541	2144	1989	1989
query3	6709	219	224	219
query4	34061	23590	23422	23422
query5	4339	454	472	454
query6	274	171	180	171
query7	4639	329	321	321
query8	302	228	225	225
query9	9584	2683	2665	2665
query10	480	273	255	255
query11	18075	15473	15202	15202
query12	150	104	106	104
query13	1640	423	433	423
query14	9685	6961	7161	6961
query15	246	176	179	176
query16	8093	467	474	467
query17	1639	582	576	576
query18	2105	312	325	312
query19	244	156	159	156
query20	115	109	109	109
query21	210	105	109	105
query22	4557	4349	4251	4251
query23	34777	33901	34022	33901
query24	11654	2900	2907	2900
query25	693	396	395	395
query26	1825	174	178	174
query27	2884	352	344	344
query28	8017	2461	2472	2461
query29	1068	446	429	429
query30	333	164	160	160
query31	1011	822	819	819
query32	100	61	61	61
query33	796	303	308	303
query34	935	528	524	524
query35	875	744	715	715
query36	1126	965	975	965
query37	174	74	75	74
query38	4003	3913	3995	3913
query39	1504	1420	1484	1420
query40	288	101	97	97
query41	52	49	49	49
query42	112	102	106	102
query43	539	492	476	476
query44	1260	821	807	807
query45	190	166	172	166
query46	1167	731	744	731
query47	1986	1815	1867	1815
query48	474	388	394	388
query49	1289	402	378	378
query50	810	418	430	418
query51	7284	7142	7134	7134
query52	107	89	96	89
query53	265	187	200	187
query54	1257	478	476	476
query55	82	78	80	78
query56	268	247	252	247
query57	1244	1119	1122	1119
query58	250	218	219	218
query59	3360	3124	2992	2992
query60	283	257	269	257
query61	113	108	150	108
query62	870	663	695	663
query63	229	197	195	195
query64	5205	672	651	651
query65	3274	3218	3188	3188
query66	1448	313	306	306
query67	16032	15723	15498	15498
query68	4401	581	591	581
query69	418	294	273	273
query70	1201	1143	1170	1143
query71	321	263	246	246
query72	6378	4075	4061	4061
query73	775	358	351	351
query74	10216	8997	8909	8909
query75	3402	2673	2643	2643
query76	2719	1093	1017	1017
query77	423	286	280	280
query78	10509	9696	9584	9584
query79	1061	609	599	599
query80	630	444	431	431
query81	506	238	239	238
query82	204	125	120	120
query83	165	146	150	146
query84	250	83	84	83
query85	985	317	299	299
query86	319	290	305	290
query87	4483	4347	4252	4252
query88	3839	2418	2384	2384
query89	393	299	294	294
query90	2028	190	194	190
query91	204	156	151	151
query92	62	55	56	55
query93	1043	565	561	561
query94	871	319	306	306
query95	364	266	274	266
query96	604	283	277	277
query97	3342	3192	3165	3165
query98	212	212	205	205
query99	1492	1326	1313	1313
Total cold run time: 299995 ms
Total hot run time: 192038 ms

@doris-robot
Copy link

doris-robot commented Jan 23, 2025

ClickBench: Total hot run time: 33.51 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 01cc988df0efe8ae64b6c42f06212bb5991df5ae, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.05
query3	0.25	0.07	0.07
query4	1.63	0.10	0.10
query5	0.52	0.54	0.52
query6	1.14	0.73	0.74
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.53	0.51
query10	0.55	0.55	0.56
query11	0.15	0.10	0.11
query12	0.15	0.12	0.11
query13	0.62	0.61	0.61
query14	2.96	2.93	2.98
query15	0.90	0.83	0.82
query16	0.39	0.37	0.39
query17	1.00	1.05	1.00
query18	0.24	0.23	0.22
query19	1.97	1.98	1.95
query20	0.01	0.01	0.01
query21	15.37	0.60	0.61
query22	2.71	2.82	1.71
query23	17.10	0.96	0.83
query24	3.01	1.24	1.48
query25	0.21	0.10	0.06
query26	0.70	0.14	0.14
query27	0.06	0.05	0.04
query28	10.20	1.10	1.08
query29	12.57	3.29	3.26
query30	0.25	0.06	0.07
query31	2.84	0.40	0.39
query32	3.23	0.48	0.49
query33	3.02	3.03	3.05
query34	17.02	4.51	4.52
query35	4.60	4.59	4.55
query36	0.67	0.51	0.49
query37	0.10	0.07	0.07
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.17	0.14	0.14
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.32 s
Total hot run time: 33.51 s

@dataroaring dataroaring merged commit 84a21bd into apache:branch-3.0 Jan 24, 2025
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring Awaiting requested review from dataroaring

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@bobhan1 @hello-stephen @doris-robot @dataroaring