Skip to content

Conversation

@hubgeter
Copy link
Contributor

@hubgeter hubgeter commented Jan 13, 2026

What problem does this PR solve?

Related PR: #58785

Problem Summary:
The performance rollback was introduced in #58785. The reason is that, prior to #58785, reading ordinary columns did not require filling and saving the RL DL. #58785 combined the logic for reading ordinary columns from a struct with the logic for reading ordinary columns, and filled and saved the RL DL to populate the null map information of the struct. This PR re-separates the reading logic.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 13, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@hubgeter
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31757 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ce50a17bdc42dd83a180ba5e7dd62f46374e2439, data reload: false

------ Round 1 ----------------------------------
q1	17603	4193	4015	4015
q2	2034	349	237	237
q3	10157	1228	725	725
q4	10226	899	336	336
q5	7533	2077	1843	1843
q6	187	170	136	136
q7	927	799	654	654
q8	9261	1424	1112	1112
q9	4895	4648	4483	4483
q10	6789	1808	1426	1426
q11	513	299	279	279
q12	669	751	566	566
q13	17801	3878	3114	3114
q14	288	296	292	292
q15	582	517	505	505
q16	662	686	659	659
q17	657	760	525	525
q18	6520	6462	6738	6462
q19	1194	1049	625	625
q20	425	380	273	273
q21	3190	2618	2475	2475
q22	1143	1113	1015	1015
Total cold run time: 103256 ms
Total hot run time: 31757 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4360	4310	4238	4238
q2	330	389	303	303
q3	2287	2723	2515	2515
q4	1382	1940	1378	1378
q5	4503	4399	4357	4357
q6	206	172	134	134
q7	2032	1939	1780	1780
q8	2543	2346	2424	2346
q9	7086	7197	7266	7197
q10	2642	2662	2215	2215
q11	555	461	464	461
q12	713	790	605	605
q13	3347	3760	3086	3086
q14	274	282	259	259
q15	522	495	485	485
q16	623	659	616	616
q17	1078	1226	1279	1226
q18	7733	7400	7446	7400
q19	849	791	824	791
q20	1880	2021	1836	1836
q21	4634	4350	4209	4209
q22	1085	1053	1021	1021
Total cold run time: 50664 ms
Total hot run time: 48458 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172742 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ce50a17bdc42dd83a180ba5e7dd62f46374e2439, data reload: false

query5	4438	630	508	508
query6	329	231	207	207
query7	4219	455	256	256
query8	334	240	235	235
query9	8685	2836	2852	2836
query10	522	378	313	313
query11	15074	14943	14829	14829
query12	175	118	113	113
query13	1272	455	367	367
query14	6122	2909	2689	2689
query14_1	2593	2588	2564	2564
query15	195	189	167	167
query16	997	478	455	455
query17	1082	677	579	579
query18	2553	444	335	335
query19	248	228	196	196
query20	124	118	120	118
query21	219	137	119	119
query22	4008	4228	3817	3817
query23	15968	15780	15416	15416
query23_1	15440	15570	15655	15570
query24	7178	1531	1147	1147
query24_1	1159	1141	1162	1141
query25	558	471	417	417
query26	1242	272	156	156
query27	2756	438	265	265
query28	4559	2133	2125	2125
query29	788	547	447	447
query30	303	243	213	213
query31	806	656	543	543
query32	96	76	77	76
query33	544	343	314	314
query34	891	867	516	516
query35	717	767	678	678
query36	906	928	821	821
query37	142	99	88	88
query38	2670	2680	2651	2651
query39	782	741	727	727
query39_1	726	726	705	705
query40	218	135	122	122
query41	73	66	65	65
query42	110	105	104	104
query43	465	440	397	397
query44	1311	734	744	734
query45	186	182	180	180
query46	821	925	561	561
query47	1418	1482	1430	1430
query48	310	326	237	237
query49	616	439	348	348
query50	608	273	210	210
query51	3825	3808	3722	3722
query52	105	109	101	101
query53	294	323	273	273
query54	309	289	274	274
query55	85	82	75	75
query56	323	320	334	320
query57	1010	1039	960	960
query58	330	263	259	259
query59	1983	2160	2044	2044
query60	325	317	312	312
query61	153	157	153	153
query62	388	368	309	309
query63	300	260	257	257
query64	4936	1275	970	970
query65	3749	3746	3712	3712
query66	1422	422	315	315
query67	14911	15084	15660	15084
query68	7237	982	691	691
query69	503	349	325	325
query70	1076	837	945	837
query71	359	307	287	287
query72	5694	3332	3408	3332
query73	769	723	298	298
query74	8791	8798	8619	8619
query75	2784	2786	2459	2459
query76	3314	1060	632	632
query77	517	392	304	304
query78	9739	9786	9264	9264
query79	1363	890	548	548
query80	652	587	486	486
query81	528	269	229	229
query82	213	147	109	109
query83	263	260	245	245
query84	261	115	90	90
query85	890	522	442	442
query86	442	297	298	297
query87	2856	2845	2726	2726
query88	2853	2121	2113	2113
query89	379	355	319	319
query90	2152	166	159	159
query91	168	165	141	141
query92	89	73	71	71
query93	1085	902	541	541
query94	580	278	309	278
query95	583	333	371	333
query96	549	471	194	194
query97	2317	2372	2279	2279
query98	232	201	200	200
query99	607	603	534	534
Total cold run time: 250428 ms
Total hot run time: 172742 ms

@hubgeter
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31866 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1473e5e9e68b778a4ffa17951a40e77c423b3158, data reload: false

------ Round 1 ----------------------------------
q1	17593	4224	4068	4068
q2	2048	368	253	253
q3	10122	1225	690	690
q4	10220	873	313	313
q5	7466	2078	1802	1802
q6	187	166	139	139
q7	927	800	648	648
q8	9266	1424	1083	1083
q9	4853	4668	4540	4540
q10	6790	1799	1384	1384
q11	524	309	282	282
q12	709	735	604	604
q13	17799	3830	3075	3075
q14	284	306	289	289
q15	582	514	502	502
q16	697	680	650	650
q17	660	818	452	452
q18	6973	6499	6921	6499
q19	1177	1056	649	649
q20	423	369	254	254
q21	3221	2694	2620	2620
q22	1098	1118	1070	1070
Total cold run time: 103619 ms
Total hot run time: 31866 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4329	4288	4286	4286
q2	323	414	338	338
q3	2298	2846	2389	2389
q4	1460	1873	1442	1442
q5	4636	4499	4379	4379
q6	225	170	122	122
q7	1985	1836	1792	1792
q8	2534	2370	2322	2322
q9	7223	7408	7029	7029
q10	2539	2765	2287	2287
q11	551	500	483	483
q12	731	750	684	684
q13	3644	3946	3156	3156
q14	295	294	261	261
q15	517	509	488	488
q16	621	669	630	630
q17	1083	1269	1298	1269
q18	7399	7311	7179	7179
q19	834	792	800	792
q20	1894	1971	1796	1796
q21	4551	4268	4092	4092
q22	1070	1027	966	966
Total cold run time: 50742 ms
Total hot run time: 48182 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173527 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1473e5e9e68b778a4ffa17951a40e77c423b3158, data reload: false

query5	4545	632	504	504
query6	334	227	233	227
query7	4230	471	253	253
query8	355	246	240	240
query9	8727	2915	2878	2878
query10	509	373	342	342
query11	15129	15260	14752	14752
query12	183	120	129	120
query13	1267	501	374	374
query14	6438	3060	2983	2983
query14_1	2703	2708	2684	2684
query15	193	196	177	177
query16	982	482	455	455
query17	1124	683	593	593
query18	2610	441	341	341
query19	242	235	234	234
query20	122	117	116	116
query21	211	134	113	113
query22	4016	4038	3987	3987
query23	15956	15537	15251	15251
query23_1	15452	15464	15435	15435
query24	7162	1542	1164	1164
query24_1	1166	1160	1166	1160
query25	535	447	405	405
query26	1243	269	164	164
query27	2749	431	266	266
query28	4566	2159	2173	2159
query29	747	522	422	422
query30	310	244	209	209
query31	818	635	554	554
query32	90	81	77	77
query33	515	350	294	294
query34	885	887	521	521
query35	732	757	701	701
query36	888	934	927	927
query37	131	102	83	83
query38	2686	2659	2711	2659
query39	787	741	750	741
query39_1	710	716	715	715
query40	221	133	117	117
query41	67	62	63	62
query42	106	105	102	102
query43	449	463	444	444
query44	1351	728	735	728
query45	188	197	178	178
query46	821	946	585	585
query47	1405	1511	1370	1370
query48	310	328	239	239
query49	614	427	353	353
query50	626	284	213	213
query51	3769	3763	3841	3763
query52	106	109	106	106
query53	296	334	274	274
query54	300	274	277	274
query55	82	81	78	78
query56	340	317	316	316
query57	997	1016	909	909
query58	271	263	266	263
query59	2143	2021	1936	1936
query60	334	335	335	335
query61	159	160	156	156
query62	417	384	312	312
query63	299	275	273	273
query64	4812	1388	1079	1079
query65	3799	3642	3751	3642
query66	1434	455	339	339
query67	15006	14897	14827	14827
query68	3846	1022	718	718
query69	519	360	339	339
query70	1071	887	830	830
query71	355	329	307	307
query72	5946	3489	3460	3460
query73	767	721	307	307
query74	8769	8785	8667	8667
query75	2805	2798	2434	2434
query76	3443	1050	678	678
query77	511	400	312	312
query78	9862	9915	9204	9204
query79	1458	874	558	558
query80	1574	577	501	501
query81	554	270	237	237
query82	413	145	109	109
query83	360	257	243	243
query84	250	113	100	100
query85	909	512	455	455
query86	411	300	294	294
query87	2885	2838	2761	2761
query88	2941	2185	2147	2147
query89	397	361	336	336
query90	1952	173	170	170
query91	172	177	143	143
query92	83	71	69	69
query93	1057	921	531	531
query94	629	341	301	301
query95	593	341	381	341
query96	543	474	197	197
query97	2332	2381	2332	2332
query98	228	201	197	197
query99	576	610	505	505
Total cold run time: 249800 ms
Total hot run time: 173527 ms

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 79.17% (19/24) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.04% (18992/35809)
Line Coverage 39.08% (176022/450430)
Region Coverage 33.68% (136348/404879)
Branch Coverage 34.69% (58929/169850)

@hubgeter hubgeter force-pushed the fix_parquet_falt_slow branch from 1473e5e to 770f06a Compare January 14, 2026 02:50
@hubgeter
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31771 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 770f06a8cd89fc9f6bc01894b335847dd4861cbb, data reload: false

------ Round 1 ----------------------------------
q1	17704	4229	4805	4229
q2	2207	355	247	247
q3	10177	1260	737	737
q4	10238	851	321	321
q5	7542	2098	1816	1816
q6	193	174	140	140
q7	930	800	645	645
q8	9265	1377	1200	1200
q9	4826	4491	4551	4491
q10	7062	1807	1423	1423
q11	548	300	293	293
q12	693	744	596	596
q13	17788	3822	3133	3133
q14	289	308	273	273
q15	608	516	505	505
q16	676	687	620	620
q17	639	751	571	571
q18	6605	6469	6450	6450
q19	1102	976	596	596
q20	385	352	244	244
q21	3065	2416	2265	2265
q22	1051	1003	976	976
Total cold run time: 103593 ms
Total hot run time: 31771 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4110	4013	4022	4013
q2	329	405	306	306
q3	2081	2616	2209	2209
q4	1319	1764	1286	1286
q5	4062	3963	4054	3963
q6	211	171	133	133
q7	1814	1836	2044	1836
q8	2625	2463	2460	2460
q9	7298	7122	7126	7122
q10	2483	2793	2271	2271
q11	571	477	466	466
q12	729	804	625	625
q13	3606	4148	3461	3461
q14	292	317	305	305
q15	552	606	517	517
q16	655	680	691	680
q17	1260	1354	1356	1354
q18	7880	7993	7646	7646
q19	900	854	855	854
q20	1991	2104	1981	1981
q21	4795	4589	4347	4347
q22	1054	1055	996	996
Total cold run time: 50617 ms
Total hot run time: 48831 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174300 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 770f06a8cd89fc9f6bc01894b335847dd4861cbb, data reload: false

query5	4445	625	496	496
query6	356	256	221	221
query7	4233	471	245	245
query8	388	254	248	248
query9	8730	2856	2899	2856
query10	499	389	328	328
query11	15258	15208	14965	14965
query12	187	122	118	118
query13	1278	488	374	374
query14	6279	2993	2851	2851
query14_1	2719	2726	2688	2688
query15	212	198	177	177
query16	1110	487	471	471
query17	1118	686	586	586
query18	2494	447	345	345
query19	231	227	230	227
query20	123	120	116	116
query21	305	139	119	119
query22	4114	3980	4042	3980
query23	16067	15604	15353	15353
query23_1	15542	15420	15510	15420
query24	7177	1554	1166	1166
query24_1	1194	1159	1192	1159
query25	541	450	397	397
query26	1246	273	194	194
query27	2706	429	279	279
query28	4546	2156	2137	2137
query29	760	526	424	424
query30	354	240	212	212
query31	778	630	549	549
query32	79	75	71	71
query33	536	339	310	310
query34	911	879	533	533
query35	730	747	681	681
query36	911	922	885	885
query37	144	96	83	83
query38	2729	2733	2657	2657
query39	771	751	728	728
query39_1	725	714	728	714
query40	219	138	120	120
query41	66	62	63	62
query42	106	103	105	103
query43	463	442	420	420
query44	1317	737	735	735
query45	188	188	178	178
query46	830	964	581	581
query47	1369	1468	1395	1395
query48	298	329	243	243
query49	608	422	345	345
query50	619	275	207	207
query51	3768	3731	3729	3729
query52	103	107	98	98
query53	303	332	268	268
query54	308	273	267	267
query55	81	78	78	78
query56	310	316	329	316
query57	988	1034	960	960
query58	282	276	262	262
query59	2113	2223	2169	2169
query60	339	329	323	323
query61	165	165	156	156
query62	388	352	313	313
query63	302	266	269	266
query64	4931	1328	992	992
query65	3808	3693	3699	3693
query66	1445	430	325	325
query67	15405	14959	15320	14959
query68	3292	990	718	718
query69	489	366	316	316
query70	1005	918	921	918
query71	346	320	296	296
query72	5668	3566	3534	3534
query73	760	722	306	306
query74	8835	8810	8573	8573
query75	2837	2838	2491	2491
query76	3243	1061	691	691
query77	578	397	311	311
query78	9738	9856	9174	9174
query79	967	897	572	572
query80	656	638	533	533
query81	497	270	241	241
query82	412	155	115	115
query83	273	269	255	255
query84	259	116	96	96
query85	955	515	465	465
query86	347	301	323	301
query87	2860	2886	2788	2788
query88	2999	2165	2151	2151
query89	396	354	333	333
query90	1957	169	165	165
query91	168	161	138	138
query92	76	75	74	74
query93	1045	894	529	529
query94	594	331	297	297
query95	581	389	321	321
query96	579	490	201	201
query97	2362	2394	2331	2331
query98	218	203	199	199
query99	600	576	591	576
Total cold run time: 248689 ms
Total hot run time: 174300 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.1 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 770f06a8cd89fc9f6bc01894b335847dd4861cbb, data reload: false

query1	0.06	0.04	0.05
query2	0.14	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.11	0.12
query5	0.29	0.24	0.25
query6	1.15	0.66	0.65
query7	0.03	0.03	0.02
query8	0.06	0.04	0.05
query9	0.59	0.50	0.50
query10	0.55	0.54	0.54
query11	0.17	0.10	0.10
query12	0.14	0.11	0.11
query13	0.59	0.58	0.60
query14	0.97	0.95	0.95
query15	0.81	0.77	0.79
query16	0.41	0.42	0.43
query17	1.09	1.07	1.08
query18	0.23	0.22	0.21
query19	1.97	1.89	1.91
query20	0.02	0.01	0.01
query21	15.47	0.26	0.14
query22	4.90	0.05	0.05
query23	15.86	0.29	0.10
query24	2.52	0.32	0.55
query25	0.09	0.11	0.08
query26	0.15	0.13	0.13
query27	0.08	0.06	0.09
query28	4.41	1.08	0.89
query29	12.56	3.91	3.17
query30	0.28	0.13	0.11
query31	2.82	0.65	0.41
query32	3.24	0.56	0.46
query33	2.98	2.99	3.00
query34	15.97	5.10	4.48
query35	4.49	4.48	4.49
query36	0.66	0.50	0.49
query37	0.15	0.06	0.06
query38	0.08	0.04	0.04
query39	0.06	0.03	0.02
query40	0.17	0.14	0.14
query41	0.10	0.03	0.03
query42	0.06	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.28 s
Total hot run time: 27.1 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 80.00% (20/25) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.01% (18981/35809)
Line Coverage 39.06% (175926/450437)
Region Coverage 33.66% (136304/404886)
Branch Coverage 34.68% (58899/169855)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (25/25) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.22% (25341/35091)
Line Coverage 59.41% (267254/449847)
Region Coverage 53.92% (220758/409391)
Branch Coverage 55.77% (95148/170616)

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 15, 2026
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit 52c75e7 into apache:master Jan 16, 2026
31 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.x dev/4.0.x-conflict reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants