Skip to content

Conversation

@suxiaogang223
Copy link
Contributor

bp #47506

remove unnecessary fields of orc_reader:
- remove `_col_name_to_file_col_name_low_case` by storing original field
name in `type_map`
- add comment to describe the the functionality of these mappings
@suxiaogang223 suxiaogang223 requested a review from morrySnow as a code owner June 20, 2025 07:32
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39780 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1f8faa7f1b2337fd18c0425ad0605e1896115b11, data reload: false

------ Round 1 ----------------------------------
q1	17589	6733	6610	6610
q2	2065	170	172	170
q3	10586	1081	1275	1081
q4	10510	749	832	749
q5	7716	2902	2799	2799
q6	217	134	131	131
q7	962	616	612	612
q8	9351	1929	2011	1929
q9	6592	6398	6379	6379
q10	7058	2249	2244	2244
q11	470	260	252	252
q12	392	206	204	204
q13	17795	2977	3050	2977
q14	244	214	215	214
q15	532	459	465	459
q16	527	411	389	389
q17	977	535	576	535
q18	7532	6881	6686	6686
q19	1396	1064	1029	1029
q20	470	205	192	192
q21	3941	3144	3195	3144
q22	1137	1036	995	995
Total cold run time: 108059 ms
Total hot run time: 39780 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6560	6588	6603	6588
q2	323	230	248	230
q3	2902	2709	2866	2709
q4	2045	1783	1846	1783
q5	5746	5763	5719	5719
q6	205	126	126	126
q7	2201	1782	1832	1782
q8	3370	3557	3546	3546
q9	8897	8827	8898	8827
q10	3557	3516	3506	3506
q11	605	492	497	492
q12	782	615	633	615
q13	10020	3215	3106	3106
q14	309	292	288	288
q15	521	468	446	446
q16	493	468	440	440
q17	1863	1622	1605	1605
q18	8119	7842	7799	7799
q19	1659	1505	1524	1505
q20	2074	1861	1884	1861
q21	5230	5168	5042	5042
q22	1140	1074	1052	1052
Total cold run time: 68621 ms
Total hot run time: 59067 ms

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 41.15% (10901/26488)
Line Coverage 31.96% (93220/291682)
Region Coverage 31.04% (48089/154935)
Branch Coverage 27.51% (24631/89538)

@doris-robot
Copy link

TPC-DS: Total hot run time: 196398 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1f8faa7f1b2337fd18c0425ad0605e1896115b11, data reload: false

query1	1271	936	909	909
query2	6218	1850	1834	1834
query3	10993	4453	4470	4453
query4	61783	30059	23587	23587
query5	5208	450	454	450
query6	412	175	197	175
query7	5483	318	306	306
query8	313	226	229	226
query9	8459	2592	2568	2568
query10	460	266	253	253
query11	17566	15242	15736	15242
query12	160	105	108	105
query13	1501	431	450	431
query14	10587	6817	7478	6817
query15	210	178	184	178
query16	7066	490	483	483
query17	1257	615	579	579
query18	1930	318	351	318
query19	209	170	169	169
query20	118	115	110	110
query21	205	108	101	101
query22	4679	4344	4691	4344
query23	34911	34306	33913	33913
query24	6183	2892	2891	2891
query25	546	403	409	403
query26	667	163	178	163
query27	2046	364	361	361
query28	4196	2169	2177	2169
query29	684	443	438	438
query30	239	159	156	156
query31	1038	806	829	806
query32	69	58	58	58
query33	475	292	293	292
query34	940	509	529	509
query35	853	749	716	716
query36	1081	936	928	928
query37	105	72	69	69
query38	4084	3937	4020	3937
query39	1514	1493	1461	1461
query40	212	96	105	96
query41	49	50	50	50
query42	115	108	112	108
query43	532	471	482	471
query44	1169	823	813	813
query45	186	170	170	170
query46	1165	731	736	731
query47	2006	1891	1946	1891
query48	476	373	377	373
query49	735	394	396	394
query50	852	422	416	416
query51	7412	7237	7216	7216
query52	101	88	91	88
query53	260	188	183	183
query54	565	465	452	452
query55	84	82	76	76
query56	270	248	237	237
query57	1315	1203	1208	1203
query58	225	208	213	208
query59	3165	3008	2984	2984
query60	300	258	269	258
query61	117	110	114	110
query62	785	662	675	662
query63	219	188	184	184
query64	1469	684	657	657
query65	3296	3154	3171	3154
query66	701	297	298	297
query67	15798	15569	15526	15526
query68	4157	573	570	570
query69	424	269	262	262
query70	1162	1139	1116	1116
query71	346	257	263	257
query72	6381	4139	3958	3958
query73	749	347	350	347
query74	10167	9000	9187	9000
query75	3347	2650	2635	2635
query76	1913	1066	1048	1048
query77	504	264	281	264
query78	10520	9620	9640	9620
query79	1431	590	599	590
query80	927	470	454	454
query81	489	219	228	219
query82	1271	95	87	87
query83	253	152	152	152
query84	287	84	80	80
query85	1007	311	297	297
query86	351	297	302	297
query87	4333	4229	4240	4229
query88	3548	2407	2342	2342
query89	430	285	291	285
query90	1990	185	184	184
query91	181	148	149	148
query92	67	48	55	48
query93	1884	550	554	550
query94	748	304	296	296
query95	356	254	254	254
query96	611	281	277	277
query97	3265	3127	3186	3127
query98	204	203	198	198
query99	1610	1291	1300	1291
Total cold run time: 314211 ms
Total hot run time: 196398 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.39 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1f8faa7f1b2337fd18c0425ad0605e1896115b11, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.02
query3	0.24	0.06	0.06
query4	1.62	0.11	0.10
query5	0.51	0.51	0.50
query6	1.14	0.74	0.73
query7	0.02	0.02	0.02
query8	0.06	0.04	0.02
query9	0.56	0.51	0.48
query10	0.54	0.55	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.59	0.59
query14	0.79	0.78	0.80
query15	0.84	0.81	0.84
query16	0.38	0.36	0.39
query17	1.05	1.03	0.98
query18	0.24	0.22	0.22
query19	1.88	1.82	1.76
query20	0.01	0.02	0.01
query21	15.39	0.58	0.57
query22	2.52	1.94	1.75
query23	17.05	0.94	0.81
query24	3.09	0.35	2.23
query25	0.09	0.23	0.11
query26	0.65	0.14	0.14
query27	0.05	0.04	0.04
query28	10.06	0.49	0.47
query29	12.59	3.26	3.25
query30	0.25	0.06	0.06
query31	2.85	0.38	0.38
query32	3.26	0.46	0.46
query33	3.00	2.96	2.96
query34	17.12	4.50	4.56
query35	4.54	4.51	4.54
query36	0.68	0.49	0.49
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.15	0.12	0.13
query41	0.07	0.02	0.02
query42	0.03	0.03	0.03
query43	0.03	0.03	0.03
Total cold run time: 104.49 s
Total hot run time: 29.39 s

@morrySnow morrySnow merged commit a3385f3 into apache:branch-3.1 Jun 20, 2025
20 of 22 checks passed
@suxiaogang223 suxiaogang223 deleted the fix_orc_reader_31 branch June 27, 2025 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants