[fix](orc) check all the cases before build_search_argument #44615

suxiaogang223 · 2024-11-26T09:11:43Z

What problem does this PR solve?

In the old logic, the check_expr_can_push_down function does not check whether the orc::Literal are constructed successfully, but only checks during build_search_argument. However, if it is found that the orc::Literal fails to be constructed after builder->startNot, it will fail because the builder cannot end startNot.
Therefore, we advance the behavior of constructing orc::Literal to the check_expr_can_push_down function and save the result to the map, so that it will never fail in the build_search_argument phase.

Related PR: #43255

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

doris-robot · 2024-11-26T09:11:49Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

suxiaogang223 · 2024-11-26T18:05:54Z

run buildall

doris-robot · 2024-11-26T18:49:55Z

TPC-H: Total hot run time: 39864 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 55170503f5afd25455e0900a2575d77ca1976281, data reload: false

------ Round 1 ----------------------------------
q1	17582	7439	7287	7287
q2	2039	173	167	167
q3	10775	1144	1160	1144
q4	10533	740	675	675
q5	7607	2721	2697	2697
q6	235	148	146	146
q7	972	624	614	614
q8	9238	1832	1925	1832
q9	6566	6377	6386	6377
q10	7045	2287	2301	2287
q11	474	266	249	249
q12	418	213	213	213
q13	17795	3017	3014	3014
q14	242	210	212	210
q15	576	542	518	518
q16	666	585	594	585
q17	971	606	533	533
q18	7181	6870	6796	6796
q19	1349	1011	936	936
q20	470	179	179	179
q21	4141	3299	3091	3091
q22	388	320	314	314
Total cold run time: 107263 ms
Total hot run time: 39864 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7224	7264	7195	7195
q2	330	228	233	228
q3	2927	2826	2946	2826
q4	2052	1794	1795	1794
q5	5731	5689	5671	5671
q6	248	138	140	138
q7	2255	1865	1833	1833
q8	3415	3569	3501	3501
q9	8837	8890	8884	8884
q10	3606	3531	3543	3531
q11	604	513	491	491
q12	828	629	641	629
q13	11335	3255	3177	3177
q14	305	279	270	270
q15	596	524	524	524
q16	696	637	634	634
q17	1846	1613	1596	1596
q18	8279	7876	7508	7508
q19	1718	1665	1533	1533
q20	2120	1853	1895	1853
q21	5483	5415	5404	5404
q22	636	565	567	565
Total cold run time: 71071 ms
Total hot run time: 59785 ms

doris-robot · 2024-11-26T19:02:33Z

TPC-DS: Total hot run time: 197432 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 55170503f5afd25455e0900a2575d77ca1976281, data reload: false

query1	1215	927	979	927
query2	6241	2109	2092	2092
query3	10803	4034	3925	3925
query4	67543	31932	23797	23797
query5	4972	473	465	465
query6	305	184	174	174
query7	4863	304	296	296
query8	288	234	232	232
query9	6250	2774	2765	2765
query10	422	273	237	237
query11	15571	15203	15882	15203
query12	167	103	107	103
query13	1040	432	420	420
query14	10529	7549	7322	7322
query15	218	183	190	183
query16	7168	458	480	458
query17	1054	585	596	585
query18	1872	316	319	316
query19	234	163	155	155
query20	130	115	118	115
query21	206	109	109	109
query22	4923	4673	4595	4595
query23	34747	34167	34453	34167
query24	5544	2529	2482	2482
query25	500	403	389	389
query26	680	145	149	145
query27	2255	287	299	287
query28	4244	2541	2505	2505
query29	691	444	428	428
query30	215	155	153	153
query31	1013	850	847	847
query32	71	59	59	59
query33	454	287	288	287
query34	939	521	535	521
query35	878	747	749	747
query36	1107	982	1001	982
query37	123	74	77	74
query38	4483	4390	4427	4390
query39	1530	1465	1489	1465
query40	210	107	105	105
query41	50	46	48	46
query42	115	140	103	103
query43	556	506	511	506
query44	1172	834	842	834
query45	183	172	167	167
query46	1159	686	710	686
query47	2063	1898	1949	1898
query48	409	313	313	313
query49	716	398	402	398
query50	882	396	399	396
query51	7397	7223	7097	7097
query52	100	88	88	88
query53	249	181	177	177
query54	519	410	390	390
query55	74	78	78	78
query56	241	280	234	234
query57	1288	1197	1164	1164
query58	212	215	211	211
query59	3223	3250	3076	3076
query60	272	247	245	245
query61	106	109	108	108
query62	809	675	671	671
query63	219	191	196	191
query64	1375	709	622	622
query65	3275	3216	3188	3188
query66	695	298	304	298
query67	16172	15708	15665	15665
query68	3632	588	561	561
query69	443	258	260	258
query70	1239	1146	1164	1146
query71	405	248	245	245
query72	6495	4087	4045	4045
query73	784	366	374	366
query74	9873	8960	9090	8960
query75	3415	2658	2666	2658
query76	1651	1052	1126	1052
query77	502	290	294	290
query78	10478	9516	9409	9409
query79	1599	604	611	604
query80	843	435	432	432
query81	517	231	223	223
query82	1281	117	123	117
query83	207	147	157	147
query84	283	72	69	69
query85	1012	296	285	285
query86	351	300	319	300
query87	4881	4597	4537	4537
query88	3779	2236	2231	2231
query89	434	297	297	297
query90	2063	190	187	187
query91	145	100	104	100
query92	65	50	53	50
query93	2220	531	542	531
query94	812	300	300	300
query95	345	256	252	252
query96	625	277	290	277
query97	2891	2687	2647	2647
query98	221	196	201	196
query99	2010	1303	1304	1303
Total cold run time: 314730 ms
Total hot run time: 197432 ms

doris-robot · 2024-11-26T19:08:03Z

ClickBench: Total hot run time: 32.52 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 55170503f5afd25455e0900a2575d77ca1976281, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.04
query3	0.24	0.08	0.07
query4	1.60	0.10	0.10
query5	0.43	0.42	0.41
query6	1.15	0.65	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.57	0.50	0.51
query10	0.55	0.56	0.55
query11	0.14	0.10	0.10
query12	0.15	0.12	0.11
query13	0.60	0.62	0.60
query14	2.85	2.73	2.72
query15	0.91	0.83	0.83
query16	0.38	0.38	0.40
query17	1.05	1.07	1.04
query18	0.22	0.21	0.23
query19	1.97	1.88	2.01
query20	0.01	0.01	0.01
query21	15.37	0.61	0.57
query22	2.73	2.02	1.43
query23	16.96	1.12	0.81
query24	2.97	2.07	1.19
query25	0.21	0.23	0.05
query26	0.63	0.13	0.13
query27	0.04	0.04	0.04
query28	9.88	1.11	1.08
query29	12.54	3.25	3.26
query30	0.26	0.08	0.07
query31	2.83	0.38	0.38
query32	3.27	0.46	0.46
query33	2.98	3.08	3.06
query34	17.18	4.44	4.48
query35	4.48	4.55	4.48
query36	0.69	0.51	0.51
query37	0.10	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.14	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.47 s
Total hot run time: 32.52 s

doris-robot · 2024-11-26T19:21:48Z

TeamCity be ut coverage result:
Function Coverage: 38.35% (9977/26017)
Line Coverage: 29.45% (83525/283588)
Region Coverage: 28.59% (42962/150293)
Branch Coverage: 25.18% (21816/86630)
Coverage Report: http://coverage.selectdb-in.cc/coverage/55170503f5afd25455e0900a2575d77ca1976281_55170503f5afd25455e0900a2575d77ca1976281/report/index.html

suxiaogang223 · 2024-11-27T02:11:57Z

run buildall

suxiaogang223 · 2024-11-27T02:27:30Z

run buildall

doris-robot · 2024-11-27T03:07:17Z

TPC-H: Total hot run time: 39745 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f06e0b50cb695b0c9af5e3ef1b627b545846ddd4, data reload: false

------ Round 1 ----------------------------------
q1	17601	7455	7266	7266
q2	2045	169	186	169
q3	10564	1094	1198	1094
q4	10589	694	747	694
q5	7603	2661	2732	2661
q6	240	146	149	146
q7	1002	631	601	601
q8	9253	1818	1926	1818
q9	6519	6385	6402	6385
q10	7042	2296	2301	2296
q11	466	261	258	258
q12	397	220	217	217
q13	17790	3069	2977	2977
q14	243	206	206	206
q15	567	545	532	532
q16	671	607	584	584
q17	974	522	567	522
q18	7198	6634	6696	6634
q19	1328	1017	960	960
q20	481	179	181	179
q21	3971	3332	3234	3234
q22	390	330	312	312
Total cold run time: 106934 ms
Total hot run time: 39745 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7219	7204	7216	7204
q2	329	231	240	231
q3	2899	2819	2972	2819
q4	2044	1791	1817	1791
q5	5715	5699	5618	5618
q6	231	146	143	143
q7	2284	1819	1797	1797
q8	3413	3537	3508	3508
q9	8890	8960	8853	8853
q10	3580	3541	3572	3541
q11	603	498	505	498
q12	816	617	625	617
q13	12816	3224	3271	3224
q14	303	269	283	269
q15	576	548	527	527
q16	714	664	646	646
q17	1851	1621	1626	1621
q18	8341	7701	7515	7515
q19	1709	1562	1549	1549
q20	2135	1851	1888	1851
q21	5579	5487	5505	5487
q22	616	583	548	548
Total cold run time: 72663 ms
Total hot run time: 59857 ms

doris-robot · 2024-11-27T03:07:19Z

TeamCity be ut coverage result:
Function Coverage: 38.36% (9980/26017)
Line Coverage: 29.47% (83562/283582)
Region Coverage: 28.59% (42969/150288)
Branch Coverage: 25.20% (21833/86626)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f06e0b50cb695b0c9af5e3ef1b627b545846ddd4_f06e0b50cb695b0c9af5e3ef1b627b545846ddd4/report/index.html

doris-robot · 2024-11-27T03:19:57Z

TPC-DS: Total hot run time: 197189 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f06e0b50cb695b0c9af5e3ef1b627b545846ddd4, data reload: false

query1	1255	949	894	894
query2	6261	2072	2050	2050
query3	10858	4186	3976	3976
query4	67679	28198	23565	23565
query5	4946	455	447	447
query6	431	204	189	189
query7	5481	309	297	297
query8	311	234	226	226
query9	8600	2752	2751	2751
query10	437	261	245	245
query11	16913	15281	15878	15281
query12	152	102	110	102
query13	1451	421	420	420
query14	10883	7607	7555	7555
query15	205	183	187	183
query16	7119	495	514	495
query17	1025	584	566	566
query18	1833	309	297	297
query19	207	160	151	151
query20	114	114	111	111
query21	206	104	107	104
query22	4712	4316	4571	4316
query23	34995	34511	34447	34447
query24	5486	2613	2591	2591
query25	472	401	414	401
query26	653	163	156	156
query27	1871	296	283	283
query28	4241	2515	2457	2457
query29	680	419	417	417
query30	215	151	148	148
query31	1038	810	855	810
query32	64	54	59	54
query33	454	287	302	287
query34	924	533	526	526
query35	863	749	744	744
query36	1133	962	978	962
query37	128	74	81	74
query38	4551	4483	4430	4430
query39	1559	1480	1574	1480
query40	210	103	105	103
query41	49	48	47	47
query42	120	104	102	102
query43	568	520	512	512
query44	1222	823	803	803
query45	190	175	167	167
query46	1129	706	722	706
query47	2017	1895	1920	1895
query48	438	322	337	322
query49	725	395	414	395
query50	839	421	393	393
query51	7381	7180	7139	7139
query52	102	90	87	87
query53	259	178	180	178
query54	525	401	415	401
query55	80	75	77	75
query56	260	234	264	234
query57	1299	1174	1153	1153
query58	218	229	223	223
query59	3396	2995	2976	2976
query60	288	269	249	249
query61	116	105	122	105
query62	770	673	659	659
query63	218	194	194	194
query64	1385	658	678	658
query65	3255	3222	3235	3222
query66	775	298	294	294
query67	16061	15420	15740	15420
query68	3783	580	581	580
query69	430	258	258	258
query70	1232	1139	1060	1060
query71	356	263	247	247
query72	6374	4021	4120	4021
query73	775	372	366	366
query74	10191	8970	9068	8970
query75	3391	2709	2657	2657
query76	1791	1105	1059	1059
query77	512	317	275	275
query78	10487	9409	9348	9348
query79	1675	612	617	612
query80	1439	421	443	421
query81	531	231	230	230
query82	1320	121	118	118
query83	272	158	158	158
query84	282	76	71	71
query85	1032	290	296	290
query86	416	276	306	276
query87	4723	4567	4627	4567
query88	3536	2249	2212	2212
query89	417	303	306	303
query90	1985	190	188	188
query91	139	103	106	103
query92	63	57	53	53
query93	2065	541	563	541
query94	845	298	305	298
query95	356	264	256	256
query96	615	287	275	275
query97	2874	2696	2679	2679
query98	224	196	196	196
query99	1724	1318	1301	1301
Total cold run time: 320156 ms
Total hot run time: 197189 ms

doris-robot · 2024-11-27T03:25:26Z

ClickBench: Total hot run time: 33.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f06e0b50cb695b0c9af5e3ef1b627b545846ddd4, data reload: false

query1	0.04	0.03	0.02
query2	0.06	0.04	0.03
query3	0.23	0.07	0.08
query4	1.60	0.10	0.10
query5	0.42	0.42	0.42
query6	1.14	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.59	0.52	0.51
query10	0.54	0.55	0.57
query11	0.14	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.61	0.60
query14	2.69	2.82	2.74
query15	0.90	0.84	0.82
query16	0.39	0.39	0.39
query17	1.06	1.05	1.06
query18	0.22	0.22	0.21
query19	1.97	1.88	2.02
query20	0.02	0.01	0.01
query21	15.36	0.60	0.57
query22	2.85	2.20	1.96
query23	16.96	1.15	0.70
query24	3.02	1.45	1.15
query25	0.14	0.19	0.12
query26	0.49	0.14	0.14
query27	0.04	0.05	0.04
query28	10.07	1.11	1.07
query29	12.53	3.25	3.23
query30	0.25	0.06	0.06
query31	2.89	0.40	0.38
query32	3.28	0.49	0.48
query33	3.02	3.03	3.09
query34	16.91	4.50	4.58
query35	4.60	4.58	4.55
query36	0.67	0.49	0.48
query37	0.09	0.06	0.07
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.15	0.13	0.13
query41	0.07	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.34 s
Total hot run time: 33.08 s

suxiaogang223 · 2024-11-27T15:18:35Z

run external

suxiaogang223 · 2024-11-27T18:54:31Z

run buildall

doris-robot · 2024-11-27T19:32:48Z

TeamCity be ut coverage result:
Function Coverage: 38.34% (9976/26018)
Line Coverage: 29.45% (83573/283743)
Region Coverage: 28.57% (42967/150387)
Branch Coverage: 25.19% (21829/86648)
Coverage Report: http://coverage.selectdb-in.cc/coverage/91c17ee8d451792d45e5aedb71098b9a9441e707_91c17ee8d451792d45e5aedb71098b9a9441e707/report/index.html

doris-robot · 2024-11-27T19:35:20Z

TPC-H: Total hot run time: 39882 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 91c17ee8d451792d45e5aedb71098b9a9441e707, data reload: false

------ Round 1 ----------------------------------
q1	17570	7461	7243	7243
q2	2051	177	160	160
q3	10663	1095	1221	1095
q4	10565	727	748	727
q5	7884	2795	2720	2720
q6	241	160	156	156
q7	979	638	615	615
q8	9246	1863	1916	1863
q9	6603	6394	6448	6394
q10	7035	2323	2379	2323
q11	460	269	259	259
q12	423	218	213	213
q13	17759	3015	3046	3015
q14	255	211	217	211
q15	581	523	522	522
q16	646	604	592	592
q17	974	574	551	551
q18	7225	6581	6762	6581
q19	1310	983	1021	983
q20	477	178	178	178
q21	3981	3265	3171	3171
q22	378	313	310	310
Total cold run time: 107306 ms
Total hot run time: 39882 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7253	7307	7229	7229
q2	320	226	225	225
q3	2916	2777	2960	2777
q4	2063	1770	1864	1770
q5	5714	5701	5614	5614
q6	223	148	146	146
q7	2276	1834	1823	1823
q8	3403	3575	3527	3527
q9	8866	8920	8883	8883
q10	3623	3537	3561	3537
q11	596	506	515	506
q12	799	602	575	575
q13	11384	3296	3271	3271
q14	302	271	279	271
q15	591	523	527	523
q16	695	642	641	641
q17	1863	1647	1607	1607
q18	8263	7792	7514	7514
q19	1767	1678	1554	1554
q20	2147	1928	1897	1897
q21	5570	5292	5350	5292
q22	679	567	606	567
Total cold run time: 71313 ms
Total hot run time: 59749 ms

doris-robot · 2024-11-27T19:48:00Z

TPC-DS: Total hot run time: 196808 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 91c17ee8d451792d45e5aedb71098b9a9441e707, data reload: false

query1	1250	944	939	939
query2	6228	2029	2029	2029
query3	10941	3958	4081	3958
query4	67593	28635	23589	23589
query5	4982	451	440	440
query6	409	207	182	182
query7	5501	307	293	293
query8	319	233	228	228
query9	8549	2730	2718	2718
query10	455	264	271	264
query11	17028	15212	16016	15212
query12	158	107	102	102
query13	1487	443	453	443
query14	10814	7136	6789	6789
query15	209	193	199	193
query16	7114	493	500	493
query17	1080	591	606	591
query18	1829	344	307	307
query19	202	163	149	149
query20	118	118	118	118
query21	209	107	113	107
query22	4571	4612	4565	4565
query23	35172	34694	34407	34407
query24	5633	2546	2539	2539
query25	505	392	397	392
query26	661	155	155	155
query27	1911	292	290	290
query28	4426	2502	2468	2468
query29	682	416	417	416
query30	220	176	149	149
query31	1024	834	867	834
query32	65	60	56	56
query33	404	287	297	287
query34	958	538	525	525
query35	900	772	762	762
query36	1081	973	975	973
query37	123	78	76	76
query38	4479	4485	4520	4485
query39	1526	1466	1485	1466
query40	206	109	98	98
query41	44	42	44	42
query42	113	98	98	98
query43	543	503	493	493
query44	1186	841	818	818
query45	187	169	171	169
query46	1159	730	713	713
query47	2067	1940	1964	1940
query48	424	338	321	321
query49	728	399	388	388
query50	849	398	416	398
query51	7373	7314	7155	7155
query52	98	89	87	87
query53	259	184	181	181
query54	526	415	388	388
query55	84	76	78	76
query56	246	236	265	236
query57	1318	1194	1178	1178
query58	223	222	219	219
query59	3237	2960	3030	2960
query60	270	250	240	240
query61	112	112	112	112
query62	772	656	677	656
query63	212	193	187	187
query64	1454	663	645	645
query65	3270	3219	3221	3219
query66	690	303	304	303
query67	16108	15636	15527	15527
query68	3733	571	579	571
query69	427	280	256	256
query70	1186	1094	1138	1094
query71	370	254	255	254
query72	6442	4110	3986	3986
query73	771	360	369	360
query74	10212	8934	8961	8934
query75	3409	2682	2650	2650
query76	1778	1109	1114	1109
query77	522	296	285	285
query78	10501	9550	9458	9458
query79	1840	599	608	599
query80	1250	423	476	423
query81	529	242	227	227
query82	460	118	115	115
query83	182	150	147	147
query84	290	73	68	68
query85	1005	308	305	305
query86	384	309	315	309
query87	4698	4476	4512	4476
query88	3570	2229	2182	2182
query89	421	312	295	295
query90	2034	192	191	191
query91	147	106	102	102
query92	70	49	52	49
query93	2546	549	548	548
query94	901	301	268	268
query95	354	251	249	249
query96	631	279	282	279
query97	2867	2685	2671	2671
query98	212	197	195	195
query99	2048	1320	1317	1317
Total cold run time: 320364 ms
Total hot run time: 196808 ms

doris-robot · 2024-11-27T19:53:30Z

ClickBench: Total hot run time: 33.13 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 91c17ee8d451792d45e5aedb71098b9a9441e707, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.04	0.03
query3	0.23	0.08	0.06
query4	1.62	0.10	0.10
query5	0.45	0.41	0.41
query6	1.13	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.52	0.49
query10	0.56	0.54	0.56
query11	0.14	0.11	0.11
query12	0.13	0.12	0.11
query13	0.60	0.59	0.61
query14	2.70	2.75	2.69
query15	0.90	0.83	0.82
query16	0.37	0.38	0.39
query17	1.06	1.06	1.05
query18	0.22	0.21	0.22
query19	1.95	1.87	1.91
query20	0.02	0.00	0.02
query21	15.42	0.62	0.60
query22	2.81	2.46	2.16
query23	17.13	0.86	0.86
query24	2.88	1.09	1.96
query25	0.20	0.06	0.08
query26	0.60	0.14	0.14
query27	0.04	0.04	0.04
query28	9.97	1.12	1.09
query29	12.59	3.28	3.20
query30	0.24	0.06	0.06
query31	2.88	0.39	0.38
query32	3.27	0.47	0.47
query33	3.01	3.08	3.04
query34	17.10	4.48	4.50
query35	4.54	4.54	4.49
query36	0.68	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.04	0.03	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.65 s
Total hot run time: 33.13 s

be/src/vec/exec/format/orc/vorc_reader.cpp

Co-authored-by: Mingyu Chen (Rayner) <yunyou@selectdb.com>

…44615) …" This reverts commit 4b15b1f.

…pache#44615) (apache#44801)" This reverts commit 4b15b1f.

Revert "branch-2.1: [enhance](orc) Optimize ORC Predicate Pushdown for OR-connected Predicate #43255 (#44438)" Revert "[fix](orc) check all the cases before build_search_argument (#44615) (#44801)"

…pache#44615) (apache#44802)" This reverts commit 88926d2.

revert: branch-3.0: [fix](orc) ignore null values when the literals of in_predicate contains #45104 (#45586) [fix](orc) check all the cases before build_search_argument (#44615) (#44802) branch-3.0: [enhance](orc) Optimize ORC Predicate Pushdown for OR-connected Predicate #43255 (#44436) re-pick: branch-3.0: [Fix](ORC) Not push down fixed char type in orc reader #45484 (#45525) --------- Co-authored-by: Socrates <suyiteng@selectdb.com>

This reverts commit d4a6fd1.

…4615) In the old logic, the `check_expr_can_push_down` function does not check whether the `orc::Literal` are constructed successfully, but only checks during `build_search_argument`. However, if it is found that the `orc::Literal` fails to be constructed after `builder->startNot`, it will fail because the builder cannot end `startNot`. Therefore, we advance the behavior of constructing `orc::Literal` to the `check_expr_can_push_down` function and save the result to the map, so that it will never fail in the `build_search_argument` phase. Related PR: apache#43255

…nected Predicate #43255 #44615 #45104 #47506 #47625 #49088 #49835 #49927 (#52192) picks: #43255 #44615 #45104 #47506 #47625 #49088 #49835 #49927 --------- Co-authored-by: Qi Chen <chenqi@selectdb.com>

) In the old logic, the `check_expr_can_push_down` function does not check whether the `orc::Literal` are constructed successfully, but only checks during `build_search_argument`. However, if it is found that the `orc::Literal` fails to be constructed after `builder->startNot`, it will fail because the builder cannot end `startNot`. Therefore, we advance the behavior of constructing `orc::Literal` to the `check_expr_can_push_down` function and save the result to the map, so that it will never fail in the `build_search_argument` phase. Related PR: apache#43255 Conflicts: be/src/vec/exec/format/orc/vorc_reader.cpp be/test/vec/exec/orc_reader_test.cpp

…pache#43255 & apache#44615

suxiaogang223 marked this pull request as draft November 26, 2024 10:46

suxiaogang223 marked this pull request as ready for review November 26, 2024 18:05

suxiaogang223 added 4 commits November 27, 2024 10:26

fix orc

86ebb61

also check for is null

d24f581

fix

22a454a

add be ut case

f06e0b5

suxiaogang223 force-pushed the fix_orc_predicate_push_down branch from d2d6724 to f06e0b5 Compare November 27, 2024 02:26

do not use get_child api

91c17ee

morningman added dev/2.1.x dev/3.0.x labels Nov 28, 2024

morningman reviewed Nov 28, 2024

View reviewed changes

be/src/vec/exec/format/orc/vorc_reader.cpp Outdated Show resolved Hide resolved

fix wrong logic

3df89ac

Co-authored-by: Mingyu Chen (Rayner) <yunyou@selectdb.com>

morningman added dev/2.1.8-merged dev/3.0.4-merged and removed dev/2.1.x dev/3.0.x dev/3.0.x-conflict dev/2.1.x-conflict labels Nov 30, 2024

morningman added a commit that referenced this pull request Dec 6, 2024

Revert "[fix](orc) check all the cases before build_search_argument (#…

0714a3b

…44615) …" This reverts commit 4b15b1f.

morningman added a commit to morningman/doris that referenced this pull request Dec 6, 2024

Revert "[fix](orc) check all the cases before build_search_argument (a…

c4c7218

…pache#44615) (apache#44801)" This reverts commit 4b15b1f.

morningman mentioned this pull request Dec 6, 2024

Revert #43255 & #44615 #45096

Merged

morningman removed the dev/2.1.8-merged label Dec 6, 2024

suxiaogang223 deleted the fix_orc_predicate_push_down branch December 12, 2024 09:27

morningman added a commit to morningman/doris that referenced this pull request Feb 8, 2025

Revert "[fix](orc) check all the cases before build_search_argument (a…

9e0d553

…pache#44615) (apache#44802)" This reverts commit 88926d2.

morningman mentioned this pull request Feb 8, 2025

[revert](orc) revert orc predicate pushdown related commits #47662

Merged

morningman added dev/3.0.x-experimental and removed dev/3.0.4-merged labels Feb 9, 2025

morningman added dev/2.1.x-experimental dev/2.1.x-lh labels Feb 17, 2025

morningman added a commit that referenced this pull request Feb 17, 2025

Revert "Revert #43255 & #44615 (#45096)"

e1f869d

This reverts commit d4a6fd1.

suxiaogang223 mentioned this pull request Jun 24, 2025

branch-3.1: [enhance](orc) Optimize ORC Predicate Pushdown for OR-connected Predicate (#43255 #44615 #45104 #47506 #47625 #49088 #49835 #49927) #52192

Merged

morrySnow added the dev/3.1.0-merged label Jun 27, 2025

morningman removed the dev/3.0.x-experimental label Jul 4, 2025

GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Sep 26, 2025

fix(decimal): fix filter error when cast to decimal(30, 6) and revert a…

c3d8568

…pache#43255 & apache#44615

[fix](orc) check all the cases before build_search_argument #44615

[fix](orc) check all the cases before build_search_argument #44615

Uh oh!

Conversation

suxiaogang223 commented Nov 26, 2024 • edited by morningman Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

doris-robot commented Nov 26, 2024

Uh oh!

suxiaogang223 commented Nov 26, 2024

Uh oh!

doris-robot commented Nov 26, 2024

Uh oh!

doris-robot commented Nov 26, 2024

Uh oh!

doris-robot commented Nov 26, 2024

Uh oh!

doris-robot commented Nov 26, 2024

Uh oh!

suxiaogang223 commented Nov 27, 2024

Uh oh!

suxiaogang223 commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

suxiaogang223 commented Nov 27, 2024

Uh oh!

suxiaogang223 commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

doris-robot commented Nov 27, 2024

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

suxiaogang223 commented Nov 26, 2024 •

edited by morningman

Loading