Skip to content

Conversation

@924060929
Copy link
Contributor

cherry pick from #55378

@924060929 924060929 requested a review from morrySnow as a code owner August 29, 2025 02:34
@Thearas
Copy link
Contributor

Thearas commented Aug 29, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32492 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 05d9dea1028bd67362050829c4d5cebba52377a6, data reload: false

------ Round 1 ----------------------------------
q1	17595	5459	5398	5398
q2	2022	400	279	279
q3	12207	1230	747	747
q4	10315	855	454	454
q5	9350	2383	2082	2082
q6	186	164	134	134
q7	878	732	630	630
q8	9363	1411	1138	1138
q9	5236	4913	4850	4850
q10	6770	2270	1842	1842
q11	499	290	280	280
q12	332	353	207	207
q13	17785	3588	3015	3015
q14	227	228	218	218
q15	517	466	463	463
q16	435	417	366	366
q17	594	841	364	364
q18	6868	6350	6327	6327
q19	1200	945	541	541
q20	322	331	218	218
q21	2970	2173	1950	1950
q22	1049	1039	989	989
Total cold run time: 106720 ms
Total hot run time: 32492 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5537	5538	5489	5489
q2	228	317	240	240
q3	2230	2639	2301	2301
q4	1353	1783	1377	1377
q5	4421	4807	5142	4807
q6	162	160	130	130
q7	2125	1991	1831	1831
q8	2704	2911	2747	2747
q9	7459	7300	7358	7300
q10	3066	3366	2869	2869
q11	565	511	500	500
q12	637	755	620	620
q13	3528	3897	3322	3322
q14	287	297	269	269
q15	526	464	485	464
q16	466	487	443	443
q17	1247	1781	1287	1287
q18	7688	7421	7360	7360
q19	800	1154	1071	1071
q20	1979	2059	1880	1880
q21	5455	5066	4805	4805
q22	1048	1036	1046	1036
Total cold run time: 53511 ms
Total hot run time: 52148 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193834 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 05d9dea1028bd67362050829c4d5cebba52377a6, data reload: false

query1	961	413	392	392
query2	6227	1885	1878	1878
query3	8680	200	199	199
query4	33111	23885	23897	23885
query5	4012	575	447	447
query6	306	199	175	175
query7	4210	486	314	314
query8	300	244	235	235
query9	9440	2612	2596	2596
query10	453	325	270	270
query11	18327	15421	15194	15194
query12	156	109	113	109
query13	1562	555	429	429
query14	9528	7926	7608	7608
query15	247	192	181	181
query16	8139	674	513	513
query17	1540	796	607	607
query18	2156	409	320	320
query19	235	190	171	171
query20	131	118	132	118
query21	217	130	118	118
query22	4491	4704	4559	4559
query23	34894	34827	34203	34203
query24	7480	2731	2749	2731
query25	546	476	419	419
query26	728	283	176	176
query27	2058	482	367	367
query28	5228	2198	2141	2141
query29	700	593	481	481
query30	241	204	158	158
query31	1085	930	820	820
query32	94	59	60	59
query33	502	385	357	357
query34	749	881	530	530
query35	803	842	735	735
query36	1024	1057	982	982
query37	109	90	65	65
query38	4052	4067	4053	4053
query39	1537	1486	1485	1485
query40	207	123	112	112
query41	50	51	47	47
query42	128	106	108	106
query43	506	545	488	488
query44	1316	848	838	838
query45	190	178	175	175
query46	880	1070	677	677
query47	1937	2023	1908	1908
query48	408	434	354	354
query49	723	511	411	411
query50	695	689	441	441
query51	7339	7318	7268	7268
query52	106	103	98	98
query53	227	271	200	200
query54	562	559	475	475
query55	85	80	85	80
query56	270	273	250	250
query57	1239	1287	1202	1202
query58	237	224	224	224
query59	2974	3175	3102	3102
query60	297	285	275	275
query61	117	109	117	109
query62	846	743	695	695
query63	237	196	197	196
query64	3802	994	637	637
query65	3413	3313	3319	3313
query66	789	405	297	297
query67	16569	15901	15546	15546
query68	7780	819	536	536
query69	500	317	258	258
query70	1217	1087	1119	1087
query71	415	286	269	269
query72	5178	3735	4002	3735
query73	641	748	352	352
query74	10142	9477	8997	8997
query75	3929	3165	2737	2737
query76	3544	1111	767	767
query77	759	371	285	285
query78	10377	10549	9569	9569
query79	2977	901	603	603
query80	718	529	447	447
query81	489	257	218	218
query82	553	118	90	90
query83	170	167	145	145
query84	282	101	84	84
query85	790	364	300	300
query86	340	319	301	301
query87	4386	4349	4273	4273
query88	3605	2386	2376	2376
query89	414	325	290	290
query90	1815	195	191	191
query91	133	159	109	109
query92	65	55	51	51
query93	1115	887	545	545
query94	648	409	310	310
query95	353	285	284	284
query96	485	610	285	285
query97	3256	3306	3233	3233
query98	219	206	196	196
query99	1514	1456	1307	1307
Total cold run time: 291217 ms
Total hot run time: 193834 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.73 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 05d9dea1028bd67362050829c4d5cebba52377a6, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.62	0.11	0.11
query5	0.52	0.53	0.53
query6	1.14	0.73	0.74
query7	0.02	0.02	0.02
query8	0.05	0.04	0.03
query9	0.56	0.51	0.50
query10	0.57	0.56	0.56
query11	0.15	0.13	0.10
query12	0.13	0.11	0.11
query13	0.62	0.61	0.60
query14	0.79	0.79	0.80
query15	0.87	0.85	0.84
query16	0.38	0.39	0.41
query17	1.07	1.04	1.02
query18	0.24	0.23	0.23
query19	1.93	1.83	1.78
query20	0.02	0.02	0.01
query21	15.39	0.99	0.60
query22	0.74	0.96	0.91
query23	14.72	1.43	0.55
query24	3.07	1.63	1.48
query25	0.30	0.08	0.09
query26	0.23	0.14	0.14
query27	0.05	0.03	0.07
query28	14.10	0.92	0.43
query29	12.57	3.94	3.27
query30	0.25	0.08	0.08
query31	2.81	0.61	0.39
query32	3.22	0.55	0.46
query33	2.99	3.02	3.12
query34	16.68	5.24	4.51
query35	4.62	4.54	4.57
query36	0.64	0.52	0.47
query37	0.09	0.07	0.06
query38	0.04	0.04	0.03
query39	0.04	0.02	0.02
query40	0.18	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.02	0.02
Total cold run time: 103.91 s
Total hot run time: 29.73 s

support simplify string range:

```sql
where dt > '20250101' and dt > '20260101' -- simplify to dt > '20260101'
```

(cherry picked from commit 9a056fe)
@924060929 924060929 force-pushed the branch-3.1-opt-string-partition branch from 05d9dea to 29a97c1 Compare August 29, 2025 10:30
@924060929
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32844 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 29a97c1f3fa4093cf77a4444cd847e9e2f7c9c2a, data reload: false

------ Round 1 ----------------------------------
q1	17601	5489	5509	5489
q2	2043	338	161	161
q3	10383	1291	766	766
q4	10218	872	466	466
q5	7655	2427	2135	2135
q6	183	167	138	138
q7	941	778	625	625
q8	9349	1494	1195	1195
q9	5312	4986	4977	4977
q10	6774	2270	1827	1827
q11	482	285	261	261
q12	332	358	208	208
q13	17778	3596	2997	2997
q14	231	240	205	205
q15	532	466	457	457
q16	410	424	370	370
q17	601	870	385	385
q18	6862	6414	6414	6414
q19	1198	967	555	555
q20	351	347	208	208
q21	3021	2208	2018	2018
q22	1048	1010	987	987
Total cold run time: 103305 ms
Total hot run time: 32844 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5578	5553	5566	5553
q2	233	340	240	240
q3	2260	2602	2271	2271
q4	1316	1789	1399	1399
q5	4415	4847	5142	4847
q6	183	172	129	129
q7	2062	1971	1798	1798
q8	2607	2823	2747	2747
q9	7223	7125	7102	7102
q10	3011	3316	2814	2814
q11	600	511	490	490
q12	658	761	612	612
q13	3401	3806	3176	3176
q14	271	287	279	279
q15	533	488	473	473
q16	440	519	434	434
q17	1224	1741	1278	1278
q18	7572	7507	7233	7233
q19	794	1137	1072	1072
q20	2030	2023	1885	1885
q21	5368	4874	4746	4746
q22	1127	1070	1065	1065
Total cold run time: 52906 ms
Total hot run time: 51643 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190964 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 29a97c1f3fa4093cf77a4444cd847e9e2f7c9c2a, data reload: false

query1	979	378	385	378
query2	6538	1959	1869	1869
query3	6707	217	224	217
query4	34367	23585	23556	23556
query5	4302	620	456	456
query6	273	201	215	201
query7	4627	500	317	317
query8	296	243	238	238
query9	9810	2617	2617	2617
query10	488	339	255	255
query11	18089	15396	14991	14991
query12	159	113	102	102
query13	1650	568	422	422
query14	9126	7707	7217	7217
query15	254	199	178	178
query16	8120	627	533	533
query17	1686	794	584	584
query18	2152	428	336	336
query19	233	200	174	174
query20	122	124	117	117
query21	214	129	110	110
query22	4663	4545	4635	4545
query23	34381	33537	33475	33475
query24	7226	2652	2613	2613
query25	519	474	409	409
query26	1198	284	173	173
query27	1962	480	344	344
query28	5125	2177	2140	2140
query29	758	582	468	468
query30	257	190	158	158
query31	971	864	831	831
query32	91	64	62	62
query33	529	366	313	313
query34	751	846	526	526
query35	765	791	731	731
query36	953	1025	921	921
query37	98	93	72	72
query38	3917	3921	3852	3852
query39	1504	1446	1423	1423
query40	211	118	109	109
query41	58	59	51	51
query42	119	104	106	104
query43	510	522	469	469
query44	1372	811	811	811
query45	184	174	175	174
query46	865	1028	679	679
query47	1873	1898	1830	1830
query48	406	420	359	359
query49	804	504	430	430
query50	674	667	433	433
query51	7114	7186	7099	7099
query52	102	103	98	98
query53	234	253	179	179
query54	548	549	466	466
query55	77	79	80	79
query56	272	267	250	250
query57	1238	1245	1155	1155
query58	234	237	222	222
query59	2926	3035	2870	2870
query60	301	278	283	278
query61	114	119	111	111
query62	801	749	677	677
query63	226	194	192	192
query64	4436	960	662	662
query65	3298	3223	3208	3208
query66	1050	409	306	306
query67	16010	15746	15533	15533
query68	7242	832	529	529
query69	479	305	264	264
query70	1205	1156	1132	1132
query71	561	308	267	267
query72	5727	3859	3785	3785
query73	634	745	352	352
query74	10207	9179	9224	9179
query75	3266	3172	2663	2663
query76	3331	1164	756	756
query77	540	381	277	277
query78	10377	10442	9558	9558
query79	3446	902	593	593
query80	716	540	458	458
query81	511	257	219	219
query82	649	125	90	90
query83	171	162	146	146
query84	243	102	77	77
query85	783	403	302	302
query86	401	313	311	311
query87	4278	4282	4241	4241
query88	4949	2388	2364	2364
query89	406	327	288	288
query90	1867	192	188	188
query91	161	146	109	109
query92	68	58	56	56
query93	1917	888	546	546
query94	693	426	306	306
query95	346	277	269	269
query96	494	611	288	288
query97	3218	3273	3082	3082
query98	230	216	198	198
query99	1597	1415	1313	1313
Total cold run time: 292630 ms
Total hot run time: 190964 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.64 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 29a97c1f3fa4093cf77a4444cd847e9e2f7c9c2a, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.63	0.10	0.10
query5	0.53	0.51	0.51
query6	1.13	0.74	0.73
query7	0.02	0.02	0.02
query8	0.05	0.03	0.03
query9	0.56	0.50	0.51
query10	0.54	0.54	0.55
query11	0.14	0.10	0.11
query12	0.13	0.11	0.13
query13	0.61	0.60	0.59
query14	0.77	0.77	0.79
query15	0.84	0.83	0.81
query16	0.39	0.39	0.40
query17	1.01	1.09	1.02
query18	0.24	0.23	0.23
query19	1.93	1.86	1.81
query20	0.02	0.01	0.01
query21	15.43	0.98	0.61
query22	0.76	0.85	0.65
query23	15.07	1.41	0.51
query24	3.60	0.92	1.23
query25	0.21	0.10	0.05
query26	0.21	0.16	0.14
query27	0.06	0.04	0.04
query28	13.43	0.95	0.44
query29	12.57	3.89	3.25
query30	0.25	0.10	0.06
query31	2.82	0.59	0.37
query32	3.22	0.54	0.45
query33	3.02	2.98	3.07
query34	16.68	5.17	4.48
query35	4.53	4.50	4.54
query36	0.65	0.50	0.48
query37	0.08	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.03	0.02
query40	0.16	0.14	0.13
query41	0.07	0.02	0.02
query42	0.03	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 103.84 s
Total hot run time: 28.64 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 66.67% (2/3) 🎉
Increment coverage report
Complete coverage report

@morrySnow morrySnow changed the title branch-3.1: [opt](nereids) support simplify string range (#55378) branch-3.1: [opt](nereids) support simplify string range #55378 Sep 4, 2025
@morrySnow morrySnow merged commit 433276d into apache:branch-3.1 Sep 4, 2025
23 checks passed
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants