Skip to content

[opt](nereids) support simplify string range #55378

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
924060929 merged 1 commit into from
Aug 29, 2025
Merged

[opt](nereids) support simplify string range #55378

924060929 merged 1 commit into from
Aug 29, 2025

Conversation

@924060929
Copy link
Contributor

@924060929 924060929 commented Aug 27, 2025 •
edited
Loading

What problem does this PR solve?

support simplify string range:

where dt > '20250101' and dt > '20260101' -- simplify to dt > '20260101'

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 27, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929
Copy link
Contributor Author

924060929 commented Aug 27, 2025

run buildall

@924060929 924060929 added usercase Important user case type label dev/2.1.x dev/3.0.x dev/3.1.x labels Aug 27, 2025
@doris-robot
Copy link

doris-robot commented Aug 27, 2025

TPC-H: Total hot run time: 34284 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aa9e1eba94488fa03391492241a3af5e99daab50, data reload: false

------ Round 1 ----------------------------------
q1	17612	5290	5096	5096
q2	2002	327	242	242
q3	10206	1369	739	739
q4	10235	1033	517	517
q5	7555	2382	2371	2371
q6	181	165	135	135
q7	929	759	629	629
q8	9330	1352	1128	1128
q9	6949	5149	5151	5149
q10	6887	2372	1959	1959
q11	485	297	308	297
q12	357	363	221	221
q13	17768	3689	3050	3050
q14	249	258	229	229
q15	552	500	501	500
q16	428	429	371	371
q17	587	870	366	366
q18	7412	7139	7180	7139
q19	1105	963	566	566
q20	348	340	228	228
q21	4148	3267	2368	2368
q22	1096	1036	984	984
Total cold run time: 106421 ms
Total hot run time: 34284 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5146	5115	5097	5097
q2	251	324	228	228
q3	2164	2708	2308	2308
q4	1381	1779	1340	1340
q5	4202	4405	4658	4405
q6	224	187	133	133
q7	2040	1984	1870	1870
q8	2676	2701	2591	2591
q9	7428	7377	7253	7253
q10	3093	3298	2923	2923
q11	586	512	499	499
q12	691	799	671	671
q13	3643	3949	3330	3330
q14	297	305	293	293
q15	566	518	497	497
q16	489	511	440	440
q17	1277	1659	1352	1352
q18	7907	7798	7658	7658
q19	825	836	909	836
q20	2020	2097	2006	2006
q21	5019	4452	4308	4308
q22	1114	1060	997	997
Total cold run time: 53039 ms
Total hot run time: 51035 ms

@doris-robot
Copy link

doris-robot commented Aug 27, 2025

TPC-DS: Total hot run time: 129149 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aa9e1eba94488fa03391492241a3af5e99daab50, data reload: false

ship_mode	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:44:00	2023-12-26 22:44:05	NULL	utf-8	NULL	NULL	
customer_address	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:58	2023-12-26 22:44:41	NULL	utf-8	NULL	NULL	
customer	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:44:00	2023-12-26 22:44:53	NULL	utf-8	NULL	NULL	
web_returns	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:58	2023-12-26 22:45:43	NULL	utf-8	NULL	NULL	
warehouse	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
web_site	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:58	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
customer_demographics	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:34	NULL	utf-8	NULL	NULL	
promotion	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:58	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
web_page	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:59	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
store_returns	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:59	2023-12-26 22:47:09	NULL	utf-8	NULL	NULL	
income_band	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:58	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
dbgen_version	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:44:00	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
catalog_page	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:58	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
reason	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
============================================
query1	1058	471	396	396
query2	6561	1816	1831	1816
query3	6751	229	230	229
query4	query5	5065	554	539	539
query6	338	233	224	224
query7	4797	455	306	306
query8	query9	11212	2931	2935	2931
query10	617	365	303	303
query11	query12	176	122	127	122
query13	2212	605	430	430
query14	8571	5825	5736	5736
query15	279	196	192	192
query16	7479	683	481	481
query17	976	788	614	614
query18	2196	459	347	347
query19	334	207	185	185
query20	134	128	126	126
query21	230	134	119	119
query22	4032	4239	4198	4198
query23	34140	33104	33026	33026
query24	query25	702	602	463	463
query26	1254	280	167	167
query27	4155	539	344	344
query28	4451	2294	2263	2263
query29	1033	602	493	493
query30	487	233	199	199
query31	query32	124	85	76	76
query33	584	404	350	350
query34	3738	882	508	508
query35	853	822	774	774
query36	1017	1016	956	956
query37	139	113	98	98
query38	4092	4154	3972	3972
query39	1488	1443	1467	1443
query40	228	142	135	135
query41	query42	139	122	122	122
query43	493	527	475	475
query44	1657	889	878	878
query45	190	180	182	180
query46	879	1010	663	663
query47	query48	387	408	334	334
query49	1248	493	437	437
query50	757	650	402	402
query51	4021	4143	4145	4143
query52	126	118	107	107
query53	244	268	202	202
query54	query55	102	94	98	94
query56	335	332	306	306
query57	query58	query59	query60	389	366	350	350
query61	163	166	165	165
query62	819	726	659	659
query63	239	255	200	200
query64	query65	4282	4186	4149	4149
query66	1842	449	344	344
query67	15456	15162	15221	15162
query68	4197	885	601	601
query69	493	341	299	299
query70	1166	1107	1095	1095
query71	375	355	328	328
query72	5738	5010	5058	5010
query73	621	617	360	360
query74	query75	3133	3147	2675	2675
query76	2783	1108	761	761
query77	497	404	328	328
query78	9495	9646	8930	8930
query79	1009	837	608	608
query80	866	596	553	553
query81	502	269	228	228
query82	1267	140	112	112
query83	query84	306	121	102	102
query85	query86	376	335	309	309
query87	4273	4257	4181	4181
query88	2827	2268	2219	2219
query89	375	325	298	298
query90	2095	240	225	225
query91	161	175	146	146
query92	95	78	78	78
query93	1108	977	651	651
query94	726	404	322	322
query95	434	336	332	332
query96	488	596	281	281
query97	2582	2668	2643	2643
query98	249	238	225	225
query99	1334	1422	1316	1316
Total cold run time: 204842 ms
Total hot run time: 129149 ms

@doris-robot
Copy link

doris-robot commented Aug 27, 2025

ClickBench: Total hot run time: 32.64 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aa9e1eba94488fa03391492241a3af5e99daab50, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.06	0.06
query3	0.26	0.09	0.08
query4	1.61	0.12	0.12
query5	0.44	0.41	0.40
query6	1.16	0.64	0.66
query7	0.03	0.03	0.03
query8	0.06	0.05	0.05
query9	0.60	0.53	0.52
query10	0.59	0.58	0.58
query11	0.17	0.12	0.12
query12	0.16	0.13	0.12
query13	0.64	0.63	0.63
query14	0.81	0.83	0.84
query15	0.88	0.87	0.87
query16	0.41	0.41	0.40
query17	1.02	1.05	1.12
query18	0.22	0.20	0.20
query19	1.95	1.89	1.81
query20	0.02	0.02	0.01
query21	15.42	1.02	0.59
query22	0.76	1.19	0.82
query23	14.81	1.42	0.62
query24	6.53	1.17	0.57
query25	0.48	0.15	0.07
query26	0.65	0.17	0.15
query27	0.07	0.05	0.06
query28	9.47	0.98	0.44
query29	12.56	3.95	3.22
query30	3.14	3.05	3.02
query31	2.83	0.58	0.38
query32	3.24	0.56	0.48
query33	3.06	3.06	3.29
query34	16.18	5.45	4.85
query35	4.98	4.89	4.93
query36	0.70	0.56	0.49
query37	0.10	0.08	0.07
query38	0.05	0.04	0.05
query39	0.04	0.03	0.03
query40	0.18	0.16	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 106.61 s
Total hot run time: 32.64 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 27, 2025

FE UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉
Increment coverage report
Complete coverage report

@924060929
Copy link
Contributor Author

924060929 commented Aug 27, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 27, 2025

TPC-H: Total hot run time: 34024 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 45cb6cdbd02df922217f29cabd8130a6d4bd57dd, data reload: false

------ Round 1 ----------------------------------
q1	17631	5146	5045	5045
q2	2007	330	217	217
q3	10238	1294	720	720
q4	10251	1022	521	521
q5	7538	2426	2310	2310
q6	183	166	137	137
q7	937	766	624	624
q8	9363	1309	1152	1152
q9	7000	5143	5161	5143
q10	6944	2352	1980	1980
q11	501	311	286	286
q12	360	357	230	230
q13	17805	3675	3027	3027
q14	234	238	228	228
q15	564	512	510	510
q16	427	430	389	389
q17	600	858	365	365
q18	7489	7100	7118	7100
q19	1438	961	551	551
q20	336	337	229	229
q21	3782	3120	2285	2285
q22	1052	1031	975	975
Total cold run time: 106680 ms
Total hot run time: 34024 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5227	5135	5122	5122
q2	244	330	226	226
q3	2176	2655	2277	2277
q4	1340	1798	1343	1343
q5	4250	4391	4594	4391
q6	216	193	148	148
q7	2044	1905	1892	1892
q8	2749	2574	2493	2493
q9	7387	7315	7398	7315
q10	3115	3360	2905	2905
q11	613	525	504	504
q12	714	784	681	681
q13	3442	3932	3365	3365
q14	301	305	294	294
q15	527	524	534	524
q16	456	492	434	434
q17	1201	1487	1422	1422
q18	7799	7699	7752	7699
q19	868	887	977	887
q20	1960	1973	1824	1824
q21	4781	4294	4267	4267
q22	1132	1037	989	989
Total cold run time: 52542 ms
Total hot run time: 51002 ms

@doris-robot
Copy link

doris-robot commented Aug 27, 2025

TPC-DS: Total hot run time: 186572 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 45cb6cdbd02df922217f29cabd8130a6d4bd57dd, data reload: false

query1	1049	430	443	430
query2	6572	1724	1746	1724
query3	6758	231	228	228
query4	26786	23690	23145	23145
query5	4390	673	538	538
query6	350	256	240	240
query7	4655	516	305	305
query8	306	268	246	246
query9	9149	2878	2966	2878
query10	503	360	318	318
query11	15513	15040	14785	14785
query12	176	125	125	125
query13	1692	586	467	467
query14	9227	5894	5836	5836
query15	214	200	179	179
query16	7577	657	517	517
query17	1225	756	636	636
query18	2031	463	343	343
query19	206	194	179	179
query20	138	125	125	125
query21	238	141	114	114
query22	4125	4248	4082	4082
query23	33726	32741	32888	32741
query24	8163	2377	2394	2377
query25	579	511	441	441
query26	1231	280	165	165
query27	2738	516	374	374
query28	4384	2262	2229	2229
query29	793	635	498	498
query30	289	245	205	205
query31	912	820	726	726
query32	95	84	78	78
query33	581	391	353	353
query34	813	861	535	535
query35	843	823	759	759
query36	977	1016	954	954
query37	132	111	92	92
query38	4086	4115	3963	3963
query39	1504	1427	1434	1427
query40	220	133	136	133
query41	67	63	63	63
query42	129	120	117	117
query43	509	546	487	487
query44	1383	874	867	867
query45	181	179	174	174
query46	854	1020	653	653
query47	1761	1776	1705	1705
query48	393	431	324	324
query49	739	508	416	416
query50	651	701	412	412
query51	4072	4106	4167	4106
query52	116	115	108	108
query53	244	271	215	215
query54	620	618	535	535
query55	100	89	95	89
query56	350	333	326	326
query57	1191	1207	1130	1130
query58	291	291	297	291
query59	2613	2790	2557	2557
query60	381	375	414	375
query61	166	155	158	155
query62	811	721	696	696
query63	234	199	192	192
query64	4446	1123	847	847
query65	4324	4201	4212	4201
query66	1132	453	343	343
query67	15521	15177	14957	14957
query68	7920	921	585	585
query69	494	328	297	297
query70	1220	1121	1103	1103
query71	454	354	332	332
query72	5938	4997	5154	4997
query73	693	638	365	365
query74	8964	9172	8865	8865
query75	3357	3093	2702	2702
query76	3343	1139	744	744
query77	660	431	346	346
query78	9537	9717	8917	8917
query79	1953	860	595	595
query80	668	594	515	515
query81	507	264	229	229
query82	200	145	126	126
query83	266	273	250	250
query84	263	122	100	100
query85	866	481	437	437
query86	388	329	307	307
query87	4260	4264	4217	4217
query88	2983	2267	2249	2249
query89	389	335	298	298
query90	1977	244	234	234
query91	167	160	135	135
query92	89	79	71	71
query93	1889	1024	673	673
query94	689	427	298	298
query95	420	393	326	326
query96	486	602	286	286
query97	2625	2683	2586	2586
query98	260	229	216	216
query99	1347	1447	1340	1340
Total cold run time: 273497 ms
Total hot run time: 186572 ms

@doris-robot
Copy link

doris-robot commented Aug 27, 2025

ClickBench: Total hot run time: 32.77 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 45cb6cdbd02df922217f29cabd8130a6d4bd57dd, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.06
query3	0.26	0.08	0.08
query4	1.60	0.12	0.11
query5	0.45	0.41	0.42
query6	1.20	0.64	0.66
query7	0.04	0.03	0.03
query8	0.05	0.04	0.04
query9	0.61	0.52	0.52
query10	0.61	0.57	0.57
query11	0.17	0.11	0.11
query12	0.15	0.13	0.13
query13	0.64	0.63	0.63
query14	0.81	0.84	0.85
query15	0.87	0.86	0.85
query16	0.39	0.39	0.41
query17	1.06	1.04	1.05
query18	0.21	0.20	0.23
query19	1.94	1.82	1.92
query20	0.02	0.01	0.01
query21	15.39	0.94	0.59
query22	0.78	0.93	0.67
query23	15.21	1.37	0.66
query24	6.91	2.20	0.77
query25	0.50	0.20	0.19
query26	0.72	0.16	0.13
query27	0.05	0.05	0.06
query28	10.30	0.87	0.44
query29	12.55	3.92	3.22
query30	3.10	3.06	2.97
query31	2.83	0.59	0.38
query32	3.24	0.56	0.48
query33	3.12	3.06	3.10
query34	16.18	5.52	4.84
query35	4.94	4.95	4.93
query36	0.71	0.50	0.51
query37	0.10	0.08	0.07
query38	0.06	0.05	0.04
query39	0.03	0.04	0.03
query40	0.19	0.15	0.14
query41	0.08	0.03	0.03
query42	0.03	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 108.29 s
Total hot run time: 32.77 s

@924060929 924060929 force-pushed the master-opt-string-partition branch from 45cb6cd to 15446ad Compare August 27, 2025 15:18
@924060929
Copy link
Contributor Author

924060929 commented Aug 27, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 27, 2025

TPC-H: Total hot run time: 34301 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 15446ad5e2db48f58301c91e1c5849ea34a61713, data reload: false

------ Round 1 ----------------------------------
q1	17644	5270	5229	5229
q2	1979	311	215	215
q3	10255	1249	711	711
q4	10225	1024	531	531
q5	7547	2437	2506	2437
q6	190	171	140	140
q7	917	775	630	630
q8	9338	1299	1135	1135
q9	6769	5084	5093	5084
q10	6949	2373	1981	1981
q11	498	303	292	292
q12	359	357	231	231
q13	17799	3609	3050	3050
q14	238	241	216	216
q15	572	503	493	493
q16	421	434	399	399
q17	619	859	360	360
q18	7596	7177	6965	6965
q19	1336	952	576	576
q20	345	344	230	230
q21	3846	2543	2391	2391
q22	1074	1039	1005	1005
Total cold run time: 106516 ms
Total hot run time: 34301 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5204	5145	5253	5145
q2	250	332	240	240
q3	2186	2660	2325	2325
q4	1352	1838	1300	1300
q5	4211	4290	4525	4290
q6	222	171	135	135
q7	2083	1936	1884	1884
q8	2676	2749	2715	2715
q9	7290	7270	7261	7261
q10	3122	3276	2901	2901
q11	585	533	533	533
q12	694	771	633	633
q13	3701	3832	3314	3314
q14	322	338	292	292
q15	526	486	465	465
q16	466	491	459	459
q17	1339	1611	1365	1365
q18	7663	7927	7571	7571
q19	835	853	928	853
q20	2077	2100	1882	1882
q21	5033	4588	4414	4414
q22	1093	1062	996	996
Total cold run time: 52930 ms
Total hot run time: 50973 ms

@doris-robot
Copy link

doris-robot commented Aug 27, 2025

TPC-DS: Total hot run time: 187304 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 15446ad5e2db48f58301c91e1c5849ea34a61713, data reload: false

query1	1070	452	419	419
query2	6554	1811	1854	1811
query3	6751	232	222	222
query4	26334	23504	23104	23104
query5	4385	668	537	537
query6	343	258	242	242
query7	4663	527	308	308
query8	313	261	255	255
query9	8634	2948	2947	2947
query10	517	360	322	322
query11	15804	15194	14957	14957
query12	191	127	124	124
query13	1706	597	477	477
query14	9507	6239	6159	6159
query15	226	205	175	175
query16	7729	665	505	505
query17	1252	779	642	642
query18	2070	446	345	345
query19	265	190	168	168
query20	142	126	124	124
query21	220	133	120	120
query22	4149	4138	4221	4138
query23	33734	32919	33074	32919
query24	8215	2400	2419	2400
query25	552	517	440	440
query26	1244	284	170	170
query27	2689	511	355	355
query28	4349	2277	2237	2237
query29	777	597	493	493
query30	303	224	198	198
query31	928	843	746	746
query32	92	82	84	82
query33	585	395	363	363
query34	802	874	521	521
query35	812	821	785	785
query36	980	1023	946	946
query37	137	117	99	99
query38	4080	4026	3996	3996
query39	1498	1431	1452	1431
query40	222	134	127	127
query41	72	62	59	59
query42	128	123	118	118
query43	533	512	482	482
query44	1410	864	895	864
query45	187	182	169	169
query46	868	1030	646	646
query47	1792	1841	1742	1742
query48	399	437	325	325
query49	774	527	423	423
query50	660	683	412	412
query51	4103	4198	4003	4003
query52	128	121	114	114
query53	252	286	202	202
query54	617	617	548	548
query55	104	98	93	93
query56	374	346	325	325
query57	1217	1209	1125	1125
query58	297	288	279	279
query59	2664	2673	2664	2664
query60	361	351	344	344
query61	166	157	161	157
query62	835	773	658	658
query63	242	208	200	200
query64	4437	1138	862	862
query65	4339	4199	4214	4199
query66	1114	430	370	370
query67	15647	15366	15122	15122
query68	9073	934	596	596
query69	493	337	302	302
query70	1234	1167	1120	1120
query71	491	374	340	340
query72	5942	4917	5094	4917
query73	767	658	365	365
query74	9257	8721	8694	8694
query75	4335	3081	2625	2625
query76	3767	1168	786	786
query77	905	439	345	345
query78	9586	9662	8907	8907
query79	4985	847	591	591
query80	730	599	513	513
query81	496	256	226	226
query82	643	144	115	115
query83	297	262	249	249
query84	306	121	93	93
query85	896	466	434	434
query86	344	341	324	324
query87	4237	4274	4136	4136
query88	3261	2246	2257	2246
query89	460	329	291	291
query90	1972	239	239	239
query91	165	172	180	172
query92	85	78	71	71
query93	2837	982	654	654
query94	702	433	329	329
query95	424	338	325	325
query96	492	588	279	279
query97	2627	2756	2589	2589
query98	243	222	219	219
query99	1511	1389	1298	1298
Total cold run time: 282107 ms
Total hot run time: 187304 ms

@doris-robot
Copy link

doris-robot commented Aug 27, 2025

ClickBench: Total hot run time: 33.04 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 15446ad5e2db48f58301c91e1c5849ea34a61713, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.05	0.05
query3	0.26	0.08	0.08
query4	1.60	0.12	0.12
query5	0.43	0.43	0.43
query6	1.20	0.64	0.66
query7	0.03	0.03	0.03
query8	0.06	0.05	0.04
query9	0.60	0.52	0.52
query10	0.58	0.57	0.58
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.64	0.63	0.62
query14	0.81	0.84	0.84
query15	0.85	0.87	0.84
query16	0.40	0.40	0.40
query17	1.05	1.06	1.07
query18	0.21	0.20	0.19
query19	1.92	1.84	1.84
query20	0.02	0.02	0.01
query21	15.41	0.98	0.58
query22	0.78	1.28	0.75
query23	14.76	1.40	0.64
query24	6.58	1.54	0.97
query25	0.51	0.18	0.14
query26	0.50	0.15	0.14
query27	0.06	0.06	0.06
query28	10.10	0.94	0.42
query29	12.74	3.91	3.23
query30	3.03	2.97	2.99
query31	2.84	0.57	0.38
query32	3.25	0.56	0.47
query33	3.08	3.07	3.08
query34	16.19	5.48	4.89
query35	4.88	4.92	4.93
query36	0.68	0.51	0.50
query37	0.10	0.08	0.07
query38	0.05	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.15	0.14
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 107.06 s
Total hot run time: 33.04 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 27, 2025

FE UT Coverage Report

Increment line coverage 66.67% (2/3) 🎉
Increment coverage report
Complete coverage report

@924060929 924060929 force-pushed the master-opt-string-partition branch from 15446ad to 923901e Compare August 28, 2025 09:30
@924060929
Copy link
Contributor Author

924060929 commented Aug 28, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 28, 2025

TPC-H: Total hot run time: 34789 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 923901ebbe6819d43266f389f49e2b8a841ba3ba, data reload: false

------ Round 1 ----------------------------------
q1	17648	5206	5066	5066
q2	2023	330	247	247
q3	10186	1298	723	723
q4	10226	1026	521	521
q5	7560	2420	2323	2323
q6	192	172	136	136
q7	925	754	631	631
q8	9342	1291	1163	1163
q9	7065	5146	5158	5146
q10	6911	2390	2004	2004
q11	479	302	316	302
q12	346	365	235	235
q13	17743	3673	3031	3031
q14	243	242	230	230
q15	554	512	497	497
q16	428	422	374	374
q17	603	851	368	368
q18	7548	7065	7023	7023
q19	1224	962	580	580
q20	351	339	232	232
q21	3699	3187	2975	2975
q22	1095	1029	982	982
Total cold run time: 106391 ms
Total hot run time: 34789 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5159	5286	5074	5074
q2	248	328	224	224
q3	2153	2711	2285	2285
q4	1381	1769	1375	1375
q5	4210	4422	4596	4422
q6	249	182	132	132
q7	2076	1945	1854	1854
q8	2766	2729	2596	2596
q9	7459	7386	7254	7254
q10	3169	3315	3189	3189
q11	642	530	514	514
q12	715	789	646	646
q13	3691	3886	3323	3323
q14	302	311	293	293
q15	551	504	511	504
q16	462	530	487	487
q17	1199	1556	1402	1402
q18	8060	7741	7634	7634
q19	837	874	914	874
q20	2021	2049	1921	1921
q21	4908	4597	4284	4284
q22	1106	1047	1005	1005
Total cold run time: 53364 ms
Total hot run time: 51292 ms

@doris-robot
Copy link

doris-robot commented Aug 28, 2025

TPC-DS: Total hot run time: 186817 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 923901ebbe6819d43266f389f49e2b8a841ba3ba, data reload: false

query1	1052	480	424	424
query2	6560	1832	1788	1788
query3	6749	232	225	225
query4	26440	23784	22908	22908
query5	4784	683	523	523
query6	349	237	218	218
query7	4647	503	304	304
query8	290	264	246	246
query9	8611	2896	2889	2889
query10	541	360	307	307
query11	15839	15275	14821	14821
query12	187	120	121	120
query13	1677	566	426	426
query14	9886	5841	5802	5802
query15	214	198	182	182
query16	7696	682	463	463
query17	1221	771	661	661
query18	2068	448	357	357
query19	209	202	181	181
query20	137	127	133	127
query21	226	132	116	116
query22	4244	4223	4118	4118
query23	34240	33092	32843	32843
query24	8121	2381	2386	2381
query25	602	521	455	455
query26	1237	285	170	170
query27	2725	507	374	374
query28	4342	2286	2243	2243
query29	746	601	487	487
query30	294	224	205	205
query31	923	825	745	745
query32	97	90	83	83
query33	577	409	350	350
query34	801	853	536	536
query35	854	815	765	765
query36	973	1055	898	898
query37	129	113	99	99
query38	4103	4094	4048	4048
query39	1546	1443	1480	1443
query40	225	138	127	127
query41	70	68	62	62
query42	132	119	126	119
query43	522	516	483	483
query44	1357	858	856	856
query45	185	183	170	170
query46	863	1040	649	649
query47	1826	1821	1798	1798
query48	393	440	332	332
query49	760	505	419	419
query50	673	683	415	415
query51	4123	4093	4123	4093
query52	119	116	110	110
query53	252	271	208	208
query54	613	614	554	554
query55	95	93	98	93
query56	349	336	336	336
query57	1209	1221	1114	1114
query58	291	282	286	282
query59	2658	2783	2588	2588
query60	355	379	348	348
query61	173	156	159	156
query62	835	733	686	686
query63	232	202	198	198
query64	4325	1203	824	824
query65	4334	4229	4143	4143
query66	1134	459	332	332
query67	15576	15348	15282	15282
query68	7919	934	588	588
query69	490	336	294	294
query70	1217	1077	1139	1077
query71	479	353	328	328
query72	5857	4959	4848	4848
query73	659	597	361	361
query74	8903	9088	9026	9026
query75	3875	3071	2637	2637
query76	3578	1157	759	759
query77	822	428	344	344
query78	9625	9789	8893	8893
query79	1953	822	613	613
query80	697	604	632	604
query81	514	270	228	228
query82	209	145	117	117
query83	279	267	260	260
query84	261	113	94	94
query85	869	479	426	426
query86	393	344	301	301
query87	4214	4259	4254	4254
query88	2870	2230	2315	2230
query89	398	361	296	296
query90	1964	228	223	223
query91	156	171	137	137
query92	96	72	73	72
query93	1884	1014	646	646
query94	649	407	333	333
query95	419	337	326	326
query96	491	589	282	282
query97	2596	2734	2599	2599
query98	256	229	228	228
query99	1373	1414	1285	1285
Total cold run time: 275581 ms
Total hot run time: 186817 ms

@doris-robot
Copy link

doris-robot commented Aug 28, 2025

ClickBench: Total hot run time: 33.4 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 923901ebbe6819d43266f389f49e2b8a841ba3ba, data reload: false

query1	0.06	0.06	0.05
query2	0.10	0.05	0.06
query3	0.26	0.09	0.09
query4	1.61	0.11	0.11
query5	0.44	0.42	0.42
query6	1.19	0.65	0.67
query7	0.04	0.03	0.03
query8	0.06	0.04	0.04
query9	0.61	0.52	0.52
query10	0.60	0.57	0.57
query11	0.16	0.12	0.11
query12	0.16	0.13	0.12
query13	0.63	0.63	0.62
query14	0.80	0.84	0.88
query15	0.87	0.86	0.87
query16	0.40	0.40	0.40
query17	1.03	1.05	1.07
query18	0.21	0.20	0.20
query19	1.97	1.88	1.85
query20	0.02	0.01	0.02
query21	15.41	0.96	0.59
query22	0.76	1.16	0.70
query23	14.97	1.39	0.63
query24	6.24	2.19	1.19
query25	0.44	0.22	0.21
query26	0.55	0.15	0.14
query27	0.07	0.06	0.05
query28	10.46	0.93	0.43
query29	12.58	3.91	3.24
query30	3.09	2.97	3.02
query31	2.83	0.60	0.39
query32	3.24	0.56	0.47
query33	3.09	3.18	3.12
query34	15.96	5.46	4.88
query35	4.94	4.92	4.98
query36	0.73	0.53	0.50
query37	0.10	0.08	0.08
query38	0.06	0.04	0.04
query39	0.03	0.04	0.03
query40	0.18	0.14	0.14
query41	0.09	0.04	0.03
query42	0.04	0.03	0.04
query43	0.04	0.04	0.03
Total cold run time: 107.12 s
Total hot run time: 33.4 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 28, 2025

FE UT Coverage Report

Increment line coverage 66.67% (2/3) 🎉
Increment coverage report
Complete coverage report

yujun777
yujun777 approved these changes Aug 28, 2025
View reviewed changes
Copy link
Contributor

@yujun777 yujun777 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Aug 28, 2025

PR approved by anyone and no changes requested.

@924060929 924060929 merged commit 9a056fe into apache:master Aug 29, 2025
25 of 26 checks passed
@924060929 924060929 deleted the master-opt-string-partition branch August 29, 2025 02:08
924060929 added a commit to 924060929/incubator-doris that referenced this pull request Aug 29, 2025
@924060929
[opt](nereids) support simplify string range (apache#55378)
d9b63af 
support simplify string range:

```sql
where dt > '20250101' and dt > '20260101' -- simplify to dt > '20260101'
```

(cherry picked from commit 9a056fe)
@924060929 924060929 mentioned this pull request Aug 29, 2025
Merged
924060929 added a commit to 924060929/incubator-doris that referenced this pull request Aug 29, 2025
@924060929
[opt](nereids) support simplify string range (apache#55378)
05d9dea 
support simplify string range:

```sql
where dt > '20250101' and dt > '20260101' -- simplify to dt > '20260101'
```

(cherry picked from commit 9a056fe)
@924060929 924060929 mentioned this pull request Aug 29, 2025
Merged
924060929 added a commit to 924060929/incubator-doris that referenced this pull request Aug 29, 2025
@924060929
[opt](nereids) support simplify string range (apache#55378)
0d00407 
support simplify string range:

```sql
where dt > '20250101' and dt > '20260101' -- simplify to dt > '20260101'
```

(cherry picked from commit 9a056fe)
@924060929 924060929 mentioned this pull request Aug 29, 2025
Merged
924060929 added a commit to 924060929/incubator-doris that referenced this pull request Aug 29, 2025
@924060929
[opt](nereids) support simplify string range (apache#55378)
bd37916 
support simplify string range:

```sql
where dt > '20250101' and dt > '20260101' -- simplify to dt > '20260101'
```

(cherry picked from commit 9a056fe)
924060929 added a commit to 924060929/incubator-doris that referenced this pull request Aug 29, 2025
@924060929
[opt](nereids) support simplify string range (apache#55378)
b7d1626 
support simplify string range:

```sql
where dt > '20250101' and dt > '20260101' -- simplify to dt > '20260101'
```

(cherry picked from commit 9a056fe)
924060929 added a commit to 924060929/incubator-doris that referenced this pull request Aug 29, 2025
@924060929
[opt](nereids) support simplify string range (apache#55378)
29a97c1 
support simplify string range:

```sql
where dt > '20250101' and dt > '20260101' -- simplify to dt > '20260101'
```

(cherry picked from commit 9a056fe)
yiguolei pushed a commit that referenced this pull request Sep 3, 2025
@924060929
branch-2.1: [opt](nereids) support simplify string range (#55378) (#5…
ee60560 
…5459)

cherry pick from #55378
morrySnow pushed a commit that referenced this pull request Sep 4, 2025
@924060929
branch-3.1: [opt](nereids) support simplify string range #55378 (#55456)
433276d 
cherry pick from #55378
dataroaring pushed a commit that referenced this pull request Sep 4, 2025
@924060929
branch-3.0: [opt](nereids) support simplify string range (#55378) (#5…
dd64463 
…5454)

cherry pick from #55378
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

+1 more reviewer

@yujun777 yujun777 yujun777 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.12-merged dev/3.1.1-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@924060929 @hello-stephen @doris-robot @yujun777 @morrySnow @yiguolei