Skip to content

branch-3.0: [opt](nereids) support simplify string range (#55378) #55454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Sep 4, 2025
Merged

branch-3.0: [opt](nereids) support simplify string range (#55378) #55454

dataroaring merged 1 commit into from
Sep 4, 2025

Conversation

@924060929
Copy link
Contributor

@924060929 924060929 commented Aug 29, 2025

cherry pick from #55378

@924060929 924060929 requested a review from dataroaring as a code owner August 29, 2025 02:29
@Thearas
Copy link
Contributor

Thearas commented Aug 29, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929
Copy link
Contributor Author

924060929 commented Aug 29, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 29, 2025

FE UT Coverage Report

Increment line coverage 66.67% (2/3) 🎉
Increment coverage report
Complete coverage report

@924060929
[opt](nereids) support simplify string range (apache#55378)
bd37916 
support simplify string range:

```sql
where dt > '20250101' and dt > '20260101' -- simplify to dt > '20260101'
```

(cherry picked from commit 9a056fe)
@924060929 924060929 force-pushed the branch-3.0-opt-string-partition branch from 85eca7d to bd37916 Compare August 29, 2025 10:29
@924060929
Copy link
Contributor Author

924060929 commented Aug 29, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

TPC-H: Total hot run time: 39749 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bd37916f1b512eee132d6b94db986bbf6dc12726, data reload: false

------ Round 1 ----------------------------------
q1	17632	6893	6710	6710
q2	2061	211	170	170
q3	10645	1129	1192	1129
q4	10511	737	728	728
q5	7735	2869	2810	2810
q6	212	131	136	131
q7	969	619	603	603
q8	9356	1967	2012	1967
q9	6676	6422	6365	6365
q10	7046	2286	2294	2286
q11	467	263	263	263
q12	396	215	217	215
q13	17793	2990	2960	2960
q14	242	198	205	198
q15	507	453	450	450
q16	482	373	375	373
q17	998	633	522	522
q18	7143	6541	6603	6541
q19	1398	1042	1115	1042
q20	481	203	209	203
q21	3941	3137	3115	3115
q22	1085	976	968	968
Total cold run time: 107776 ms
Total hot run time: 39749 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6598	6550	6549	6549
q2	333	225	237	225
q3	2863	2885	2931	2885
q4	2077	1826	1866	1826
q5	5716	5769	5718	5718
q6	215	133	136	133
q7	2199	1805	1793	1793
q8	3348	3500	3567	3500
q9	8798	8934	8859	8859
q10	3572	3515	3506	3506
q11	581	482	505	482
q12	829	624	596	596
q13	7892	3194	3132	3132
q14	308	263	271	263
q15	529	473	471	471
q16	527	449	478	449
q17	1907	1685	1643	1643
q18	8580	8246	7756	7756
q19	1663	1460	1634	1460
q20	2047	1870	1888	1870
q21	5146	5126	5036	5036
q22	1114	1050	1009	1009
Total cold run time: 66842 ms
Total hot run time: 59161 ms

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

TPC-DS: Total hot run time: 193010 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bd37916f1b512eee132d6b94db986bbf6dc12726, data reload: false

query1	956	403	421	403
query2	6358	1929	1890	1890
query3	8687	211	208	208
query4	33369	23473	23575	23473
query5	3703	466	442	442
query6	289	189	196	189
query7	4200	319	329	319
query8	310	230	229	229
query9	9394	2595	2571	2571
query10	476	274	265	265
query11	18125	15184	15240	15184
query12	162	99	104	99
query13	1561	457	430	430
query14	8825	7540	7108	7108
query15	263	177	177	177
query16	7994	452	498	452
query17	1632	634	582	582
query18	2121	323	335	323
query19	270	171	165	165
query20	123	117	115	115
query21	208	108	109	108
query22	4769	4593	4562	4562
query23	34678	34495	34018	34018
query24	11387	2931	2995	2931
query25	667	432	449	432
query26	1682	181	180	180
query27	2892	372	367	367
query28	7892	2184	2158	2158
query29	988	487	445	445
query30	275	166	161	161
query31	1037	819	864	819
query32	108	60	64	60
query33	798	317	312	312
query34	965	542	522	522
query35	850	741	774	741
query36	1093	948	967	948
query37	138	71	65	65
query38	4106	3983	3953	3953
query39	1527	1463	1457	1457
query40	259	101	103	101
query41	54	47	50	47
query42	120	100	100	100
query43	507	476	480	476
query44	1249	817	809	809
query45	198	175	169	169
query46	1188	739	751	739
query47	2044	1897	1956	1897
query48	477	391	388	388
query49	1109	427	394	394
query50	833	437	433	433
query51	7497	7293	7284	7284
query52	99	94	93	93
query53	263	190	189	189
query54	1206	483	484	483
query55	83	80	81	80
query56	281	251	261	251
query57	1343	1205	1199	1199
query58	228	222	215	215
query59	3283	3124	3139	3124
query60	298	273	261	261
query61	120	114	123	114
query62	851	707	697	697
query63	228	195	197	195
query64	4942	686	663	663
query65	3400	3296	3345	3296
query66	1391	312	304	304
query67	15950	15568	15438	15438
query68	4830	608	582	582
query69	450	269	267	267
query70	1179	1091	1070	1070
query71	344	261	257	257
query72	6240	4150	4059	4059
query73	768	356	351	351
query74	10424	9079	9153	9079
query75	3357	2613	2662	2613
query76	2723	1173	1073	1073
query77	373	297	275	275
query78	10613	9694	9591	9591
query79	2184	596	606	596
query80	841	439	430	430
query81	541	218	212	212
query82	461	86	86	86
query83	246	143	139	139
query84	236	80	78	78
query85	1466	311	293	293
query86	456	284	274	274
query87	4384	4338	4270	4270
query88	4483	2399	2416	2399
query89	412	307	302	302
query90	1986	181	181	181
query91	186	149	154	149
query92	59	46	49	46
query93	2167	568	579	568
query94	797	312	307	307
query95	363	265	271	265
query96	617	286	290	286
query97	3317	3123	3130	3123
query98	227	197	198	197
query99	1541	1310	1290	1290
Total cold run time: 302650 ms
Total hot run time: 193010 ms

@doris-robot
Copy link

doris-robot commented Aug 29, 2025

ClickBench: Total hot run time: 30.19 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bd37916f1b512eee132d6b94db986bbf6dc12726, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.02
query3	0.23	0.06	0.06
query4	1.62	0.11	0.10
query5	0.53	0.51	0.51
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.57	0.50	0.50
query10	0.55	0.55	0.56
query11	0.14	0.10	0.11
query12	0.14	0.12	0.10
query13	0.61	0.60	0.59
query14	0.78	0.82	0.79
query15	0.85	0.83	0.83
query16	0.41	0.38	0.39
query17	1.07	1.06	1.06
query18	0.24	0.24	0.22
query19	1.93	1.87	1.75
query20	0.01	0.01	0.01
query21	15.38	0.59	0.58
query22	2.49	2.39	1.41
query23	17.06	0.90	0.95
query24	3.54	1.76	1.13
query25	0.25	0.18	0.08
query26	0.42	0.16	0.15
query27	0.05	0.04	0.05
query28	9.24	0.55	0.49
query29	12.57	3.34	3.31
query30	0.25	0.06	0.06
query31	2.84	0.40	0.40
query32	3.24	0.46	0.46
query33	3.06	3.00	3.04
query34	17.05	4.49	4.59
query35	4.63	4.66	4.53
query36	0.67	0.50	0.49
query37	0.08	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.03
query40	0.16	0.12	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 104.17 s
Total hot run time: 30.19 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 29, 2025

FE UT Coverage Report

Increment line coverage 66.67% (2/3) 🎉
Increment coverage report
Complete coverage report

dataroaring
dataroaring approved these changes Sep 4, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit dd64463 into apache:branch-3.0 Sep 4, 2025
24 of 25 checks passed
@924060929 924060929 deleted the branch-3.0-opt-string-partition branch September 4, 2025 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@924060929 @Thearas @hello-stephen @doris-robot @dataroaring