Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FE](session-variable) Add a debug variable of constant fold #35584

Merged
merged 1 commit into from
May 30, 2024

Conversation

zclllyybb
Copy link
Contributor

@zclllyybb zclllyybb commented May 29, 2024

Proposed changes

Issue Number: close #xxx

which force skip constant fold in nereids. make it easier for test, especially for SqlLogic platform

mysql> desc select date_add('2020-12-12', interval 1 day);
+---------------------------------+
| Explain String(Nereids Planner) |
+---------------------------------+
| PLAN FRAGMENT 0                 |
|   OUTPUT EXPRS:                 |
|     '2020-12-13 00:00:00'[#0]   |
|   PARTITION: UNPARTITIONED      |
|                                 |
|   HAS_COLO_PLAN_NODE: false     |
|                                 |
|   VRESULT SINK                  |
|      MYSQL_PROTOCAL             |
|                                 |
|   0:VUNION(32)                  |
|      constant exprs:            |
|          '2020-12-13 00:00:00'  |
+---------------------------------+
13 rows in set (0.18 sec)

mysql> set debug_skip_fold_constant=true;
Query OK, 0 rows affected (0.10 sec)

mysql> desc select date_add('2020-12-12', interval 1 day);
+-----------------------------------------------------------+
| Explain String(Nereids Planner)                           |
+-----------------------------------------------------------+
| PLAN FRAGMENT 0                                           |
|   OUTPUT EXPRS:                                           |
|     days_add(cast('2020-12-12' as DATETIMEV2(0)), 1)[#0]  |
|   PARTITION: UNPARTITIONED                                |
|                                                           |
|   HAS_COLO_PLAN_NODE: false                               |
|                                                           |
|   VRESULT SINK                                            |
|      MYSQL_PROTOCAL                                       |
|                                                           |
|   0:VUNION(30)                                            |
|      constant exprs:                                      |
|          days_add(CAST('2020-12-12' AS DATETIMEV2(0)), 1) |
+-----------------------------------------------------------+
13 rows in set (0.11 sec)

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 42492 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fb50a707372fba66295117c79f422636ab7f4d83, data reload: false

------ Round 1 ----------------------------------
q1	17618	4365	4365	4365
q2	2037	203	202	202
q3	10469	1306	1222	1222
q4	10197	822	873	822
q5	7500	2756	2741	2741
q6	224	134	137	134
q7	970	634	629	629
q8	9218	2183	2136	2136
q9	9507	6682	6802	6682
q10	9276	3971	3898	3898
q11	483	251	249	249
q12	422	229	221	221
q13	18488	3252	3175	3175
q14	269	223	236	223
q15	532	472	496	472
q16	535	416	409	409
q17	1031	677	691	677
q18	8271	7923	7901	7901
q19	3396	1563	1565	1563
q20	643	323	328	323
q21	5185	4160	4171	4160
q22	376	301	288	288
Total cold run time: 116647 ms
Total hot run time: 42492 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4640	4453	4445	4445
q2	397	277	278	277
q3	3181	2966	2891	2891
q4	2039	1766	1705	1705
q5	5405	5528	5537	5528
q6	222	129	130	129
q7	2292	1847	1849	1847
q8	3331	3467	3531	3467
q9	8753	8721	8774	8721
q10	4089	3889	3690	3690
q11	593	508	494	494
q12	763	607	634	607
q13	16997	3193	3182	3182
q14	296	264	284	264
q15	528	498	494	494
q16	509	449	448	448
q17	1914	1574	1559	1559
q18	7965	7520	7525	7520
q19	2776	1659	1629	1629
q20	2072	1818	1833	1818
q21	12906	4997	4757	4757
q22	586	502	496	496
Total cold run time: 82254 ms
Total hot run time: 55968 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169893 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fb50a707372fba66295117c79f422636ab7f4d83, data reload: false

query1	935	386	373	373
query2	7446	2426	2361	2361
query3	6643	208	207	207
query4	19562	17335	17240	17240
query5	4174	431	411	411
query6	251	157	147	147
query7	4595	301	288	288
query8	241	178	186	178
query9	8449	2485	2457	2457
query10	464	300	265	265
query11	10737	10248	10069	10069
query12	135	86	88	86
query13	1633	365	362	362
query14	10256	6888	7551	6888
query15	219	175	170	170
query16	7744	284	262	262
query17	1609	520	517	517
query18	1950	274	270	270
query19	196	148	146	146
query20	90	82	83	82
query21	195	140	131	131
query22	4134	3953	3915	3915
query23	33751	33011	33106	33011
query24	7699	2842	2899	2842
query25	577	350	365	350
query26	707	153	157	153
query27	2201	321	321	321
query28	5328	2084	2090	2084
query29	862	605	641	605
query30	244	150	151	150
query31	965	763	751	751
query32	97	54	55	54
query33	535	268	268	268
query34	865	486	488	486
query35	697	626	584	584
query36	1039	915	918	915
query37	104	70	69	69
query38	2919	2797	2755	2755
query39	853	799	805	799
query40	199	125	123	123
query41	45	45	46	45
query42	104	97	96	96
query43	608	568	564	564
query44	1111	730	746	730
query45	183	171	167	167
query46	1066	741	727	727
query47	1845	1795	1778	1778
query48	361	313	319	313
query49	849	389	437	389
query50	773	387	384	384
query51	6815	6748	6651	6651
query52	97	93	89	89
query53	350	294	288	288
query54	545	464	445	445
query55	72	74	74	74
query56	261	238	255	238
query57	1097	1046	1081	1046
query58	227	206	215	206
query59	3466	3293	3207	3207
query60	279	261	262	261
query61	94	89	89	89
query62	618	456	479	456
query63	312	295	290	290
query64	8511	2231	1772	1772
query65	3174	3100	3150	3100
query66	786	339	314	314
query67	15220	14828	14982	14828
query68	4537	558	549	549
query69	471	270	271	270
query70	1196	1147	1192	1147
query71	399	268	264	264
query72	7631	5702	2731	2731
query73	724	327	322	322
query74	6041	5637	5595	5595
query75	3394	2638	2626	2626
query76	2639	1009	943	943
query77	618	271	278	271
query78	10193	9905	9775	9775
query79	2064	518	513	513
query80	802	429	434	429
query81	478	217	217	217
query82	1152	94	89	89
query83	189	168	171	168
query84	252	90	86	86
query85	1272	275	270	270
query86	424	281	313	281
query87	3280	3100	3158	3100
query88	3965	2440	2447	2440
query89	470	394	386	386
query90	2024	191	189	189
query91	169	95	96	95
query92	56	49	47	47
query93	2372	519	509	509
query94	1174	186	185	185
query95	403	321	309	309
query96	595	279	267	267
query97	3164	3042	3037	3037
query98	245	220	216	216
query99	1174	882	875	875
Total cold run time: 263736 ms
Total hot run time: 169893 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.39 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fb50a707372fba66295117c79f422636ab7f4d83, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.24	0.05	0.04
query4	1.66	0.08	0.07
query5	0.50	0.49	0.50
query6	1.12	0.73	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.53	0.49	0.49
query10	0.56	0.56	0.55
query11	0.16	0.12	0.11
query12	0.15	0.13	0.11
query13	0.60	0.58	0.59
query14	0.77	0.82	0.78
query15	0.82	0.81	0.81
query16	0.37	0.37	0.37
query17	1.01	1.04	0.98
query18	0.20	0.26	0.24
query19	1.90	1.69	1.79
query20	0.01	0.02	0.01
query21	15.44	0.70	0.67
query22	3.96	6.83	1.69
query23	18.31	1.37	1.25
query24	1.67	0.25	0.22
query25	0.15	0.09	0.08
query26	0.26	0.16	0.17
query27	0.09	0.07	0.08
query28	13.43	1.00	1.01
query29	12.73	3.32	3.27
query30	0.25	0.06	0.06
query31	2.88	0.37	0.37
query32	3.29	0.47	0.47
query33	2.88	2.91	2.90
query34	17.26	4.48	4.42
query35	4.47	4.55	4.58
query36	0.67	0.51	0.47
query37	0.19	0.15	0.15
query38	0.15	0.14	0.15
query39	0.04	0.03	0.04
query40	0.16	0.14	0.14
query41	0.10	0.05	0.04
query42	0.06	0.04	0.04
query43	0.04	0.04	0.03
Total cold run time: 109.28 s
Total hot run time: 30.39 s

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40693 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 411275a5c4a5dabd8db0a85212de1ee3a9b875d4, data reload: false

------ Round 1 ----------------------------------
q1	17614	4390	4228	4228
q2	2035	196	190	190
q3	10461	1233	1179	1179
q4	10201	839	814	814
q5	7481	2723	2651	2651
q6	227	129	129	129
q7	972	615	585	585
q8	9217	2122	2065	2065
q9	9740	6570	6668	6570
q10	9052	3858	3893	3858
q11	444	239	239	239
q12	429	220	229	220
q13	17353	3272	3184	3184
q14	268	223	237	223
q15	534	487	475	475
q16	509	390	400	390
q17	964	711	690	690
q18	8361	7797	7675	7675
q19	5518	1609	1584	1584
q20	633	313	318	313
q21	5141	4190	3103	3103
q22	387	335	328	328
Total cold run time: 117541 ms
Total hot run time: 40693 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4572	4356	4396	4356
q2	366	269	281	269
q3	3083	2861	2854	2854
q4	1937	1661	1819	1661
q5	5384	5387	5493	5387
q6	211	124	123	123
q7	2216	1844	1806	1806
q8	3242	3395	3418	3395
q9	8598	8577	8641	8577
q10	4100	3853	3758	3758
q11	586	479	496	479
q12	756	595	599	595
q13	15919	3122	3131	3122
q14	280	273	267	267
q15	526	489	516	489
q16	500	432	431	431
q17	1800	1516	1506	1506
q18	8044	7674	7463	7463
q19	1665	1613	1560	1560
q20	2003	1781	1781	1781
q21	10473	4603	4747	4603
q22	564	543	514	514
Total cold run time: 76825 ms
Total hot run time: 54996 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 167844 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 411275a5c4a5dabd8db0a85212de1ee3a9b875d4, data reload: false

query1	930	376	365	365
query2	6448	2558	2382	2382
query3	6640	212	213	212
query4	19816	17297	17257	17257
query5	4147	424	408	408
query6	255	178	153	153
query7	4576	291	284	284
query8	332	277	281	277
query9	8513	2367	2356	2356
query10	450	277	254	254
query11	10648	10016	10057	10016
query12	127	89	86	86
query13	1645	357	354	354
query14	10086	6855	5940	5940
query15	242	184	188	184
query16	7742	264	251	251
query17	1755	516	501	501
query18	1957	268	267	267
query19	201	155	150	150
query20	93	82	84	82
query21	206	139	131	131
query22	4409	4052	3918	3918
query23	33471	33061	32934	32934
query24	6922	2824	2781	2781
query25	569	351	356	351
query26	706	150	155	150
query27	2141	310	321	310
query28	4418	2050	2038	2038
query29	842	605	595	595
query30	243	153	151	151
query31	974	774	741	741
query32	86	53	55	53
query33	506	271	263	263
query34	869	467	464	464
query35	708	596	579	579
query36	1037	949	926	926
query37	104	67	65	65
query38	2907	2776	2758	2758
query39	838	790	811	790
query40	197	122	125	122
query41	51	53	51	51
query42	103	96	95	95
query43	604	567	531	531
query44	1080	717	728	717
query45	198	174	170	170
query46	1053	763	702	702
query47	1867	1744	1805	1744
query48	350	294	292	292
query49	850	373	385	373
query50	777	380	378	378
query51	6841	6672	6724	6672
query52	105	89	90	89
query53	352	289	283	283
query54	527	420	435	420
query55	74	72	73	72
query56	262	235	250	235
query57	1105	1046	1053	1046
query58	225	228	202	202
query59	3487	3111	3207	3111
query60	274	258	253	253
query61	125	92	87	87
query62	589	444	444	444
query63	313	282	287	282
query64	8482	2173	1669	1669
query65	3152	3076	3098	3076
query66	793	332	329	329
query67	15327	14801	14950	14801
query68	4500	533	526	526
query69	449	273	275	273
query70	1128	1137	1167	1137
query71	375	276	269	269
query72	7336	5682	2750	2750
query73	716	314	314	314
query74	6037	5700	5580	5580
query75	3281	2645	2595	2595
query76	2242	1034	958	958
query77	405	259	260	259
query78	10230	9756	9707	9707
query79	2074	505	505	505
query80	988	430	426	426
query81	520	220	221	220
query82	1296	90	96	90
query83	216	176	175	175
query84	245	84	80	80
query85	1357	269	262	262
query86	464	316	303	303
query87	3363	3086	3131	3086
query88	3748	2344	2315	2315
query89	477	390	385	385
query90	2051	188	189	188
query91	119	95	95	95
query92	60	48	48	48
query93	2486	494	487	487
query94	1237	182	180	180
query95	394	306	294	294
query96	579	267	265	265
query97	3192	3032	2999	2999
query98	242	223	210	210
query99	1141	847	840	840
Total cold run time: 260670 ms
Total hot run time: 167844 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.39 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 411275a5c4a5dabd8db0a85212de1ee3a9b875d4, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.04
query4	1.69	0.06	0.06
query5	0.51	0.48	0.50
query6	1.14	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.54	0.50	0.50
query10	0.53	0.55	0.53
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.61
query14	0.76	0.77	0.77
query15	0.82	0.79	0.80
query16	0.36	0.38	0.37
query17	0.98	0.94	0.98
query18	0.22	0.26	0.21
query19	1.74	1.64	1.70
query20	0.02	0.01	0.01
query21	15.49	0.69	0.68
query22	4.62	6.71	1.87
query23	18.27	1.35	1.32
query24	2.00	0.23	0.21
query25	0.15	0.08	0.08
query26	0.27	0.16	0.17
query27	0.08	0.08	0.08
query28	13.27	1.02	0.99
query29	13.14	3.30	3.30
query30	0.24	0.05	0.05
query31	2.89	0.39	0.37
query32	3.27	0.47	0.47
query33	2.91	2.90	2.92
query34	17.30	4.37	4.45
query35	4.45	4.51	4.59
query36	0.68	0.46	0.46
query37	0.18	0.16	0.15
query38	0.16	0.15	0.14
query39	0.04	0.04	0.03
query40	0.16	0.14	0.14
query41	0.09	0.04	0.04
query42	0.05	0.05	0.04
query43	0.04	0.03	0.04
Total cold run time: 110.37 s
Total hot run time: 30.39 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 30, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@morrySnow
Copy link
Contributor

run external

@zhangstar333 zhangstar333 merged commit 94db52c into apache:master May 30, 2024
27 of 29 checks passed
@zclllyybb zclllyybb deleted the skip_fold branch May 30, 2024 13:48
dataroaring pushed a commit that referenced this pull request May 31, 2024
which force skip constant fold in nereids. make it easier for test,
especially for SqlLogic platform
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Sep 20, 2024
…35584)

which force skip constant fold in nereids. make it easier for test,
especially for SqlLogic platform
LiBinfeng-01 pushed a commit to LiBinfeng-01/doris that referenced this pull request Oct 8, 2024
…35584)

which force skip constant fold in nereids. make it easier for test,
especially for SqlLogic platform
LiBinfeng-01 pushed a commit to LiBinfeng-01/doris that referenced this pull request Oct 10, 2024
…35584)

which force skip constant fold in nereids. make it easier for test,
especially for SqlLogic platform
LiBinfeng-01 pushed a commit to LiBinfeng-01/doris that referenced this pull request Oct 11, 2024
…35584)

which force skip constant fold in nereids. make it easier for test,
especially for SqlLogic platform
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.0-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants