Skip to content

Conversation

@morrySnow
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #33087

Problem Summary:

CSE should not extract ArrayItemSlot and ArrayItemReference. Because they could not be computed out of Lambda expression.
NOTICE: currently, we could not extract common expression from Lambda, because ArrayItemSlot in Lambda are not same.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32603 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ea3dac8820f99d249b6a4d4a9e8f613fc1919fc2, data reload: false

------ Round 1 ----------------------------------
q1	24076	5011	4965	4965
q2	2041	285	169	169
q3	10413	1253	676	676
q4	10221	1002	536	536
q5	7508	2447	2406	2406
q6	181	163	132	132
q7	896	750	601	601
q8	9307	1304	1148	1148
q9	5039	4922	4706	4706
q10	6799	2312	1886	1886
q11	468	269	261	261
q12	342	352	212	212
q13	17785	3637	3103	3103
q14	222	241	210	210
q15	551	507	489	489
q16	638	663	575	575
q17	582	849	353	353
q18	7024	6594	6468	6468
q19	1070	966	549	549
q20	319	338	201	201
q21	2789	2146	1979	1979
q22	1048	1023	978	978
Total cold run time: 109319 ms
Total hot run time: 32603 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5131	5109	5067	5067
q2	236	329	233	233
q3	2145	2652	2285	2285
q4	1442	1817	1402	1402
q5	4231	4125	4393	4125
q6	216	166	123	123
q7	1975	1964	1746	1746
q8	2626	2472	2465	2465
q9	7127	7159	7238	7159
q10	2977	3220	2837	2837
q11	587	492	488	488
q12	691	756	611	611
q13	3504	3828	3265	3265
q14	278	287	269	269
q15	566	484	501	484
q16	648	698	618	618
q17	1121	1592	1364	1364
q18	7822	7554	7508	7508
q19	800	895	1010	895
q20	1971	2057	1886	1886
q21	5326	4832	4735	4735
q22	1075	1047	1024	1024
Total cold run time: 52495 ms
Total hot run time: 50589 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184858 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ea3dac8820f99d249b6a4d4a9e8f613fc1919fc2, data reload: false

query1	1009	490	471	471
query2	6553	1953	1899	1899
query3	6817	217	217	217
query4	26645	23616	23651	23616
query5	4737	669	481	481
query6	315	198	173	173
query7	4612	494	284	284
query8	277	230	219	219
query9	8636	2494	2492	2492
query10	462	319	246	246
query11	15885	15111	14787	14787
query12	164	106	104	104
query13	1652	515	387	387
query14	9897	6566	6231	6231
query15	202	186	169	169
query16	7253	632	472	472
query17	1157	735	535	535
query18	1963	389	286	286
query19	191	173	148	148
query20	116	116	111	111
query21	213	119	102	102
query22	4253	4332	4297	4297
query23	33938	32884	33279	32884
query24	7704	2366	2381	2366
query25	514	455	387	387
query26	1229	270	157	157
query27	2229	477	329	329
query28	4084	2363	2403	2363
query29	738	563	416	416
query30	290	216	190	190
query31	944	881	754	754
query32	71	66	62	62
query33	551	361	346	346
query34	785	832	501	501
query35	779	817	751	751
query36	956	985	877	877
query37	120	101	76	76
query38	4177	4117	4071	4071
query39	1446	1409	1384	1384
query40	213	117	101	101
query41	55	51	54	51
query42	116	105	101	101
query43	510	500	479	479
query44	1273	778	771	771
query45	175	172	167	167
query46	845	1027	618	618
query47	1773	1830	1726	1726
query48	363	411	294	294
query49	810	502	413	413
query50	689	733	402	402
query51	4199	4186	4203	4186
query52	114	110	103	103
query53	230	257	188	188
query54	504	530	419	419
query55	84	86	83	83
query56	277	271	255	255
query57	1104	1118	1090	1090
query58	242	241	229	229
query59	2764	2857	2518	2518
query60	284	294	261	261
query61	145	124	122	122
query62	792	726	654	654
query63	222	188	188	188
query64	4223	989	656	656
query65	4427	4319	4360	4319
query66	1185	433	302	302
query67	15638	15393	15214	15214
query68	5304	876	504	504
query69	453	303	262	262
query70	1216	1126	1090	1090
query71	399	292	276	276
query72	5493	3558	3803	3558
query73	731	718	340	340
query74	8992	9240	8682	8682
query75	3169	3180	2695	2695
query76	3255	1186	731	731
query77	492	372	282	282
query78	9870	10167	9273	9273
query79	1357	824	579	579
query80	1402	546	435	435
query81	507	266	213	213
query82	412	128	93	93
query83	180	177	147	147
query84	243	89	78	78
query85	752	438	300	300
query86	320	325	293	293
query87	4443	4484	4404	4404
query88	2843	2188	2192	2188
query89	379	310	281	281
query90	1738	211	209	209
query91	144	141	110	110
query92	66	65	58	58
query93	1162	1035	592	592
query94	679	423	296	296
query95	357	276	268	268
query96	480	560	268	268
query97	3301	3515	3311	3311
query98	225	202	196	196
query99	1313	1392	1295	1295
Total cold run time: 268497 ms
Total hot run time: 184858 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.24 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ea3dac8820f99d249b6a4d4a9e8f613fc1919fc2, data reload: false

query1	0.04	0.04	0.04
query2	0.13	0.10	0.11
query3	0.24	0.19	0.20
query4	1.59	0.20	0.19
query5	0.59	0.58	0.59
query6	1.18	0.71	0.71
query7	0.02	0.01	0.01
query8	0.04	0.04	0.04
query9	0.59	0.52	0.51
query10	0.59	0.60	0.56
query11	0.16	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.66	2.73	2.68
query15	0.90	0.84	0.87
query16	0.40	0.39	0.38
query17	1.02	1.01	1.02
query18	0.21	0.19	0.19
query19	1.92	1.90	1.88
query20	0.01	0.01	0.01
query21	15.36	0.94	0.54
query22	0.74	1.29	1.11
query23	15.09	1.37	0.63
query24	6.73	2.40	0.41
query25	0.35	0.15	0.15
query26	0.66	0.16	0.13
query27	0.05	0.04	0.05
query28	9.65	0.85	0.43
query29	12.54	3.89	3.31
query30	0.26	0.09	0.06
query31	2.86	0.61	0.39
query32	3.23	0.54	0.46
query33	3.03	2.98	2.98
query34	15.74	5.15	4.54
query35	4.54	4.55	4.56
query36	0.65	0.49	0.48
query37	0.08	0.06	0.07
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 104.98 s
Total hot run time: 31.24 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 18, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit 9d4faa5 into apache:master Mar 18, 2025
26 of 27 checks passed
@morrySnow morrySnow deleted the fix_cse_lambda branch March 18, 2025 12:45
@924060929 924060929 added usercase Important user case type label dev/2.1.x dev/3.0.x labels Apr 10, 2025
924060929 pushed a commit to 924060929/incubator-doris that referenced this pull request Apr 10, 2025
…pache#49166)

### What problem does this PR solve?

Related PR: apache#33087

Problem Summary:

CSE should not extract ArrayItemSlot and ArrayItemReference. Because
they could not be computed out of Lambda expression.
NOTICE: currently, we could not extract common expression from Lambda,
because ArrayItemSlot in Lambda are not same.

(cherry picked from commit 9d4faa5)
924060929 pushed a commit to 924060929/incubator-doris that referenced this pull request Apr 10, 2025
…pache#49166)

### What problem does this PR solve?

Related PR: apache#33087

Problem Summary:

CSE should not extract ArrayItemSlot and ArrayItemReference. Because
they could not be computed out of Lambda expression.
NOTICE: currently, we could not extract common expression from Lambda,
because ArrayItemSlot in Lambda are not same.

(cherry picked from commit 9d4faa5)
yiguolei pushed a commit that referenced this pull request Apr 10, 2025
…xpressions (#49166) (#49942)

cherry pick from #49166

Co-authored-by: morrySnow <zhangwenxin@selectdb.com>
dataroaring pushed a commit that referenced this pull request Apr 22, 2025
…xpressions #49166 (#49941)

cherry pick from #49166

Co-authored-by: morrySnow <zhangwenxin@selectdb.com>
@yiguolei yiguolei mentioned this pull request May 13, 2025
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
…pache#49166)

### What problem does this PR solve?

Related PR: apache#33087

Problem Summary:

CSE should not extract ArrayItemSlot and ArrayItemReference. Because
they could not be computed out of Lambda expression.
NOTICE: currently, we could not extract common expression from Lambda,
because ArrayItemSlot in Lambda are not same.
@gavinchou gavinchou mentioned this pull request Jun 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.10-merged dev/3.0.6-merged reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants