[fix](rbo) use wrong child's output of set operation #59105

morrySnow · 2025-12-17T04:11:35Z

What problem does this PR solve?

Related PR: #22203

Problem Summary:

This pull request refines the logic in the EliminateEmptyRelation rewrite rule, focusing on how set operations (UNION and EXCEPT) handle empty relations in query plans. The changes improve efficiency and code clarity by processing only when necessary, optimizing builder usage, and simplifying project and aggregate node construction.

Set Operation Handling Improvements:

Updated the logic for UNION nodes to only process and eliminate empty relation children when at least one is present, reducing unnecessary computation. Builders are now initialized with expected sizes for efficiency, and redundant code for handling constant expressions was removed.
Improved handling of EXCEPT nodes by checking for empty relations only among the right-hand children, and optimizing builder initialization. The construction of aggregate and project nodes now uses the correct outputs directly, streamlining the process.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-12-17T04:11:40Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

morrySnow · 2025-12-17T04:11:42Z

run buildall

doris-robot · 2025-12-17T04:31:27Z

TPC-H: Total hot run time: 35024 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 240567e8a63d41a225d4b78e318f30dd1d245d05, data reload: false

------ Round 1 ----------------------------------
q1	17633	4299	4055	4055
q2	2016	360	250	250
q3	10199	1324	750	750
q4	10229	875	316	316
q5	7612	2141	1968	1968
q6	190	171	136	136
q7	1017	838	719	719
q8	9373	1408	1152	1152
q9	7062	5306	5383	5306
q10	6788	2402	1960	1960
q11	538	336	304	304
q12	660	739	573	573
q13	17784	3697	3070	3070
q14	282	286	270	270
q15	573	515	524	515
q16	709	680	632	632
q17	695	832	535	535
q18	7460	7093	7166	7093
q19	1102	962	600	600
q20	397	369	256	256
q21	4194	3920	3585	3585
q22	1045	1033	979	979
Total cold run time: 107558 ms
Total hot run time: 35024 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4098	4033	4065	4033
q2	339	400	321	321
q3	2119	2676	2283	2283
q4	1346	1757	1283	1283
q5	4273	4619	4761	4619
q6	231	178	132	132
q7	2057	1936	1805	1805
q8	2705	2522	2558	2522
q9	7788	7728	7667	7667
q10	3073	3339	2846	2846
q11	592	507	495	495
q12	757	758	586	586
q13	3565	3872	3327	3327
q14	299	324	283	283
q15	557	514	516	514
q16	654	688	631	631
q17	1200	1413	1417	1413
q18	8004	7738	7685	7685
q19	880	870	924	870
q20	2072	2061	1962	1962
q21	4886	4577	4139	4139
q22	1070	1025	986	986
Total cold run time: 52565 ms
Total hot run time: 50402 ms

doris-robot · 2025-12-17T04:42:33Z

TPC-DS: Total hot run time: 178287 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 240567e8a63d41a225d4b78e318f30dd1d245d05, data reload: false

query5	4443	602	427	427
query6	326	225	219	219
query7	4228	468	265	265
query8	309	252	235	235
query9	8754	2519	2553	2519
query10	465	405	328	328
query11	15194	14772	14504	14504
query12	178	113	113	113
query13	1244	485	392	392
query14	5480	2997	2772	2772
query14_1	2674	2634	2641	2634
query15	212	199	189	189
query16	926	479	424	424
query17	979	703	609	609
query18	2433	449	351	351
query19	237	240	206	206
query20	121	116	116	116
query21	218	150	119	119
query22	4128	4195	4149	4149
query23	16596	16297	15905	15905
query23_1	16092	16084	16104	16084
query24	7435	1662	1232	1232
query24_1	1260	1238	1249	1238
query25	593	508	456	456
query26	1246	272	159	159
query27	2766	457	315	315
query28	4482	2141	2139	2139
query29	854	610	429	429
query30	314	241	211	211
query31	808	703	650	650
query32	76	70	66	66
query33	524	337	309	309
query34	938	909	534	534
query35	779	821	720	720
query36	882	904	837	837
query37	127	97	74	74
query38	2855	2902	2807	2807
query39	762	744	715	715
query39_1	698	707	710	707
query40	220	139	120	120
query41	66	64	62	62
query42	108	104	106	104
query43	432	435	415	415
query44	1363	751	740	740
query45	197	192	182	182
query46	872	977	618	618
query47	1700	1696	1631	1631
query48	322	326	248	248
query49	632	445	350	350
query50	644	300	210	210
query51	3895	3849	3873	3849
query52	102	110	102	102
query53	315	349	295	295
query54	334	262	243	243
query55	83	74	72	72
query56	297	288	297	288
query57	1153	1137	1073	1073
query58	271	266	247	247
query59	2409	2491	2300	2300
query60	302	308	284	284
query61	160	155	155	155
query62	697	654	612	612
query63	329	294	309	294
query64	5134	1294	983	983
query65	4013	3965	3931	3931
query66	1478	438	318	318
query67	15625	15075	15043	15043
query68	6688	989	732	732
query69	507	341	310	310
query70	1058	1029	921	921
query71	367	300	276	276
query72	6211	4955	5123	4955
query73	696	612	319	319
query74	8883	8716	8622	8622
query75	3182	3161	2804	2804
query76	3835	1123	739	739
query77	530	375	298	298
query78	9509	9616	8935	8935
query79	1612	922	617	617
query80	1433	657	550	550
query81	570	265	232	232
query82	407	127	103	103
query83	264	254	240	240
query84	309	124	107	107
query85	957	511	461	461
query86	459	288	285	285
query87	3104	3046	2941	2941
query88	3262	2252	2241	2241
query89	466	415	391	391
query90	2050	158	151	151
query91	172	165	155	155
query92	77	67	63	63
query93	1423	902	560	560
query94	528	298	263	263
query95	578	329	310	310
query96	602	456	207	207
query97	2278	2315	2213	2213
query98	215	196	191	191
query99	1288	1304	1162	1162
Total cold run time: 259387 ms
Total hot run time: 178287 ms

doris-robot · 2025-12-17T04:47:35Z

ClickBench: Total hot run time: 27.19 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 240567e8a63d41a225d4b78e318f30dd1d245d05, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.05	0.05
query3	0.25	0.08	0.08
query4	1.60	0.11	0.12
query5	0.28	0.26	0.26
query6	1.18	0.64	0.63
query7	0.03	0.03	0.02
query8	0.05	0.04	0.04
query9	0.57	0.51	0.49
query10	0.57	0.56	0.54
query11	0.16	0.10	0.11
query12	0.14	0.12	0.12
query13	0.61	0.60	0.58
query14	0.98	0.97	0.97
query15	0.81	0.80	0.80
query16	0.42	0.39	0.41
query17	1.05	1.07	1.01
query18	0.23	0.21	0.21
query19	1.92	1.75	1.81
query20	0.02	0.01	0.01
query21	15.45	0.29	0.15
query22	5.05	0.06	0.04
query23	16.16	0.28	0.10
query24	1.23	0.50	0.25
query25	0.09	0.08	0.08
query26	0.15	0.13	0.14
query27	0.08	0.05	0.05
query28	3.69	1.22	1.02
query29	12.64	3.99	3.24
query30	0.28	0.13	0.11
query31	2.82	0.62	0.41
query32	3.22	0.54	0.45
query33	3.04	3.07	3.00
query34	16.57	5.17	4.58
query35	4.56	4.54	4.54
query36	0.67	0.51	0.48
query37	0.10	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.14	0.13
query41	0.09	0.03	0.03
query42	0.05	0.03	0.02
query43	0.04	0.03	0.04
Total cold run time: 97.26 s
Total hot run time: 27.19 s

morrySnow · 2025-12-17T07:54:22Z

run buildall

doris-robot · 2025-12-17T09:13:54Z

TPC-H: Total hot run time: 36472 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 711fba5e489f2c8841df0b615d7c31cb5bce679e, data reload: false

------ Round 1 ----------------------------------
q1	17635	4211	4148	4148
q2	2026	354	244	244
q3	10174	1421	735	735
q4	10230	904	318	318
q5	7569	2183	1917	1917
q6	190	173	134	134
q7	1011	857	725	725
q8	9397	1429	1130	1130
q9	7180	5307	5347	5307
q10	6868	2380	1956	1956
q11	518	324	302	302
q12	733	757	573	573
q13	17770	3731	3071	3071
q14	294	306	277	277
q15	606	528	512	512
q16	680	690	654	654
q17	708	813	558	558
q18	7537	7862	7888	7862
q19	1207	1032	640	640
q20	444	389	264	264
q21	4603	4362	4100	4100
q22	1134	1098	1045	1045
Total cold run time: 108514 ms
Total hot run time: 36472 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4375	4281	4321	4281
q2	335	416	334	334
q3	2708	2797	2402	2402
q4	1479	1863	1440	1440
q5	4633	4492	4487	4487
q6	211	162	124	124
q7	2046	1933	1834	1834
q8	2655	2619	2468	2468
q9	7578	7386	7528	7386
q10	3064	3114	2671	2671
q11	566	493	469	469
q12	670	734	545	545
q13	3354	3754	3048	3048
q14	279	283	255	255
q15	539	505	499	499
q16	611	643	639	639
q17	1110	1355	1353	1353
q18	7284	7148	7003	7003
q19	855	811	825	811
q20	1905	1967	1789	1789
q21	4726	4254	4169	4169
q22	1106	1012	969	969
Total cold run time: 52089 ms
Total hot run time: 48976 ms

doris-robot · 2025-12-17T09:24:59Z

TPC-DS: Total hot run time: 175826 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 711fba5e489f2c8841df0b615d7c31cb5bce679e, data reload: false

query5	4877	615	452	452
query6	343	243	231	231
query7	4229	479	281	281
query8	311	252	253	252
query9	8788	2539	2539	2539
query10	538	392	342	342
query11	15603	14789	14604	14604
query12	177	123	120	120
query13	1263	506	403	403
query14	6184	3074	2812	2812
query14_1	2734	2775	2664	2664
query15	208	202	181	181
query16	918	473	436	436
query17	1118	734	610	610
query18	2551	448	345	345
query19	241	236	212	212
query20	126	120	113	113
query21	221	142	124	124
query22	3966	3976	3944	3944
query23	16597	16294	15800	15800
query23_1	16004	15976	15904	15904
query24	7328	1667	1247	1247
query24_1	1249	1240	1251	1240
query25	596	512	456	456
query26	1255	279	171	171
query27	2761	471	307	307
query28	4461	2151	2141	2141
query29	842	573	465	465
query30	313	250	215	215
query31	825	721	636	636
query32	82	68	68	68
query33	563	355	303	303
query34	914	924	540	540
query35	779	857	712	712
query36	860	906	815	815
query37	130	92	78	78
query38	2861	2893	2865	2865
query39	746	753	728	728
query39_1	698	708	692	692
query40	233	133	118	118
query41	67	61	59	59
query42	106	105	104	104
query43	427	440	394	394
query44	1329	749	751	749
query45	185	186	184	184
query46	873	970	607	607
query47	1624	1708	1599	1599
query48	326	329	241	241
query49	615	448	355	355
query50	651	306	220	220
query51	3826	3854	3855	3854
query52	110	109	97	97
query53	330	343	292	292
query54	311	255	246	246
query55	83	79	73	73
query56	296	306	297	297
query57	1148	1129	1129	1129
query58	291	253	258	253
query59	2441	2400	2404	2400
query60	309	309	294	294
query61	165	157	157	157
query62	712	668	630	630
query63	331	293	296	293
query64	4970	1342	1018	1018
query65	4056	3960	4014	3960
query66	1436	435	314	314
query67	15534	14856	14669	14669
query68	7545	1013	736	736
query69	502	352	310	310
query70	1056	965	951	951
query71	362	309	271	271
query72	6096	4988	2644	2644
query73	684	592	316	316
query74	8801	8943	8532	8532
query75	3198	3157	2814	2814
query76	3934	1116	733	733
query77	533	407	299	299
query78	9419	9473	8881	8881
query79	1618	951	622	622
query80	889	657	557	557
query81	538	265	234	234
query82	411	140	105	105
query83	254	262	244	244
query84	266	124	104	104
query85	913	513	480	480
query86	430	296	267	267
query87	2999	3055	3030	3030
query88	3419	2320	2312	2312
query89	471	430	394	394
query90	2120	165	155	155
query91	181	166	148	148
query92	86	66	67	66
query93	2162	930	553	553
query94	531	300	257	257
query95	572	385	311	311
query96	593	482	217	217
query97	2271	2346	2244	2244
query98	212	190	194	190
query99	1302	1305	1219	1219
Total cold run time: 261428 ms
Total hot run time: 175826 ms

doris-robot · 2025-12-17T09:30:06Z

ClickBench: Total hot run time: 27.53 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 711fba5e489f2c8841df0b615d7c31cb5bce679e, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.05	0.04
query3	0.26	0.09	0.09
query4	1.61	0.11	0.11
query5	0.28	0.27	0.26
query6	1.15	0.65	0.63
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.57	0.50	0.48
query10	0.56	0.55	0.55
query11	0.16	0.11	0.11
query12	0.15	0.13	0.12
query13	0.63	0.61	0.62
query14	1.00	0.98	0.97
query15	0.83	0.80	0.80
query16	0.40	0.40	0.40
query17	1.06	1.02	1.04
query18	0.24	0.20	0.21
query19	1.94	1.81	1.81
query20	0.01	0.01	0.01
query21	15.44	0.29	0.14
query22	4.90	0.05	0.05
query23	16.08	0.28	0.10
query24	1.42	0.28	1.87
query25	0.11	0.06	0.07
query26	0.15	0.13	0.12
query27	0.07	0.06	0.08
query28	6.10	1.20	1.02
query29	12.63	4.02	3.40
query30	0.28	0.13	0.12
query31	2.82	0.62	0.40
query32	3.23	0.59	0.47
query33	3.00	3.00	3.10
query34	16.77	5.19	4.58
query35	4.57	4.58	4.64
query36	0.66	0.50	0.49
query37	0.11	0.07	0.07
query38	0.07	0.04	0.04
query39	0.05	0.03	0.03
query40	0.18	0.13	0.12
query41	0.08	0.03	0.03
query42	0.04	0.03	0.04
query43	0.04	0.03	0.03
Total cold run time: 99.89 s
Total hot run time: 27.53 s

hello-stephen · 2025-12-17T10:47:09Z

FE UT Coverage Report

Increment line coverage 55.56% (20/36) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2025-12-17T12:07:30Z

FE Regression Coverage Report

Increment line coverage 97.22% (35/36) 🎉
Increment coverage report
Complete coverage report

github-actions · 2025-12-18T01:47:01Z

PR approved by anyone and no changes requested.

github-actions · 2025-12-18T02:28:39Z

PR approved by at least one committer and no changes requested.

Related PR: #22203 Problem Summary: This pull request refines the logic in the `EliminateEmptyRelation` rewrite rule, focusing on how set operations (`UNION` and `EXCEPT`) handle empty relations in query plans. The changes improve efficiency and code clarity by processing only when necessary, optimizing builder usage, and simplifying project and aggregate node construction. **Set Operation Handling Improvements:** * Updated the logic for `UNION` nodes to only process and eliminate empty relation children when at least one is present, reducing unnecessary computation. Builders are now initialized with expected sizes for efficiency, and redundant code for handling constant expressions was removed. * Improved handling of `EXCEPT` nodes by checking for empty relations only among the right-hand children, and optimizing builder initialization. The construction of aggregate and project nodes now uses the correct outputs directly, streamlining the process.

picked from #59105

… (#59131) Cherry-picked from #59105 Co-authored-by: morrySnow <zhangwenxin@selectdb.com>

… (#59141) picked from #59105

morrySnow added dev/3.1.x dev/4.0.x labels Dec 17, 2025

[fix](rbo) use wrong child's output of set operation

711fba5

morrySnow force-pushed the fix_eli_emp_set branch from 240567e to 711fba5 Compare December 17, 2025 07:54

morrySnow marked this pull request as ready for review December 17, 2025 07:54

morrySnow added the p0_w label Dec 18, 2025

feiniaofeiafei approved these changes Dec 18, 2025

View reviewed changes

github-actions bot added the reviewed label Dec 18, 2025

starocean999 approved these changes Dec 18, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 18, 2025

morrySnow merged commit bb6b39b into apache:master Dec 18, 2025
29 checks passed

morrySnow deleted the fix_eli_emp_set branch December 18, 2025 02:30

github-actions bot added the dev/3.1.x-conflict label Dec 18, 2025

github-actions bot mentioned this pull request Dec 18, 2025

branch-4.0: [fix](rbo) use wrong child's output of set operation #59105 #59131

Merged

morrySnow added a commit that referenced this pull request Dec 18, 2025

branch-3.1: [fix](rbo) use wrong child's output of set operation #59105

cc73877

picked from #59105

morrySnow mentioned this pull request Dec 18, 2025

branch-3.1: [fix](rbo) use wrong child's output of set operation #59105 #59141

Merged

yiguolei pushed a commit that referenced this pull request Dec 18, 2025

branch-4.0: [fix](rbo) use wrong child's output of set operation #59105…

41e95bd

… (#59131) Cherry-picked from #59105 Co-authored-by: morrySnow <zhangwenxin@selectdb.com>

yiguolei added dev/4.0.3-merged and removed dev/4.0.x labels Dec 18, 2025

morrySnow added a commit that referenced this pull request Dec 19, 2025

branch-3.1: [fix](rbo) use wrong child's output of set operation #59105…

7227338

… (#59141) picked from #59105

morrySnow added dev/3.1.4-merged and removed dev/3.1.x dev/3.1.x-conflict labels Dec 19, 2025

[fix](rbo) use wrong child's output of set operation #59105

[fix](rbo) use wrong child's output of set operation #59105

Uh oh!

Conversation

morrySnow commented Dec 17, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Dec 17, 2025

Uh oh!

morrySnow commented Dec 17, 2025

Uh oh!

doris-robot commented Dec 17, 2025

Uh oh!

doris-robot commented Dec 17, 2025

Uh oh!

doris-robot commented Dec 17, 2025

Uh oh!

morrySnow commented Dec 17, 2025

Uh oh!

doris-robot commented Dec 17, 2025

Uh oh!

doris-robot commented Dec 17, 2025

Uh oh!

doris-robot commented Dec 17, 2025

Uh oh!

hello-stephen commented Dec 17, 2025

FE UT Coverage Report

Uh oh!

hello-stephen commented Dec 17, 2025

FE Regression Coverage Report

Uh oh!

github-actions bot commented Dec 18, 2025

Uh oh!

github-actions bot commented Dec 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants