Skip to content

[opt](nereids)push down assert one row join #57414

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
englefly merged 1 commit into from
Oct 31, 2025
Merged

[opt](nereids)push down assert one row join #57414

englefly merged 1 commit into from
Oct 31, 2025

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented Oct 28, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@englefly
Copy link
Contributor Author

englefly commented Oct 28, 2025

run buildall

@Thearas
Copy link
Contributor

Thearas commented Oct 28, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

doris-robot commented Oct 28, 2025

TPC-DS: Total hot run time: 190579 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cc72db438bdf04a06b361580f5d1bfa81991f30a, data reload: false

query1	1067	422	399	399
query2	6555	1678	1745	1678
query3	6758	229	220	220
query4	26275	23955	23454	23454
query5	5174	651	488	488
query6	362	250	236	236
query7	4650	500	309	309
query8	338	279	266	266
query9	8743	2617	2613	2613
query10	562	349	303	303
query11	15677	15209	14840	14840
query12	191	122	123	122
query13	1694	586	469	469
query14	12476	9543	9478	9478
query15	262	198	185	185
query16	7798	704	533	533
query17	1620	836	700	700
query18	2103	445	374	374
query19	266	229	193	193
query20	136	151	130	130
query21	286	142	120	120
query22	4590	4798	4431	4431
query23	35244	34747	34186	34186
query24	8338	2587	2519	2519
query25	577	533	504	504
query26	1277	278	189	189
query27	2802	524	367	367
query28	4366	2254	2246	2246
query29	796	677	514	514
query30	334	235	221	221
query31	949	858	792	792
query32	90	85	79	79
query33	606	403	329	329
query34	820	883	527	527
query35	819	824	742	742
query36	957	975	926	926
query37	142	114	89	89
query38	3532	3573	3472	3472
query39	1482	1407	1433	1407
query40	221	123	120	120
query41	60	59	59	59
query42	120	113	110	110
query43	485	507	458	458
query44	1221	740	740	740
query45	191	176	173	173
query46	889	991	632	632
query47	1746	1770	1680	1680
query48	394	454	337	337
query49	774	498	425	425
query50	657	691	408	408
query51	3888	4058	3895	3895
query52	115	119	105	105
query53	244	279	203	203
query54	307	286	291	286
query55	92	87	82	82
query56	332	338	328	328
query57	1169	1173	1120	1120
query58	291	290	288	288
query59	2564	2632	2513	2513
query60	367	343	325	325
query61	161	158	152	152
query62	794	729	682	682
query63	247	196	195	195
query64	4406	1156	878	878
query65	4060	3970	4021	3970
query66	1097	438	348	348
query67	15297	15205	14927	14927
query68	8025	924	600	600
query69	482	336	301	301
query70	1386	1295	1231	1231
query71	499	344	326	326
query72	5949	4877	4841	4841
query73	677	573	364	364
query74	8898	8922	8613	8613
query75	3881	3313	2874	2874
query76	3611	1180	799	799
query77	789	437	325	325
query78	9575	9755	8931	8931
query79	2015	851	602	602
query80	680	595	487	487
query81	503	264	233	233
query82	440	157	142	142
query83	266	270	258	258
query84	262	112	99	99
query85	913	489	430	430
query86	342	311	303	303
query87	3720	3718	3665	3665
query88	3308	2287	2296	2287
query89	411	339	297	297
query90	2021	226	226	226
query91	169	174	138	138
query92	85	72	70	70
query93	1147	973	649	649
query94	695	438	337	337
query95	397	322	318	318
query96	509	586	309	309
query97	2948	2993	2874	2874
query98	245	215	218	215
query99	1460	1413	1346	1346
Total cold run time: 279211 ms
Total hot run time: 190579 ms

@doris-robot
Copy link

doris-robot commented Oct 28, 2025

ClickBench: Total hot run time: 27.71 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cc72db438bdf04a06b361580f5d1bfa81991f30a, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.08	0.09
query4	1.61	0.11	0.12
query5	0.28	0.26	0.25
query6	1.17	0.67	0.64
query7	0.03	0.03	0.03
query8	0.06	0.05	0.05
query9	0.62	0.52	0.53
query10	0.58	0.58	0.58
query11	0.17	0.12	0.12
query12	0.15	0.13	0.13
query13	0.64	0.60	0.61
query14	1.00	1.02	1.01
query15	0.85	0.85	0.87
query16	0.39	0.40	0.39
query17	1.03	1.04	1.03
query18	0.22	0.20	0.20
query19	1.93	1.74	1.80
query20	0.01	0.01	0.02
query21	15.46	0.19	0.12
query22	5.09	0.07	0.04
query23	15.67	0.26	0.10
query24	2.86	0.64	1.17
query25	0.08	0.06	0.06
query26	0.13	0.14	0.14
query27	0.07	0.06	0.05
query28	5.16	1.15	0.93
query29	12.56	3.93	3.23
query30	0.28	0.14	0.11
query31	2.82	0.59	0.38
query32	3.24	0.55	0.46
query33	3.21	3.06	3.04
query34	15.84	5.18	4.52
query35	4.59	4.62	4.60
query36	0.69	0.52	0.48
query37	0.10	0.07	0.07
query38	0.07	0.04	0.03
query39	0.03	0.03	0.03
query40	0.18	0.14	0.15
query41	0.09	0.03	0.03
query42	0.05	0.04	0.03
query43	0.04	0.04	0.04
Total cold run time: 99.47 s
Total hot run time: 27.71 s

@englefly
Copy link
Contributor Author

englefly commented Oct 29, 2025

run p0

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 29, 2025

FE Regression Coverage Report

Increment line coverage 69.61% (71/102) 🎉
Increment coverage report
Complete coverage report

@englefly englefly changed the title [opt](nereids)push down asser one row join [opt](nereids)push down assert one row join Oct 30, 2025
@englefly englefly marked this pull request as ready for review October 30, 2025 03:06
@github-actions
Copy link
Contributor

github-actions bot commented Oct 30, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 30, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 30, 2025

PR approved by anyone and no changes requested.

@englefly englefly merged commit e4c029b into apache:master Oct 31, 2025
30 of 31 checks passed
dwdwqfwe pushed a commit to dwdwqfwe/doris that referenced this pull request Oct 31, 2025
@englefly @dwdwqfwe
[opt](nereids)push down assert one row join (apache#57414)
6c4a38f 
### What problem does this PR solve?
* Push down join when one child is LogicalAssertNumRows.
 * select * from T1 join T2 where T1.b=T2.b and T1.a > (select x from T3 ...)
 *
 * <pre>
 * Case 1: Push to left child
 * Before:
 *     topJoin(T1.a > x)
 *       |-- Project
 *       |     `-- bottomJoin(T1.b = T2.b)
 *       |           |-- Scan(T1)
 *       |           `-- Scan(T2)
 *       `-- LogicalAssertNumRows(output=(x, ...))
 *
 * After:
 *     Project
 *       |-- topJoin(T1.b = T2.b)
 *               |-- bottomJoin(T1.a > x)
 *                   |-- Scan(T1)
 *                   `-- LogicalAssertNumRows(output=(x, ...))
 *               `-- Scan(T2)
 *
 * Case 2: Push to right child
 * Before:
 *     topJoin(T2.a > x)
 *       |-- Project
 *       |     `-- bottomJoin(T1.b = T2.b)
 *       |           |-- Scan(T1)
 *       |           `-- Scan(T2)
 *       `-- LogicalAssertNumRows(output=(x, ...))
 *
 * After:
 *     Project
 *       |-- topJoin(T1.b = T2.b)
 *              |--Scan(T1)
 *             `-- bottomJoin(T2.a > x)
 *                   |-- Scan(T2)
 *                   `-- LogicalAssertNumRows(output=(x, ...))
@starocean999 starocean999 mentioned this pull request Nov 11, 2025
Merged
16 tasks
@englefly englefly mentioned this pull request Nov 20, 2025
Merged
16 tasks
englefly added a commit that referenced this pull request Nov 24, 2025
@englefly
[fix](nereids)PushDownJoinOnAssertNumRows rule process project alias …
7c176af 
…bug (#58174)

### What problem does this PR solve?
pr #57414 will generate error project: LogicalProject(a as b, b as c),
in which 'b as c' is not correct, because 'b' is not in output of
project.child()
pr #57804 fixed above issue, but made another issue, that is when the
bottom project is not resued in result plan, some aliases definitions
are lost.
nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025
@englefly @nagisa-kunhah
[fix](nereids)PushDownJoinOnAssertNumRows rule process project alias …
32b294b 
…bug (apache#58174)

### What problem does this PR solve?
pr apache#57414 will generate error project: LogicalProject(a as b, b as c),
in which 'b as c' is not correct, because 'b' is not in output of
project.child()
pr apache#57804 fixed above issue, but made another issue, that is when the
bottom project is not resued in result plan, some aliases definitions
are lost.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

@Jibing-Li Jibing-Li Jibing-Li approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@englefly @Thearas @doris-robot @hello-stephen @starocean999 @Jibing-Li