Skip to content

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented Jan 8, 2025

pick#46551

What problem does this PR solve?

Plan node is not good to be hash map key, because two plan nodes in different tree level may be regarded as "equal". for example, in following tree, topn1.equals(topn2) may be true.
Topn filter generator should distinguish them, and hence topn
node is not suitable to be used as hash map key.

topn1
-->some node
-->topn2
-->other node

Related PR: #31485

(cherry picked from commit 811f936)

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

…ode (apache#46551)

### What problem does this PR solve?
Plan node is not good to be hash map key, because two plan nodes in
different tree level may be regarded as "equal". for example, in
following tree, topn1.equals(topn2) may be true.
 Topn filter generator should distinguish them, and hence topn
node is not suitable to be used as hash map key.

topn1
    -->some node
        -->topn2
             -->other node

Related PR: apache#31485

(cherry picked from commit 811f936)
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly
Copy link
Contributor Author

englefly commented Jan 8, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40715 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0503704267d006c371e170b84b6aa5a72312a5fc, data reload: false

------ Round 1 ----------------------------------
q1	17591	7399	7242	7242
q2	2057	174	171	171
q3	10627	1108	1187	1108
q4	10518	715	783	715
q5	7755	2826	2750	2750
q6	235	147	146	146
q7	969	609	601	601
q8	9366	1936	1918	1918
q9	6556	6412	6416	6412
q10	7032	2319	2331	2319
q11	468	260	265	260
q12	405	211	210	210
q13	17799	2986	3022	2986
q14	243	202	201	201
q15	584	526	521	521
q16	679	617	608	608
q17	958	608	523	523
q18	7304	6672	6727	6672
q19	1383	1114	1082	1082
q20	485	199	199	199
q21	4091	3099	3259	3099
q22	1083	1016	972	972
Total cold run time: 108188 ms
Total hot run time: 40715 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7237	7199	7233	7199
q2	326	235	229	229
q3	2897	2976	2946	2946
q4	1996	1819	1865	1819
q5	5683	5775	5750	5750
q6	226	139	141	139
q7	2267	1794	1873	1794
q8	3423	3547	3550	3547
q9	8848	8910	8795	8795
q10	3530	3552	3525	3525
q11	593	504	511	504
q12	827	581	595	581
q13	9357	3167	3229	3167
q14	298	284	277	277
q15	582	538	522	522
q16	732	694	676	676
q17	1846	1614	1591	1591
q18	8213	7665	7668	7665
q19	1641	1627	1561	1561
q20	2102	1852	1876	1852
q21	5550	5303	5239	5239
q22	1146	1031	1021	1021
Total cold run time: 69320 ms
Total hot run time: 60399 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197558 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0503704267d006c371e170b84b6aa5a72312a5fc, data reload: false

query1	1309	922	920	920
query2	6234	2142	2062	2062
query3	10858	4200	4312	4200
query4	66138	29550	23545	23545
query5	4898	466	446	446
query6	411	178	174	174
query7	5596	323	326	323
query8	315	242	233	233
query9	9004	2682	2677	2677
query10	450	275	266	266
query11	17532	15253	15871	15253
query12	159	106	119	106
query13	1491	440	432	432
query14	10995	7674	7118	7118
query15	202	184	187	184
query16	7090	501	518	501
query17	1109	625	618	618
query18	1420	340	331	331
query19	221	169	168	168
query20	125	113	115	113
query21	211	105	106	105
query22	4646	4405	4401	4401
query23	34803	34128	34513	34128
query24	6129	2968	3066	2968
query25	558	441	438	438
query26	666	176	175	175
query27	1927	350	355	350
query28	4405	2464	2442	2442
query29	697	480	450	450
query30	248	172	166	166
query31	1017	832	821	821
query32	76	57	64	57
query33	410	296	290	290
query34	946	531	539	531
query35	855	728	716	716
query36	1082	973	952	952
query37	126	76	78	76
query38	4133	4009	4088	4009
query39	1538	1523	1479	1479
query40	215	106	101	101
query41	51	51	48	48
query42	117	104	105	104
query43	556	505	519	505
query44	1240	836	834	834
query45	194	173	167	167
query46	1157	771	732	732
query47	1976	1879	1928	1879
query48	514	385	403	385
query49	722	385	411	385
query50	834	424	436	424
query51	7333	7168	7009	7009
query52	103	86	88	86
query53	268	183	182	182
query54	555	465	451	451
query55	78	77	80	77
query56	271	240	242	240
query57	1240	1116	1114	1114
query58	208	202	205	202
query59	3200	2969	3012	2969
query60	289	270	241	241
query61	108	107	129	107
query62	772	675	664	664
query63	222	207	205	205
query64	1383	657	642	642
query65	3263	3220	3202	3202
query66	643	303	323	303
query67	15894	15504	15653	15504
query68	4103	579	589	579
query69	430	269	281	269
query70	1154	1149	1124	1124
query71	358	262	255	255
query72	6398	4074	3983	3983
query73	763	348	348	348
query74	9991	9073	8971	8971
query75	3350	2642	2682	2642
query76	1876	1121	1123	1121
query77	477	294	273	273
query78	10666	9547	9556	9547
query79	1316	605	615	605
query80	869	431	434	431
query81	510	240	238	238
query82	707	119	121	119
query83	163	143	152	143
query84	289	87	76	76
query85	840	290	292	290
query86	328	281	303	281
query87	4386	4206	4339	4206
query88	3921	2388	2353	2353
query89	428	289	286	286
query90	2034	188	192	188
query91	192	146	151	146
query92	68	48	51	48
query93	1535	547	554	547
query94	811	296	291	291
query95	349	254	253	253
query96	607	277	279	277
query97	3312	3196	3187	3187
query98	217	219	194	194
query99	1600	1301	1313	1301
Total cold run time: 317484 ms
Total hot run time: 197558 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.71 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0503704267d006c371e170b84b6aa5a72312a5fc, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.06	0.06
query4	1.64	0.10	0.10
query5	0.51	0.52	0.53
query6	1.13	0.72	0.75
query7	0.02	0.02	0.01
query8	0.04	0.05	0.03
query9	0.57	0.50	0.51
query10	0.56	0.55	0.57
query11	0.14	0.10	0.10
query12	0.13	0.12	0.12
query13	0.60	0.59	0.60
query14	2.91	2.92	2.93
query15	0.89	0.83	0.82
query16	0.40	0.38	0.37
query17	1.05	1.08	1.06
query18	0.23	0.21	0.22
query19	1.97	1.81	2.01
query20	0.01	0.00	0.01
query21	15.35	0.60	0.58
query22	2.76	1.89	1.99
query23	16.94	1.14	0.81
query24	2.62	1.58	0.67
query25	0.23	0.09	0.16
query26	0.43	0.15	0.14
query27	0.04	0.03	0.04
query28	10.67	1.10	1.08
query29	12.59	3.26	3.27
query30	0.24	0.06	0.06
query31	2.84	0.39	0.37
query32	3.27	0.46	0.45
query33	2.95	2.95	3.04
query34	16.97	4.47	4.52
query35	4.46	4.52	4.54
query36	0.69	0.51	0.48
query37	0.09	0.07	0.06
query38	0.06	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.66 s
Total hot run time: 32.71 s

@englefly englefly merged commit a7765d2 into apache:branch-3.0 Jan 8, 2025
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants