Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nereids)subquery unnesting get wrong result if correlated conjuncts is not slot_a = slot_b #37644

Merged
merged 5 commits into from
Jul 17, 2024

Conversation

starocean999
Copy link
Contributor

sql:
select * from t1 where t1.c1 < (select max(t2.c1) from t2 where t1.c2 = abs(t2.c2));
before this pr, we would use the equivalent sql as bellow, which is wrong
select t1.* from t1 semi join ( select t2.c2 as c2, max(t2.c1) as c1 from t2 group by t2.c ) tx on t1.c1 < tx.c1 and t1.c2 = tx.c2;
the pr fix the problem and get the correct one as:
select t1.* from t1 semi join ( select abs(t2.c2) as c2, max(t2.c1) as c1 from t2 group by abs(t2.c) ) tx on t1.c1 < tx.c1 and t1.c2 = tx.c2;

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40164 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aa0510a792e5a5689a663ef4a1256c953281ace3, data reload: false

------ Round 1 ----------------------------------
q1	18017	4443	4386	4386
q2	2219	200	186	186
q3	10448	1234	1102	1102
q4	10202	893	826	826
q5	7532	2705	2599	2599
q6	223	134	134	134
q7	966	605	595	595
q8	9234	2070	2102	2070
q9	8806	6555	6547	6547
q10	8709	3762	3797	3762
q11	448	249	243	243
q12	393	233	230	230
q13	17772	3030	3020	3020
q14	276	234	244	234
q15	528	484	505	484
q16	476	378	380	378
q17	973	668	701	668
q18	8087	7408	7530	7408
q19	4251	1436	1439	1436
q20	701	336	324	324
q21	4820	3202	3312	3202
q22	388	333	330	330
Total cold run time: 115469 ms
Total hot run time: 40164 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4359	4288	4197	4197
q2	364	275	260	260
q3	2996	2762	2750	2750
q4	1883	1582	1600	1582
q5	5276	5318	5320	5318
q6	216	129	132	129
q7	2126	1723	1764	1723
q8	3187	3323	3302	3302
q9	8430	8407	8403	8403
q10	3900	3753	3665	3665
q11	594	491	505	491
q12	809	606	607	606
q13	17562	3005	3008	3005
q14	308	276	272	272
q15	520	486	492	486
q16	462	402	424	402
q17	1772	1449	1428	1428
q18	7537	7469	7303	7303
q19	1899	1547	1481	1481
q20	1966	1789	1774	1774
q21	4850	4673	4711	4673
q22	599	531	555	531
Total cold run time: 71615 ms
Total hot run time: 53781 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174237 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aa0510a792e5a5689a663ef4a1256c953281ace3, data reload: false

query1	915	366	368	366
query2	6446	2474	2392	2392
query3	6653	207	226	207
query4	27895	17535	17415	17415
query5	4230	504	491	491
query6	296	175	171	171
query7	4595	305	293	293
query8	322	305	308	305
query9	8519	2459	2406	2406
query10	435	277	271	271
query11	12026	10058	10132	10058
query12	129	80	78	78
query13	1624	362	358	358
query14	10164	7590	7226	7226
query15	255	189	190	189
query16	7932	331	309	309
query17	1788	538	510	510
query18	2006	298	269	269
query19	200	147	147	147
query20	90	79	79	79
query21	209	127	125	125
query22	4456	4086	4124	4086
query23	33873	32974	32974	32974
query24	11872	2796	2904	2796
query25	682	358	363	358
query26	1807	147	144	144
query27	2979	266	269	266
query28	7796	2088	2083	2083
query29	1094	612	613	612
query30	289	148	147	147
query31	968	764	759	759
query32	92	53	54	53
query33	769	299	302	299
query34	938	494	482	482
query35	678	565	580	565
query36	1112	935	932	932
query37	192	81	81	81
query38	2886	2749	2761	2749
query39	880	783	785	783
query40	284	115	116	115
query41	53	52	52	52
query42	118	99	102	99
query43	632	535	573	535
query44	1213	766	731	731
query45	197	167	161	161
query46	1079	754	707	707
query47	1851	1782	1786	1782
query48	368	291	306	291
query49	1202	408	403	403
query50	764	393	398	393
query51	6829	6838	6714	6714
query52	104	96	94	94
query53	358	294	306	294
query54	926	446	437	437
query55	76	75	78	75
query56	289	264	272	264
query57	1162	1093	1045	1045
query58	246	238	243	238
query59	3506	3273	3246	3246
query60	307	276	297	276
query61	151	94	93	93
query62	842	649	640	640
query63	324	287	293	287
query64	10490	2195	5204	2195
query65	3177	3098	3124	3098
query66	1368	326	338	326
query67	15476	14809	14923	14809
query68	4554	541	541	541
query69	535	319	327	319
query70	1189	1112	1182	1112
query71	407	276	274	274
query72	7129	5752	5584	5584
query73	744	322	322	322
query74	5930	5582	5532	5532
query75	3440	2664	2698	2664
query76	2702	939	864	864
query77	492	304	301	301
query78	9569	10660	9266	9266
query79	2929	521	517	517
query80	1251	485	461	461
query81	577	220	221	220
query82	788	133	136	133
query83	275	163	162	162
query84	280	85	88	85
query85	1357	318	323	318
query86	489	321	307	307
query87	3303	3109	3156	3109
query88	4083	2397	2441	2397
query89	488	399	387	387
query90	1875	192	189	189
query91	128	98	102	98
query92	59	49	51	49
query93	4298	527	518	518
query94	1209	217	211	211
query95	410	321	324	321
query96	613	268	273	268
query97	3174	3013	3053	3013
query98	217	196	194	194
query99	1558	1267	1296	1267
Total cold run time: 289536 ms
Total hot run time: 174237 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.47 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aa0510a792e5a5689a663ef4a1256c953281ace3, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.07	0.07
query5	0.49	0.49	0.50
query6	1.13	0.72	0.72
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.55	0.48	0.48
query10	0.54	0.55	0.53
query11	0.16	0.11	0.11
query12	0.14	0.12	0.13
query13	0.59	0.59	0.59
query14	0.76	0.77	0.79
query15	0.84	0.81	0.81
query16	0.35	0.37	0.37
query17	0.96	1.04	0.97
query18	0.22	0.22	0.21
query19	1.91	1.74	1.73
query20	0.02	0.01	0.01
query21	15.41	0.73	0.66
query22	4.56	7.39	1.81
query23	18.28	1.30	1.21
query24	2.08	0.25	0.23
query25	0.16	0.09	0.09
query26	0.29	0.22	0.21
query27	0.46	0.24	0.24
query28	13.20	1.02	0.99
query29	12.63	3.33	3.36
query30	0.24	0.05	0.06
query31	2.88	0.38	0.38
query32	3.27	0.48	0.48
query33	2.87	2.96	2.91
query34	17.04	4.31	4.29
query35	4.41	4.39	4.41
query36	0.66	0.46	0.46
query37	0.18	0.16	0.15
query38	0.15	0.15	0.15
query39	0.04	0.03	0.04
query40	0.15	0.12	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.89 s
Total hot run time: 30.47 s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40233 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 32d16199f6660852ce4a89cf98b582885057fa3e, data reload: false

------ Round 1 ----------------------------------
q1	17712	5415	4361	4361
q2	2636	213	191	191
q3	11422	1212	1109	1109
q4	10544	798	815	798
q5	7557	2710	2802	2710
q6	225	140	143	140
q7	973	619	615	615
q8	9346	2095	2084	2084
q9	8690	6573	6520	6520
q10	8638	3844	3785	3785
q11	444	242	245	242
q12	396	235	236	235
q13	17778	2970	2986	2970
q14	281	232	236	232
q15	527	494	481	481
q16	501	383	376	376
q17	958	653	670	653
q18	8066	7433	7383	7383
q19	7248	1477	1554	1477
q20	700	332	336	332
q21	4963	3204	3245	3204
q22	389	338	335	335
Total cold run time: 119994 ms
Total hot run time: 40233 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4374	4256	4233	4233
q2	374	283	264	264
q3	2991	2780	2741	2741
q4	1900	1608	1595	1595
q5	5278	5326	5335	5326
q6	225	133	133	133
q7	2140	1760	1701	1701
q8	3160	3327	3262	3262
q9	8437	8317	8380	8317
q10	3894	3697	3705	3697
q11	598	495	493	493
q12	809	644	637	637
q13	16381	2986	3002	2986
q14	312	267	286	267
q15	515	483	482	482
q16	468	434	432	432
q17	1764	1488	1449	1449
q18	7774	7589	7338	7338
q19	1696	1657	1622	1622
q20	1960	1795	1803	1795
q21	4661	4863	4739	4739
q22	631	542	554	542
Total cold run time: 70342 ms
Total hot run time: 54051 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172484 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 32d16199f6660852ce4a89cf98b582885057fa3e, data reload: false

query1	896	378	367	367
query2	6438	2482	2265	2265
query3	6647	205	226	205
query4	24557	17278	17321	17278
query5	4182	489	472	472
query6	260	182	153	153
query7	4599	292	292	292
query8	316	318	279	279
query9	8489	2439	2430	2430
query10	439	285	267	267
query11	11605	10207	9997	9997
query12	128	87	86	86
query13	1642	374	366	366
query14	10425	7930	6427	6427
query15	225	183	188	183
query16	7624	335	316	316
query17	1802	576	543	543
query18	1393	279	281	279
query19	198	160	159	159
query20	92	83	84	83
query21	217	136	133	133
query22	4494	4126	4063	4063
query23	33629	33089	33429	33089
query24	11091	2927	2881	2881
query25	631	432	366	366
query26	1226	143	145	143
query27	2876	268	273	268
query28	7420	2103	2087	2087
query29	872	637	607	607
query30	279	145	143	143
query31	959	748	743	743
query32	98	52	53	52
query33	758	294	292	292
query34	954	475	484	475
query35	680	604	566	566
query36	1081	959	945	945
query37	141	77	77	77
query38	2869	2748	2740	2740
query39	826	776	790	776
query40	280	117	117	117
query41	55	51	56	51
query42	112	97	96	96
query43	610	543	519	519
query44	1222	749	735	735
query45	194	163	161	161
query46	1090	715	713	713
query47	1880	1802	1794	1794
query48	364	298	300	298
query49	1069	409	413	409
query50	780	415	396	396
query51	6940	6841	6782	6782
query52	119	87	96	87
query53	350	293	284	284
query54	962	439	445	439
query55	78	72	76	72
query56	284	275	282	275
query57	1143	1053	1055	1053
query58	243	264	239	239
query59	3409	2995	3233	2995
query60	305	281	299	281
query61	95	93	95	93
query62	845	632	657	632
query63	321	294	287	287
query64	9596	2209	1667	1667
query65	3131	3107	3111	3107
query66	897	340	334	334
query67	15499	14986	14963	14963
query68	4625	524	550	524
query69	482	317	308	308
query70	1157	1157	1144	1144
query71	386	277	273	273
query72	7067	5167	5762	5167
query73	739	323	323	323
query74	6020	5518	5478	5478
query75	3386	2695	2721	2695
query76	2806	946	1000	946
query77	449	307	299	299
query78	9808	10326	9408	9408
query79	6137	526	536	526
query80	1229	480	471	471
query81	581	223	217	217
query82	591	139	134	134
query83	283	168	168	168
query84	278	89	88	88
query85	1399	317	296	296
query86	501	311	322	311
query87	3266	3104	3143	3104
query88	4609	2398	2370	2370
query89	495	390	425	390
query90	1878	191	189	189
query91	130	100	104	100
query92	64	49	48	48
query93	4062	514	501	501
query94	1220	215	205	205
query95	413	322	322	322
query96	592	283	277	277
query97	3157	3022	3007	3007
query98	225	205	192	192
query99	1627	1265	1231	1231
Total cold run time: 284593 ms
Total hot run time: 172484 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.58 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 32d16199f6660852ce4a89cf98b582885057fa3e, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.04	0.04
query4	1.68	0.08	0.07
query5	0.51	0.50	0.50
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.48	0.49
query10	0.54	0.55	0.54
query11	0.16	0.12	0.12
query12	0.16	0.12	0.12
query13	0.59	0.58	0.59
query14	0.75	0.80	0.75
query15	0.84	0.82	0.82
query16	0.36	0.35	0.37
query17	1.03	1.04	1.02
query18	0.22	0.21	0.21
query19	1.88	1.74	1.74
query20	0.02	0.01	0.01
query21	15.39	0.72	0.66
query22	4.28	7.05	1.96
query23	18.28	1.40	1.17
query24	2.10	0.23	0.22
query25	0.15	0.09	0.09
query26	0.30	0.22	0.21
query27	0.45	0.23	0.23
query28	13.26	1.02	1.00
query29	12.64	3.34	3.27
query30	0.25	0.06	0.05
query31	2.87	0.40	0.40
query32	3.28	0.47	0.47
query33	2.89	2.90	2.92
query34	17.08	4.32	4.31
query35	4.45	4.38	4.44
query36	0.66	0.47	0.49
query37	0.19	0.16	0.16
query38	0.16	0.14	0.15
query39	0.04	0.03	0.04
query40	0.15	0.13	0.12
query41	0.09	0.05	0.05
query42	0.05	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.88 s
Total hot run time: 30.58 s

Comment on lines 106 to 114
newAggOutput.addAll(newGroupby.stream().map(expression -> {
if (expression instanceof Slot) {
return (NamedExpression) expression;
} else {
Alias alias = new Alias(expression);
unCorrelatedExprToSlot.put(expression, alias.toSlot());
return alias;
}
}).collect(ImmutableList.toImmutableList()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do not use stream api

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40013 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5502b65a01be055bc02ff21fc3655663a982f8e7, data reload: false

------ Round 1 ----------------------------------
q1	17609	4333	4282	4282
q2	2014	193	197	193
q3	10439	1151	1144	1144
q4	10195	853	792	792
q5	7612	2675	2638	2638
q6	219	137	138	137
q7	955	613	600	600
q8	9203	2065	2078	2065
q9	8797	6505	6502	6502
q10	8842	3763	3777	3763
q11	452	235	237	235
q12	409	237	228	228
q13	17766	3017	2981	2981
q14	281	233	227	227
q15	527	472	484	472
q16	496	385	373	373
q17	956	670	739	670
q18	8006	7445	7355	7355
q19	7244	1486	1476	1476
q20	695	322	321	321
q21	4904	3220	3228	3220
q22	391	344	339	339
Total cold run time: 118012 ms
Total hot run time: 40013 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4366	4294	4229	4229
q2	369	280	274	274
q3	2963	2907	2940	2907
q4	2034	1683	1653	1653
q5	5582	5526	5469	5469
q6	223	138	149	138
q7	2195	1861	1828	1828
q8	3263	3378	3403	3378
q9	8715	8792	8828	8792
q10	4067	3825	3762	3762
q11	600	518	494	494
q12	840	648	648	648
q13	17374	3179	3161	3161
q14	311	299	282	282
q15	524	501	498	498
q16	509	432	418	418
q17	1796	1512	1493	1493
q18	8281	8102	7810	7810
q19	1855	1719	1552	1552
q20	2182	1860	1873	1860
q21	5109	4731	4914	4731
q22	616	565	577	565
Total cold run time: 73774 ms
Total hot run time: 55942 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174594 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5502b65a01be055bc02ff21fc3655663a982f8e7, data reload: false

query1	915	374	358	358
query2	6357	2574	2299	2299
query3	6638	214	223	214
query4	24631	17707	17336	17336
query5	3627	489	499	489
query6	264	197	156	156
query7	4583	291	281	281
query8	323	294	293	293
query9	8734	2456	2453	2453
query10	419	272	274	272
query11	11228	10029	9995	9995
query12	111	90	80	80
query13	1627	358	374	358
query14	10184	7521	7116	7116
query15	237	182	190	182
query16	7372	308	321	308
query17	1357	530	526	526
query18	1672	284	291	284
query19	202	151	150	150
query20	89	81	81	81
query21	204	128	127	127
query22	4305	4076	4022	4022
query23	33872	33885	33519	33519
query24	11068	2940	2980	2940
query25	638	415	416	415
query26	704	155	152	152
query27	2258	287	281	281
query28	5840	2180	2168	2168
query29	914	679	649	649
query30	268	157	157	157
query31	956	754	733	733
query32	95	55	55	55
query33	684	317	311	311
query34	897	512	494	494
query35	721	622	618	618
query36	1085	975	981	975
query37	152	87	88	87
query38	2994	2892	2868	2868
query39	906	870	849	849
query40	204	126	121	121
query41	52	48	57	48
query42	119	98	102	98
query43	584	552	547	547
query44	1075	727	719	719
query45	199	166	172	166
query46	1092	751	722	722
query47	1869	1768	1775	1768
query48	367	297	300	297
query49	864	412	432	412
query50	771	399	404	399
query51	6791	6856	6657	6657
query52	107	98	98	98
query53	378	297	286	286
query54	872	462	457	457
query55	78	77	77	77
query56	300	291	298	291
query57	1147	1052	1067	1052
query58	256	247	262	247
query59	3340	3223	3199	3199
query60	319	295	300	295
query61	122	119	119	119
query62	820	664	646	646
query63	322	288	286	286
query64	9290	2232	1667	1667
query65	3161	3114	3074	3074
query66	754	329	337	329
query67	15801	15021	14905	14905
query68	8107	547	541	541
query69	744	491	374	374
query70	1205	1102	1154	1102
query71	531	270	287	270
query72	9100	5860	5796	5796
query73	866	331	327	327
query74	5987	5495	5529	5495
query75	5025	2680	2699	2680
query76	4685	965	954	954
query77	787	304	301	301
query78	9750	8889	10093	8889
query79	9817	529	522	522
query80	1212	494	464	464
query81	581	233	225	225
query82	650	136	136	136
query83	325	165	162	162
query84	272	84	87	84
query85	1293	350	295	295
query86	408	331	281	281
query87	3318	3136	3119	3119
query88	4787	2462	2473	2462
query89	505	376	376	376
query90	2063	196	204	196
query91	132	101	105	101
query92	68	47	48	47
query93	5512	509	506	506
query94	1282	216	215	215
query95	410	330	332	330
query96	608	273	273	273
query97	3204	2981	3042	2981
query98	219	204	200	200
query99	1707	1257	1260	1257
Total cold run time: 295345 ms
Total hot run time: 174594 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.83 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5502b65a01be055bc02ff21fc3655663a982f8e7, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.66	0.09	0.07
query5	0.49	0.48	0.49
query6	1.14	0.73	0.72
query7	0.02	0.01	0.01
query8	0.06	0.04	0.05
query9	0.56	0.50	0.49
query10	0.54	0.54	0.55
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.58	0.59	0.59
query14	0.76	0.77	0.83
query15	0.84	0.80	0.81
query16	0.37	0.37	0.37
query17	0.95	0.99	1.01
query18	0.23	0.23	0.22
query19	1.90	1.74	1.74
query20	0.02	0.01	0.01
query21	15.78	0.78	0.65
query22	4.25	6.80	2.23
query23	18.26	1.29	1.24
query24	2.13	0.24	0.21
query25	0.15	0.09	0.09
query26	0.29	0.21	0.21
query27	0.46	0.24	0.23
query28	13.30	1.02	1.00
query29	12.61	3.31	3.36
query30	0.25	0.06	0.05
query31	2.87	0.38	0.38
query32	3.28	0.47	0.48
query33	2.88	2.85	2.93
query34	16.94	4.28	4.32
query35	4.40	4.45	4.41
query36	0.65	0.46	0.49
query37	0.19	0.16	0.15
query38	0.16	0.15	0.15
query39	0.04	0.04	0.03
query40	0.15	0.13	0.12
query41	0.09	0.05	0.04
query42	0.06	0.04	0.06
query43	0.05	0.04	0.04
Total cold run time: 110 s
Total hot run time: 30.83 s

@starocean999
Copy link
Contributor Author

run External

@starocean999
Copy link
Contributor Author

run external

@starocean999
Copy link
Contributor Author

run buildall

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40208 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e4e11cf58f7f3f7d89f4aacc106d34f1d51ed876, data reload: false

------ Round 1 ----------------------------------
q1	17629	4356	4289	4289
q2	2013	189	208	189
q3	10437	1236	1117	1117
q4	10177	775	858	775
q5	7662	2683	2696	2683
q6	215	135	138	135
q7	950	601	610	601
q8	9233	2086	2060	2060
q9	9097	6665	6599	6599
q10	8771	3826	3807	3807
q11	437	242	244	242
q12	412	231	225	225
q13	18235	3044	3020	3020
q14	280	231	231	231
q15	523	486	491	486
q16	518	381	388	381
q17	962	754	759	754
q18	8125	7546	7416	7416
q19	8056	1289	1483	1289
q20	662	319	326	319
q21	4909	3317	3299	3299
q22	360	291	298	291
Total cold run time: 119663 ms
Total hot run time: 40208 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4498	4359	4333	4333
q2	380	294	283	283
q3	3285	3109	3084	3084
q4	2093	1884	1841	1841
q5	5693	5632	5598	5598
q6	227	133	137	133
q7	2215	1862	1859	1859
q8	3319	3446	3425	3425
q9	8726	8917	8755	8755
q10	4127	3769	3842	3769
q11	601	487	501	487
q12	824	669	634	634
q13	15855	3197	3171	3171
q14	308	307	268	268
q15	531	476	497	476
q16	479	456	449	449
q17	1837	1528	1519	1519
q18	8078	7963	7883	7883
q19	1782	1668	1563	1563
q20	2138	1893	1856	1856
q21	7778	4916	4638	4638
q22	608	524	507	507
Total cold run time: 75382 ms
Total hot run time: 56531 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173244 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e4e11cf58f7f3f7d89f4aacc106d34f1d51ed876, data reload: false

query1	916	363	359	359
query2	6440	2003	1956	1956
query3	6628	209	217	209
query4	26792	17589	17215	17215
query5	3649	466	481	466
query6	253	179	164	164
query7	4580	291	290	290
query8	237	192	195	192
query9	8599	2413	2396	2396
query10	437	288	291	288
query11	10708	10224	10211	10211
query12	118	83	87	83
query13	1649	378	369	369
query14	10167	6976	7934	6976
query15	221	167	169	167
query16	7891	316	313	313
query17	1808	544	529	529
query18	2052	277	270	270
query19	194	149	150	149
query20	89	83	78	78
query21	205	128	125	125
query22	4259	4240	4057	4057
query23	34170	33999	33640	33640
query24	11027	2871	2933	2871
query25	615	380	400	380
query26	707	151	149	149
query27	2235	275	288	275
query28	6694	2121	2082	2082
query29	886	630	650	630
query30	252	149	154	149
query31	957	783	758	758
query32	98	56	56	56
query33	751	296	322	296
query34	993	487	498	487
query35	698	583	583	583
query36	1137	984	964	964
query37	166	85	83	83
query38	3011	2851	2812	2812
query39	883	860	861	860
query40	197	118	124	118
query41	45	44	43	43
query42	115	98	98	98
query43	521	498	458	458
query44	1237	728	731	728
query45	196	160	154	154
query46	1086	770	750	750
query47	1840	1787	1763	1763
query48	368	299	295	295
query49	846	425	413	413
query50	788	394	386	386
query51	6854	6735	6787	6735
query52	109	90	106	90
query53	361	296	290	290
query54	960	446	449	446
query55	77	73	72	72
query56	279	271	295	271
query57	1148	1064	1040	1040
query58	249	248	254	248
query59	3017	2769	2750	2750
query60	342	277	268	268
query61	96	93	90	90
query62	816	667	664	664
query63	315	287	299	287
query64	9187	2214	1689	1689
query65	3151	3091	3140	3091
query66	704	328	375	328
query67	15576	15131	15153	15131
query68	4423	537	553	537
query69	570	406	334	334
query70	1197	1140	1164	1140
query71	419	281	277	277
query72	6984	5270	5736	5270
query73	750	326	324	324
query74	6309	5685	5748	5685
query75	3488	2677	2704	2677
query76	2681	931	884	884
query77	482	313	289	289
query78	10520	8985	10857	8985
query79	4072	517	521	517
query80	1716	477	478	477
query81	575	216	232	216
query82	697	136	130	130
query83	310	167	166	166
query84	276	90	89	89
query85	1668	343	358	343
query86	474	332	321	321
query87	3295	3105	3111	3105
query88	3686	2369	2346	2346
query89	481	382	385	382
query90	1807	190	191	190
query91	127	99	104	99
query92	63	50	50	50
query93	4157	518	517	517
query94	1165	214	214	214
query95	408	316	321	316
query96	606	267	274	267
query97	3195	2994	3028	2994
query98	214	203	197	197
query99	1537	1273	1288	1273
Total cold run time: 282004 ms
Total hot run time: 173244 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.56 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e4e11cf58f7f3f7d89f4aacc106d34f1d51ed876, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.06
query4	1.66	0.07	0.07
query5	0.51	0.49	0.50
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.53	0.49	0.49
query10	0.55	0.55	0.54
query11	0.15	0.12	0.11
query12	0.15	0.13	0.12
query13	0.59	0.58	0.58
query14	0.75	0.76	0.79
query15	0.85	0.81	0.80
query16	0.37	0.37	0.37
query17	0.96	0.99	1.00
query18	0.22	0.21	0.21
query19	1.74	1.73	1.74
query20	0.01	0.01	0.01
query21	15.39	0.76	0.67
query22	4.80	7.32	1.88
query23	18.23	1.30	1.23
query24	2.20	0.22	0.22
query25	0.16	0.08	0.08
query26	0.32	0.21	0.22
query27	0.45	0.24	0.23
query28	13.28	1.02	1.00
query29	12.61	3.26	3.28
query30	0.25	0.06	0.05
query31	2.84	0.39	0.40
query32	3.26	0.47	0.48
query33	2.95	2.89	2.87
query34	17.04	4.35	4.36
query35	4.50	4.45	4.44
query36	0.66	0.47	0.48
query37	0.20	0.16	0.16
query38	0.15	0.16	0.15
query39	0.04	0.03	0.04
query40	0.15	0.12	0.11
query41	0.09	0.04	0.05
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.24 s
Total hot run time: 30.56 s

yiguolei pushed a commit that referenced this pull request Jul 16, 2024
…ncts is not slot_a = slot_b (#37683)

pick from master #37644

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39864 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f7c068f50a41f3b0ef54e9eb449cae8cc3ea2688, data reload: false

------ Round 1 ----------------------------------
q1	17628	4481	4288	4288
q2	2016	192	183	183
q3	10524	1310	1127	1127
q4	10204	750	893	750
q5	7557	2687	2762	2687
q6	220	138	139	138
q7	1024	634	616	616
q8	9215	2130	2060	2060
q9	8813	6584	6586	6584
q10	8903	3786	3875	3786
q11	447	235	245	235
q12	443	233	239	233
q13	17841	3002	2990	2990
q14	279	242	241	241
q15	522	475	489	475
q16	515	387	382	382
q17	974	647	674	647
q18	7961	7466	7403	7403
q19	5678	1449	1288	1288
q20	646	317	344	317
q21	5064	3154	3347	3154
q22	355	280	286	280
Total cold run time: 116829 ms
Total hot run time: 39864 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4385	4283	4225	4225
q2	382	282	273	273
q3	3047	2896	2941	2896
q4	2003	1722	1714	1714
q5	5619	5531	5544	5531
q6	227	139	140	139
q7	2186	1889	1841	1841
q8	3331	3499	3422	3422
q9	8810	8817	8931	8817
q10	4103	3962	3833	3833
q11	645	527	512	512
q12	829	624	647	624
q13	17184	3159	3206	3159
q14	326	290	288	288
q15	526	506	485	485
q16	496	444	444	444
q17	1840	1548	1529	1529
q18	8208	7837	7760	7760
q19	3027	1587	1712	1587
q20	2162	1902	1852	1852
q21	5171	4976	4753	4753
q22	614	511	552	511
Total cold run time: 75121 ms
Total hot run time: 56195 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173195 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f7c068f50a41f3b0ef54e9eb449cae8cc3ea2688, data reload: false

query1	914	372	364	364
query2	6423	1807	1797	1797
query3	6657	212	218	212
query4	28350	17541	17470	17470
query5	3596	467	467	467
query6	255	175	160	160
query7	4580	297	288	288
query8	237	184	199	184
query9	8554	2378	2336	2336
query10	433	287	270	270
query11	12293	10022	10072	10022
query12	124	88	86	86
query13	1657	375	372	372
query14	10123	7568	7512	7512
query15	217	171	172	171
query16	7761	317	312	312
query17	1799	563	542	542
query18	1986	307	289	289
query19	206	151	159	151
query20	90	83	85	83
query21	211	128	125	125
query22	4400	4092	3932	3932
query23	34055	33681	33471	33471
query24	11035	2915	2980	2915
query25	650	412	428	412
query26	1226	156	152	152
query27	2507	279	275	275
query28	7037	2040	2042	2040
query29	900	632	633	632
query30	256	151	168	151
query31	971	762	765	762
query32	98	50	56	50
query33	758	301	294	294
query34	1011	497	497	497
query35	696	588	595	588
query36	1111	1007	976	976
query37	168	87	88	87
query38	2943	2932	2864	2864
query39	912	843	873	843
query40	205	127	118	118
query41	45	46	44	44
query42	110	106	100	100
query43	484	464	468	464
query44	1252	723	737	723
query45	189	161	161	161
query46	1068	757	768	757
query47	1858	1779	1760	1760
query48	375	298	296	296
query49	841	409	416	409
query50	775	396	396	396
query51	7015	6800	6869	6800
query52	105	91	91	91
query53	353	288	291	288
query54	881	454	441	441
query55	76	72	71	71
query56	290	266	282	266
query57	1122	1048	1061	1048
query58	232	258	251	251
query59	2879	2605	2829	2605
query60	303	268	269	268
query61	120	89	90	89
query62	810	651	655	651
query63	314	283	284	283
query64	9432	2224	1673	1673
query65	3143	3088	3128	3088
query66	741	325	325	325
query67	15363	14933	15117	14933
query68	8290	536	529	529
query69	763	433	355	355
query70	1203	1178	1146	1146
query71	509	281	274	274
query72	8312	6061	5335	5335
query73	1169	325	326	325
query74	6114	5647	5693	5647
query75	4566	2692	2717	2692
query76	4821	956	1011	956
query77	750	299	303	299
query78	9710	9252	8843	8843
query79	8086	516	515	515
query80	1035	471	471	471
query81	558	224	223	223
query82	545	139	128	128
query83	297	161	161	161
query84	271	91	83	83
query85	1066	311	302	302
query86	349	309	294	294
query87	3328	3116	3104	3104
query88	4441	2479	2472	2472
query89	525	389	385	385
query90	1992	190	188	188
query91	124	98	101	98
query92	62	51	50	50
query93	5949	498	499	498
query94	1259	214	211	211
query95	401	321	312	312
query96	614	277	279	277
query97	3140	3033	3024	3024
query98	213	206	204	204
query99	1577	1241	1280	1241
Total cold run time: 299620 ms
Total hot run time: 173195 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.53 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f7c068f50a41f3b0ef54e9eb449cae8cc3ea2688, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.09	0.09
query5	0.50	0.49	0.47
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.56	0.48	0.49
query10	0.54	0.55	0.55
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.58
query14	0.75	0.79	0.77
query15	0.86	0.81	0.83
query16	0.36	0.37	0.36
query17	0.94	0.94	0.98
query18	0.22	0.21	0.22
query19	1.82	1.77	1.73
query20	0.01	0.01	0.01
query21	15.40	0.75	0.67
query22	3.99	7.12	1.86
query23	18.29	1.47	1.29
query24	2.07	0.24	0.22
query25	0.16	0.08	0.08
query26	0.29	0.21	0.22
query27	0.45	0.23	0.23
query28	13.24	1.02	1.00
query29	12.62	3.30	3.29
query30	0.25	0.05	0.05
query31	2.88	0.38	0.39
query32	3.28	0.49	0.47
query33	2.89	2.86	2.84
query34	17.58	4.37	4.32
query35	4.46	4.43	4.49
query36	0.64	0.47	0.48
query37	0.19	0.16	0.16
query38	0.16	0.15	0.14
query39	0.04	0.03	0.03
query40	0.16	0.12	0.12
query41	0.09	0.05	0.06
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.94 s
Total hot run time: 30.53 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 16, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

morrySnow pushed a commit that referenced this pull request Jul 17, 2024
@starocean999 starocean999 merged commit 5670e1a into apache:master Jul 17, 2024
27 of 29 checks passed
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
…ncts is not slot_a = slot_b (#37644)

sql:
`select * from t1 where t1.c1 < (select max(t2.c1) from t2 where t1.c2 =
abs(t2.c2));`
before this pr, we would use the equivalent sql as bellow, which is
wrong
`select t1.* from t1 semi join ( select t2.c2 as c2, max(t2.c1) as c1
from t2 group by t2.c ) tx on t1.c1 < tx.c1 and t1.c2 = tx.c2;`
the pr fix the problem and get the correct one as:
`select t1.* from t1 semi join ( select abs(t2.c2) as c2, max(t2.c1) as
c1 from t2 group by abs(t2.c) ) tx on t1.c1 < tx.c1 and t1.c2 = tx.c2;`
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants