Skip to content

[fix](sql_select_limit) sql_select_limit should not affect DML #53379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 2 commits into from
Jul 18, 2025
Merged

[fix](sql_select_limit) sql_select_limit should not affect DML #53379

morrySnow merged 2 commits into from
Jul 18, 2025

Conversation

@morrySnow
Copy link
Contributor

@morrySnow morrySnow commented Jul 16, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #53354

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 16, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow
Copy link
Contributor Author

morrySnow commented Jul 16, 2025

run buildall

@morrySnow morrySnow added usercase Important user case type label dev/2.1.x dev/3.0.x dev/3.1.x labels Jul 16, 2025
@doris-robot
Copy link

doris-robot commented Jul 16, 2025

TPC-H: Total hot run time: 34137 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3fefcf620a20e94a0b735498c9476f7b1e590b10, data reload: false

------ Round 1 ----------------------------------
q1	17572	5396	5101	5101
q2	1920	283	180	180
q3	10344	1268	724	724
q4	10255	1021	522	522
q5	8347	2360	2373	2360
q6	207	158	130	130
q7	912	754	592	592
q8	9311	1353	1134	1134
q9	7160	5141	5172	5141
q10	6958	2381	1997	1997
q11	503	290	270	270
q12	360	367	215	215
q13	17757	3741	3118	3118
q14	229	219	211	211
q15	573	473	481	473
q16	423	444	394	394
q17	611	859	366	366
q18	7724	7082	7101	7082
q19	1584	945	563	563
q20	349	328	219	219
q21	4118	3201	2368	2368
q22	1064	1074	977	977
Total cold run time: 108281 ms
Total hot run time: 34137 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5250	5085	5107	5085
q2	242	328	220	220
q3	2191	2642	2284	2284
q4	1365	1789	1315	1315
q5	4378	4471	4574	4471
q6	225	171	133	133
q7	1995	1997	1792	1792
q8	2662	2696	2671	2671
q9	7325	7401	7222	7222
q10	3103	3333	2832	2832
q11	587	513	492	492
q12	680	776	602	602
q13	3695	4043	3412	3412
q14	292	318	267	267
q15	534	468	487	468
q16	459	511	528	511
q17	1254	1568	1334	1334
q18	7892	7755	7407	7407
q19	788	764	760	760
q20	1932	2003	1846	1846
q21	4810	4433	4303	4303
q22	1086	1072	997	997
Total cold run time: 52745 ms
Total hot run time: 50424 ms

@doris-robot
Copy link

doris-robot commented Jul 16, 2025

TPC-DS: Total hot run time: 187353 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3fefcf620a20e94a0b735498c9476f7b1e590b10, data reload: false

query1	993	368	389	368
query2	6502	1652	1641	1641
query3	6745	211	206	206
query4	26381	23677	23199	23199
query5	4376	601	440	440
query6	319	223	199	199
query7	4624	502	297	297
query8	275	226	216	216
query9	8603	2631	2633	2631
query10	468	326	274	274
query11	15968	15023	14895	14895
query12	168	107	100	100
query13	1634	535	397	397
query14	9311	5804	5875	5804
query15	209	193	162	162
query16	7359	596	498	498
query17	1214	708	573	573
query18	1999	399	298	298
query19	198	187	154	154
query20	119	117	110	110
query21	218	124	102	102
query22	4276	4329	4183	4183
query23	34120	33263	33252	33252
query24	8475	2387	2395	2387
query25	537	460	397	397
query26	1250	261	150	150
query27	2762	516	345	345
query28	4345	2140	2150	2140
query29	751	570	423	423
query30	276	224	194	194
query31	926	848	772	772
query32	73	60	60	60
query33	557	357	310	310
query34	815	842	512	512
query35	818	804	724	724
query36	967	992	920	920
query37	109	98	78	78
query38	4129	4126	4094	4094
query39	1531	1452	1450	1450
query40	216	121	111	111
query41	66	61	63	61
query42	122	111	109	109
query43	518	495	463	463
query44	1342	844	822	822
query45	177	177	168	168
query46	849	1031	631	631
query47	1751	1815	1739	1739
query48	391	429	306	306
query49	747	483	432	432
query50	629	694	409	409
query51	5530	5519	5497	5497
query52	111	104	96	96
query53	217	268	193	193
query54	567	563	487	487
query55	83	78	85	78
query56	304	297	287	287
query57	1213	1198	1138	1138
query58	260	261	267	261
query59	2559	2661	2486	2486
query60	334	324	303	303
query61	123	120	123	120
query62	784	716	673	673
query63	224	201	190	190
query64	4316	977	653	653
query65	4241	4189	4220	4189
query66	1094	435	334	334
query67	16047	15839	15654	15654
query68	7884	864	521	521
query69	478	300	267	267
query70	1216	1136	1105	1105
query71	433	319	296	296
query72	5550	4699	4822	4699
query73	663	612	354	354
query74	9069	9170	9119	9119
query75	3497	3198	2667	2667
query76	3321	1167	725	725
query77	732	397	307	307
query78	10021	10298	9364	9364
query79	2254	822	597	597
query80	585	535	442	442
query81	501	264	227	227
query82	456	124	95	95
query83	299	258	230	230
query84	248	106	86	86
query85	813	368	310	310
query86	386	304	296	296
query87	4454	4428	4285	4285
query88	3960	2292	2291	2291
query89	382	323	305	305
query90	1889	210	208	208
query91	143	132	112	112
query92	77	63	56	56
query93	1869	950	588	588
query94	696	418	303	303
query95	372	296	275	275
query96	493	569	277	277
query97	2716	2739	2604	2604
query98	232	207	213	207
query99	1380	1402	1290	1290
Total cold run time: 276284 ms
Total hot run time: 187353 ms

@doris-robot
Copy link

doris-robot commented Jul 16, 2025

ClickBench: Total hot run time: 33.06 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3fefcf620a20e94a0b735498c9476f7b1e590b10, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.05
query3	0.25	0.07	0.07
query4	1.61	0.12	0.11
query5	0.42	0.44	0.43
query6	1.17	0.66	0.66
query7	0.02	0.01	0.02
query8	0.05	0.04	0.03
query9	0.61	0.52	0.52
query10	0.58	0.56	0.56
query11	0.15	0.11	0.11
query12	0.15	0.11	0.11
query13	0.63	0.61	0.61
query14	0.81	0.80	0.82
query15	0.90	0.86	0.87
query16	0.40	0.40	0.39
query17	1.09	1.10	1.06
query18	0.23	0.21	0.22
query19	1.96	1.86	1.90
query20	0.01	0.01	0.01
query21	15.39	0.89	0.55
query22	0.76	1.17	0.72
query23	14.89	1.39	0.58
query24	7.04	1.27	1.42
query25	0.55	0.26	0.12
query26	0.55	0.16	0.13
query27	0.06	0.04	0.04
query28	9.85	0.96	0.44
query29	12.64	4.00	3.34
query30	3.03	2.96	2.93
query31	2.82	0.60	0.38
query32	3.24	0.55	0.47
query33	3.14	3.09	3.15
query34	16.05	5.48	4.83
query35	4.92	4.92	4.87
query36	0.69	0.50	0.49
query37	0.09	0.07	0.07
query38	0.06	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.15	0.13
query41	0.07	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.28 s
Total hot run time: 33.06 s

starocean999
starocean999 previously approved these changes Jul 17, 2025
View reviewed changes
@morrySnow
Copy link
Contributor Author

morrySnow commented Jul 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

TPC-H: Total hot run time: 34003 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 764f1d82e04001d43e0a15fc9918d1d85cdf4c6c, data reload: false

------ Round 1 ----------------------------------
q1	17590	5407	5254	5254
q2	1920	277	177	177
q3	10328	1294	707	707
q4	10232	1002	531	531
q5	7564	2416	2317	2317
q6	180	160	129	129
q7	903	752	596	596
q8	9307	1274	1043	1043
q9	7018	5154	5088	5088
q10	6881	2375	1975	1975
q11	474	291	269	269
q12	344	347	214	214
q13	17778	3728	3098	3098
q14	219	229	222	222
q15	538	473	477	473
q16	426	443	371	371
q17	579	871	360	360
q18	7516	7130	7153	7130
q19	1231	961	556	556
q20	335	338	227	227
q21	3728	2573	2301	2301
q22	1097	1003	965	965
Total cold run time: 106188 ms
Total hot run time: 34003 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5344	5385	5450	5385
q2	239	326	219	219
q3	2181	2680	2286	2286
q4	1357	1808	1316	1316
q5	4152	4468	4606	4468
q6	214	171	128	128
q7	1995	2067	1836	1836
q8	2665	2517	2519	2517
q9	7375	7395	7377	7377
q10	3143	3413	2841	2841
q11	571	515	489	489
q12	715	797	626	626
q13	3656	3974	3468	3468
q14	307	306	311	306
q15	535	471	495	471
q16	452	500	441	441
q17	1167	1539	1445	1445
q18	8027	7510	7688	7510
q19	786	773	815	773
q20	1929	1963	1840	1840
q21	4713	4368	4255	4255
q22	1115	1040	981	981
Total cold run time: 52638 ms
Total hot run time: 50978 ms

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

TPC-DS: Total hot run time: 186633 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 764f1d82e04001d43e0a15fc9918d1d85cdf4c6c, data reload: false

query1	995	407	406	406
query2	6550	1647	1648	1647
query3	6734	218	227	218
query4	26260	23462	23072	23072
query5	4359	604	428	428
query6	289	212	196	196
query7	4623	524	279	279
query8	277	214	215	214
query9	8598	2601	2604	2601
query10	474	356	256	256
query11	15677	15020	14742	14742
query12	156	104	97	97
query13	1629	540	400	400
query14	8522	5734	5773	5734
query15	193	199	172	172
query16	7133	668	481	481
query17	1209	723	571	571
query18	1968	401	298	298
query19	191	198	158	158
query20	128	122	108	108
query21	209	125	106	106
query22	4164	4127	4018	4018
query23	34177	33364	33277	33277
query24	8494	2389	2356	2356
query25	563	478	382	382
query26	1250	280	148	148
query27	2761	532	335	335
query28	4327	2150	2138	2138
query29	771	601	433	433
query30	278	213	193	193
query31	930	839	769	769
query32	71	67	66	66
query33	552	363	305	305
query34	823	888	516	516
query35	825	847	725	725
query36	1021	1007	922	922
query37	111	101	78	78
query38	4139	4226	4138	4138
query39	1506	1426	1455	1426
query40	204	121	108	108
query41	56	56	54	54
query42	128	107	105	105
query43	491	511	477	477
query44	1368	803	800	800
query45	179	183	163	163
query46	882	1067	616	616
query47	1789	1810	1763	1763
query48	383	423	297	297
query49	720	477	390	390
query50	667	733	410	410
query51	5619	5591	5456	5456
query52	112	114	97	97
query53	256	258	190	190
query54	583	567	503	503
query55	82	79	80	79
query56	304	322	311	311
query57	1178	1208	1122	1122
query58	255	248	259	248
query59	2528	2595	2477	2477
query60	345	314	307	307
query61	125	123	124	123
query62	800	749	649	649
query63	233	190	193	190
query64	4447	1005	648	648
query65	4256	4166	4181	4166
query66	1147	416	315	315
query67	15837	15918	15510	15510
query68	4517	931	532	532
query69	512	313	279	279
query70	1262	1129	1151	1129
query71	399	324	293	293
query72	5925	4884	4940	4884
query73	688	703	349	349
query74	8887	9137	8958	8958
query75	3356	3239	2677	2677
query76	3172	1306	741	741
query77	475	370	289	289
query78	10025	10029	9312	9312
query79	2433	866	586	586
query80	913	508	419	419
query81	507	270	231	231
query82	393	132	94	94
query83	240	252	225	225
query84	244	104	86	86
query85	846	360	327	327
query86	416	330	280	280
query87	4544	4555	4458	4458
query88	3529	2263	2245	2245
query89	393	315	277	277
query90	1929	211	202	202
query91	148	141	107	107
query92	78	66	56	56
query93	2675	957	590	590
query94	751	432	312	312
query95	381	301	277	277
query96	497	628	289	289
query97	2669	2792	2680	2680
query98	235	215	201	201
query99	1357	1452	1265	1265
Total cold run time: 272307 ms
Total hot run time: 186633 ms

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

ClickBench: Total hot run time: 32.8 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 764f1d82e04001d43e0a15fc9918d1d85cdf4c6c, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.05	0.04
query3	0.24	0.07	0.08
query4	1.61	0.12	0.11
query5	0.45	0.42	0.43
query6	1.19	0.65	0.68
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.59	0.52	0.52
query10	0.58	0.56	0.57
query11	0.17	0.11	0.11
query12	0.15	0.12	0.11
query13	0.63	0.61	0.61
query14	0.81	0.81	0.81
query15	0.90	0.87	0.88
query16	0.38	0.39	0.40
query17	1.06	1.06	1.03
query18	0.22	0.21	0.22
query19	1.94	1.81	1.86
query20	0.01	0.01	0.02
query21	15.40	0.94	0.56
query22	0.75	1.34	0.75
query23	14.74	1.34	0.64
query24	7.03	1.23	0.96
query25	0.48	0.11	0.08
query26	0.63	0.18	0.15
query27	0.06	0.05	0.05
query28	9.20	0.87	0.45
query29	12.54	3.99	3.31
query30	3.01	2.96	2.91
query31	2.83	0.58	0.38
query32	3.23	0.55	0.47
query33	3.04	3.18	3.14
query34	16.17	5.49	4.86
query35	4.88	4.86	4.91
query36	0.68	0.51	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.18	0.15	0.14
query41	0.08	0.02	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.29 s
Total hot run time: 32.8 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 18, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 18, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 18, 2025

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit c60378e into apache:master Jul 18, 2025
26 of 28 checks passed
@morrySnow morrySnow deleted the fix_select_limit_dml branch July 18, 2025 06:32
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jul 18, 2025
@morrySnow
branch-3.0: [fix](sql_select_limit) sql_select_limit should not affec…
0a57b36 
…t DML apache#53379

cherry picked from apache#53379
@morrySnow morrySnow mentioned this pull request Jul 18, 2025
Merged
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jul 18, 2025
@morrySnow
branch-3.1: [fix](sql_select_limit) sql_select_limit should not affec…
6686bcc 
…t DML apache#53379

cherry picked from apache#53379
@morrySnow morrySnow mentioned this pull request Jul 18, 2025
Merged
morrySnow added a commit that referenced this pull request Jul 18, 2025
@morrySnow
branch-3.1: [fix](sql_select_limit) sql_select_limit should not affec…
f8d3e85 
…t DML #53379 (#53534)

cherry picked from #53379
dataroaring pushed a commit that referenced this pull request Jul 21, 2025
@morrySnow
branch-3.0: [fix](sql_select_limit) sql_select_limit should not affec…
484807a 
…t DML #53379 (#53528)

cherry picked from #53379
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jul 23, 2025
@morrySnow
branch-2.1: [fix](sql_select_limit) sql_select_limit should not affec…
7c2b884 
…t DML apache#53379

cherry picked from apache#53379
@morrySnow morrySnow mentioned this pull request Jul 23, 2025
Merged
yiguolei pushed a commit that referenced this pull request Jul 25, 2025
@morrySnow
branch-2.1: [fix](sql_select_limit) sql_select_limit should not affec…
850360c 
…t DML #53379 (#53764)

cherry picked from #53379
@gavinchou gavinchou mentioned this pull request Aug 15, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@924060929 924060929 924060929 approved these changes

@starocean999 starocean999 starocean999 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.11-merged dev/3.0.7-merged dev/3.1.0-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug] sql_select_limit for INSERT behaves different on master/non-master fe

7 participants

@morrySnow @hello-stephen @doris-robot @924060929 @starocean999 @yiguolei @dataroaring