Skip to content

Conversation

@github-actions
Copy link
Contributor

Cherry-picked from #51649

…lection strategy as cumulative compaction (#51649)

1. Make base compaction use the same tablet selection strategy as
cumulative compaction.
2. Modify some parameters to make them clearer
@github-actions github-actions bot requested a review from dataroaring as a code owner June 26, 2025 14:39
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Jun 26, 2025
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39936 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 29706b0f5be233a8a147bb88cd736579ab486d97, data reload: false

------ Round 1 ----------------------------------
q1	17616	6967	6664	6664
q2	2064	167	165	165
q3	10791	1092	1217	1092
q4	10544	761	827	761
q5	7745	2923	2827	2827
q6	217	137	133	133
q7	976	614	609	609
q8	9579	1953	2003	1953
q9	6839	6446	6438	6438
q10	6976	2267	2326	2267
q11	460	260	254	254
q12	397	222	208	208
q13	17793	3016	3002	3002
q14	248	204	202	202
q15	502	453	469	453
q16	482	393	374	374
q17	977	533	584	533
q18	7212	6825	6684	6684
q19	1392	1064	1102	1064
q20	491	211	199	199
q21	3925	3056	3247	3056
q22	1095	998	1032	998
Total cold run time: 108321 ms
Total hot run time: 39936 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6718	6608	6847	6608
q2	329	234	232	232
q3	2887	2739	2953	2739
q4	2009	1805	1737	1737
q5	5715	5742	5681	5681
q6	211	132	132	132
q7	2258	1801	1806	1801
q8	3370	3572	3527	3527
q9	8893	8706	8873	8706
q10	3544	3570	3524	3524
q11	614	498	504	498
q12	801	601	607	601
q13	6157	3135	3216	3135
q14	294	270	278	270
q15	514	453	465	453
q16	503	437	447	437
q17	1839	1617	1629	1617
q18	8292	7854	7870	7854
q19	1700	1565	1611	1565
q20	2064	1844	1808	1808
q21	5241	5025	5055	5025
q22	1160	1073	1018	1018
Total cold run time: 65113 ms
Total hot run time: 58968 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197918 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 29706b0f5be233a8a147bb88cd736579ab486d97, data reload: false

query1	1287	889	884	884
query2	6312	1887	1838	1838
query3	10957	4305	4562	4305
query4	62087	29440	24032	24032
query5	5278	447	447	447
query6	390	172	175	172
query7	5478	299	299	299
query8	299	217	213	213
query9	8515	2581	2558	2558
query10	466	289	253	253
query11	17677	15236	15957	15236
query12	156	103	104	103
query13	1440	453	444	444
query14	10231	7141	7222	7141
query15	199	185	173	173
query16	7156	524	494	494
query17	1130	571	546	546
query18	1888	308	309	308
query19	204	167	153	153
query20	114	107	114	107
query21	223	102	104	102
query22	4720	4622	4618	4618
query23	34889	34287	34484	34287
query24	6162	2924	2908	2908
query25	546	425	418	418
query26	706	168	170	168
query27	2152	372	363	363
query28	4317	2201	2154	2154
query29	725	474	467	467
query30	239	159	162	159
query31	976	834	858	834
query32	71	64	58	58
query33	444	314	298	298
query34	930	532	512	512
query35	846	748	728	728
query36	1083	958	962	958
query37	113	70	69	69
query38	4078	4006	4054	4006
query39	1607	1465	1508	1465
query40	203	106	107	106
query41	51	50	51	50
query42	122	106	101	101
query43	519	488	480	480
query44	1165	815	812	812
query45	187	177	167	167
query46	1168	735	738	735
query47	2018	1926	1964	1926
query48	480	379	399	379
query49	739	434	415	415
query50	857	431	428	428
query51	7470	7224	7192	7192
query52	104	90	90	90
query53	263	194	191	191
query54	592	472	470	470
query55	78	75	80	75
query56	268	255	255	255
query57	1328	1236	1195	1195
query58	206	206	220	206
query59	3336	2980	2947	2947
query60	267	251	252	251
query61	111	113	105	105
query62	791	692	665	665
query63	217	187	183	183
query64	1364	664	636	636
query65	3261	3200	3202	3200
query66	698	306	289	289
query67	15925	15670	15466	15466
query68	4152	589	567	567
query69	435	259	258	258
query70	1116	1096	1092	1092
query71	353	255	259	255
query72	6418	4112	4005	4005
query73	752	345	355	345
query74	10185	9319	9281	9281
query75	3252	2653	2632	2632
query76	2006	1024	1045	1024
query77	524	269	272	269
query78	10626	9552	9533	9533
query79	1692	585	605	585
query80	1154	428	428	428
query81	509	220	221	220
query82	885	84	86	84
query83	160	141	142	141
query84	283	79	79	79
query85	941	295	289	289
query86	349	272	288	272
query87	4458	4298	4253	4253
query88	3886	2381	2354	2354
query89	416	292	287	287
query90	2046	186	186	186
query91	177	143	143	143
query92	61	51	52	51
query93	1757	582	556	556
query94	764	301	246	246
query95	355	249	250	249
query96	618	284	281	281
query97	3331	3196	3162	3162
query98	213	205	199	199
query99	1607	1343	1299	1299
Total cold run time: 315360 ms
Total hot run time: 197918 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.58 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 29706b0f5be233a8a147bb88cd736579ab486d97, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.07
query4	1.63	0.11	0.10
query5	0.51	0.50	0.51
query6	1.13	0.72	0.73
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.55	0.51	0.50
query10	0.56	0.58	0.56
query11	0.13	0.10	0.10
query12	0.14	0.11	0.10
query13	0.61	0.60	0.60
query14	0.78	0.80	0.81
query15	0.84	0.82	0.82
query16	0.39	0.38	0.38
query17	0.98	0.99	1.05
query18	0.23	0.20	0.21
query19	1.97	1.78	1.88
query20	0.01	0.00	0.01
query21	15.40	0.60	0.58
query22	2.45	1.81	2.10
query23	17.11	0.88	0.87
query24	2.95	1.69	1.31
query25	0.12	0.20	0.08
query26	0.48	0.13	0.14
query27	0.06	0.04	0.04
query28	9.67	0.54	0.46
query29	12.58	3.22	3.25
query30	0.25	0.06	0.07
query31	2.85	0.38	0.38
query32	3.25	0.46	0.45
query33	3.01	3.01	3.03
query34	16.97	4.53	4.58
query35	4.57	4.54	4.54
query36	0.67	0.47	0.47
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.17	0.13	0.13
query41	0.07	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 103.72 s
Total hot run time: 30.58 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 0460f62 into branch-3.0 Jun 30, 2025
22 of 25 checks passed
@github-actions github-actions bot deleted the auto-pick-51649-branch-3.0 branch June 30, 2025 07:52
koarz pushed a commit to koarz/doris that referenced this pull request Jul 3, 2025
…me tablet selection strategy as cumulative compaction apache#51649 (apache#52389)

Cherry-picked from apache#51649

Co-authored-by: abmdocrt <lianyukang@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants