Skip to content

branch-3.1: [fix](compaction) skip tablets with compaction score 0 during compaction #55550 #55570

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Sep 4, 2025
Merged

branch-3.1: [fix](compaction) skip tablets with compaction score 0 during compaction #55550 #55570

morrySnow merged 1 commit into from
Sep 4, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Sep 2, 2025

Cherry-picked from #55550

@github-actions github-actions bot requested a review from morrySnow as a code owner September 2, 2025 05:23
@Thearas
Copy link
Contributor

Thearas commented Sep 2, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Sep 2, 2025
@dataroaring dataroaring reopened this Sep 2, 2025
@Thearas
Copy link
Contributor

Thearas commented Sep 2, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 2, 2025

TPC-H: Total hot run time: 32765 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cbb2378730fcad76ce10d0a1ade44485df27d5c7, data reload: false

------ Round 1 ----------------------------------
q1	17579	5451	5517	5451
q2	2037	415	283	283
q3	12112	1254	734	734
q4	10313	866	446	446
q5	9570	2370	2130	2130
q6	189	160	131	131
q7	901	727	615	615
q8	9342	1402	1119	1119
q9	5201	4947	4913	4913
q10	6779	2282	1837	1837
q11	485	277	266	266
q12	342	367	208	208
q13	17769	3606	3027	3027
q14	230	225	211	211
q15	532	480	470	470
q16	435	428	384	384
q17	600	854	364	364
q18	6849	6480	6511	6480
q19	1236	962	539	539
q20	334	327	195	195
q21	2797	2126	1987	1987
q22	1035	1012	975	975
Total cold run time: 106667 ms
Total hot run time: 32765 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5530	5477	5504	5477
q2	233	329	236	236
q3	2279	2656	2307	2307
q4	1315	1768	1385	1385
q5	4382	4971	4977	4971
q6	178	165	128	128
q7	2077	1983	1839	1839
q8	2633	2833	2713	2713
q9	7477	7387	7414	7387
q10	3100	3311	2763	2763
q11	604	538	498	498
q12	712	779	623	623
q13	3445	3785	3288	3288
q14	290	301	279	279
q15	547	478	465	465
q16	450	503	454	454
q17	1276	1757	1266	1266
q18	7596	7442	7301	7301
q19	835	1090	1150	1090
q20	1995	2077	1904	1904
q21	5303	4900	4530	4530
q22	1058	1019	1015	1015
Total cold run time: 53315 ms
Total hot run time: 51919 ms

@doris-robot
Copy link

doris-robot commented Sep 2, 2025

TPC-DS: Total hot run time: 192560 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cbb2378730fcad76ce10d0a1ade44485df27d5c7, data reload: false

query1	961	405	397	397
query2	6218	2019	1935	1935
query3	8685	201	197	197
query4	33379	23793	23478	23478
query5	3707	586	427	427
query6	272	188	173	173
query7	4196	482	305	305
query8	298	234	240	234
query9	9309	2596	2575	2575
query10	493	321	253	253
query11	18034	15460	15198	15198
query12	165	107	101	101
query13	1546	551	435	435
query14	10321	7043	6907	6907
query15	249	193	181	181
query16	8099	683	483	483
query17	1600	766	593	593
query18	2174	428	339	339
query19	234	189	168	168
query20	128	129	128	128
query21	206	131	116	116
query22	4647	4731	4506	4506
query23	35004	35030	34022	34022
query24	7257	2713	2695	2695
query25	495	485	427	427
query26	739	253	173	173
query27	1991	490	348	348
query28	5317	2247	2210	2210
query29	619	571	494	494
query30	277	198	168	168
query31	971	901	830	830
query32	88	60	57	57
query33	517	373	314	314
query34	749	850	510	510
query35	776	802	732	732
query36	1016	1060	928	928
query37	110	91	64	64
query38	4020	3930	3978	3930
query39	1512	1483	1506	1483
query40	210	127	107	107
query41	50	48	48	48
query42	129	114	120	114
query43	500	504	464	464
query44	1358	819	810	810
query45	181	172	171	171
query46	872	1063	679	679
query47	1972	1989	1960	1960
query48	418	438	357	357
query49	736	487	403	403
query50	697	693	450	450
query51	7199	7194	7183	7183
query52	104	107	92	92
query53	232	254	199	199
query54	547	551	489	489
query55	84	81	77	77
query56	271	277	259	259
query57	1288	1268	1198	1198
query58	223	220	210	210
query59	3114	3204	3196	3196
query60	317	275	274	274
query61	152	107	138	107
query62	800	773	715	715
query63	240	197	191	191
query64	4223	999	657	657
query65	3378	3292	3322	3292
query66	804	425	305	305
query67	16365	15759	15663	15663
query68	7474	822	558	558
query69	503	315	268	268
query70	1163	1112	1112	1112
query71	372	286	259	259
query72	5712	3989	3886	3886
query73	634	741	359	359
query74	10509	9145	9005	9005
query75	3259	3140	2659	2659
query76	3226	1185	770	770
query77	669	378	287	287
query78	10309	10516	9590	9590
query79	3417	913	590	590
query80	643	520	439	439
query81	502	257	219	219
query82	567	124	91	91
query83	159	154	147	147
query84	235	95	83	83
query85	780	376	299	299
query86	389	324	308	308
query87	4288	4295	4227	4227
query88	5295	2452	2429	2429
query89	407	329	295	295
query90	1759	193	193	193
query91	139	150	108	108
query92	68	56	52	52
query93	2080	869	559	559
query94	679	410	314	314
query95	345	272	271	271
query96	504	622	294	294
query97	3170	3285	3189	3189
query98	235	214	206	206
query99	2049	1439	1310	1310
Total cold run time: 294383 ms
Total hot run time: 192560 ms

@doris-robot
Copy link

doris-robot commented Sep 2, 2025

ClickBench: Total hot run time: 29.27 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cbb2378730fcad76ce10d0a1ade44485df27d5c7, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.06
query4	1.62	0.11	0.11
query5	0.50	0.50	0.50
query6	1.12	0.72	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.60	0.50	0.49
query10	0.55	0.54	0.55
query11	0.14	0.14	0.10
query12	0.14	0.11	0.11
query13	0.63	0.60	0.58
query14	0.77	0.81	0.80
query15	0.84	0.83	0.82
query16	0.38	0.40	0.42
query17	1.09	1.09	1.10
query18	0.23	0.23	0.23
query19	1.97	1.87	1.86
query20	0.02	0.02	0.01
query21	15.37	0.93	0.57
query22	0.74	0.96	0.66
query23	14.97	1.37	0.51
query24	3.52	1.36	1.90
query25	0.12	0.06	0.15
query26	0.36	0.15	0.13
query27	0.07	0.05	0.04
query28	14.23	0.90	0.42
query29	12.57	3.99	3.31
query30	0.26	0.09	0.06
query31	2.83	0.62	0.39
query32	3.23	0.53	0.46
query33	3.00	2.97	3.06
query34	16.30	5.19	4.46
query35	4.54	4.58	4.55
query36	0.66	0.51	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.26 s
Total hot run time: 29.27 s

@doris-robot
Copy link

doris-robot commented Sep 2, 2025

BE UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.48% (12722/27973)
Line Coverage 36.36% (113434/311937)
Region Coverage 34.00% (64942/191013)
Branch Coverage 31.05% (34094/109816)

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 2, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.33% (20994/27503)
Line Coverage 69.68% (216623/310886)
Region Coverage 67.65% (129710/191751)
Branch Coverage 61.16% (67469/110310)

dataroaring
dataroaring approved these changes Sep 2, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morrySnow morrySnow merged commit 3a0bef9 into branch-3.1 Sep 4, 2025
22 of 23 checks passed
@github-actions github-actions bot deleted the auto-pick-55550-branch-3.1 branch September 4, 2025 02:37
@morrySnow morrySnow mentioned this pull request Sep 22, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@Thearas @doris-robot @hello-stephen @dataroaring @morrySnow @luwei16