Skip to content

[fix](compaction) fix time series compaction time threshold (#50306) #50391

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Apr 25, 2025
Merged

[fix](compaction) fix time series compaction time threshold (#50306) #50391

dataroaring merged 1 commit into from
Apr 25, 2025

Conversation

@csun5285
Copy link
Contributor

@csun5285 csun5285 commented Apr 24, 2025

pick from master #50306

@csun5285 csun5285 requested a review from dataroaring as a code owner April 24, 2025 11:24
@csun5285
Copy link
Contributor Author

csun5285 commented Apr 24, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Apr 24, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@csun5285
Copy link
Contributor Author

csun5285 commented Apr 24, 2025

run buildall

@csun5285
[fix](compaction) fix time series compaction time threshold (apache#5…
bfa0e9f 
…0306)

fix time series compaction time threshold

fix pick

fix build
@csun5285
Copy link
Contributor Author

csun5285 commented Apr 24, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Apr 24, 2025

TPC-H: Total hot run time: 39902 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bfa0e9ff45b14750a895c611c714a2f3857742ce, data reload: false

------ Round 1 ----------------------------------
q1	17635	6838	6574	6574
q2	2062	176	180	176
q3	10691	1074	1098	1074
q4	10556	766	699	699
q5	7719	2848	2823	2823
q6	219	130	129	129
q7	967	600	601	600
q8	9374	1893	2039	1893
q9	6485	6364	6405	6364
q10	7038	2211	2295	2211
q11	460	255	257	255
q12	397	217	207	207
q13	17795	2951	3002	2951
q14	242	205	208	205
q15	510	459	475	459
q16	647	587	579	579
q17	988	558	589	558
q18	7079	6528	6535	6528
q19	1455	1073	1101	1073
q20	485	224	222	222
q21	4105	3444	3331	3331
q22	1134	991	1012	991
Total cold run time: 108043 ms
Total hot run time: 39902 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6712	6525	6583	6525
q2	330	229	232	229
q3	2855	2753	2903	2753
q4	2039	1783	1800	1783
q5	5794	5755	5710	5710
q6	212	125	129	125
q7	2199	1785	1785	1785
q8	3343	3514	3513	3513
q9	8935	8778	8908	8778
q10	3584	3516	3519	3516
q11	593	492	490	490
q12	788	582	631	582
q13	10923	3161	3113	3113
q14	300	266	267	266
q15	520	456	472	456
q16	692	636	664	636
q17	1839	1642	1593	1593
q18	8183	7798	7699	7699
q19	1705	1558	1366	1366
q20	2041	1790	1858	1790
q21	5526	5379	5213	5213
q22	1154	1108	1016	1016
Total cold run time: 70267 ms
Total hot run time: 58937 ms

@doris-robot
Copy link

doris-robot commented Apr 24, 2025

TPC-DS: Total hot run time: 197374 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bfa0e9ff45b14750a895c611c714a2f3857742ce, data reload: false

query1	1322	906	913	906
query2	6260	2001	2020	2001
query3	10812	4343	4341	4341
query4	60941	29438	23388	23388
query5	5227	462	438	438
query6	399	171	172	171
query7	5451	324	314	314
query8	310	231	225	225
query9	8583	2633	2625	2625
query10	463	269	262	262
query11	17739	15256	16080	15256
query12	151	103	107	103
query13	1402	459	425	425
query14	10616	6970	7526	6970
query15	203	175	180	175
query16	7149	496	523	496
query17	1137	578	575	575
query18	1897	324	335	324
query19	207	165	162	162
query20	125	109	110	109
query21	214	105	106	105
query22	4520	4379	4633	4379
query23	34628	34330	33989	33989
query24	6119	2874	2971	2874
query25	548	434	436	434
query26	652	170	180	170
query27	2217	377	350	350
query28	4255	2472	2434	2434
query29	700	499	453	453
query30	258	169	174	169
query31	1017	813	850	813
query32	68	54	60	54
query33	428	297	278	278
query34	919	521	526	521
query35	860	737	703	703
query36	1041	936	949	936
query37	116	73	67	67
query38	4049	3990	3945	3945
query39	1481	1509	1486	1486
query40	204	98	97	97
query41	50	47	48	47
query42	112	99	100	99
query43	512	471	500	471
query44	1154	824	835	824
query45	195	167	166	166
query46	1149	741	745	741
query47	2009	1906	1939	1906
query48	490	385	389	385
query49	731	385	385	385
query50	866	447	443	443
query51	7450	7396	7326	7326
query52	101	89	88	88
query53	259	187	182	182
query54	579	454	472	454
query55	76	76	79	76
query56	272	245	257	245
query57	1274	1145	1140	1140
query58	230	212	221	212
query59	3192	3119	3062	3062
query60	304	256	248	248
query61	109	107	107	107
query62	749	669	666	666
query63	218	188	183	183
query64	1446	652	668	652
query65	3296	3200	3229	3200
query66	671	299	302	299
query67	15917	15715	15812	15715
query68	4139	589	589	589
query69	436	267	272	267
query70	1104	1103	1057	1057
query71	329	265	266	265
query72	6341	4065	4008	4008
query73	759	344	350	344
query74	10051	9174	8932	8932
query75	3358	2619	2679	2619
query76	1962	1106	1168	1106
query77	492	274	279	274
query78	10631	9583	9564	9564
query79	1499	596	597	596
query80	868	418	424	418
query81	516	242	233	233
query82	1257	89	84	84
query83	241	149	147	147
query84	284	80	79	79
query85	879	312	287	287
query86	338	310	303	303
query87	4477	4211	4239	4211
query88	3905	2374	2379	2374
query89	409	294	292	292
query90	1978	186	184	184
query91	189	152	150	150
query92	66	50	50	50
query93	1878	560	561	560
query94	714	301	306	301
query95	349	261	261	261
query96	609	283	282	282
query97	3307	3196	3188	3188
query98	215	208	204	204
query99	1572	1300	1314	1300
Total cold run time: 313251 ms
Total hot run time: 197374 ms

@hello-stephen
Copy link
Contributor

hello-stephen commented Apr 24, 2025

BE UT Coverage Report

Increment line coverage 100.00% (20/20) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 40.22% (10557/26247)
Line Coverage 31.00% (89332/288214)
Region Coverage 30.12% (46080/152979)
Branch Coverage 26.63% (23559/88464)

@doris-robot
Copy link

doris-robot commented Apr 24, 2025

ClickBench: Total hot run time: 32.3 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bfa0e9ff45b14750a895c611c714a2f3857742ce, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.62	0.11	0.10
query5	0.52	0.52	0.52
query6	1.15	0.72	0.73
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.50	0.49
query10	0.56	0.55	0.54
query11	0.15	0.11	0.10
query12	0.13	0.12	0.13
query13	0.61	0.59	0.60
query14	2.73	2.70	2.79
query15	0.89	0.84	0.84
query16	0.39	0.37	0.38
query17	1.04	1.04	1.03
query18	0.26	0.23	0.23
query19	1.94	1.85	2.12
query20	0.02	0.00	0.01
query21	15.36	0.60	0.59
query22	2.86	2.02	1.96
query23	17.10	0.84	0.83
query24	3.06	0.84	2.24
query25	0.25	0.23	0.18
query26	0.34	0.15	0.15
query27	0.03	0.05	0.04
query28	9.38	0.52	0.47
query29	12.61	3.26	3.25
query30	0.26	0.06	0.07
query31	2.85	0.39	0.38
query32	3.25	0.48	0.46
query33	2.97	3.06	2.99
query34	17.15	4.48	4.46
query35	4.48	4.49	4.53
query36	0.67	0.48	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.12	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.07 s
Total hot run time: 32.3 s

dataroaring
dataroaring approved these changes Apr 25, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 0fb1395 into apache:branch-3.0 Apr 25, 2025
21 of 23 checks passed
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 25, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 25, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 25, 2025

PR approved by anyone and no changes requested.

dataroaring pushed a commit that referenced this pull request Apr 25, 2025
@csun5285
[fix](compaction) fix time series compaction time threshold (#50306) (#…
e568976 
…50391)

pick from master #50306
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@csun5285 @hello-stephen @doris-robot @dataroaring