Skip to content

Conversation

@csun5285
Copy link
Contributor

What problem does this PR solve?

fix time series compaction time threshold

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@csun5285
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34071 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 76b1ff887dd9a4340d259bb3f63e964430a9f7c6, data reload: false

------ Round 1 ----------------------------------
q1	25814	5054	4965	4965
q2	2058	276	179	179
q3	10406	1281	683	683
q4	10222	998	534	534
q5	7534	2378	2330	2330
q6	177	163	132	132
q7	915	734	633	633
q8	9315	1266	1097	1097
q9	6805	5094	5103	5094
q10	6806	2314	1885	1885
q11	470	279	272	272
q12	347	351	224	224
q13	17785	3654	3197	3197
q14	223	244	209	209
q15	537	497	512	497
q16	447	457	405	405
q17	615	868	382	382
q18	7620	7272	7164	7164
q19	1236	946	554	554
q20	331	319	219	219
q21	3996	3346	2458	2458
q22	1031	998	958	958
Total cold run time: 114690 ms
Total hot run time: 34071 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5080	5058	5028	5028
q2	238	330	239	239
q3	2156	2624	2288	2288
q4	1485	1830	1405	1405
q5	4412	4339	4386	4339
q6	215	171	129	129
q7	1951	2110	1728	1728
q8	2583	2588	2512	2512
q9	7253	7208	7086	7086
q10	3003	3123	2727	2727
q11	578	512	499	499
q12	673	770	630	630
q13	3497	3872	3295	3295
q14	301	312	289	289
q15	523	475	478	475
q16	457	504	456	456
q17	1149	1574	1386	1386
q18	7752	7568	7448	7448
q19	827	800	882	800
q20	2018	1964	1837	1837
q21	5277	4880	4742	4742
q22	1076	1067	1008	1008
Total cold run time: 52504 ms
Total hot run time: 50346 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192658 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 76b1ff887dd9a4340d259bb3f63e964430a9f7c6, data reload: false

query1	1406	1075	1072	1072
query2	6303	1891	1844	1844
query3	11137	4690	4633	4633
query4	26419	23339	23501	23339
query5	5644	647	487	487
query6	313	190	185	185
query7	3986	493	275	275
query8	287	236	237	236
query9	8511	2526	2545	2526
query10	542	321	258	258
query11	15178	14970	14850	14850
query12	157	115	106	106
query13	1563	527	397	397
query14	9021	6195	6174	6174
query15	198	195	160	160
query16	7230	612	484	484
query17	1125	783	614	614
query18	2031	405	297	297
query19	197	181	164	164
query20	123	125	117	117
query21	206	130	111	111
query22	4553	4453	4320	4320
query23	34580	33751	33338	33338
query24	8510	2501	2422	2422
query25	520	461	400	400
query26	1199	279	154	154
query27	2861	523	338	338
query28	4599	2144	2100	2100
query29	759	578	439	439
query30	274	215	198	198
query31	859	846	761	761
query32	72	64	62	62
query33	537	352	322	322
query34	801	850	525	525
query35	831	854	804	804
query36	981	1042	879	879
query37	115	103	82	82
query38	4244	4189	4180	4180
query39	1547	1443	1449	1443
query40	218	118	104	104
query41	53	54	60	54
query42	130	106	109	106
query43	509	514	491	491
query44	1283	814	807	807
query45	193	175	172	172
query46	865	1047	667	667
query47	1883	1895	1825	1825
query48	379	415	331	331
query49	761	516	414	414
query50	682	692	403	403
query51	4258	4235	4173	4173
query52	111	110	101	101
query53	229	259	193	193
query54	591	579	539	539
query55	87	90	84	84
query56	308	313	296	296
query57	1190	1190	1157	1157
query58	266	264	255	255
query59	2819	2861	2724	2724
query60	336	327	324	324
query61	157	150	150	150
query62	825	736	697	697
query63	226	188	196	188
query64	4111	1126	775	775
query65	4423	4465	4385	4385
query66	1066	438	321	321
query67	16052	15512	15422	15422
query68	8880	877	515	515
query69	502	294	270	270
query70	1221	1128	1082	1082
query71	472	315	282	282
query72	5370	4656	4879	4656
query73	695	610	351	351
query74	8930	9169	8845	8845
query75	4320	3205	2650	2650
query76	3810	1184	782	782
query77	798	365	295	295
query78	10024	10206	9269	9269
query79	2165	865	558	558
query80	577	497	430	430
query81	465	259	215	215
query82	420	121	96	96
query83	283	255	232	232
query84	293	116	90	90
query85	746	344	302	302
query86	329	299	300	299
query87	4353	4412	4347	4347
query88	3039	2169	2207	2169
query89	406	315	282	282
query90	1912	204	201	201
query91	138	145	105	105
query92	71	57	59	57
query93	1150	950	570	570
query94	640	420	296	296
query95	361	283	287	283
query96	480	552	270	270
query97	3161	3202	3128	3128
query98	230	204	211	204
query99	1450	1402	1273	1273
Total cold run time: 281121 ms
Total hot run time: 192658 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.74 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 76b1ff887dd9a4340d259bb3f63e964430a9f7c6, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.11	0.12
query3	0.26	0.20	0.20
query4	1.60	0.19	0.19
query5	0.60	0.59	0.59
query6	1.19	0.72	0.73
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.53	0.52
query10	0.57	0.58	0.57
query11	0.16	0.10	0.11
query12	0.15	0.11	0.12
query13	0.61	0.59	0.59
query14	1.17	1.21	1.17
query15	0.88	0.84	0.85
query16	0.39	0.37	0.38
query17	1.06	1.03	1.01
query18	0.21	0.20	0.19
query19	1.93	1.82	1.74
query20	0.02	0.01	0.02
query21	15.39	0.91	0.56
query22	0.74	1.06	0.62
query23	15.14	1.38	0.61
query24	7.03	1.18	1.03
query25	0.51	0.26	0.08
query26	0.59	0.15	0.14
query27	0.06	0.05	0.04
query28	10.21	0.88	0.43
query29	12.55	4.07	3.36
query30	0.25	0.09	0.06
query31	2.81	0.58	0.37
query32	3.23	0.53	0.46
query33	3.06	3.03	3.11
query34	15.79	5.09	4.48
query35	4.56	4.55	4.57
query36	0.66	0.50	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.04
Total cold run time: 104.67 s
Total hot run time: 29.74 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 66.67% (20/30) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.55% (14538/27148)
Line Coverage 42.38% (126020/297364)
Region Coverage 41.20% (64419/156354)
Branch Coverage 35.78% (32380/90510)

@hello-stephen
Copy link
Contributor

BE Regression P0 && UT Coverage Report

Increment line coverage 66.67% (20/30) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage
Line Coverage
Region Coverage
Branch Coverage

@dataroaring dataroaring added the usercase Important user case type label label Apr 23, 2025
Copy link
Contributor

@zzzxl1993 zzzxl1993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@hello-stephen
Copy link
Contributor

BE Regression P0 && UT Coverage Report

Increment line coverage 66.67% (20/30) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage
Line Coverage
Region Coverage
Branch Coverage

@csun5285
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34422 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3896a4152ba999a6323fb161ec6efc3b70cfb317, data reload: false

------ Round 1 ----------------------------------
q1	26182	4994	4963	4963
q2	2082	281	183	183
q3	10380	1220	704	704
q4	10227	988	525	525
q5	7521	2330	2341	2330
q6	179	160	131	131
q7	884	751	598	598
q8	9301	1252	1047	1047
q9	6955	5087	5054	5054
q10	6797	2284	1888	1888
q11	474	287	256	256
q12	352	342	214	214
q13	17749	3660	3077	3077
q14	238	219	204	204
q15	537	498	501	498
q16	435	435	410	410
q17	580	839	350	350
q18	7578	7115	7156	7115
q19	1435	947	548	548
q20	331	324	225	225
q21	3799	3343	3162	3162
q22	1047	1014	940	940
Total cold run time: 115063 ms
Total hot run time: 34422 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5069	5035	5025	5025
q2	249	333	232	232
q3	2144	2597	2293	2293
q4	1387	1752	1406	1406
q5	4443	4393	4346	4346
q6	214	173	127	127
q7	1979	1930	1755	1755
q8	2561	2501	2497	2497
q9	7258	7140	7131	7131
q10	2943	3181	2729	2729
q11	593	503	481	481
q12	685	781	618	618
q13	3461	3906	3309	3309
q14	296	307	275	275
q15	528	492	516	492
q16	446	499	464	464
q17	1158	1585	1345	1345
q18	7679	7594	7395	7395
q19	795	775	809	775
q20	1970	1963	1832	1832
q21	5290	4890	4897	4890
q22	1096	1096	1015	1015
Total cold run time: 52244 ms
Total hot run time: 50432 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191444 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3896a4152ba999a6323fb161ec6efc3b70cfb317, data reload: false

query1	1381	1106	1090	1090
query2	6330	1810	1777	1777
query3	11004	4360	4171	4171
query4	55887	24640	23073	23073
query5	5155	472	466	466
query6	387	203	196	196
query7	5071	492	287	287
query8	321	237	220	220
query9	6627	2532	2548	2532
query10	402	314	249	249
query11	15028	15005	14762	14762
query12	159	113	107	107
query13	1150	531	412	412
query14	10020	6359	6364	6359
query15	199	208	177	177
query16	7025	663	523	523
query17	1083	758	622	622
query18	1531	446	324	324
query19	198	192	173	173
query20	131	130	125	125
query21	213	127	111	111
query22	4366	4458	4252	4252
query23	34031	33645	33686	33645
query24	7006	2415	2376	2376
query25	482	456	406	406
query26	658	279	152	152
query27	2176	498	331	331
query28	3105	2160	2147	2147
query29	569	584	432	432
query30	272	218	189	189
query31	883	875	764	764
query32	70	66	66	66
query33	479	355	310	310
query34	753	857	536	536
query35	780	837	741	741
query36	957	986	896	896
query37	116	96	74	74
query38	4214	4240	4128	4128
query39	1481	1457	1467	1457
query40	200	120	105	105
query41	52	53	52	52
query42	124	109	105	105
query43	482	489	477	477
query44	1340	834	801	801
query45	176	178	164	164
query46	830	1020	640	640
query47	1863	1878	1793	1793
query48	379	406	311	311
query49	705	537	432	432
query50	652	694	405	405
query51	4236	4261	4098	4098
query52	108	108	101	101
query53	230	268	187	187
query54	580	565	505	505
query55	80	85	81	81
query56	317	306	294	294
query57	1209	1191	1131	1131
query58	280	251	259	251
query59	2649	2801	2649	2649
query60	332	309	308	308
query61	127	150	124	124
query62	751	770	675	675
query63	227	184	191	184
query64	1452	1028	689	689
query65	4503	4274	4283	4274
query66	735	413	298	298
query67	16082	15530	15189	15189
query68	7722	879	514	514
query69	532	313	259	259
query70	1180	1096	1104	1096
query71	478	307	289	289
query72	5754	4530	4715	4530
query73	1217	631	346	346
query74	8936	9230	8955	8955
query75	3738	3198	2753	2753
query76	4290	1194	753	753
query77	625	361	280	280
query78	9907	10085	9298	9298
query79	3207	803	546	546
query80	674	507	438	438
query81	472	253	217	217
query82	491	122	103	103
query83	363	255	230	230
query84	284	102	85	85
query85	860	349	305	305
query86	413	315	310	310
query87	4417	4379	4342	4342
query88	3223	2196	2207	2196
query89	408	329	281	281
query90	1983	205	208	205
query91	136	141	109	109
query92	71	55	56	55
query93	1599	931	576	576
query94	684	438	294	294
query95	369	288	277	277
query96	480	562	276	276
query97	3169	3219	3090	3090
query98	231	196	204	196
query99	1422	1376	1259	1259
Total cold run time: 301736 ms
Total hot run time: 191444 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.65 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3896a4152ba999a6323fb161ec6efc3b70cfb317, data reload: false

query1	0.04	0.03	0.03
query2	0.13	0.11	0.12
query3	0.25	0.20	0.20
query4	1.59	0.20	0.20
query5	0.58	0.58	0.58
query6	1.19	0.71	0.72
query7	0.03	0.01	0.01
query8	0.04	0.03	0.04
query9	0.57	0.54	0.51
query10	0.57	0.56	0.56
query11	0.15	0.11	0.10
query12	0.14	0.12	0.11
query13	0.60	0.60	0.60
query14	1.15	1.17	1.17
query15	0.87	0.85	0.86
query16	0.38	0.39	0.40
query17	1.08	1.02	1.02
query18	0.21	0.20	0.20
query19	1.93	1.77	1.83
query20	0.01	0.01	0.01
query21	15.39	0.93	0.56
query22	0.74	1.06	0.80
query23	14.86	1.37	0.68
query24	7.51	1.02	0.62
query25	0.48	0.21	0.07
query26	0.66	0.16	0.13
query27	0.05	0.05	0.05
query28	9.10	0.85	0.44
query29	12.54	3.94	3.31
query30	0.25	0.09	0.06
query31	2.84	0.58	0.37
query32	3.22	0.54	0.46
query33	3.09	3.07	3.08
query34	16.01	5.12	4.54
query35	4.50	4.53	4.53
query36	0.66	0.49	0.48
query37	0.08	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.03	0.03
query40	0.16	0.13	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 103.86 s
Total hot run time: 29.65 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 67.74% (21/31) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.54% (14537/27150)
Line Coverage 42.37% (126033/297456)
Region Coverage 41.18% (64409/156409)
Branch Coverage 35.76% (32377/90540)

@hello-stephen
Copy link
Contributor

BE Regression P0 && UT Coverage Report

Increment line coverage 67.74% (21/31) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage
Line Coverage
Region Coverage
Branch Coverage

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 24, 2025
@dataroaring dataroaring merged commit 5da1025 into apache:master Apr 24, 2025
23 of 24 checks passed
csun5285 added a commit to csun5285/doris that referenced this pull request Apr 24, 2025
csun5285 added a commit to csun5285/doris that referenced this pull request Apr 24, 2025
…0306)

fix time series compaction time threshold

fix pick

fix build
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.6-merged p0_b reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants