Skip to content

Conversation

@deardeng
Copy link
Contributor

@deardeng deardeng commented Jan 3, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@deardeng deardeng marked this pull request as draft January 3, 2025 11:44
@dataroaring dataroaring changed the title [Feature](cloud) Support clean tablet file cache when tablet drop [fix](cloud) Support clean tablet file cache when tablet drop Jan 3, 2025
@deardeng deardeng marked this pull request as ready for review January 8, 2025 11:46
@deardeng
Copy link
Contributor Author

deardeng commented Jan 8, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32550 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ecfaf53340a114da632778a02eff56b6a04ca005, data reload: false

------ Round 1 ----------------------------------
q1	17590	6243	6035	6035
q2	2050	299	170	170
q3	10420	1245	708	708
q4	10218	861	424	424
q5	7541	2209	2006	2006
q6	204	186	151	151
q7	899	771	619	619
q8	9239	1376	1183	1183
q9	5240	4907	4869	4869
q10	6761	2301	1833	1833
q11	466	276	269	269
q12	355	366	216	216
q13	17785	3676	2960	2960
q14	238	240	213	213
q15	552	499	488	488
q16	634	629	597	597
q17	570	858	341	341
q18	7068	6430	6428	6428
q19	1223	967	546	546
q20	307	328	191	191
q21	2849	2270	1996	1996
q22	364	323	307	307
Total cold run time: 102573 ms
Total hot run time: 32550 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6198	6201	6225	6201
q2	236	330	240	240
q3	2269	2638	2333	2333
q4	1395	1763	1353	1353
q5	4315	4751	4734	4734
q6	187	183	143	143
q7	2110	2000	1800	1800
q8	2596	2805	2701	2701
q9	7275	7186	7173	7173
q10	3014	3316	2780	2780
q11	581	525	489	489
q12	666	758	600	600
q13	3525	3853	3176	3176
q14	277	295	284	284
q15	568	520	501	501
q16	654	681	641	641
q17	1234	1737	1242	1242
q18	7947	7471	7234	7234
q19	847	1074	1159	1074
q20	1986	2039	1883	1883
q21	5642	5149	5100	5100
q22	628	586	582	582
Total cold run time: 54150 ms
Total hot run time: 52264 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 188592 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ecfaf53340a114da632778a02eff56b6a04ca005, data reload: false

query1	970	382	359	359
query2	6512	2297	2298	2297
query3	6712	213	208	208
query4	33581	23281	23232	23232
query5	4331	607	447	447
query6	293	196	186	186
query7	4651	497	314	314
query8	310	262	238	238
query9	9864	2690	2675	2675
query10	470	308	244	244
query11	18155	15333	14988	14988
query12	157	105	103	103
query13	1671	548	408	408
query14	10322	7522	7329	7329
query15	258	188	187	187
query16	8732	575	435	435
query17	1643	736	569	569
query18	2132	411	306	306
query19	206	182	157	157
query20	120	122	118	118
query21	257	118	102	102
query22	4333	4563	4082	4082
query23	33879	33030	33006	33006
query24	6317	2276	2252	2252
query25	492	441	379	379
query26	1170	268	149	149
query27	2032	446	320	320
query28	5479	2447	2441	2441
query29	757	548	429	429
query30	228	182	145	145
query31	953	848	782	782
query32	89	60	56	56
query33	503	349	289	289
query34	771	809	508	508
query35	783	810	711	711
query36	1023	998	939	939
query37	118	98	77	77
query38	4035	4183	3961	3961
query39	1461	1408	1416	1408
query40	198	112	99	99
query41	47	44	47	44
query42	120	103	114	103
query43	510	521	478	478
query44	1309	819	808	808
query45	172	170	156	156
query46	867	1027	637	637
query47	1881	1843	1771	1771
query48	376	400	328	328
query49	788	474	386	386
query50	619	639	384	384
query51	6841	6931	6892	6892
query52	100	104	87	87
query53	223	245	187	187
query54	470	495	390	390
query55	76	78	79	78
query56	269	265	256	256
query57	1169	1169	1066	1066
query58	246	229	234	229
query59	2952	3149	2859	2859
query60	282	270	277	270
query61	110	111	112	111
query62	812	782	675	675
query63	225	200	194	194
query64	4252	995	636	636
query65	3251	3145	3185	3145
query66	1062	408	315	315
query67	15924	15694	15504	15504
query68	8546	711	522	522
query69	465	295	257	257
query70	1238	1055	1122	1055
query71	430	329	257	257
query72	6526	3841	3847	3841
query73	671	778	363	363
query74	9900	9023	9100	9023
query75	3964	3133	2615	2615
query76	3674	1156	767	767
query77	763	386	266	266
query78	10064	11530	9615	9615
query79	2448	766	613	613
query80	591	543	427	427
query81	481	271	231	231
query82	288	145	124	124
query83	166	169	144	144
query84	235	93	70	70
query85	728	383	328	328
query86	357	330	312	312
query87	4520	4580	4274	4274
query88	4237	2238	2192	2192
query89	397	311	306	306
query90	1931	193	188	188
query91	137	137	105	105
query92	72	59	52	52
query93	970	778	532	532
query94	670	397	302	302
query95	345	266	255	255
query96	477	654	277	277
query97	2799	2946	2797	2797
query98	239	204	195	195
query99	1817	1482	1363	1363
Total cold run time: 291067 ms
Total hot run time: 188592 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.87% (10130/26061)
Line Coverage: 29.92% (85730/286511)
Region Coverage: 29.02% (43732/150720)
Branch Coverage: 25.55% (22322/87354)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ecfaf53340a114da632778a02eff56b6a04ca005_ecfaf53340a114da632778a02eff56b6a04ca005/report/index.html

@doris-robot
Copy link

ClickBench: Total hot run time: 30.72 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ecfaf53340a114da632778a02eff56b6a04ca005, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.04	0.03
query3	0.24	0.07	0.07
query4	1.61	0.11	0.10
query5	0.42	0.42	0.42
query6	1.14	0.67	0.65
query7	0.02	0.01	0.01
query8	0.04	0.04	0.03
query9	0.58	0.51	0.50
query10	0.54	0.56	0.55
query11	0.16	0.10	0.10
query12	0.13	0.11	0.12
query13	0.61	0.62	0.60
query14	2.81	2.85	2.73
query15	0.90	0.82	0.83
query16	0.38	0.37	0.39
query17	0.99	1.05	1.04
query18	0.22	0.21	0.22
query19	1.98	2.02	1.85
query20	0.01	0.01	0.02
query21	15.37	0.94	0.59
query22	0.75	0.86	0.64
query23	15.24	1.43	0.59
query24	2.69	0.71	0.45
query25	0.22	0.12	0.05
query26	0.35	0.14	0.14
query27	0.05	0.06	0.05
query28	12.83	1.58	1.05
query29	12.55	3.85	3.21
query30	0.24	0.09	0.08
query31	2.81	0.58	0.38
query32	3.23	0.53	0.46
query33	3.03	3.19	3.05
query34	16.64	5.14	4.49
query35	4.53	4.47	4.50
query36	0.64	0.49	0.48
query37	0.10	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 104.54 s
Total hot run time: 30.72 s

@deardeng
Copy link
Contributor Author

deardeng commented Jan 9, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32774 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 79d549207889eeb13dc09fcac949c6883368df30, data reload: false

------ Round 1 ----------------------------------
q1	17588	6831	6123	6123
q2	2050	317	176	176
q3	10396	1259	775	775
q4	10264	872	432	432
q5	8503	2190	1969	1969
q6	207	178	147	147
q7	899	764	599	599
q8	9233	1383	1192	1192
q9	5268	4900	4841	4841
q10	6760	2311	1849	1849
q11	484	272	292	272
q12	353	358	230	230
q13	17768	3688	3066	3066
q14	226	247	225	225
q15	558	509	500	500
q16	631	617	581	581
q17	583	860	341	341
q18	6931	6504	6351	6351
q19	1712	1034	601	601
q20	320	313	190	190
q21	2916	2232	2005	2005
q22	360	347	309	309
Total cold run time: 104010 ms
Total hot run time: 32774 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6291	6225	6208	6208
q2	244	326	231	231
q3	2257	2706	2330	2330
q4	1439	1819	1385	1385
q5	4328	4761	4851	4761
q6	195	182	141	141
q7	2058	1963	1800	1800
q8	2610	2880	2726	2726
q9	7327	7261	7313	7261
q10	3128	3294	2783	2783
q11	586	511	489	489
q12	679	766	665	665
q13	3523	3915	3208	3208
q14	294	301	269	269
q15	562	502	510	502
q16	658	691	645	645
q17	1216	1736	1253	1253
q18	7684	7659	7217	7217
q19	888	1226	1115	1115
q20	1978	2001	1878	1878
q21	5798	5216	5146	5146
q22	606	617	610	610
Total cold run time: 54349 ms
Total hot run time: 52623 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.88% (10131/26057)
Line Coverage: 29.93% (85708/286391)
Region Coverage: 29.03% (43732/150667)
Branch Coverage: 25.56% (22319/87312)
Coverage Report: http://coverage.selectdb-in.cc/coverage/79d549207889eeb13dc09fcac949c6883368df30_79d549207889eeb13dc09fcac949c6883368df30/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 195072 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 79d549207889eeb13dc09fcac949c6883368df30, data reload: false

query1	1290	942	930	930
query2	6402	2344	2288	2288
query3	11130	4818	4867	4818
query4	32637	23988	23658	23658
query5	3941	627	466	466
query6	282	195	206	195
query7	3988	494	296	296
query8	310	248	226	226
query9	9412	2662	2645	2645
query10	490	298	253	253
query11	17768	15235	15050	15050
query12	155	102	100	100
query13	1567	512	386	386
query14	9505	7325	7395	7325
query15	244	200	196	196
query16	8376	625	489	489
query17	1581	809	606	606
query18	2139	424	353	353
query19	210	217	173	173
query20	119	115	114	114
query21	211	135	112	112
query22	4573	4581	4346	4346
query23	34135	33345	33277	33277
query24	6466	2277	2314	2277
query25	491	452	397	397
query26	855	277	150	150
query27	2023	479	332	332
query28	5415	2541	2491	2491
query29	671	555	436	436
query30	211	173	145	145
query31	950	885	809	809
query32	92	57	57	57
query33	475	346	298	298
query34	762	849	504	504
query35	808	820	774	774
query36	996	1023	944	944
query37	130	105	74	74
query38	4009	4230	4010	4010
query39	1493	1442	1490	1442
query40	230	117	103	103
query41	47	47	43	43
query42	126	105	100	100
query43	512	536	501	501
query44	1321	824	827	824
query45	177	172	165	165
query46	880	1061	656	656
query47	1894	1918	1821	1821
query48	386	426	338	338
query49	751	474	395	395
query50	637	693	398	398
query51	7097	7123	6999	6999
query52	99	97	91	91
query53	224	262	186	186
query54	476	511	410	410
query55	84	81	77	77
query56	246	264	226	226
query57	1199	1218	1125	1125
query58	264	246	215	215
query59	3154	3276	3283	3276
query60	287	267	280	267
query61	113	114	120	114
query62	854	789	715	715
query63	226	193	189	189
query64	3648	1017	671	671
query65	3344	3223	3254	3223
query66	813	403	300	300
query67	16396	15806	15341	15341
query68	8102	710	513	513
query69	491	291	254	254
query70	1190	1124	1060	1060
query71	428	275	250	250
query72	6392	3800	3896	3800
query73	635	749	348	348
query74	10299	9177	8684	8684
query75	4079	3114	2614	2614
query76	3629	1159	763	763
query77	765	358	279	279
query78	10145	10289	9365	9365
query79	3504	815	596	596
query80	679	516	449	449
query81	487	259	228	228
query82	616	144	114	114
query83	161	161	148	148
query84	237	83	74	74
query85	781	346	359	346
query86	405	302	300	300
query87	4320	4300	4206	4206
query88	4649	2164	2133	2133
query89	409	328	286	286
query90	1785	183	192	183
query91	135	136	102	102
query92	66	54	53	53
query93	2503	864	536	536
query94	641	396	294	294
query95	337	265	256	256
query96	497	626	276	276
query97	2794	2947	2775	2775
query98	215	208	198	198
query99	1366	1481	1378	1378
Total cold run time: 294539 ms
Total hot run time: 195072 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.05 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 79d549207889eeb13dc09fcac949c6883368df30, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.42	0.41	0.42
query6	1.16	0.66	0.64
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.62	0.51	0.50
query10	0.57	0.56	0.54
query11	0.14	0.10	0.10
query12	0.13	0.10	0.10
query13	0.61	0.60	0.59
query14	2.76	2.84	2.72
query15	0.89	0.83	0.82
query16	0.38	0.39	0.39
query17	1.02	1.01	1.06
query18	0.22	0.21	0.21
query19	1.83	1.84	2.00
query20	0.01	0.01	0.02
query21	15.36	0.99	0.57
query22	0.76	0.89	0.97
query23	14.94	1.39	0.61
query24	3.25	1.81	1.32
query25	0.21	0.09	0.07
query26	0.21	0.15	0.13
query27	0.04	0.07	0.05
query28	14.69	1.47	1.05
query29	12.63	3.98	3.28
query30	0.26	0.08	0.06
query31	2.82	0.59	0.38
query32	3.22	0.55	0.46
query33	3.20	3.19	3.22
query34	16.84	5.10	4.56
query35	4.57	4.47	4.52
query36	0.64	0.49	0.49
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.13	0.14
query41	0.07	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.89 s
Total hot run time: 32.05 s

@dataroaring dataroaring added dev/3.0.x usercase Important user case type label labels Jan 9, 2025
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 13, 2025
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit be2932a into apache:master Jan 13, 2025
27 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.4-merged reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants