[tool](filecache) add debugging facilities for lru dump #54412

freemandealer · 2025-08-06T13:19:05Z

lru tools for examing file cache
lru dump file backup
debug VLOG
manual dump API

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-08-06T13:19:12Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

freemandealer · 2025-08-06T13:19:20Z

run buildall

freemandealer · 2025-08-06T13:22:21Z

modification on be/src/io/cache/cache_lru_dumper.{h,cpp} are applied on [DO_NOT_MERGE]enhancement Persist LRU information for file cache (pick #49456), URL: #52807 but tools are not.

doris-robot · 2025-08-06T13:58:28Z

TPC-H: Total hot run time: 33649 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b4aac0fbe94e95b748736292d26cf464c7b520d2, data reload: false

------ Round 1 ----------------------------------
q1	17585	5256	5102	5102
q2	1938	284	186	186
q3	10336	1337	700	700
q4	10228	1026	531	531
q5	7502	2337	2299	2299
q6	181	164	132	132
q7	889	757	618	618
q8	9305	1269	1045	1045
q9	6755	5055	5156	5055
q10	7033	2404	2000	2000
q11	461	279	272	272
q12	359	388	242	242
q13	17813	3456	2985	2985
q14	230	231	229	229
q15	533	462	464	462
q16	423	440	393	393
q17	582	838	362	362
q18	7431	6967	6970	6967
q19	1499	989	572	572
q20	333	315	212	212
q21	3590	3014	2302	2302
q22	1073	1032	983	983
Total cold run time: 106079 ms
Total hot run time: 33649 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5373	5224	5369	5224
q2	243	303	215	215
q3	2087	2561	2213	2213
q4	1326	1757	1330	1330
q5	4095	4319	4466	4319
q6	221	181	133	133
q7	2077	1930	1686	1686
q8	2767	2601	2397	2397
q9	7227	7531	7288	7288
q10	3152	3360	2969	2969
q11	565	503	524	503
q12	809	806	620	620
q13	3478	3718	3290	3290
q14	296	293	277	277
q15	498	463	462	462
q16	455	511	455	455
q17	1201	1432	1541	1432
q18	8737	7538	7996	7538
q19	11102	1074	1266	1074
q20	2024	1906	1757	1757
q21	14937	4203	4251	4203
q22	1062	1011	1002	1002
Total cold run time: 73732 ms
Total hot run time: 50387 ms

doris-robot · 2025-08-06T14:09:59Z

TPC-DS: Total hot run time: 160686 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b4aac0fbe94e95b748736292d26cf464c7b520d2, data reload: false

reason	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
============================================
query1	1007	385	404	385
query2	6529	2033	1720	1720
query3	6756	238	228	228
query4	26733	23828	22925	22925
query5	4367	658	514	514
query6	332	246	241	241
query7	4630	519	293	293
query8	297	247	228	228
query9	8628	3017	2957	2957
query10	486	353	303	303
query11	15501	15425	14721	14721
query12	187	145	135	135
query13	1656	570	409	409
query14	9274	5951	5987	5951
query15	218	190	174	174
query16	7941	661	453	453
query17	1617	792	610	610
query18	2099	433	310	310
query19	300	209	195	195
query20	161	142	138	138
query21	217	124	106	106
query22	3851	4099	3813	3813
query23	34374	34249	34326	34249
query24	7661	2442	2400	2400
query25	538	507	444	444
query26	720	299	162	162
query27	2161	480	355	355
query28	3956	2353	2355	2353
query29	708	609	488	488
query30	304	236	190	190
query31	885	816	714	714
query32	93	78	77	77
query33	594	416	369	369
query34	822	843	528	528
query35	784	853	765	765
query36	1030	1067	947	947
query37	131	109	88	88
query38	3978	3971	3905	3905
query39	1456	1384	1397	1384
query40	235	147	140	140
query41	63	61	59	59
query42	142	124	125	124
query43	531	549	493	493
query44	1423	886	919	886
query45	192	186	179	179
query46	951	1087	678	678
query47	1768	1827	1743	1743
query48	416	430	317	317
query49	671	516	415	415
query50	657	701	424	424
query51	4226	4211	4182	4182
query52	121	130	119	119
query53	265	294	218	218
query54	664	646	571	571
query55	95	90	91	90
query56	367	366	357	357
query57	1211	1199	1129	1129
query58	340	329	352	329
query59	2623	2760	2560	2560
query60	412	395	390	390
query61	122	123	118	118
query62	820	740	651	651
query63	252	220	220	220
query64	3081	1084	768	768
query65	4288	4157	4146	4146
query66	908	462	336	336
query67	query68	16865	881	879	879
query69	1158	300	307	300
query70	1485	1132	1143	1132
query71	724	342	352	342
query72	9166	2271	2139	2139
query73	3006	708	357	357
query74	9013	8898	8673	8673
query75	7619	3115	2671	2671
query76	8826	1216	789	789
query77	1173	405	331	331
query78	query79	18036	603	578	578
query80	2039	580	490	490
query81	569	262	227	227
query82	473	159	118	118
query83	360	285	279	279
query84	301	105	86	86
query85	1770	394	333	333
query86	361	313	308	308
query87	4244	4183	4085	4085
query88	5743	2268	2284	2268
query89	544	375	324	324
query90	2595	243	244	243
query91	145	141	114	114
query92	87	75	71	71
query93	6964	994	658	658
query94	1181	390	282	282
query95	423	334	331	331
query96	510	589	287	287
query97	2687	2776	2597	2597
query98	253	231	247	231
query99	1487	1339	1294	1294
Total cold run time: 299633 ms
Total hot run time: 160686 ms

doris-robot · 2025-08-06T14:15:28Z

ClickBench: Total hot run time: 32.84 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b4aac0fbe94e95b748736292d26cf464c7b520d2, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.24	0.07	0.06
query4	1.63	0.11	0.11
query5	0.42	0.42	0.42
query6	1.17	0.65	0.66
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.56	0.48	0.47
query10	0.53	0.52	0.53
query11	0.15	0.11	0.10
query12	0.14	0.10	0.10
query13	0.64	0.64	0.65
query14	0.96	1.04	1.25
query15	0.93	0.88	0.90
query16	0.39	0.39	0.40
query17	1.11	1.10	1.05
query18	0.22	0.20	0.20
query19	1.97	1.96	1.98
query20	0.02	0.01	0.02
query21	15.36	0.87	0.55
query22	0.79	1.20	0.79
query23	14.78	1.16	0.64
query24	6.25	2.31	0.46
query25	0.47	0.19	0.08
query26	0.69	0.16	0.14
query27	0.06	0.05	0.05
query28	9.66	0.83	0.44
query29	12.81	3.83	3.29
query30	3.08	2.99	2.96
query31	2.82	0.56	0.40
query32	3.24	0.56	0.50
query33	3.05	3.10	3.17
query34	16.13	5.33	4.90
query35	4.91	4.97	4.95
query36	0.70	0.51	0.50
query37	0.10	0.08	0.07
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.18	0.14	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 106.59 s
Total hot run time: 32.84 s

freemandealer · 2025-08-07T07:54:57Z

run buildall

doris-robot · 2025-08-07T09:03:28Z

TPC-H: Total hot run time: 34003 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0ec19626252fb1e06965a2263d0089a4704d760a, data reload: false

------ Round 1 ----------------------------------
q1	17579	5615	5225	5225
q2	1931	288	186	186
q3	10303	1349	718	718
q4	10229	993	536	536
q5	7490	2196	2377	2196
q6	178	165	128	128
q7	887	808	610	610
q8	9308	1319	1168	1168
q9	6911	5171	5117	5117
q10	6896	2376	1982	1982
q11	458	296	267	267
q12	357	367	218	218
q13	17750	3502	3022	3022
q14	241	231	231	231
q15	528	472	466	466
q16	426	432	375	375
q17	594	841	351	351
q18	7230	7158	7058	7058
q19	1098	989	569	569
q20	337	317	227	227
q21	2816	3073	2338	2338
q22	1051	1033	1015	1015
Total cold run time: 104598 ms
Total hot run time: 34003 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5345	5311	5238	5238
q2	231	304	223	223
q3	2067	2588	2196	2196
q4	1328	1726	1316	1316
q5	4138	4337	4569	4337
q6	217	176	134	134
q7	1947	1980	1853	1853
q8	2575	2541	2469	2469
q9	7355	7373	7336	7336
q10	3234	3376	2916	2916
q11	535	503	490	490
q12	726	837	621	621
q13	3466	4160	3202	3202
q14	312	303	299	299
q15	511	470	454	454
q16	460	529	466	466
q17	1171	1554	1427	1427
q18	14330	7847	7832	7832
q19	12287	989	1082	989
q20	2101	2028	1846	1846
q21	14855	4397	4348	4348
q22	1025	1027	966	966
Total cold run time: 80216 ms
Total hot run time: 50958 ms

doris-robot · 2025-08-07T09:15:21Z

TPC-DS: Total hot run time: 170659 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0ec19626252fb1e06965a2263d0089a4704d760a, data reload: false

============================================
query1	1000	391	430	391
query2	6514	1797	1697	1697
query3	6737	232	222	222
query4	27091	23268	23036	23036
query5	4358	638	535	535
query6	354	235	236	235
query7	4638	541	303	303
query8	278	261	232	232
query9	8581	2979	3006	2979
query10	456	342	294	294
query11	16113	14938	14871	14871
query12	198	137	134	134
query13	1666	553	429	429
query14	8533	5848	5924	5848
query15	219	198	172	172
query16	7791	671	493	493
query17	1631	805	658	658
query18	2073	449	334	334
query19	297	218	187	187
query20	152	148	139	139
query21	214	127	116	116
query22	3973	3927	3927	3927
query23	34574	34635	34305	34305
query24	8106	2375	2481	2375
query25	559	511	439	439
query26	950	298	167	167
query27	2624	505	364	364
query28	4226	2362	2340	2340
query29	697	613	489	489
query30	306	225	201	201
query31	899	827	725	725
query32	90	82	85	82
query33	569	432	390	390
query34	820	860	521	521
query35	819	851	753	753
query36	1032	1041	898	898
query37	139	112	99	99
query38	3957	3974	3981	3974
query39	1466	1379	1380	1379
query40	248	143	134	134
query41	61	58	58	58
query42	139	122	129	122
query43	537	529	496	496
query44	1424	892	875	875
query45	205	190	190	190
query46	959	1069	703	703
query47	1791	1822	1783	1783
query48	409	440	324	324
query49	703	500	433	433
query50	677	709	427	427
query51	4183	4240	4133	4133
query52	136	139	122	122
query53	268	292	217	217
query54	659	659	565	565
query55	93	94	90	90
query56	360	359	358	358
query57	1183	1209	1101	1101
query58	350	361	337	337
query59	2602	2642	2571	2571
query60	434	413	396	396
query61	125	126	120	120
query62	811	752	651	651
query63	257	211	218	211
query64	3525	1096	757	757
query65	4260	4162	4125	4125
query66	952	455	330	330
query67	query68	18350	933	996	933
query69	1016	291	295	291
query70	1360	1125	1121	1121
query71	679	325	324	324
query72	9074	2309	2253	2253
query73	4096	675	358	358
query74	9018	8942	8862	8862
query75	7307	3171	2694	2694
query76	8514	1254	814	814
query77	1162	419	345	345
query78	9777	11116	9125	9125
query79	15688	700	590	590
query80	1709	582	470	470
query81	557	268	235	235
query82	462	158	125	125
query83	375	297	271	271
query84	302	97	80	80
query85	1091	379	348	348
query86	366	337	298	298
query87	4352	4164	4109	4109
query88	5523	2307	2239	2239
query89	502	378	321	321
query90	2566	235	247	235
query91	146	150	112	112
query92	89	72	68	68
query93	6725	977	656	656
query94	1103	400	282	282
query95	428	405	326	326
query96	504	600	280	280
query97	2676	2726	2615	2615
query98	254	242	224	224
query99	1471	1409	1293	1293
Total cold run time: 309293 ms
Total hot run time: 170659 ms

doris-robot · 2025-08-07T09:20:59Z

ClickBench: Total hot run time: 33.23 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0ec19626252fb1e06965a2263d0089a4704d760a, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.05	0.04
query3	0.24	0.07	0.07
query4	1.62	0.11	0.11
query5	0.41	0.44	0.44
query6	1.18	0.67	0.68
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.57	0.47	0.48
query10	0.52	0.53	0.54
query11	0.16	0.11	0.11
query12	0.15	0.11	0.11
query13	0.65	0.66	0.65
query14	0.98	1.20	1.00
query15	0.93	0.89	0.89
query16	0.38	0.40	0.40
query17	1.06	1.06	1.08
query18	0.22	0.20	0.20
query19	1.92	1.87	1.94
query20	0.02	0.02	0.01
query21	15.43	0.81	0.55
query22	0.77	1.19	0.80
query23	14.79	1.18	0.59
query24	6.66	0.88	0.64
query25	0.52	0.22	0.06
query26	0.53	0.15	0.13
query27	0.06	0.06	0.05
query28	9.96	0.88	0.45
query29	12.60	3.88	3.38
query30	3.05	2.97	2.93
query31	2.82	0.57	0.39
query32	3.26	0.57	0.49
query33	3.08	3.21	3.22
query34	16.26	5.36	4.98
query35	4.92	5.01	5.03
query36	0.70	0.51	0.50
query37	0.10	0.07	0.07
query38	0.07	0.04	0.05
query39	0.04	0.03	0.03
query40	0.17	0.14	0.12
query41	0.08	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 107.16 s
Total hot run time: 33.23 s

freemandealer · 2025-08-07T10:21:28Z

depend on: #54456

freemandealer · 2025-08-07T14:25:47Z

run buildall

doris-robot · 2025-08-07T15:08:27Z

TPC-H: Total hot run time: 33870 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 03d9f8601c0b8e08129c1cc6f6e7dd9555a5ef03, data reload: false

------ Round 1 ----------------------------------
q1	17602	5266	5409	5266
q2	1933	303	184	184
q3	10293	1412	752	752
q4	10225	1022	512	512
q5	7518	2296	2384	2296
q6	186	169	133	133
q7	878	784	634	634
q8	9322	1292	1052	1052
q9	6830	5029	5162	5029
q10	6939	2363	1975	1975
q11	487	282	271	271
q12	353	366	235	235
q13	17770	3448	2979	2979
q14	244	256	217	217
q15	533	474	465	465
q16	420	414	370	370
q17	574	837	352	352
q18	7284	7097	7091	7091
q19	1525	994	535	535
q20	324	311	207	207
q21	3459	2490	2283	2283
q22	1113	1057	1032	1032
Total cold run time: 105812 ms
Total hot run time: 33870 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5481	5422	5407	5407
q2	244	318	216	216
q3	2085	2577	2184	2184
q4	1311	1747	1320	1320
q5	4141	4432	4525	4432
q6	221	178	135	135
q7	1979	1917	1786	1786
q8	2528	2694	2492	2492
q9	7462	7391	7260	7260
q10	3207	3368	2952	2952
q11	562	502	494	494
q12	714	768	712	712
q13	3696	3732	3450	3450
q14	277	315	299	299
q15	507	459	473	459
q16	472	511	462	462
q17	1191	1517	1444	1444
q18	10321	7771	8240	7771
q19	12396	886	894	886
q20	3676	2055	1856	1856
q21	14887	4354	4268	4268
q22	1145	1051	998	998
Total cold run time: 78503 ms
Total hot run time: 51283 ms

dataroaring

LGTM

github-actions · 2025-08-07T15:17:29Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-08-07T15:17:31Z

PR approved by anyone and no changes requested.

doris-robot · 2025-08-07T15:20:28Z

TPC-DS: Total hot run time: 161629 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 03d9f8601c0b8e08129c1cc6f6e7dd9555a5ef03, data reload: false

reason	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
============================================
query1	1006	413	417	413
query2	6530	1884	1708	1708
query3	6751	222	221	221
query4	26879	23854	23399	23399
query5	4373	643	541	541
query6	331	246	240	240
query7	4638	531	291	291
query8	294	247	240	240
query9	8611	2947	2972	2947
query10	495	366	296	296
query11	16038	15130	14807	14807
query12	192	139	134	134
query13	1665	554	405	405
query14	8711	5972	5910	5910
query15	211	191	179	179
query16	7128	674	478	478
query17	972	799	642	642
query18	2042	445	328	328
query19	233	218	183	183
query20	157	142	140	140
query21	224	130	114	114
query22	3928	3984	3813	3813
query23	34879	34641	34682	34641
query24	8121	2427	2441	2427
query25	563	512	456	456
query26	1121	307	164	164
query27	2697	512	352	352
query28	4347	2333	2317	2317
query29	820	618	503	503
query30	299	233	192	192
query31	874	829	700	700
query32	96	79	83	79
query33	562	430	377	377
query34	833	859	527	527
query35	830	833	767	767
query36	1025	1033	961	961
query37	135	119	95	95
query38	3980	3937	3936	3936
query39	1446	1399	1373	1373
query40	251	149	135	135
query41	66	61	57	57
query42	140	129	129	129
query43	525	537	497	497
query44	1408	883	875	875
query45	197	193	187	187
query46	943	1067	677	677
query47	1792	1840	1721	1721
query48	396	427	307	307
query49	733	511	418	418
query50	711	700	403	403
query51	4223	4203	4184	4184
query52	123	135	122	122
query53	264	293	219	219
query54	658	657	570	570
query55	91	92	88	88
query56	361	353	379	353
query57	1192	1229	1156	1156
query58	345	348	338	338
query59	2666	2618	2485	2485
query60	427	412	386	386
query61	129	161	129	129
query62	807	733	677	677
query63	261	213	225	213
query64	4416	1112	786	786
query65	4285	4129	4196	4129
query66	1166	445	331	331
query67	query68	19368	1047	636	636
query69	994	324	295	295
query70	1438	1107	1093	1093
query71	665	347	323	323
query72	9073	2345	4814	2345
query73	4692	781	361	361
query74	8997	8824	8581	8581
query75	6996	3149	2674	2674
query76	8325	1226	833	833
query77	1162	428	336	336
query78	9701	11378	query79	16294	633	599	599
query80	3688	614	493	493
query81	544	260	241	241
query82	534	157	119	119
query83	380	291	272	272
query84	299	97	85	85
query85	971	405	347	347
query86	356	314	299	299
query87	4301	4206	4179	4179
query88	5461	2266	2247	2247
query89	499	375	319	319
query90	2449	234	238	234
query91	152	156	113	113
query92	123	75	72	72
query93	7189	1034	656	656
query94	1422	421	294	294
query95	437	327	325	325
query96	513	602	284	284
query97	2738	2730	2596	2596
query98	251	236	223	223
query99	1470	1440	1287	1287
Total cold run time: 304394 ms
Total hot run time: 161629 ms

doris-robot · 2025-08-07T15:25:56Z

ClickBench: Total hot run time: 32.9 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 03d9f8601c0b8e08129c1cc6f6e7dd9555a5ef03, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.04
query3	0.24	0.06	0.07
query4	1.64	0.11	0.11
query5	0.41	0.42	0.45
query6	1.16	0.66	0.68
query7	0.02	0.01	0.02
query8	0.05	0.03	0.04
query9	0.56	0.50	0.47
query10	0.53	0.54	0.53
query11	0.16	0.10	0.10
query12	0.14	0.11	0.11
query13	0.67	0.66	0.63
query14	0.94	1.05	1.16
query15	0.92	0.87	0.91
query16	0.40	0.38	0.39
query17	1.08	1.05	1.08
query18	0.22	0.20	0.20
query19	2.05	1.90	1.88
query20	0.01	0.02	0.01
query21	15.37	0.87	0.55
query22	0.78	1.17	0.70
query23	14.85	1.18	0.63
query24	6.75	1.42	0.51
query25	0.50	0.28	0.13
query26	0.60	0.16	0.12
query27	0.06	0.06	0.05
query28	10.17	0.87	0.46
query29	12.67	3.78	3.28
query30	3.02	2.99	2.97
query31	2.82	0.57	0.40
query32	3.24	0.57	0.50
query33	3.05	3.17	3.20
query34	15.98	5.36	4.98
query35	4.89	4.94	4.97
query36	0.69	0.52	0.50
query37	0.10	0.07	0.08
query38	0.06	0.05	0.04
query39	0.03	0.02	0.03
query40	0.16	0.14	0.14
query41	0.08	0.03	0.03
query42	0.04	0.02	0.03
query43	0.04	0.04	0.04
Total cold run time: 107.26 s
Total hot run time: 32.9 s

freemandealer · 2025-08-13T07:56:32Z

run buildall

doris-robot · 2025-08-13T09:15:05Z

TPC-H: Total hot run time: 34002 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b367dd9c47918370fb46859986cbf1de2ab0ddd5, data reload: false

------ Round 1 ----------------------------------
q1	17611	5312	5136	5136
q2	1933	288	176	176
q3	10313	1319	723	723
q4	10216	1007	514	514
q5	7511	2455	2332	2332
q6	197	166	133	133
q7	907	744	599	599
q8	9311	1290	1117	1117
q9	7169	5234	5126	5126
q10	6973	2374	1963	1963
q11	538	295	273	273
q12	363	352	217	217
q13	17783	3646	3048	3048
q14	229	237	209	209
q15	555	483	487	483
q16	432	435	372	372
q17	605	862	350	350
q18	7601	7063	7088	7063
q19	1178	971	568	568
q20	357	346	220	220
q21	3735	3210	2406	2406
q22	1071	1024	974	974
Total cold run time: 106588 ms
Total hot run time: 34002 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5258	5145	5114	5114
q2	245	324	221	221
q3	2188	2668	2356	2356
q4	1404	1823	1366	1366
q5	4225	4676	4579	4579
q6	211	170	126	126
q7	2076	1946	1800	1800
q8	2653	2739	2604	2604
q9	7260	7317	7339	7317
q10	3102	3294	2859	2859
q11	595	516	503	503
q12	704	776	627	627
q13	3511	3939	3367	3367
q14	300	298	292	292
q15	533	495	497	495
q16	452	483	459	459
q17	1196	1619	1431	1431
q18	7974	7571	7461	7461
q19	843	896	978	896
q20	2009	2107	1843	1843
q21	4828	4278	4318	4278
q22	1068	1052	1024	1024
Total cold run time: 52635 ms
Total hot run time: 51018 ms

doris-robot · 2025-08-13T09:26:39Z

TPC-DS: Total hot run time: 185265 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b367dd9c47918370fb46859986cbf1de2ab0ddd5, data reload: false

query1	994	390	416	390
query2	6544	1755	1800	1755
query3	6751	223	221	221
query4	26456	23147	23392	23147
query5	4391	624	514	514
query6	312	210	213	210
query7	4625	497	293	293
query8	268	223	207	207
query9	8547	2879	2854	2854
query10	487	353	277	277
query11	15815	15070	14860	14860
query12	160	120	116	116
query13	1665	544	423	423
query14	8634	5773	5727	5727
query15	212	201	169	169
query16	7302	647	512	512
query17	1224	718	604	604
query18	1987	423	318	318
query19	197	208	167	167
query20	126	121	115	115
query21	212	135	110	110
query22	4244	4355	4319	4319
query23	34047	33362	33171	33171
query24	8145	2353	2376	2353
query25	546	463	415	415
query26	1249	268	156	156
query27	2731	510	351	351
query28	4457	2265	2246	2246
query29	765	556	447	447
query30	286	224	188	188
query31	903	806	701	701
query32	80	73	75	73
query33	566	367	369	367
query34	789	852	500	500
query35	809	830	726	726
query36	958	1013	899	899
query37	123	107	89	89
query38	4012	3971	3912	3912
query39	1457	1419	1405	1405
query40	214	129	114	114
query41	60	56	54	54
query42	120	108	125	108
query43	477	495	483	483
query44	1321	853	838	838
query45	178	170	163	163
query46	862	1012	641	641
query47	1749	1841	1736	1736
query48	382	415	311	311
query49	711	524	399	399
query50	639	694	404	404
query51	4055	4107	4173	4107
query52	115	118	102	102
query53	239	256	196	196
query54	593	585	520	520
query55	86	81	85	81
query56	330	306	305	305
query57	1213	1191	1118	1118
query58	277	275	268	268
query59	2693	2784	2686	2686
query60	344	336	351	336
query61	126	125	121	121
query62	801	722	669	669
query63	224	193	184	184
query64	4375	1049	714	714
query65	4324	4225	4176	4176
query66	1163	424	336	336
query67	15654	15295	15095	15095
query68	8313	905	582	582
query69	488	333	281	281
query70	1251	1167	1151	1151
query71	494	333	321	321
query72	5576	4743	4864	4743
query73	723	625	351	351
query74	9277	9066	8682	8682
query75	3932	3072	2618	2618
query76	3746	1155	750	750
query77	800	397	324	324
query78	9639	9694	8876	8876
query79	2196	831	599	599
query80	606	541	462	462
query81	474	256	216	216
query82	424	136	111	111
query83	258	251	237	237
query84	250	105	89	89
query85	790	375	339	339
query86	341	330	309	309
query87	4299	4311	4228	4228
query88	3398	2183	2172	2172
query89	397	319	280	280
query90	1928	218	213	213
query91	139	139	113	113
query92	87	72	68	68
query93	1343	1048	643	643
query94	681	413	292	292
query95	395	323	306	306
query96	478	575	272	272
query97	2649	2659	2554	2554
query98	250	232	221	221
query99	1423	1418	1290	1290
Total cold run time: 273446 ms
Total hot run time: 185265 ms

doris-robot · 2025-08-13T09:32:07Z

ClickBench: Total hot run time: 32.39 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b367dd9c47918370fb46859986cbf1de2ab0ddd5, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.24	0.07	0.07
query4	1.62	0.10	0.10
query5	0.44	0.42	0.40
query6	1.18	0.64	0.66
query7	0.03	0.02	0.02
query8	0.05	0.04	0.04
query9	0.60	0.53	0.52
query10	0.57	0.56	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.11
query13	0.63	0.61	0.61
query14	0.79	0.82	0.84
query15	0.90	0.86	0.84
query16	0.38	0.39	0.40
query17	1.01	1.04	1.03
query18	0.21	0.20	0.19
query19	1.89	1.79	1.79
query20	0.01	0.01	0.01
query21	15.42	0.95	0.55
query22	0.77	1.22	0.74
query23	14.80	1.38	0.65
query24	6.84	1.98	0.54
query25	0.47	0.11	0.09
query26	0.58	0.16	0.12
query27	0.06	0.05	0.05
query28	10.01	0.93	0.44
query29	12.55	3.93	3.28
query30	3.06	3.01	2.91
query31	2.82	0.60	0.39
query32	3.26	0.55	0.48
query33	3.13	3.17	3.14
query34	16.12	5.42	4.94
query35	4.88	4.94	4.92
query36	0.69	0.51	0.49
query37	0.10	0.07	0.07
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.15	0.14
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.97 s
Total hot run time: 32.39 s

hello-stephen · 2025-09-02T07:59:37Z

BE Regression && UT Coverage Report

Increment line coverage 55.84% (43/77) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	70.29% (22899/32580)
Line Coverage	56.60% (238179/420814)
Region Coverage	51.97% (197867/380707)
Branch Coverage	53.67% (85375/159063)

dataroaring

LGTM

github-actions · 2025-09-02T11:20:06Z

PR approved by at least one committer and no changes requested.

hello-stephen · 2025-09-02T11:24:38Z

BE Regression && UT Coverage Report

Increment line coverage 55.84% (43/77) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	70.29% (22899/32580)
Line Coverage	56.60% (238179/420814)
Region Coverage	51.97% (197867/380707)
Branch Coverage	53.67% (85375/159063)

freemandealer · 2025-09-12T02:07:42Z

run buildall

doris-robot · 2025-09-12T03:39:02Z

TPC-H: Total hot run time: 34923 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e57cb7525b0a56d802065aa448f2dafac64bf343, data reload: false

------ Round 1 ----------------------------------
q1	17615	5242	5128	5128
q2	1989	335	211	211
q3	10236	1302	721	721
q4	10230	1057	523	523
q5	7524	2440	2344	2344
q6	182	170	138	138
q7	911	762	628	628
q8	9375	1341	1064	1064
q9	7043	5231	5285	5231
q10	6975	2408	1970	1970
q11	491	325	289	289
q12	375	369	226	226
q13	17792	3651	3017	3017
q14	242	248	215	215
q15	566	498	488	488
q16	988	999	970	970
q17	599	873	376	376
q18	7674	7173	7214	7173
q19	1096	959	549	549
q20	356	342	241	241
q21	3933	3102	2401	2401
q22	1081	1062	1020	1020
Total cold run time: 107273 ms
Total hot run time: 34923 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5093	5074	5064	5064
q2	247	337	232	232
q3	2161	2716	2318	2318
q4	1372	1803	1386	1386
q5	4263	4595	4503	4503
q6	227	170	135	135
q7	2080	1996	1849	1849
q8	2640	2526	2646	2526
q9	7425	7321	7405	7321
q10	3151	3285	2896	2896
q11	590	525	497	497
q12	720	792	619	619
q13	3848	3830	3232	3232
q14	305	315	279	279
q15	530	510	482	482
q16	1109	1123	1061	1061
q17	1230	1544	1440	1440
q18	7894	8019	7636	7636
q19	837	797	824	797
q20	1950	1951	1826	1826
q21	4824	4373	4284	4284
q22	1072	1030	1021	1021
Total cold run time: 53568 ms
Total hot run time: 51404 ms

doris-robot · 2025-09-12T03:50:38Z

TPC-DS: Total hot run time: 189055 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e57cb7525b0a56d802065aa448f2dafac64bf343, data reload: false

query1	1083	429	435	429
query2	6554	1676	1719	1676
query3	6751	227	229	227
query4	26258	23105	22949	22949
query5	4408	641	523	523
query6	342	261	244	244
query7	4655	519	306	306
query8	330	276	272	272
query9	8644	2930	2945	2930
query10	478	361	297	297
query11	15424	15013	14846	14846
query12	177	128	122	122
query13	1677	582	425	425
query14	10169	9146	9194	9146
query15	211	188	181	181
query16	7309	721	472	472
query17	1230	740	626	626
query18	2012	439	376	376
query19	204	197	170	170
query20	131	130	124	124
query21	224	132	118	118
query22	4034	4307	4088	4088
query23	33574	33242	32908	32908
query24	8156	2369	2416	2369
query25	596	538	482	482
query26	1252	276	171	171
query27	2727	525	374	374
query28	4485	2271	2250	2250
query29	848	625	506	506
query30	299	237	202	202
query31	910	801	727	727
query32	90	88	86	86
query33	595	433	353	353
query34	799	858	516	516
query35	829	837	772	772
query36	996	1026	928	928
query37	126	119	94	94
query38	3499	3517	3448	3448
query39	1516	1476	1448	1448
query40	233	137	133	133
query41	63	63	64	63
query42	140	125	122	122
query43	516	525	479	479
query44	1372	877	877	877
query45	191	176	170	170
query46	863	1020	654	654
query47	1788	1819	1743	1743
query48	394	418	319	319
query49	749	509	413	413
query50	653	695	418	418
query51	4040	3921	3884	3884
query52	120	117	109	109
query53	249	273	205	205
query54	627	610	564	564
query55	105	99	98	98
query56	349	367	344	344
query57	1221	1191	1123	1123
query58	300	292	306	292
query59	2601	2711	2550	2550
query60	378	358	357	357
query61	165	159	162	159
query62	808	759	672	672
query63	233	204	196	196
query64	4540	1161	842	842
query65	4088	3967	4036	3967
query66	1168	436	356	356
query67	15898	15105	15456	15105
query68	6163	956	591	591
query69	524	335	295	295
query70	1395	1218	1268	1218
query71	552	348	332	332
query72	5488	5051	4950	4950
query73	639	593	360	360
query74	9040	9169	8877	8877
query75	3338	3225	2762	2762
query76	3146	1173	779	779
query77	508	414	342	342
query78	9489	9770	8938	8938
query79	1517	813	604	604
query80	716	597	513	513
query81	491	251	219	219
query82	240	169	140	140
query83	283	273	264	264
query84	272	121	106	106
query85	979	471	448	448
query86	338	328	323	323
query87	3717	3745	3614	3614
query88	2791	2195	2210	2195
query89	475	339	295	295
query90	1771	237	223	223
query91	166	179	135	135
query92	82	78	75	75
query93	1126	975	660	660
query94	656	416	315	315
query95	412	332	329	329
query96	485	597	281	281
query97	2883	2971	2895	2895
query98	243	218	218	218
query99	1325	1419	1364	1364
Total cold run time: 268704 ms
Total hot run time: 189055 ms

doris-robot · 2025-09-12T03:56:06Z

ClickBench: Total hot run time: 29.76 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e57cb7525b0a56d802065aa448f2dafac64bf343, data reload: false

query1	0.06	0.04	0.04
query2	0.09	0.06	0.05
query3	0.25	0.09	0.08
query4	1.61	0.12	0.11
query5	0.28	0.26	0.26
query6	1.17	0.65	0.64
query7	0.03	0.03	0.03
query8	0.06	0.04	0.04
query9	0.64	0.53	0.52
query10	0.58	0.59	0.59
query11	0.17	0.12	0.11
query12	0.15	0.12	0.12
query13	0.62	0.62	0.63
query14	1.01	1.03	1.03
query15	0.86	0.86	0.86
query16	0.41	0.40	0.40
query17	1.05	1.07	1.08
query18	0.23	0.20	0.20
query19	1.98	1.89	1.84
query20	0.02	0.01	0.01
query21	15.41	1.12	0.58
query22	0.76	1.25	0.83
query23	14.87	1.39	0.64
query24	7.12	0.80	0.49
query25	0.49	0.24	0.07
query26	0.66	0.17	0.14
query27	0.08	0.05	0.05
query28	9.38	0.94	0.43
query29	12.57	3.91	3.26
query30	0.29	0.13	0.11
query31	2.83	0.60	0.38
query32	3.36	0.59	0.47
query33	3.09	3.07	3.12
query34	16.25	5.45	4.83
query35	4.90	4.91	4.91
query36	0.70	0.53	0.50
query37	0.11	0.07	0.07
query38	0.07	0.05	0.04
query39	0.04	0.03	0.03
query40	0.20	0.15	0.15
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.04
Total cold run time: 104.61 s
Total hot run time: 29.76 s

doris-robot · 2025-09-12T05:12:40Z

BE UT Coverage Report

Increment line coverage 55.84% (43/77) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.21% (17426/33378)
Line Coverage	37.43% (158383/423177)
Region Coverage	32.02% (120868/377505)
Branch Coverage	33.38% (53001/158797)

hello-stephen · 2025-09-12T06:14:40Z

BE Regression && UT Coverage Report

Increment line coverage 55.84% (43/77) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	70.76% (23093/32637)
Line Coverage	57.19% (241439/422163)
Region Coverage	52.60% (201186/382496)
Branch Coverage	54.19% (86438/159498)

freemandealer · 2025-09-15T07:56:07Z

run external

dataroaring

LGTM

github-actions · 2025-09-16T06:58:52Z

PR approved by at least one committer and no changes requested.

1. lru tools for examing file cache 2. lru dump file backup 3. debug VLOG 4. manual dump API Signed-off-by: zhengyu <zhangzhengyu@selectdb.com> Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>

…54412 (#58871) picked from #54412 Signed-off-by: zhengyu <zhangzhengyu@selectdb.com> Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>

freemandealer requested review from dataroaring and gavinchou as code owners August 6, 2025 13:19

freemandealer requested review from CalvinKirs, morningman and w41ter as code owners August 7, 2025 14:22

freemandealer force-pushed the lru-tool branch from dceb010 to 03d9f86 Compare August 7, 2025 14:24

dataroaring previously approved these changes Aug 7, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 7, 2025

github-actions bot added the reviewed label Aug 7, 2025

dataroaring previously approved these changes Sep 2, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 2, 2025

Merge branch 'master' into lru-tool

e57cb75

freemandealer dismissed dataroaring’s stale review via e57cb75 September 12, 2025 02:07

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Sep 12, 2025

dataroaring approved these changes Sep 16, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 16, 2025

gavinchou approved these changes Sep 16, 2025

View reviewed changes

gavinchou added the dev/4.0.x label Sep 16, 2025

dataroaring merged commit 0406ebd into apache:master Sep 16, 2025
28 of 31 checks passed

github-actions bot added the dev/3.1.x-conflict label Sep 16, 2025

yiguolei removed the dev/4.0.x label Sep 29, 2025

morrySnow mentioned this pull request Dec 10, 2025

branch-3.1: [tool](filecache) add debugging facilities for lru dump #54412 #58871

Merged

morrySnow added dev/3.1.4-merged and removed dev/3.1.x dev/3.1.x-conflict labels Dec 10, 2025

[tool](filecache) add debugging facilities for lru dump #54412

[tool](filecache) add debugging facilities for lru dump #54412

Uh oh!

Conversation

freemandealer commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Aug 6, 2025

Uh oh!

freemandealer commented Aug 6, 2025

Uh oh!

freemandealer commented Aug 6, 2025

Uh oh!

doris-robot commented Aug 6, 2025

Uh oh!

doris-robot commented Aug 6, 2025

Uh oh!

doris-robot commented Aug 6, 2025

Uh oh!

freemandealer commented Aug 7, 2025

Uh oh!

doris-robot commented Aug 7, 2025

Uh oh!

doris-robot commented Aug 7, 2025

Uh oh!

doris-robot commented Aug 7, 2025

Uh oh!

freemandealer commented Aug 7, 2025

Uh oh!

freemandealer commented Aug 7, 2025

Uh oh!

doris-robot commented Aug 7, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Aug 7, 2025

Uh oh!

github-actions bot commented Aug 7, 2025

Uh oh!

doris-robot commented Aug 7, 2025

Uh oh!

doris-robot commented Aug 7, 2025

Uh oh!

freemandealer commented Aug 13, 2025

Uh oh!

doris-robot commented Aug 13, 2025

Uh oh!

doris-robot commented Aug 13, 2025

Uh oh!

doris-robot commented Aug 13, 2025

Uh oh!

hello-stephen commented Sep 2, 2025

BE Regression && UT Coverage Report

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Sep 2, 2025

Uh oh!

hello-stephen commented Sep 2, 2025

BE Regression && UT Coverage Report

Uh oh!

freemandealer commented Sep 12, 2025

Uh oh!

doris-robot commented Sep 12, 2025

Uh oh!

doris-robot commented Sep 12, 2025

Uh oh!

doris-robot commented Sep 12, 2025

Uh oh!

doris-robot commented Sep 12, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Sep 12, 2025

BE Regression && UT Coverage Report

freemandealer commented Aug 6, 2025 •

edited

Loading