[opt](filecache) schema change adaptive to file cache #57470

HaolinGuan7211 · 2025-10-29T12:16:06Z

No description provided.

Thearas · 2025-10-29T12:16:11Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

gavinchou · 2025-11-04T06:57:57Z

be/src/cloud/cloud_index_change_compaction.cpp

            DCHECK_EQ(_output_rowset->version(), _input_rowsets[0]->version());
            // MUST NOT move input rowset to stale path.
-            cloud_tablet()->add_rowsets({_output_rowset}, true, wrlock);
+            cloud_tablet()->add_rowsets({_output_rowset}, true, wrlock, true);


This should be also fixed, we cannot always warm up data when doing index compaction.

in a separate PR

HaolinGuan7211 · 2025-11-05T11:17:07Z

run buildall

HaolinGuan7211 · 2025-11-05T12:24:50Z

run buildall

doris-robot · 2025-11-05T13:20:46Z

TPC-DS: Total hot run time: 190149 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 797459739f4c86b7a781bf72a21236a93535dcbc, data reload: false

query1	1033	430	409	409
query2	6566	1672	1754	1672
query3	6748	236	228	228
query4	26568	24025	23513	23513
query5	5715	631	480	480
query6	344	231	223	223
query7	4650	503	299	299
query8	318	264	259	259
query9	8745	2587	2596	2587
query10	552	333	300	300
query11	15684	15053	14879	14879
query12	183	122	118	118
query13	1685	568	460	460
query14	11611	9410	9447	9410
query15	211	189	175	175
query16	7689	709	546	546
query17	1630	819	685	685
query18	2117	526	384	384
query19	253	226	192	192
query20	141	140	138	138
query21	221	155	153	153
query22	4743	4725	4748	4725
query23	35008	33684	33868	33684
query24	8648	2588	2521	2521
query25	605	518	465	465
query26	1268	288	158	158
query27	2738	524	368	368
query28	4436	2233	2211	2211
query29	815	700	516	516
query30	299	236	211	211
query31	943	869	799	799
query32	81	67	69	67
query33	574	363	325	325
query34	805	849	528	528
query35	783	858	750	750
query36	962	990	894	894
query37	122	106	88	88
query38	3628	3576	3537	3537
query39	1471	1417	1414	1414
query40	220	128	116	116
query41	61	57	59	57
query42	121	109	108	108
query43	485	498	462	462
query44	1236	736	750	736
query45	181	178	172	172
query46	884	1004	641	641
query47	1739	1752	1691	1691
query48	404	415	323	323
query49	785	486	409	409
query50	639	683	410	410
query51	4003	3997	3856	3856
query52	109	106	105	105
query53	235	265	199	199
query54	305	281	271	271
query55	87	87	84	84
query56	314	333	297	297
query57	1181	1182	1106	1106
query58	303	273	275	273
query59	2575	2674	2472	2472
query60	356	353	313	313
query61	159	157	161	157
query62	800	730	703	703
query63	234	203	198	198
query64	4439	1162	873	873
query65	4059	3972	3955	3955
query66	1037	436	337	337
query67	15256	15051	14866	14866
query68	8557	952	595	595
query69	548	321	300	300
query70	1358	1290	1361	1290
query71	512	354	320	320
query72	5832	4929	4897	4897
query73	680	570	364	364
query74	8949	9168	9057	9057
query75	4165	3303	2823	2823
query76	3872	1190	751	751
query77	798	416	327	327
query78	9527	9826	8909	8909
query79	2014	793	586	586
query80	668	549	505	505
query81	475	262	224	224
query82	418	158	135	135
query83	274	258	247	247
query84	255	110	95	95
query85	875	481	442	442
query86	347	303	294	294
query87	3715	3725	3597	3597
query88	3375	2254	2218	2218
query89	396	322	300	300
query90	2024	219	218	218
query91	174	168	136	136
query92	83	65	63	63
query93	1154	979	646	646
query94	677	424	343	343
query95	404	319	321	319
query96	492	579	279	279
query97	2951	2958	2854	2854
query98	244	211	202	202
query99	1513	1428	1252	1252
Total cold run time: 280126 ms
Total hot run time: 190149 ms

doris-robot · 2025-11-05T13:25:55Z

ClickBench: Total hot run time: 27.81 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 797459739f4c86b7a781bf72a21236a93535dcbc, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.04	0.05
query3	0.25	0.08	0.09
query4	1.61	0.12	0.12
query5	0.27	0.25	0.26
query6	1.19	0.66	0.65
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.60	0.54	0.51
query10	0.58	0.58	0.57
query11	0.17	0.12	0.12
query12	0.15	0.12	0.12
query13	0.62	0.60	0.61
query14	1.02	0.99	0.99
query15	0.85	0.85	0.85
query16	0.41	0.40	0.39
query17	1.03	1.05	1.04
query18	0.22	0.21	0.20
query19	1.93	1.82	1.81
query20	0.02	0.02	0.02
query21	15.44	0.18	0.13
query22	5.09	0.08	0.05
query23	15.67	0.25	0.11
query24	2.60	0.76	1.11
query25	0.08	0.06	0.05
query26	0.14	0.12	0.12
query27	0.07	0.05	0.05
query28	5.83	1.13	0.92
query29	12.62	3.93	3.16
query30	0.28	0.15	0.11
query31	2.82	0.58	0.38
query32	3.23	0.56	0.47
query33	3.08	3.00	3.01
query34	15.90	5.15	4.62
query35	4.56	4.56	4.57
query36	0.68	0.50	0.50
query37	0.10	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.15	0.13
query41	0.09	0.03	0.03
query42	0.04	0.02	0.02
query43	0.05	0.03	0.03
Total cold run time: 99.8 s
Total hot run time: 27.81 s

hello-stephen · 2025-11-05T14:51:02Z

BE UT Coverage Report

Increment line coverage 3.57% (1/28) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.76% (18189/34472)
Line Coverage	38.19% (165691/433846)
Region Coverage	33.23% (129064/388426)
Branch Coverage	33.96% (55396/163106)

dataroaring · 2025-11-12T18:50:51Z

run buildall

doris-robot · 2025-11-12T20:40:27Z

TPC-H: Total hot run time: 35645 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 32b3c20cd1fd863c4bb68c7b4fb1b6d72449c83a, data reload: false

------ Round 1 ----------------------------------
q1	17637	5259	5128	5128
q2	2011	311	208	208
q3	10257	1292	710	710
q4	10234	926	370	370
q5	7523	2483	2393	2393
q6	185	170	135	135
q7	968	781	637	637
q8	9371	1401	1163	1163
q9	7063	5207	5130	5130
q10	6900	2248	1834	1834
q11	507	299	285	285
q12	372	377	231	231
q13	17785	3747	3079	3079
q14	243	243	212	212
q15	589	518	520	518
q16	1023	1045	938	938
q17	610	896	371	371
q18	7692	7744	8084	7744
q19	1123	1013	604	604
q20	383	365	247	247
q21	4584	3736	2643	2643
q22	1147	1144	1065	1065
Total cold run time: 108207 ms
Total hot run time: 35645 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5467	5306	5364	5306
q2	253	337	236	236
q3	2464	2913	2478	2478
q4	1450	1909	1532	1532
q5	4587	4385	4323	4323
q6	218	168	125	125
q7	2095	2074	1793	1793
q8	2670	2718	2594	2594
q9	7351	7256	6787	6787
q10	2898	3093	2640	2640
q11	574	516	488	488
q12	645	729	573	573
q13	3240	3685	3049	3049
q14	275	298	260	260
q15	531	499	505	499
q16	1026	1060	995	995
q17	1127	1454	1378	1378
q18	7283	7016	7079	7016
q19	771	799	957	799
q20	1885	1992	1829	1829
q21	4935	4296	4399	4296
q22	1101	1048	989	989
Total cold run time: 52846 ms
Total hot run time: 49985 ms

doris-robot · 2025-11-12T20:52:10Z

TPC-DS: Total hot run time: 187610 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 32b3c20cd1fd863c4bb68c7b4fb1b6d72449c83a, data reload: false

query1	1045	407	387	387
query2	6588	1707	1701	1701
query3	6754	229	223	223
query4	26347	23795	23068	23068
query5	4438	660	477	477
query6	348	274	225	225
query7	4648	491	294	294
query8	295	239	239	239
query9	8684	2603	2543	2543
query10	500	339	314	314
query11	15844	15037	14918	14918
query12	170	119	109	109
query13	1669	557	425	425
query14	11085	9252	9208	9208
query15	199	187	178	178
query16	7555	679	522	522
query17	1222	749	622	622
query18	2001	420	318	318
query19	211	201	168	168
query20	125	122	121	121
query21	211	133	112	112
query22	4105	3968	4091	3968
query23	33851	33070	32835	32835
query24	8458	2475	2403	2403
query25	623	507	443	443
query26	1231	274	162	162
query27	2759	501	353	353
query28	4387	2230	2206	2206
query29	807	616	481	481
query30	293	233	202	202
query31	944	810	728	728
query32	82	77	74	74
query33	581	374	330	330
query34	797	863	516	516
query35	796	832	771	771
query36	959	993	892	892
query37	115	112	88	88
query38	3500	3498	3447	3447
query39	1489	1420	1403	1403
query40	224	135	120	120
query41	66	62	63	62
query42	131	125	111	111
query43	488	492	486	486
query44	1243	750	741	741
query45	184	180	171	171
query46	892	1018	645	645
query47	1793	1795	1735	1735
query48	403	431	329	329
query49	792	545	423	423
query50	654	697	407	407
query51	3906	4056	3861	3861
query52	113	119	104	104
query53	255	266	191	191
query54	336	315	289	289
query55	89	89	87	87
query56	360	339	316	316
query57	1207	1189	1143	1143
query58	303	295	288	288
query59	2620	2784	2603	2603
query60	367	369	345	345
query61	200	200	189	189
query62	825	749	669	669
query63	230	201	202	201
query64	4588	1308	993	993
query65	4042	3939	3943	3939
query66	1164	458	362	362
query67	15654	15129	14989	14989
query68	8253	943	601	601
query69	489	337	298	298
query70	1342	1280	1297	1280
query71	461	341	315	315
query72	5880	4943	4952	4943
query73	625	581	364	364
query74	9245	9046	8671	8671
query75	3539	3352	2849	2849
query76	3471	1227	744	744
query77	723	400	316	316
query78	9684	9603	8874	8874
query79	2661	839	604	604
query80	703	576	509	509
query81	513	261	230	230
query82	483	163	136	136
query83	268	262	250	250
query84	259	119	98	98
query85	943	490	446	446
query86	383	304	282	282
query87	3729	3707	3603	3603
query88	3800	2237	2231	2231
query89	388	337	291	291
query90	1980	216	215	215
query91	168	175	140	140
query92	90	69	61	61
query93	2382	973	648	648
query94	749	447	355	355
query95	409	329	314	314
query96	487	581	289	289
query97	2948	2988	2873	2873
query98	246	213	207	207
query99	1423	1412	1300	1300
Total cold run time: 277569 ms
Total hot run time: 187610 ms

doris-robot · 2025-11-12T20:57:16Z

ClickBench: Total hot run time: 27.32 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 32b3c20cd1fd863c4bb68c7b4fb1b6d72449c83a, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.05	0.04
query3	0.26	0.08	0.08
query4	1.62	0.12	0.11
query5	0.27	0.25	0.26
query6	1.17	0.64	0.64
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.58	0.53	0.52
query10	0.58	0.57	0.59
query11	0.16	0.11	0.12
query12	0.16	0.12	0.12
query13	0.62	0.60	0.60
query14	0.98	0.99	1.01
query15	0.84	0.82	0.83
query16	0.39	0.39	0.39
query17	1.03	1.02	1.01
query18	0.21	0.20	0.19
query19	1.90	1.84	1.79
query20	0.01	0.01	0.01
query21	15.43	0.20	0.13
query22	5.10	0.07	0.05
query23	15.69	0.26	0.10
query24	2.94	0.68	0.39
query25	0.09	0.07	0.06
query26	0.14	0.14	0.14
query27	0.07	0.05	0.05
query28	4.03	1.13	0.92
query29	12.56	3.82	3.16
query30	0.29	0.14	0.12
query31	2.81	0.61	0.38
query32	3.23	0.56	0.48
query33	3.00	3.02	3.05
query34	15.85	5.15	4.54
query35	4.55	4.62	4.56
query36	0.68	0.50	0.52
query37	0.09	0.07	0.07
query38	0.06	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.14	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 97.98 s
Total hot run time: 27.32 s

hello-stephen · 2025-11-12T22:17:51Z

BE UT Coverage Report

Increment line coverage 3.57% (1/28) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.81% (18280/34613)
Line Coverage	38.18% (166191/435294)
Region Coverage	33.17% (129201/389465)
Branch Coverage	33.91% (55446/163499)

hello-stephen · 2025-11-12T23:30:33Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/28) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.54% (19570/34013)
Line Coverage	42.98% (187371/435970)
Region Coverage	38.00% (150101/394976)
Branch Coverage	38.44% (63260/164574)

freemandealer · 2025-11-13T14:29:55Z

be/src/cloud/cloud_schema_change_job.cpp

+    double input_hit_rate = static_cast<double>(cached_index_size + cached_data_size) / total_size;
+
+    if (input_hit_rate > config::file_cache_keep_schema_change_output_min_hit_ratio) {
+        return true;


please add LOG(INFO) here to clarify the decision

tabletid, xxx_size, ratio, etc

Ok, I wiil add LOG here.

github-actions · 2025-11-13T14:30:50Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-11-13T14:30:52Z

PR approved by anyone and no changes requested.

hello-stephen · 2025-11-18T08:16:23Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/27) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.54% (19570/34013)
Line Coverage	42.98% (187371/435970)
Region Coverage	38.00% (150101/394976)
Branch Coverage	38.44% (63260/164574)

hello-stephen · 2025-11-25T15:08:10Z

BE Regression && UT Coverage Report

Increment line coverage 94.59% (35/37) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.59% (24470/34182)
Line Coverage	58.07% (254847/438856)
Region Coverage	53.21% (212475/399309)
Branch Coverage	54.73% (90816/165924)

HaolinGuan7211 · 2025-11-26T10:42:50Z

run buildall

doris-robot · 2025-11-26T12:32:23Z

TPC-H: Total hot run time: 34471 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c013bd24b7e7e5c9639585a9573ccd1e112e84db, data reload: false

------ Round 1 ----------------------------------
q1	17671	5058	4929	4929
q2	2040	325	211	211
q3	10233	1319	745	745
q4	10233	971	386	386
q5	7464	2427	2365	2365
q6	197	175	141	141
q7	950	774	635	635
q8	9348	1365	1117	1117
q9	7403	5393	5387	5387
q10	6914	2279	1826	1826
q11	515	303	287	287
q12	358	370	226	226
q13	17774	3660	3014	3014
q14	235	250	228	228
q15	598	521	517	517
q16	1068	1009	973	973
q17	608	867	395	395
q18	7960	7135	6995	6995
q19	1097	973	533	533
q20	343	346	232	232
q21	3795	3277	2352	2352
q22	1035	997	977	977
Total cold run time: 107839 ms
Total hot run time: 34471 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4989	4963	4969	4963
q2	337	388	312	312
q3	2224	2684	2308	2308
q4	1336	1769	1357	1357
q5	4287	4552	4578	4552
q6	214	176	130	130
q7	2053	1996	1831	1831
q8	2681	2639	2628	2628
q9	7537	7510	7504	7504
q10	3154	3320	2834	2834
q11	630	531	482	482
q12	687	785	629	629
q13	3599	3925	3379	3379
q14	278	286	306	286
q15	598	530	510	510
q16	1070	1135	1328	1135
q17	1158	1534	1532	1532
q18	7953	7818	7588	7588
q19	835	861	1079	861
q20	1977	2045	1860	1860
q21	4772	4409	4324	4324
q22	1089	1098	1000	1000
Total cold run time: 53458 ms
Total hot run time: 52005 ms

HaolinGuan7211 · 2025-11-26T12:33:20Z

run cloud_p0

doris-robot · 2025-11-26T12:43:53Z

TPC-DS: Total hot run time: 183967 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c013bd24b7e7e5c9639585a9573ccd1e112e84db, data reload: false

query1	1053	472	387	387
query2	6559	1642	1605	1605
query3	6754	225	228	225
query4	25470	23285	22431	22431
query5	4432	609	463	463
query6	358	234	226	226
query7	4637	501	292	292
query8	309	253	242	242
query9	8688	2551	2559	2551
query10	523	342	316	316
query11	15023	14941	14569	14569
query12	184	125	121	121
query13	1704	599	461	461
query14	10080	8899	8805	8805
query15	210	205	188	188
query16	7403	694	528	528
query17	1262	777	647	647
query18	2011	456	344	344
query19	215	206	184	184
query20	132	130	126	126
query21	217	139	120	120
query22	3876	4020	3909	3909
query23	33309	32074	32190	32074
query24	8566	2376	2430	2376
query25	627	584	487	487
query26	1241	277	170	170
query27	2737	502	352	352
query28	4404	2147	2125	2125
query29	856	652	532	532
query30	305	243	211	211
query31	841	705	641	641
query32	82	80	71	71
query33	622	385	338	338
query34	795	918	530	530
query35	815	838	744	744
query36	916	936	850	850
query37	126	106	85	85
query38	3353	3461	3260	3260
query39	1511	1406	1433	1406
query40	224	129	121	121
query41	65	66	61	61
query42	124	111	110	110
query43	460	459	440	440
query44	1238	755	749	749
query45	192	189	188	188
query46	891	996	635	635
query47	1695	1703	1678	1678
query48	391	423	330	330
query49	783	522	408	408
query50	663	680	402	402
query51	4147	3864	3887	3864
query52	110	113	103	103
query53	248	281	195	195
query54	318	287	275	275
query55	99	96	93	93
query56	329	354	321	321
query57	1169	1177	1119	1119
query58	295	276	267	267
query59	2457	2559	2355	2355
query60	378	350	337	337
query61	165	159	156	156
query62	804	729	654	654
query63	221	197	197	197
query64	4568	1225	885	885
query65	4039	3974	3975	3974
query66	1189	454	334	334
query67	15196	15091	15058	15058
query68	8450	964	648	648
query69	511	341	310	310
query70	1291	1263	1205	1205
query71	483	328	308	308
query72	5606	4910	4896	4896
query73	700	568	348	348
query74	8724	8751	8335	8335
query75	3938	3340	2833	2833
query76	3736	1126	692	692
query77	843	407	311	311
query78	9397	9867	8926	8926
query79	2071	834	585	585
query80	630	571	487	487
query81	491	268	233	233
query82	426	162	137	137
query83	274	261	254	254
query84	259	124	101	101
query85	928	492	447	447
query86	330	313	280	280
query87	3533	3542	3366	3366
query88	3486	2239	2217	2217
query89	397	328	303	303
query90	1930	221	206	206
query91	190	172	143	143
query92	82	71	66	66
query93	1423	988	657	657
query94	727	463	348	348
query95	498	398	391	391
query96	499	573	281	281
query97	2917	3035	2890	2890
query98	249	218	209	209
query99	1390	1396	1253	1253
Total cold run time: 271073 ms
Total hot run time: 183967 ms

doris-robot · 2025-11-26T12:49:02Z

ClickBench: Total hot run time: 28.02 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c013bd24b7e7e5c9639585a9573ccd1e112e84db, data reload: false

query1	0.05	0.06	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.08
query4	1.62	0.11	0.10
query5	0.27	0.25	0.25
query6	1.18	0.64	0.68
query7	0.04	0.03	0.02
query8	0.06	0.05	0.05
query9	0.60	0.54	0.52
query10	0.57	0.58	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.62	0.60
query14	1.01	1.00	1.00
query15	0.84	0.84	0.84
query16	0.41	0.40	0.39
query17	1.05	1.03	1.02
query18	0.22	0.20	0.20
query19	1.93	1.81	1.81
query20	0.01	0.01	0.01
query21	15.46	0.19	0.13
query22	4.94	0.07	0.05
query23	15.66	0.26	0.10
query24	2.92	1.35	0.90
query25	0.09	0.07	0.06
query26	0.13	0.14	0.13
query27	0.07	0.06	0.10
query28	5.27	1.14	0.92
query29	12.63	3.87	3.20
query30	0.28	0.15	0.12
query31	2.82	0.61	0.39
query32	3.24	0.56	0.47
query33	2.98	3.04	3.03
query34	15.90	5.20	4.55
query35	4.55	4.62	4.64
query36	0.67	0.51	0.50
query37	0.11	0.07	0.07
query38	0.07	0.05	0.04
query39	0.03	0.03	0.03
query40	0.18	0.13	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.03	0.04
Total cold run time: 99.33 s
Total hot run time: 28.02 s

HaolinGuan7211 · 2025-11-26T14:41:24Z

run cloud_p0

HaolinGuan7211 · 2025-11-27T10:01:59Z

run buildall

doris-robot · 2025-11-27T11:54:55Z

TPC-H: Total hot run time: 34263 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c013bd24b7e7e5c9639585a9573ccd1e112e84db, data reload: false

------ Round 1 ----------------------------------
q1	17658	5026	4873	4873
q2	2034	314	206	206
q3	10264	1302	727	727
q4	10242	984	387	387
q5	7526	2469	2301	2301
q6	186	177	142	142
q7	953	782	653	653
q8	9352	1371	1033	1033
q9	7150	5288	5324	5288
q10	6872	2234	1799	1799
q11	516	305	304	304
q12	333	369	233	233
q13	17782	3634	3010	3010
q14	233	242	213	213
q15	582	519	526	519
q16	1042	1006	963	963
q17	592	811	462	462
q18	7541	7122	7074	7074
q19	1123	942	589	589
q20	346	354	237	237
q21	3616	3344	2286	2286
q22	1050	1020	964	964
Total cold run time: 106993 ms
Total hot run time: 34263 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4942	4953	4885	4885
q2	332	394	310	310
q3	2173	2680	2279	2279
q4	1321	1762	1277	1277
q5	4212	4417	4544	4417
q6	214	176	125	125
q7	2063	1994	1859	1859
q8	2676	2510	2475	2475
q9	7620	7569	7589	7569
q10	3102	3278	2823	2823
q11	588	518	502	502
q12	719	749	606	606
q13	3546	3836	3293	3293
q14	279	302	288	288
q15	576	516	519	516
q16	1106	1144	1091	1091
q17	1318	1477	1424	1424
q18	7841	7831	7510	7510
q19	795	851	789	789
q20	1883	1990	1806	1806
q21	4605	4190	4157	4157
q22	1077	1032	975	975
Total cold run time: 52988 ms
Total hot run time: 50976 ms

doris-robot · 2025-11-27T12:06:25Z

TPC-DS: Total hot run time: 184847 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c013bd24b7e7e5c9639585a9573ccd1e112e84db, data reload: false

query1	1054	439	393	393
query2	6572	1577	1549	1549
query3	6754	225	223	223
query4	25243	22972	23049	22972
query5	4877	629	484	484
query6	340	241	220	220
query7	4658	513	300	300
query8	305	245	242	242
query9	8726	2577	2551	2551
query10	542	346	305	305
query11	15214	15021	15022	15021
query12	184	121	116	116
query13	1707	570	447	447
query14	11065	8703	8705	8703
query15	222	201	185	185
query16	7661	699	536	536
query17	1202	783	645	645
query18	2058	438	372	372
query19	215	209	181	181
query20	135	129	125	125
query21	219	136	118	118
query22	3889	3979	3931	3931
query23	33516	32255	32170	32170
query24	8493	2419	2433	2419
query25	636	548	481	481
query26	1253	282	214	214
query27	2656	489	345	345
query28	4327	2130	2126	2126
query29	786	617	496	496
query30	303	240	204	204
query31	829	698	619	619
query32	81	75	74	74
query33	602	375	328	328
query34	815	865	526	526
query35	796	832	731	731
query36	903	913	848	848
query37	124	107	86	86
query38	3296	3321	3267	3267
query39	1470	1400	1402	1400
query40	225	132	118	118
query41	63	61	63	61
query42	135	115	113	113
query43	437	444	413	413
query44	1280	769	772	769
query45	199	192	184	184
query46	866	983	637	637
query47	1734	1764	1665	1665
query48	399	416	313	313
query49	772	513	400	400
query50	651	680	415	415
query51	3878	3992	3928	3928
query52	110	112	101	101
query53	233	261	192	192
query54	299	293	276	276
query55	95	94	90	90
query56	345	338	332	332
query57	1150	1164	1088	1088
query58	295	282	278	278
query59	2337	2391	2286	2286
query60	355	347	361	347
query61	166	157	156	156
query62	764	722	651	651
query63	225	196	199	196
query64	4491	1177	880	880
query65	4106	3996	3950	3950
query66	1106	453	375	375
query67	15552	15005	14953	14953
query68	8288	959	620	620
query69	505	345	312	312
query70	1291	1237	1251	1237
query71	441	328	314	314
query72	5925	4824	4785	4785
query73	662	559	335	335
query74	8849	8685	8612	8612
query75	3451	3274	2776	2776
query76	3400	1115	706	706
query77	730	391	314	314
query78	9416	9622	8882	8882
query79	1968	835	575	575
query80	622	573	504	504
query81	507	274	236	236
query82	215	163	136	136
query83	270	256	252	252
query84	263	114	96	96
query85	893	500	442	442
query86	374	309	278	278
query87	3438	3516	3403	3403
query88	3675	2242	2254	2242
query89	382	334	294	294
query90	1974	220	208	208
query91	177	171	141	141
query92	83	64	67	64
query93	1962	999	649	649
query94	705	443	339	339
query95	497	416	405	405
query96	501	547	289	289
query97	2890	2931	2844	2844
query98	244	215	212	212
query99	1304	1374	1225	1225
Total cold run time: 271898 ms
Total hot run time: 184847 ms

doris-robot · 2025-11-27T12:11:30Z

ClickBench: Total hot run time: 27.55 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c013bd24b7e7e5c9639585a9573ccd1e112e84db, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.12	0.11
query5	0.28	0.26	0.26
query6	1.20	0.65	0.64
query7	0.04	0.03	0.03
query8	0.06	0.05	0.05
query9	0.57	0.52	0.51
query10	0.56	0.55	0.55
query11	0.16	0.11	0.11
query12	0.15	0.11	0.11
query13	0.63	0.61	0.62
query14	1.01	0.99	0.97
query15	0.82	0.80	0.82
query16	0.39	0.42	0.43
query17	1.02	1.05	0.97
query18	0.23	0.22	0.21
query19	1.92	1.88	1.89
query20	0.02	0.01	0.01
query21	15.45	0.25	0.15
query22	4.89	0.05	0.04
query23	16.06	0.27	0.10
query24	1.82	0.57	0.72
query25	0.08	0.05	0.10
query26	0.14	0.13	0.13
query27	0.07	0.04	0.05
query28	5.10	1.18	1.03
query29	12.57	3.99	3.16
query30	0.27	0.13	0.12
query31	2.83	0.63	0.39
query32	3.23	0.54	0.46
query33	3.13	3.00	3.08
query34	16.76	5.20	4.53
query35	4.49	4.52	4.55
query36	0.64	0.48	0.50
query37	0.11	0.07	0.07
query38	0.08	0.04	0.04
query39	0.05	0.03	0.02
query40	0.16	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.03	0.04
Total cold run time: 99.17 s
Total hot run time: 27.55 s

gavinchou · 2025-11-28T07:14:27Z

run check_coverage

hello-stephen · 2025-11-28T07:22:01Z

skip check_coverage

Cherry-pick from #57470

(#58622) Cherry-pick from #57470

gavinchou changed the title ~~[feat] : schema change adaptive to file cache~~ [opt](filecache) schema change adaptive to file cache Nov 4, 2025

gavinchou reviewed Nov 4, 2025

View reviewed changes

HaolinGuan7211 marked this pull request as ready for review November 5, 2025 01:04

HaolinGuan7211 requested a review from dataroaring as a code owner November 5, 2025 01:04

gavinchou previously approved these changes Nov 6, 2025

View reviewed changes

gavinchou added dev/3.1.x dev/4.0.x labels Nov 6, 2025

HaolinGuan7211 force-pushed the ghl/sc_cache_adaptive branch from 7974597 to 915dcbd Compare November 6, 2025 07:27

HaolinGuan7211 dismissed gavinchou’s stale review via fecf849 November 6, 2025 07:32

HaolinGuan7211 force-pushed the ghl/sc_cache_adaptive branch from fecf849 to c33eb68 Compare November 6, 2025 07:38

gavinchou previously approved these changes Nov 10, 2025

View reviewed changes

freemandealer reviewed Nov 13, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 13, 2025

github-actions bot added the reviewed label Nov 13, 2025

HaolinGuan7211 dismissed gavinchou’s stale review via ebc578d November 21, 2025 03:35

fix coredump

c013bd2

gavinchou approved these changes Nov 28, 2025

View reviewed changes

gavinchou merged commit 6327351 into apache:master Nov 28, 2025
25 of 26 checks passed

github-actions bot added dev/3.1.x-conflict dev/4.0.x-conflict labels Nov 28, 2025

HaolinGuan7211 added a commit to HaolinGuan7211/doris that referenced this pull request Dec 2, 2025

[opt](filecache) schema change adaptive to file cache (apache#57470)

29b0807

This was referenced Dec 2, 2025

branch-3.1: [opt](filecache) schema change adaptive to file cache #57540 #58622

Merged

[opt](filecache) schema change adaptive to file cache #58620

Merged

yiguolei pushed a commit that referenced this pull request Dec 12, 2025

[opt](filecache) schema change adaptive to file cache (#58620)

0f22ae9

Cherry-pick from #57470

yiguolei added dev/4.0.3-merged and removed dev/4.0.x dev/4.0.x-conflict labels Dec 12, 2025

nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025

[opt](filecache) schema change adaptive to file cache (apache#57470)

9a35ec0

morrySnow pushed a commit that referenced this pull request Dec 18, 2025

branch-3.1: [opt](filecache) schema change adaptive to file cache #57540

50b13fc

(#58622) Cherry-pick from #57470

morrySnow added dev/3.1.4-merged and removed dev/3.1.x dev/3.1.x-conflict labels Dec 18, 2025

[opt](filecache) schema change adaptive to file cache #57470

[opt](filecache) schema change adaptive to file cache #57470

Uh oh!

Conversation

HaolinGuan7211 commented Oct 29, 2025

Uh oh!

Thearas commented Oct 29, 2025

Uh oh!

gavinchou Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

gavinchou Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

HaolinGuan7211 commented Nov 5, 2025

Uh oh!

HaolinGuan7211 commented Nov 5, 2025

Uh oh!

doris-robot commented Nov 5, 2025

Uh oh!

doris-robot commented Nov 5, 2025

Uh oh!

hello-stephen commented Nov 5, 2025

BE UT Coverage Report

Uh oh!

dataroaring commented Nov 12, 2025

Uh oh!

doris-robot commented Nov 12, 2025

Uh oh!

doris-robot commented Nov 12, 2025

Uh oh!

doris-robot commented Nov 12, 2025

Uh oh!

hello-stephen commented Nov 12, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Nov 12, 2025

BE Regression && UT Coverage Report

Uh oh!

freemandealer Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

freemandealer Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

HaolinGuan7211 Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 13, 2025

Uh oh!

github-actions bot commented Nov 13, 2025

Uh oh!

hello-stephen commented Nov 18, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Nov 25, 2025

BE Regression && UT Coverage Report

Uh oh!

HaolinGuan7211 commented Nov 26, 2025

Uh oh!

doris-robot commented Nov 26, 2025

Uh oh!

HaolinGuan7211 commented Nov 26, 2025

Uh oh!

doris-robot commented Nov 26, 2025

Uh oh!

doris-robot commented Nov 26, 2025

Uh oh!

HaolinGuan7211 commented Nov 26, 2025

Uh oh!

HaolinGuan7211 commented Nov 27, 2025

Uh oh!

doris-robot commented Nov 27, 2025

Uh oh!

doris-robot commented Nov 27, 2025

Uh oh!

doris-robot commented Nov 27, 2025

Uh oh!

gavinchou commented Nov 28, 2025

Uh oh!