Skip to content

[fix](olap) Should erase the segment footer cache after compaction #59101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
mrhhsg merged 1 commit into from
Dec 18, 2025
Merged

[fix](olap) Should erase the segment footer cache after compaction #59101

mrhhsg merged 1 commit into from
Dec 18, 2025
+113 −2

Conversation

@mrhhsg
Copy link
Member

@mrhhsg mrhhsg commented Dec 17, 2025 •
edited
Loading

What problem does this PR solve?

The cache should be erased after compaction.

During compaction, segment data files are renamed. The renamed files may hit the footer cache of the previous files (since the cache key is composed of the file path and file size), so these caches need to be invalidated.

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@mrhhsg
Copy link
Member Author

mrhhsg commented Dec 17, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

liaoxin01
liaoxin01 previously approved these changes Dec 17, 2025
View reviewed changes
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 17, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 17, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 17, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Dec 17, 2025

TPC-H: Total hot run time: 35381 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d9a3d16fa5b174624aa6fbefa16c7f747f53454, data reload: false

------ Round 1 ----------------------------------
q1	17645	4307	4145	4145
q2	2013	350	234	234
q3	10199	1317	770	770
q4	10232	944	318	318
q5	7548	2142	2012	2012
q6	193	171	138	138
q7	1031	861	725	725
q8	9358	1527	1148	1148
q9	7252	5315	5335	5315
q10	6868	2415	1963	1963
q11	541	328	300	300
q12	674	737	567	567
q13	17793	3721	3110	3110
q14	290	322	271	271
q15	611	528	511	511
q16	695	671	623	623
q17	720	802	549	549
q18	7628	7189	7178	7178
q19	1162	962	619	619
q20	396	353	259	259
q21	4276	3948	3687	3687
q22	1029	993	939	939
Total cold run time: 108154 ms
Total hot run time: 35381 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4146	4084	4051	4051
q2	317	394	332	332
q3	2157	2715	2289	2289
q4	1341	1771	1330	1330
q5	4262	4773	4700	4700
q6	233	182	131	131
q7	2037	1969	1925	1925
q8	2638	2556	2537	2537
q9	7662	7611	7590	7590
q10	3095	3342	2788	2788
q11	600	515	495	495
q12	731	896	629	629
q13	3545	4121	3287	3287
q14	270	290	275	275
q15	574	528	514	514
q16	712	720	671	671
q17	1224	1469	1394	1394
q18	7837	7622	7533	7533
q19	918	1012	895	895
q20	2019	2064	1951	1951
q21	4831	4284	4086	4086
q22	1086	1027	991	991
Total cold run time: 52235 ms
Total hot run time: 50394 ms

@doris-robot
Copy link

doris-robot commented Dec 17, 2025

TPC-DS: Total hot run time: 178498 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d9a3d16fa5b174624aa6fbefa16c7f747f53454, data reload: false

query5	5001	613	458	458
query6	325	234	227	227
query7	4237	482	281	281
query8	323	265	248	248
query9	8788	2575	2583	2575
query10	543	359	342	342
query11	15321	15234	14707	14707
query12	193	120	113	113
query13	1262	493	404	404
query14	6637	3006	2804	2804
query14_1	2658	2610	2652	2610
query15	208	192	173	173
query16	885	490	453	453
query17	1075	698	571	571
query18	2703	437	343	343
query19	230	234	197	197
query20	122	112	119	112
query21	213	139	120	120
query22	3978	3993	3881	3881
query23	16600	16141	16011	16011
query23_1	16082	16305	16039	16039
query24	7329	1660	1250	1250
query24_1	1246	1219	1235	1219
query25	565	526	463	463
query26	1240	274	168	168
query27	2734	463	325	325
query28	4478	2187	2177	2177
query29	830	570	476	476
query30	314	248	209	209
query31	868	719	616	616
query32	80	71	76	71
query33	557	359	305	305
query34	907	920	546	546
query35	813	807	726	726
query36	881	911	844	844
query37	142	98	79	79
query38	2818	2863	2764	2764
query39	748	739	712	712
query39_1	689	692	692	692
query40	228	139	123	123
query41	64	61	61	61
query42	109	105	104	104
query43	429	435	396	396
query44	1316	760	752	752
query45	190	188	182	182
query46	879	983	608	608
query47	1702	1714	1629	1629
query48	345	345	246	246
query49	620	434	352	352
query50	647	296	212	212
query51	3831	3940	3798	3798
query52	110	109	100	100
query53	326	357	292	292
query54	281	254	268	254
query55	85	74	74	74
query56	302	302	293	293
query57	1136	1126	1087	1087
query58	273	256	255	255
query59	2295	2551	2419	2419
query60	303	311	292	292
query61	166	155	158	155
query62	705	659	619	619
query63	329	298	301	298
query64	4917	1303	989	989
query65	4053	3981	3917	3917
query66	1378	444	324	324
query67	15385	15003	14757	14757
query68	3163	1042	736	736
query69	477	351	309	309
query70	1041	990	993	990
query71	331	313	272	272
query72	6141	4966	4937	4937
query73	498	574	316	316
query74	8824	8810	8582	8582
query75	3209	3175	2787	2787
query76	3556	1137	736	736
query77	519	395	297	297
query78	9563	9750	8877	8877
query79	1027	891	619	619
query80	979	671	557	557
query81	520	264	237	237
query82	203	131	105	105
query83	386	261	245	245
query84	261	121	111	111
query85	1015	514	465	465
query86	316	291	288	288
query87	3015	3086	2908	2908
query88	3207	2288	2291	2288
query89	463	429	398	398
query90	1887	158	152	152
query91	187	164	145	145
query92	70	70	62	62
query93	955	911	567	567
query94	512	304	269	269
query95	589	384	308	308
query96	599	464	210	210
query97	2273	2352	2268	2268
query98	204	196	196	196
query99	1277	1335	1248	1248
Total cold run time: 254425 ms
Total hot run time: 178498 ms

@doris-robot
Copy link

doris-robot commented Dec 17, 2025

ClickBench: Total hot run time: 27.26 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9d9a3d16fa5b174624aa6fbefa16c7f747f53454, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.04	0.05
query3	0.26	0.09	0.08
query4	1.61	0.11	0.11
query5	0.28	0.25	0.25
query6	1.15	0.63	0.62
query7	0.03	0.02	0.03
query8	0.06	0.04	0.04
query9	0.58	0.51	0.51
query10	0.55	0.55	0.56
query11	0.16	0.11	0.11
query12	0.15	0.12	0.10
query13	0.61	0.60	0.62
query14	1.00	0.97	0.99
query15	0.82	0.80	0.80
query16	0.39	0.39	0.41
query17	0.99	1.04	1.01
query18	0.24	0.22	0.21
query19	1.88	1.85	1.79
query20	0.02	0.02	0.01
query21	15.46	0.29	0.14
query22	4.75	0.06	0.05
query23	15.94	0.28	0.10
query24	0.97	0.26	0.66
query25	0.11	0.06	0.06
query26	0.14	0.13	0.13
query27	0.09	0.05	0.06
query28	4.06	1.23	1.03
query29	12.59	3.96	3.27
query30	0.27	0.15	0.12
query31	2.84	0.62	0.40
query32	3.24	0.56	0.46
query33	3.08	3.02	3.09
query34	16.99	5.10	4.54
query35	4.64	4.55	4.53
query36	0.67	0.49	0.49
query37	0.10	0.06	0.07
query38	0.07	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 97.32 s
Total hot run time: 27.26 s

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 17, 2025
@mrhhsg
Copy link
Member Author

mrhhsg commented Dec 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Dec 17, 2025

TPC-H: Total hot run time: 35621 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a85b117d68ca863357a91ce1776d5e896d8e7cb4, data reload: false

------ Round 1 ----------------------------------
q1	17642	4200	4176	4176
q2	2079	348	235	235
q3	10152	1381	779	779
q4	10232	930	322	322
q5	7557	2097	2033	2033
q6	192	170	141	141
q7	1007	863	723	723
q8	9375	1474	1301	1301
q9	7243	5375	5326	5326
q10	6904	2386	1942	1942
q11	544	327	299	299
q12	702	740	568	568
q13	17789	3719	3052	3052
q14	290	305	270	270
q15	593	523	517	517
q16	703	691	625	625
q17	696	792	572	572
q18	7456	7049	6965	6965
q19	1143	957	639	639
q20	412	363	254	254
q21	4266	3939	3919	3919
q22	1023	994	963	963
Total cold run time: 108000 ms
Total hot run time: 35621 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4126	4032	4044	4032
q2	339	402	329	329
q3	2167	2669	2281	2281
q4	1363	1770	1319	1319
q5	4309	4637	4743	4637
q6	232	183	135	135
q7	2047	1964	1843	1843
q8	2709	2555	2564	2555
q9	7534	7541	7602	7541
q10	3125	3213	2854	2854
q11	622	502	491	491
q12	689	887	776	776
q13	3553	3890	3298	3298
q14	307	333	296	296
q15	565	521	518	518
q16	656	694	634	634
q17	1255	1669	1431	1431
q18	7921	7780	7582	7582
q19	956	904	920	904
q20	1927	1966	1864	1864
q21	4790	4256	4204	4204
q22	1122	1011	959	959
Total cold run time: 52314 ms
Total hot run time: 50483 ms

@doris-robot
Copy link

doris-robot commented Dec 17, 2025

TPC-DS: Total hot run time: 178325 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a85b117d68ca863357a91ce1776d5e896d8e7cb4, data reload: false

query5	4425	604	449	449
query6	334	236	223	223
query7	4228	464	277	277
query8	322	270	242	242
query9	8783	2539	2537	2537
query10	530	385	349	349
query11	15313	14931	14634	14634
query12	171	117	116	116
query13	1270	505	402	402
query14	5844	3089	2816	2816
query14_1	2726	2690	2664	2664
query15	218	194	175	175
query16	939	463	435	435
query17	1117	726	576	576
query18	2415	431	333	333
query19	236	231	205	205
query20	121	114	107	107
query21	217	137	119	119
query22	3950	3868	3813	3813
query23	16616	16157	15976	15976
query23_1	15923	16087	16080	16080
query24	7423	1662	1266	1266
query24_1	1240	1243	1272	1243
query25	553	489	429	429
query26	1242	266	164	164
query27	2767	461	310	310
query28	4468	2142	2139	2139
query29	840	583	485	485
query30	317	271	214	214
query31	816	686	623	623
query32	77	71	69	69
query33	528	337	291	291
query34	916	884	539	539
query35	784	808	738	738
query36	849	933	805	805
query37	136	95	81	81
query38	2882	2911	2907	2907
query39	777	738	707	707
query39_1	695	707	697	697
query40	238	140	121	121
query41	66	62	61	61
query42	108	108	108	108
query43	425	435	406	406
query44	1385	762	746	746
query45	194	189	180	180
query46	905	991	620	620
query47	1630	1683	1644	1644
query48	324	330	258	258
query49	658	465	353	353
query50	662	295	223	223
query51	3834	3821	3761	3761
query52	112	110	98	98
query53	347	358	299	299
query54	284	270	256	256
query55	78	72	72	72
query56	307	315	301	301
query57	1140	1117	1078	1078
query58	270	256	265	256
query59	2406	2506	2286	2286
query60	332	321	295	295
query61	163	159	157	157
query62	694	680	633	633
query63	333	304	307	304
query64	5084	1320	1037	1037
query65	4052	4016	3947	3947
query66	1463	428	322	322
query67	14864	14752	14575	14575
query68	2828	1037	768	768
query69	463	374	333	333
query70	1035	991	979	979
query71	324	318	293	293
query72	5893	5062	5054	5054
query73	487	545	302	302
query74	8875	8850	8528	8528
query75	3120	3178	2808	2808
query76	2890	1150	769	769
query77	369	417	284	284
query78	9574	9791	8800	8800
query79	1088	904	620	620
query80	1193	654	560	560
query81	552	266	237	237
query82	438	134	107	107
query83	369	257	247	247
query84	267	124	105	105
query85	937	525	484	484
query86	391	291	294	291
query87	3082	3152	3012	3012
query88	3360	2338	2294	2294
query89	473	425	397	397
query90	2014	163	158	158
query91	172	168	148	148
query92	71	72	69	69
query93	1128	964	563	563
query94	527	324	269	269
query95	575	394	309	309
query96	598	478	214	214
query97	2272	2341	2235	2235
query98	210	193	191	191
query99	1259	1289	1230	1230
Total cold run time: 252133 ms
Total hot run time: 178325 ms

@doris-robot
Copy link

doris-robot commented Dec 17, 2025

ClickBench: Total hot run time: 27.39 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a85b117d68ca863357a91ce1776d5e896d8e7cb4, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.61	0.11	0.12
query5	0.26	0.26	0.26
query6	1.17	0.66	0.62
query7	0.03	0.03	0.02
query8	0.06	0.04	0.04
query9	0.57	0.50	0.50
query10	0.56	0.54	0.55
query11	0.16	0.11	0.10
query12	0.15	0.12	0.12
query13	0.62	0.60	0.59
query14	0.98	0.98	0.98
query15	0.81	0.81	0.82
query16	0.40	0.41	0.44
query17	1.02	1.03	0.98
query18	0.24	0.22	0.22
query19	1.96	1.85	1.78
query20	0.02	0.01	0.02
query21	15.44	0.28	0.15
query22	4.71	0.05	0.05
query23	15.96	0.29	0.11
query24	1.10	0.54	0.60
query25	0.11	0.06	0.06
query26	0.15	0.14	0.13
query27	0.06	0.07	0.07
query28	3.96	1.19	1.02
query29	12.63	3.96	3.17
query30	0.27	0.14	0.12
query31	2.81	0.64	0.39
query32	3.25	0.55	0.45
query33	2.98	2.99	2.96
query34	16.93	5.19	4.54
query35	4.56	4.52	4.53
query36	0.65	0.50	0.49
query37	0.11	0.07	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.15	0.13
query41	0.08	0.04	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 97.14 s
Total hot run time: 27.39 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 17, 2025

BE UT Coverage Report

Increment line coverage 91.43% (64/70) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.40% (18833/35267)
Line Coverage 39.20% (174433/444997)
Region Coverage 33.76% (134991/399837)
Branch Coverage 34.67% (58139/167708)

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 17, 2025

BE Regression && UT Coverage Report

Increment line coverage 93.10% (27/29) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.30% (24985/34559)
Line Coverage 59.06% (262415/444342)
Region Coverage 54.00% (218363/404402)
Branch Coverage 55.46% (93442/168494)

@mrhhsg
Copy link
Member Author

mrhhsg commented Dec 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Dec 17, 2025

BE UT Coverage Report

Increment line coverage 92.21% (71/77) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.40% (18830/35261)
Line Coverage 39.21% (174417/444882)
Region Coverage 33.77% (134996/399734)
Branch Coverage 34.68% (58157/167692)

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 17, 2025

BE Regression && UT Coverage Report

Increment line coverage 94.44% (34/36) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.24% (24960/34553)
Line Coverage 58.96% (261938/444227)
Region Coverage 53.78% (217449/404299)
Branch Coverage 55.36% (93265/168478)

@doris-robot
Copy link

doris-robot commented Dec 17, 2025

TPC-H: Total hot run time: 36203 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 817347ae808de62750edd85a770b8020f942bcec, data reload: false

------ Round 1 ----------------------------------
q1	17650	4258	4075	4075
q2	2030	347	227	227
q3	10171	1314	744	744
q4	10222	826	314	314
q5	7531	2125	1979	1979
q6	189	171	141	141
q7	1005	874	723	723
q8	9352	1443	1178	1178
q9	6913	5374	5356	5356
q10	6846	2394	1945	1945
q11	500	342	297	297
q12	642	714	610	610
q13	17776	3676	3014	3014
q14	292	289	275	275
q15	606	521	512	512
q16	694	694	634	634
q17	708	867	484	484
q18	7763	7539	7585	7539
q19	1449	998	644	644
q20	432	396	269	269
q21	4569	4285	4231	4231
q22	1120	1019	1012	1012
Total cold run time: 108460 ms
Total hot run time: 36203 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4447	4380	4349	4349
q2	353	402	312	312
q3	2350	3092	2549	2549
q4	1420	1914	1390	1390
q5	4596	4458	4470	4458
q6	211	164	126	126
q7	2017	1943	1809	1809
q8	2683	2577	2633	2577
q9	7361	7716	7379	7379
q10	2907	3097	2707	2707
q11	574	487	465	465
q12	640	685	562	562
q13	3319	3707	3029	3029
q14	264	274	254	254
q15	545	501	499	499
q16	616	653	615	615
q17	1106	1460	1376	1376
q18	7159	7226	7053	7053
q19	874	827	831	827
q20	1942	1982	1863	1863
q21	4755	4358	4344	4344
q22	1113	1039	1020	1020
Total cold run time: 51252 ms
Total hot run time: 49563 ms

@doris-robot
Copy link

doris-robot commented Dec 17, 2025

TPC-DS: Total hot run time: 178356 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 817347ae808de62750edd85a770b8020f942bcec, data reload: false

query5	4866	634	434	434
query6	335	240	234	234
query7	4220	462	278	278
query8	306	250	235	235
query9	8787	2548	2549	2548
query10	535	366	326	326
query11	15138	14904	15044	14904
query12	176	115	110	110
query13	1288	493	373	373
query14	6262	3029	2743	2743
query14_1	2642	2632	2672	2632
query15	213	199	179	179
query16	943	465	477	465
query17	1145	713	593	593
query18	2576	442	353	353
query19	245	236	209	209
query20	120	114	113	113
query21	219	140	115	115
query22	4090	3993	3961	3961
query23	16370	16421	16285	16285
query23_1	16439	16160	16160	16160
query24	7346	1662	1246	1246
query24_1	1264	1205	1256	1205
query25	587	538	407	407
query26	1240	274	159	159
query27	2764	462	310	310
query28	4488	2132	2126	2126
query29	781	542	438	438
query30	315	247	217	217
query31	819	710	622	622
query32	77	69	69	69
query33	543	340	278	278
query34	930	911	530	530
query35	781	824	725	725
query36	862	893	816	816
query37	129	94	79	79
query38	2828	2860	2868	2860
query39	760	739	706	706
query39_1	704	715	860	715
query40	227	143	122	122
query41	65	63	61	61
query42	109	105	105	105
query43	429	447	390	390
query44	1343	744	778	744
query45	192	189	179	179
query46	878	983	615	615
query47	1667	1684	1626	1626
query48	317	331	246	246
query49	613	457	350	350
query50	668	304	211	211
query51	3867	3903	3857	3857
query52	109	109	98	98
query53	331	349	293	293
query54	285	256	251	251
query55	83	77	78	77
query56	299	309	294	294
query57	1161	1131	1068	1068
query58	282	258	255	255
query59	2394	2447	2396	2396
query60	332	316	295	295
query61	167	156	156	156
query62	725	660	629	629
query63	333	292	300	292
query64	4986	1337	1025	1025
query65	4030	3965	3956	3956
query66	1433	429	326	326
query67	14942	14899	14683	14683
query68	3222	1066	745	745
query69	483	360	311	311
query70	1057	1021	963	963
query71	351	305	284	284
query72	5889	5066	4890	4890
query73	652	542	305	305
query74	8828	8710	8297	8297
query75	3187	3158	2796	2796
query76	3800	1122	725	725
query77	526	407	300	300
query78	9361	9670	8857	8857
query79	1065	874	622	622
query80	1535	654	564	564
query81	559	273	241	241
query82	426	134	102	102
query83	362	257	242	242
query84	256	118	101	101
query85	977	507	475	475
query86	384	277	290	277
query87	3051	2998	2895	2895
query88	3256	2291	2303	2291
query89	456	426	393	393
query90	1961	163	151	151
query91	171	181	160	160
query92	69	62	66	62
query93	1103	924	579	579
query94	522	304	280	280
query95	572	327	365	327
query96	603	460	209	209
query97	2261	2336	2207	2207
query98	203	196	189	189
query99	1276	1327	1255	1255
Total cold run time: 254612 ms
Total hot run time: 178356 ms

@doris-robot
Copy link

doris-robot commented Dec 17, 2025

ClickBench: Total hot run time: 27.45 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 817347ae808de62750edd85a770b8020f942bcec, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.25	0.09	0.08
query4	1.60	0.11	0.11
query5	0.27	0.25	0.26
query6	1.17	0.63	0.64
query7	0.03	0.02	0.03
query8	0.05	0.04	0.05
query9	0.59	0.49	0.52
query10	0.55	0.56	0.56
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.62	0.59	0.60
query14	0.97	0.99	0.97
query15	0.81	0.81	0.80
query16	0.41	0.38	0.41
query17	1.06	1.02	0.99
query18	0.23	0.22	0.22
query19	1.84	1.73	1.72
query20	0.02	0.02	0.02
query21	15.44	0.26	0.14
query22	4.95	0.06	0.05
query23	15.97	0.28	0.11
query24	1.45	0.60	0.46
query25	0.11	0.06	0.05
query26	0.15	0.13	0.12
query27	0.06	0.10	0.05
query28	3.71	1.22	1.04
query29	12.60	4.17	3.27
query30	0.28	0.14	0.11
query31	2.81	0.61	0.40
query32	3.23	0.55	0.45
query33	2.97	3.06	3.06
query34	16.95	5.12	4.52
query35	4.56	4.62	4.61
query36	0.66	0.51	0.49
query37	0.12	0.06	0.07
query38	0.07	0.04	0.03
query39	0.04	0.02	0.03
query40	0.17	0.14	0.13
query41	0.09	0.03	0.03
query42	0.05	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.4 s
Total hot run time: 27.45 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 18, 2025

BE Regression && UT Coverage Report

Increment line coverage 94.44% (34/36) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.22% (24955/34553)
Line Coverage 58.95% (261861/444227)
Region Coverage 53.76% (217358/404299)
Branch Coverage 55.33% (93227/168478)

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 18, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 18, 2025

PR approved by at least one committer and no changes requested.

@hello-stephen
Copy link
Contributor

hello-stephen commented Dec 18, 2025

BE Regression && UT Coverage Report

Increment line coverage 95.00% (38/40) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.24% (24961/34553)
Line Coverage 58.96% (261931/444227)
Region Coverage 53.77% (217411/404299)
Branch Coverage 55.36% (93263/168478)

@mrhhsg mrhhsg merged commit 89d2308 into apache:master Dec 18, 2025
26 of 28 checks passed
@mrhhsg mrhhsg deleted the fix_compaction branch December 18, 2025 03:38
github-actions bot pushed a commit that referenced this pull request Dec 19, 2025
@mrhhsg
[fix](olap) Should erase the segment footer cache after compaction (#…
07c2e07 
…59101)

### What problem does this PR solve?

The cache should be erased after compaction.

During compaction, segment data files are renamed. The renamed files may
hit the footer cache of the previous files (since the cache key is
composed of the file path and file size), so these caches need to be
invalidated.

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Dec 19, 2025
Merged
yiguolei pushed a commit that referenced this pull request Dec 19, 2025
@github-actions @mrhhsg
branch-4.0: [fix](olap) Should erase the segment footer cache after c…
1aa4dc5 
…ompaction #59101 (#59172)

Cherry-picked from #59101

Co-authored-by: Jerry Hu <hushenggang@selectdb.com>
mrhhsg added a commit to mrhhsg/doris that referenced this pull request Dec 19, 2025
@mrhhsg
[fix](olap) Should erase the segment footer cache after compaction (a…
e35f248 
…pache#59101)

The cache should be erased after compaction.

During compaction, segment data files are renamed. The renamed files may
hit the footer cache of the previous files (since the cache key is
composed of the file path and file size), so these caches need to be
invalidated.

Related PR: #xxx

Problem Summary:

None

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@mrhhsg mrhhsg mentioned this pull request Dec 19, 2025
Merged
mrhhsg added a commit to mrhhsg/doris that referenced this pull request Dec 19, 2025
@mrhhsg
[fix](olap) Should erase the segment footer cache after compaction (a…
7cfbf6a 
…pache#59101)

The cache should be erased after compaction.

During compaction, segment data files are renamed. The renamed files may
hit the footer cache of the previous files (since the cache key is
composed of the file path and file size), so these caches need to be
invalidated.

Related PR: #xxx

Problem Summary:

None

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
mrhhsg added a commit to mrhhsg/doris that referenced this pull request Dec 19, 2025
@mrhhsg
[fix](olap) Should erase the segment footer cache after compaction (a…
d93c438 
…pache#59101)

The cache should be erased after compaction.

During compaction, segment data files are renamed. The renamed files may
hit the footer cache of the previous files (since the cache key is
composed of the file path and file size), so these caches need to be
invalidated.

Related PR: #xxx

Problem Summary:

None

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
morrySnow pushed a commit that referenced this pull request Dec 22, 2025
@mrhhsg
branch-3.1: [fix](olap) Should erase the segment footer cache after c…
ab87b37 
…ompaction #59101 (#59185)

picked from #59101
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liaoxin01 liaoxin01 liaoxin01 approved these changes

+1 more reviewer

@zhiqiang-hhhh zhiqiang-hhhh zhiqiang-hhhh approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.4-merged dev/4.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@mrhhsg @hello-stephen @doris-robot @liaoxin01 @zhiqiang-hhhh @yiguolei @morrySnow