Skip to content

[fix](inverted index) fix index memeory leak for inverted index #53235

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
hello-stephen merged 8 commits into from
Jul 16, 2025
Merged

[fix](inverted index) fix index memeory leak for inverted index #53235

hello-stephen merged 8 commits into from
Jul 16, 2025

Conversation

@airborne12
Copy link
Member

@airborne12 airborne12 commented Jul 14, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 14, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@airborne12
Copy link
Member Author

airborne12 commented Jul 14, 2025

run buildall

@airborne12
Copy link
Member Author

airborne12 commented Jul 15, 2025

run buildall

@airborne12
Copy link
Member Author

airborne12 commented Jul 15, 2025

run buildall

@airborne12
Copy link
Member Author

airborne12 commented Jul 15, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

TPC-H: Total hot run time: 33690 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e8be39e3bd348db1f591d2494be02fad6e9bf864, data reload: false

------ Round 1 ----------------------------------
q1	17579	5319	5099	5099
q2	1958	294	194	194
q3	10271	1263	738	738
q4	10226	1004	556	556
q5	7495	2455	2377	2377
q6	181	159	127	127
q7	929	773	599	599
q8	9319	1317	1204	1204
q9	6787	5241	5143	5143
q10	6933	2385	1989	1989
q11	483	289	270	270
q12	356	373	215	215
q13	17756	3715	3095	3095
q14	224	239	214	214
q15	542	463	492	463
q16	420	437	371	371
q17	609	897	368	368
q18	7570	7352	7172	7172
q19	1087	946	586	586
q20	347	348	221	221
q21	4293	3183	2372	2372
q22	361	329	317	317
Total cold run time: 105726 ms
Total hot run time: 33690 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5198	5162	5191	5162
q2	249	332	224	224
q3	2179	2703	2309	2309
q4	1368	1789	1378	1378
q5	4267	4424	4673	4424
q6	214	167	125	125
q7	2006	1980	1786	1786
q8	2627	2662	2686	2662
q9	7384	7292	7328	7292
q10	3180	3259	2870	2870
q11	588	520	498	498
q12	732	776	668	668
q13	3675	4103	3423	3423
q14	305	321	302	302
q15	527	471	474	471
q16	506	541	468	468
q17	1208	1581	1437	1437
q18	8050	7743	7625	7625
q19	860	925	1108	925
q20	1988	2098	1899	1899
q21	5026	4706	4552	4552
q22	644	626	566	566
Total cold run time: 52781 ms
Total hot run time: 51066 ms

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

TPC-DS: Total hot run time: 187801 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e8be39e3bd348db1f591d2494be02fad6e9bf864, data reload: false

query1	1008	370	382	370
query2	6499	1712	1733	1712
query3	6741	216	222	216
query4	26316	23722	23163	23163
query5	4384	583	452	452
query6	319	215	221	215
query7	4626	499	299	299
query8	273	225	209	209
query9	8624	2642	2645	2642
query10	469	322	271	271
query11	15895	15094	14860	14860
query12	152	109	99	99
query13	1657	543	409	409
query14	9033	5886	5891	5886
query15	213	194	180	180
query16	7209	665	438	438
query17	1210	741	625	625
query18	1990	417	309	309
query19	204	191	168	168
query20	122	119	114	114
query21	212	123	110	110
query22	4835	4908	4710	4710
query23	34862	33327	33335	33327
query24	8477	2341	2377	2341
query25	519	471	379	379
query26	1245	265	147	147
query27	2773	502	343	343
query28	4262	2126	2082	2082
query29	736	545	435	435
query30	277	223	194	194
query31	916	847	764	764
query32	72	60	62	60
query33	576	368	297	297
query34	805	851	540	540
query35	768	819	729	729
query36	963	980	890	890
query37	112	92	73	73
query38	4150	4130	4078	4078
query39	1479	1441	1441	1441
query40	206	113	105	105
query41	82	53	51	51
query42	125	104	108	104
query43	517	517	495	495
query44	1319	828	827	827
query45	173	171	166	166
query46	851	1004	625	625
query47	1770	1840	1767	1767
query48	383	421	310	310
query49	732	476	377	377
query50	636	687	427	427
query51	5545	5590	5362	5362
query52	109	103	98	98
query53	219	267	179	179
query54	570	562	495	495
query55	85	82	86	82
query56	297	299	282	282
query57	1206	1196	1130	1130
query58	264	253	254	253
query59	2777	2758	2640	2640
query60	327	316	297	297
query61	129	149	125	125
query62	755	700	659	659
query63	220	178	190	178
query64	4293	972	643	643
query65	4260	4171	4225	4171
query66	1149	405	311	311
query67	15977	16176	15565	15565
query68	9289	896	545	545
query69	472	310	261	261
query70	1212	1119	1168	1119
query71	478	340	306	306
query72	5279	4740	4709	4709
query73	704	592	347	347
query74	9200	9120	8825	8825
query75	4309	3186	2750	2750
query76	4293	1139	726	726
query77	970	390	305	305
query78	10112	10117	9423	9423
query79	2571	785	571	571
query80	628	502	445	445
query81	493	249	219	219
query82	501	131	99	99
query83	277	252	234	234
query84	291	104	88	88
query85	779	361	326	326
query86	414	310	299	299
query87	4449	4451	4351	4351
query88	3243	2277	2250	2250
query89	393	310	288	288
query90	1881	203	202	202
query91	137	140	114	114
query92	72	59	56	56
query93	1904	971	588	588
query94	666	411	313	313
query95	372	364	286	286
query96	501	565	277	277
query97	2759	2791	2644	2644
query98	227	203	208	203
query99	1486	1399	1331	1331
Total cold run time: 280321 ms
Total hot run time: 187801 ms

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

ClickBench: Total hot run time: 32.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e8be39e3bd348db1f591d2494be02fad6e9bf864, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.24	0.07	0.07
query4	1.62	0.11	0.10
query5	0.42	0.46	0.41
query6	1.18	0.66	0.65
query7	0.03	0.02	0.02
query8	0.05	0.04	0.03
query9	0.61	0.53	0.51
query10	0.58	0.57	0.57
query11	0.15	0.11	0.11
query12	0.15	0.12	0.11
query13	0.63	0.61	0.61
query14	0.80	0.82	0.82
query15	0.90	0.89	0.85
query16	0.39	0.38	0.41
query17	1.09	1.04	1.07
query18	0.23	0.22	0.21
query19	1.98	1.80	1.83
query20	0.02	0.01	0.02
query21	15.40	0.89	0.54
query22	0.77	1.42	0.67
query23	14.76	1.32	0.64
query24	7.07	1.88	0.91
query25	0.50	0.32	0.10
query26	0.57	0.17	0.14
query27	0.07	0.05	0.05
query28	9.27	0.81	0.44
query29	12.53	4.00	3.31
query30	3.04	2.99	2.91
query31	2.82	0.58	0.39
query32	3.26	0.57	0.47
query33	3.13	3.19	3.10
query34	15.97	5.37	4.87
query35	4.86	4.91	4.87
query36	0.67	0.52	0.49
query37	0.09	0.07	0.07
query38	0.06	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.14
query41	0.08	0.03	0.04
query42	0.03	0.02	0.03
query43	0.04	0.04	0.03
Total cold run time: 106.38 s
Total hot run time: 32.63 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 15, 2025

BE UT Coverage Report

Increment line coverage 100.00% (10/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.52% (15741/27365)
Line Coverage 46.20% (140380/303852)
Region Coverage 35.45% (104591/295006)
Branch Coverage 38.07% (46097/121073)

@airborne12
Copy link
Member Author

airborne12 commented Jul 15, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

TPC-H: Total hot run time: 32944 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7ee62e8175cd423e27a3834b07a73ee9ab50184c, data reload: false

------ Round 1 ----------------------------------
q1	17578	5193	5053	5053
q2	1944	283	187	187
q3	10293	1299	699	699
q4	10246	1023	507	507
q5	8135	2236	2387	2236
q6	208	163	126	126
q7	889	748	600	600
q8	9332	1294	1011	1011
q9	7230	5049	5099	5049
q10	6874	2381	1947	1947
q11	473	301	266	266
q12	342	351	212	212
q13	17759	3686	3106	3106
q14	243	225	217	217
q15	568	474	471	471
q16	420	431	372	372
q17	579	885	357	357
q18	7519	7261	7191	7191
q19	1289	935	531	531
q20	326	347	224	224
q21	3708	2523	2296	2296
q22	353	314	286	286
Total cold run time: 106308 ms
Total hot run time: 32944 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5138	5087	5142	5087
q2	238	319	217	217
q3	2165	2668	2287	2287
q4	1346	1773	1302	1302
q5	4234	4176	4557	4176
q6	216	166	126	126
q7	2123	1978	1806	1806
q8	2663	2694	2541	2541
q9	7272	7290	7265	7265
q10	3106	3322	2859	2859
q11	578	519	501	501
q12	688	770	628	628
q13	3614	3986	3250	3250
q14	296	326	281	281
q15	550	499	491	491
q16	437	488	443	443
q17	1201	1554	1589	1554
q18	7698	7826	7514	7514
q19	800	822	898	822
q20	1927	1952	1816	1816
q21	4716	4366	4293	4293
q22	635	592	534	534
Total cold run time: 51641 ms
Total hot run time: 49793 ms

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

TPC-DS: Total hot run time: 188809 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7ee62e8175cd423e27a3834b07a73ee9ab50184c, data reload: false

query1	1015	396	394	394
query2	6502	1618	1623	1618
query3	6747	212	210	210
query4	26366	24117	23679	23679
query5	4308	576	444	444
query6	311	226	207	207
query7	4633	488	284	284
query8	263	222	215	215
query9	8636	2602	2624	2602
query10	479	348	285	285
query11	15640	15112	14874	14874
query12	159	109	105	105
query13	1654	530	406	406
query14	8625	6197	6163	6163
query15	216	203	184	184
query16	7760	436	262	262
query17	1324	718	615	615
query18	2018	412	307	307
query19	206	195	161	161
query20	127	120	120	120
query21	213	142	114	114
query22	4251	4220	4180	4180
query23	34324	33449	33376	33376
query24	8574	2370	2378	2370
query25	518	482	407	407
query26	1222	268	147	147
query27	2753	511	344	344
query28	4306	2137	2087	2087
query29	716	541	441	441
query30	284	235	188	188
query31	937	815	751	751
query32	73	67	67	67
query33	571	338	288	288
query34	794	855	527	527
query35	613	639	567	567
query36	953	1028	894	894
query37	115	101	77	77
query38	4291	4146	4256	4146
query39	1682	1430	1467	1430
query40	214	123	111	111
query41	56	54	53	53
query42	121	107	111	107
query43	515	511	472	472
query44	1323	826	823	823
query45	198	174	166	166
query46	830	1039	633	633
query47	1758	1812	1744	1744
query48	383	418	302	302
query49	743	448	384	384
query50	626	704	430	430
query51	5507	5552	5321	5321
query52	114	113	95	95
query53	221	262	185	185
query54	574	565	491	491
query55	80	79	82	79
query56	304	317	312	312
query57	1195	1196	1157	1157
query58	270	252	258	252
query59	2618	2663	2485	2485
query60	321	322	302	302
query61	122	138	121	121
query62	788	728	632	632
query63	221	195	193	193
query64	4322	1204	892	892
query65	4279	4181	4226	4181
query66	1091	414	327	327
query67	15795	15858	15447	15447
query68	8134	884	532	532
query69	506	307	268	268
query70	1250	1123	1141	1123
query71	465	326	306	306
query72	5793	4668	4757	4668
query73	695	590	357	357
query74	8962	9042	9076	9042
query75	3812	3228	2820	2820
query76	3618	1157	704	704
query77	798	381	294	294
query78	10967	11241	10393	10393
query79	1952	902	594	594
query80	588	513	434	434
query81	467	266	226	226
query82	432	126	106	106
query83	250	249	232	232
query84	242	101	84	84
query85	846	366	316	316
query86	337	302	282	282
query87	4436	4400	4267	4267
query88	3322	2316	2297	2297
query89	386	317	294	294
query90	1924	222	219	219
query91	139	140	111	111
query92	72	61	60	60
query93	1068	963	595	595
query94	687	319	199	199
query95	380	297	284	284
query96	508	571	284	284
query97	2753	2848	2706	2706
query98	234	211	200	200
query99	1673	1410	1279	1279
Total cold run time: 276464 ms
Total hot run time: 188809 ms

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

ClickBench: Total hot run time: 32.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7ee62e8175cd423e27a3834b07a73ee9ab50184c, data reload: false

query1	0.05	0.04	0.03
query2	0.08	0.04	0.04
query3	0.24	0.08	0.08
query4	1.61	0.11	0.11
query5	0.42	0.42	0.40
query6	1.18	0.67	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.60	0.53	0.51
query10	0.56	0.56	0.56
query11	0.17	0.11	0.11
query12	0.15	0.11	0.11
query13	0.63	0.60	0.61
query14	0.79	0.80	0.84
query15	0.91	0.88	0.87
query16	0.39	0.38	0.39
query17	1.06	1.05	1.09
query18	0.23	0.21	0.21
query19	1.92	1.83	1.83
query20	0.02	0.01	0.01
query21	15.40	0.90	0.53
query22	0.74	1.23	0.68
query23	14.87	1.30	0.62
query24	7.26	1.25	0.65
query25	0.47	0.20	0.16
query26	0.55	0.18	0.14
query27	0.06	0.05	0.06
query28	9.96	0.85	0.45
query29	12.52	3.95	3.36
query30	3.05	3.00	2.98
query31	2.81	0.58	0.38
query32	3.24	0.56	0.47
query33	3.14	3.07	3.19
query34	15.99	5.36	4.81
query35	4.85	4.80	4.79
query36	0.71	0.51	0.49
query37	0.09	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.15	0.13
query41	0.08	0.02	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.18 s
Total hot run time: 32.36 s

@airborne12
Copy link
Member Author

airborne12 commented Jul 15, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

TPC-H: Total hot run time: 33263 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 351ccc0f13e5ee996fa76162f40c495c1d9530b9, data reload: false

------ Round 1 ----------------------------------
q1	17565	5204	5043	5043
q2	1941	280	179	179
q3	10295	1259	717	717
q4	10268	991	546	546
q5	8112	2429	2326	2326
q6	172	162	125	125
q7	892	748	615	615
q8	9306	1253	1121	1121
q9	6962	5189	5113	5113
q10	6915	2357	1966	1966
q11	469	291	273	273
q12	341	347	214	214
q13	17773	3664	3023	3023
q14	224	227	217	217
q15	535	477	473	473
q16	422	417	368	368
q17	606	871	374	374
q18	7538	7231	7156	7156
q19	1209	934	544	544
q20	354	337	226	226
q21	3906	3189	2345	2345
q22	364	317	299	299
Total cold run time: 106169 ms
Total hot run time: 33263 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5158	5107	5106	5106
q2	240	323	225	225
q3	2131	2674	2328	2328
q4	1341	1816	1322	1322
q5	4199	4292	4468	4292
q6	215	169	129	129
q7	1975	1942	1842	1842
q8	2599	2583	2602	2583
q9	7331	7378	7261	7261
q10	3144	3306	2892	2892
q11	626	582	500	500
q12	690	828	622	622
q13	3496	3934	3339	3339
q14	301	291	282	282
q15	521	482	475	475
q16	472	492	467	467
q17	1148	1588	1388	1388
q18	7910	7727	7652	7652
q19	784	789	823	789
q20	1901	1965	1917	1917
q21	4647	4291	4239	4239
q22	615	562	578	562
Total cold run time: 51444 ms
Total hot run time: 50212 ms

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

TPC-DS: Total hot run time: 188930 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 351ccc0f13e5ee996fa76162f40c495c1d9530b9, data reload: false

query1	1013	381	409	381
query2	6630	1649	1651	1649
query3	6749	214	214	214
query4	26688	23808	23352	23352
query5	4360	577	423	423
query6	305	215	219	215
query7	4624	520	291	291
query8	262	216	215	215
query9	8618	2633	2657	2633
query10	460	336	284	284
query11	15309	15548	14796	14796
query12	164	104	105	104
query13	1638	541	398	398
query14	9731	6006	6006	6006
query15	208	189	169	169
query16	7578	429	248	248
query17	1338	687	576	576
query18	1992	381	304	304
query19	204	188	159	159
query20	143	116	116	116
query21	209	134	111	111
query22	4398	4427	4366	4366
query23	34027	33187	33267	33187
query24	8400	2356	2378	2356
query25	515	476	379	379
query26	1239	264	153	153
query27	2760	499	333	333
query28	4313	2119	2085	2085
query29	702	544	436	436
query30	288	222	190	190
query31	945	827	783	783
query32	74	65	66	65
query33	568	376	282	282
query34	802	841	514	514
query35	597	651	573	573
query36	952	996	898	898
query37	112	107	83	83
query38	4083	4142	4124	4124
query39	1518	1436	1456	1436
query40	210	118	106	106
query41	60	55	53	53
query42	121	107	110	107
query43	487	523	474	474
query44	1373	847	840	840
query45	175	170	162	162
query46	855	1029	633	633
query47	1797	1820	1751	1751
query48	385	417	320	320
query49	718	488	422	422
query50	646	686	418	418
query51	5447	5620	5521	5521
query52	117	108	97	97
query53	223	262	175	175
query54	561	560	501	501
query55	82	84	80	80
query56	320	290	301	290
query57	1222	1262	1114	1114
query58	274	258	271	258
query59	2541	2744	2607	2607
query60	348	335	326	326
query61	155	153	148	148
query62	786	710	667	667
query63	223	193	185	185
query64	4495	1333	951	951
query65	4263	4189	4179	4179
query66	1061	407	318	318
query67	16089	15841	15641	15641
query68	8572	920	540	540
query69	513	309	266	266
query70	1243	1141	1155	1141
query71	530	309	299	299
query72	5543	4641	4638	4638
query73	702	572	347	347
query74	8908	9135	9051	9051
query75	3971	3213	2719	2719
query76	3620	1149	757	757
query77	786	381	360	360
query78	10931	11166	10422	10422
query79	1881	851	587	587
query80	600	577	435	435
query81	472	261	235	235
query82	431	123	96	96
query83	252	245	227	227
query84	242	108	85	85
query85	789	371	310	310
query86	338	336	294	294
query87	4468	4449	4385	4385
query88	3232	2304	2287	2287
query89	392	313	283	283
query90	1965	220	214	214
query91	141	154	123	123
query92	79	59	53	53
query93	1044	979	608	608
query94	689	319	196	196
query95	375	294	291	291
query96	514	570	284	284
query97	2707	2771	2666	2666
query98	225	216	210	210
query99	1470	1394	1290	1290
Total cold run time: 277175 ms
Total hot run time: 188930 ms

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

ClickBench: Total hot run time: 32.33 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 351ccc0f13e5ee996fa76162f40c495c1d9530b9, data reload: false

query1	0.05	0.03	0.04
query2	0.08	0.04	0.04
query3	0.26	0.08	0.08
query4	1.64	0.11	0.11
query5	0.45	0.43	0.44
query6	1.17	0.66	0.66
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.60	0.52	0.50
query10	0.58	0.57	0.56
query11	0.16	0.11	0.10
query12	0.14	0.12	0.11
query13	0.62	0.61	0.62
query14	0.82	0.78	0.82
query15	0.91	0.87	0.88
query16	0.39	0.38	0.39
query17	1.05	1.06	1.10
query18	0.22	0.21	0.21
query19	1.96	1.84	1.83
query20	0.02	0.01	0.01
query21	15.41	0.86	0.55
query22	0.76	1.16	0.77
query23	14.75	1.37	0.61
query24	7.21	1.07	0.62
query25	0.56	0.19	0.06
query26	0.54	0.16	0.14
query27	0.06	0.06	0.05
query28	9.83	0.94	0.44
query29	12.57	4.08	3.35
query30	2.99	2.95	2.90
query31	2.82	0.60	0.39
query32	3.23	0.54	0.47
query33	3.19	3.10	3.16
query34	16.07	5.35	4.80
query35	4.83	4.82	4.86
query36	0.70	0.50	0.50
query37	0.09	0.07	0.07
query38	0.06	0.04	0.04
query39	0.04	0.02	0.03
query40	0.17	0.15	0.14
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.22 s
Total hot run time: 32.33 s

@airborne12
Copy link
Member Author

airborne12 commented Jul 15, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

TPC-H: Total hot run time: 33256 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 13254a57d0f1e4316c95c4cb317d1a6b300af4c4, data reload: false

------ Round 1 ----------------------------------
q1	17581	5229	5120	5120
q2	1929	275	188	188
q3	10312	1243	738	738
q4	10213	974	530	530
q5	7833	2373	2324	2324
q6	206	162	127	127
q7	907	736	607	607
q8	9309	1278	1065	1065
q9	7108	5098	5050	5050
q10	6944	2396	1957	1957
q11	477	293	274	274
q12	332	348	216	216
q13	17759	3689	3099	3099
q14	230	219	213	213
q15	565	483	472	472
q16	430	421	367	367
q17	600	860	365	365
q18	7533	7169	7056	7056
q19	1337	938	572	572
q20	351	324	223	223
q21	3866	3090	2394	2394
q22	360	332	299	299
Total cold run time: 106182 ms
Total hot run time: 33256 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5251	5108	5123	5108
q2	243	326	223	223
q3	2126	2626	2309	2309
q4	1342	1765	1355	1355
q5	4178	4339	4461	4339
q6	219	178	124	124
q7	2038	1951	1793	1793
q8	2632	2512	2476	2476
q9	7471	7302	7327	7302
q10	3064	3290	2823	2823
q11	568	539	502	502
q12	684	774	841	774
q13	3469	3857	3375	3375
q14	297	304	284	284
q15	521	479	475	475
q16	453	498	437	437
q17	1186	1576	1380	1380
q18	8043	7731	7619	7619
q19	786	768	815	768
q20	1926	1937	1842	1842
q21	4711	4322	4275	4275
q22	633	614	543	543
Total cold run time: 51841 ms
Total hot run time: 50126 ms

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

TPC-DS: Total hot run time: 188052 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 13254a57d0f1e4316c95c4cb317d1a6b300af4c4, data reload: false

query1	1006	382	391	382
query2	6538	1681	1640	1640
query3	6779	215	214	214
query4	26951	23834	23572	23572
query5	4345	590	423	423
query6	297	212	223	212
query7	4622	489	290	290
query8	273	230	216	216
query9	8645	2627	2613	2613
query10	482	345	275	275
query11	15607	15067	14845	14845
query12	149	108	99	99
query13	1642	507	389	389
query14	8562	6021	6080	6021
query15	209	192	175	175
query16	7470	421	247	247
query17	1360	695	554	554
query18	1990	398	287	287
query19	184	198	153	153
query20	122	118	117	117
query21	212	123	107	107
query22	4101	4162	4035	4035
query23	33784	33183	33200	33183
query24	8507	2330	2364	2330
query25	556	460	391	391
query26	1231	264	143	143
query27	2772	525	350	350
query28	4327	2144	2106	2106
query29	736	547	441	441
query30	286	222	188	188
query31	920	847	742	742
query32	74	64	68	64
query33	547	340	286	286
query34	797	836	526	526
query35	624	667	570	570
query36	944	985	898	898
query37	115	105	79	79
query38	4120	4183	4023	4023
query39	1490	1418	1430	1418
query40	210	114	101	101
query41	56	55	51	51
query42	124	109	102	102
query43	506	487	467	467
query44	1308	823	824	823
query45	176	174	174	174
query46	838	1010	629	629
query47	1818	1830	1751	1751
query48	378	428	310	310
query49	731	482	397	397
query50	625	749	427	427
query51	5411	5608	5494	5494
query52	104	109	94	94
query53	217	247	183	183
query54	617	553	494	494
query55	82	81	82	81
query56	303	305	274	274
query57	1172	1219	1130	1130
query58	261	253	260	253
query59	2585	2593	2472	2472
query60	334	321	318	318
query61	132	122	119	119
query62	809	707	654	654
query63	214	199	185	185
query64	4413	1161	820	820
query65	4267	4163	4163	4163
query66	1103	426	373	373
query67	16063	15669	15576	15576
query68	5661	881	527	527
query69	508	307	272	272
query70	1196	1141	1139	1139
query71	392	319	308	308
query72	5669	4750	4849	4750
query73	658	670	356	356
query74	8866	9076	9047	9047
query75	3173	3226	2705	2705
query76	3155	1146	703	703
query77	488	379	305	305
query78	10892	11138	10213	10213
query79	996	818	580	580
query80	830	542	487	487
query81	503	263	225	225
query82	411	133	101	101
query83	265	261	244	244
query84	249	112	94	94
query85	853	461	313	313
query86	344	339	298	298
query87	4483	4419	4396	4396
query88	2875	2285	2304	2285
query89	379	320	289	289
query90	1759	220	222	220
query91	142	140	111	111
query92	64	65	63	63
query93	1017	960	593	593
query94	685	319	199	199
query95	385	296	292	292
query96	505	559	280	280
query97	2738	2739	2662	2662
query98	234	215	202	202
query99	1315	1388	1274	1274
Total cold run time: 269923 ms
Total hot run time: 188052 ms

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

ClickBench: Total hot run time: 32.21 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 13254a57d0f1e4316c95c4cb317d1a6b300af4c4, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.24	0.07	0.07
query4	1.63	0.11	0.12
query5	0.42	0.46	0.42
query6	1.17	0.66	0.66
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.60	0.51	0.54
query10	0.58	0.57	0.56
query11	0.16	0.11	0.11
query12	0.15	0.11	0.11
query13	0.62	0.60	0.61
query14	0.79	0.81	0.83
query15	0.88	0.89	0.87
query16	0.38	0.40	0.40
query17	1.05	1.02	1.04
query18	0.23	0.20	0.20
query19	1.92	1.85	1.87
query20	0.01	0.01	0.01
query21	15.41	0.87	0.52
query22	0.75	1.13	0.63
query23	15.08	1.35	0.61
query24	6.95	1.86	0.65
query25	0.48	0.34	0.09
query26	0.62	0.16	0.14
query27	0.07	0.06	0.05
query28	9.38	0.88	0.44
query29	12.56	4.03	3.32
query30	3.05	3.03	2.96
query31	2.82	0.57	0.39
query32	3.23	0.56	0.47
query33	3.04	3.09	3.10
query34	16.13	5.37	4.74
query35	4.82	4.83	4.84
query36	0.69	0.51	0.51
query37	0.09	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.15	0.14
query41	0.07	0.03	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 106.56 s
Total hot run time: 32.21 s

@doris-robot
Copy link

doris-robot commented Jul 15, 2025

BE UT Coverage Report

Increment line coverage 80.00% (20/25) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.52% (15743/27370)
Line Coverage 46.17% (140428/304181)
Region Coverage 35.48% (104719/295182)
Branch Coverage 38.07% (46151/121234)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 15, 2025

BE Regression && UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage
Line Coverage
Region Coverage
Branch Coverage

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 16, 2025

BE Regression && UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage
Line Coverage
Region Coverage
Branch Coverage

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 16, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 16, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 16, 2025

PR approved by anyone and no changes requested.

shuke987
shuke987 approved these changes Jul 16, 2025
View reviewed changes
Copy link
Collaborator

@shuke987 shuke987 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hello-stephen hello-stephen merged commit 0f5ca8c into apache:master Jul 16, 2025
28 of 31 checks passed
zzzxl1993
zzzxl1993 reviewed Jul 16, 2025
View reviewed changes
Copy link
Contributor

@zzzxl1993 zzzxl1993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@airborne12 airborne12 deleted the fix-memory-leak branch July 16, 2025 07:16
@airborne12 airborne12 mentioned this pull request Jul 17, 2025
Merged
@airborne12 airborne12 changed the title [fix](inverted index) fix index memeory leak for inverted index branch-3.1: [fix](inverted index) fix index memeory leak for inverted index Jul 17, 2025
@airborne12 airborne12 changed the title branch-3.1: [fix](inverted index) fix index memeory leak for inverted index [fix](inverted index) fix index memeory leak for inverted index Jul 17, 2025
morrySnow pushed a commit that referenced this pull request Jul 18, 2025
@airborne12
branch-3.1: [fix](inverted index) fix memory leak in inverted index p…
cbdb8e2 
…rocedure #53235 (#53425)

pick and refact from #53235
@airborne12 airborne12 mentioned this pull request Jul 23, 2025
Merged
@gavinchou gavinchou mentioned this pull request Aug 15, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zzzxl1993 zzzxl1993 zzzxl1993 left review comments

@hello-stephen hello-stephen hello-stephen approved these changes

+1 more reviewer

@shuke987 shuke987 shuke987 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.7-merged dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@airborne12 @hello-stephen @doris-robot @zzzxl1993 @shuke987 @morrySnow