Skip to content

branch-3.0: [fix](inverted index) fix memory leak in inverted index procedure #53487

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 3 commits into from
Jul 18, 2025
Merged

branch-3.0: [fix](inverted index) fix memory leak in inverted index procedure #53487

dataroaring merged 3 commits into from
Jul 18, 2025

Conversation

@airborne12
Copy link
Member

@airborne12 airborne12 commented Jul 17, 2025 •
edited
Loading

pick from #53235

@airborne12 airborne12 requested a review from dataroaring as a code owner July 17, 2025 15:05
@airborne12
Copy link
Member Author

airborne12 commented Jul 17, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@airborne12
Copy link
Member Author

airborne12 commented Jul 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

TPC-H: Total hot run time: 40189 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8aa03ba92116e14d32bbdfb1a0f06347335149c7, data reload: false

------ Round 1 ----------------------------------
q1	17665	6944	6678	6678
q2	2097	205	172	172
q3	10608	1874	1195	1195
q4	10444	783	685	685
q5	7721	2886	2872	2872
q6	215	131	133	131
q7	979	625	599	599
q8	9336	1994	2040	1994
q9	6653	6428	6443	6428
q10	6969	2261	2332	2261
q11	459	259	263	259
q12	397	221	215	215
q13	17761	2989	2992	2989
q14	240	215	211	211
q15	510	465	466	465
q16	480	398	380	380
q17	987	538	583	538
q18	7522	6774	6776	6774
q19	1392	1077	992	992
q20	473	199	206	199
q21	3895	3196	3203	3196
q22	1103	958	956	956
Total cold run time: 107906 ms
Total hot run time: 40189 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6637	6622	6598	6598
q2	326	234	227	227
q3	2988	2981	2980	2980
q4	2029	1876	1826	1826
q5	5725	5752	5760	5752
q6	206	129	130	129
q7	2280	1849	1844	1844
q8	3369	3526	3552	3526
q9	8844	8986	8864	8864
q10	3577	3526	3545	3526
q11	601	497	511	497
q12	793	597	620	597
q13	9168	3129	3171	3129
q14	300	288	267	267
q15	521	463	471	463
q16	500	444	446	444
q17	1856	1641	1636	1636
q18	8407	7812	7774	7774
q19	1677	1564	1531	1531
q20	2080	1894	1892	1892
q21	5171	5078	4887	4887
q22	1129	1041	1020	1020
Total cold run time: 68184 ms
Total hot run time: 59409 ms

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

TPC-DS: Total hot run time: 196028 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8aa03ba92116e14d32bbdfb1a0f06347335149c7, data reload: false

query1	1305	923	881	881
query2	6269	1952	1874	1874
query3	10838	4281	4438	4281
query4	32814	23911	23399	23399
query5	4230	448	440	440
query6	293	186	177	177
query7	3996	319	324	319
query8	305	228	220	220
query9	9496	2587	2569	2569
query10	478	266	256	256
query11	18500	15020	15319	15020
query12	156	108	106	106
query13	1565	444	411	411
query14	8807	7539	6776	6776
query15	253	184	181	181
query16	8032	477	483	477
query17	1631	576	602	576
query18	2181	336	312	312
query19	277	166	166	166
query20	121	120	124	120
query21	212	116	107	107
query22	4719	4438	4466	4438
query23	35102	33874	34229	33874
query24	11403	2941	2935	2935
query25	683	415	417	415
query26	1413	173	169	169
query27	2810	358	371	358
query28	7444	2136	2146	2136
query29	906	476	478	476
query30	276	164	166	164
query31	1051	798	867	798
query32	95	59	63	59
query33	787	313	310	310
query34	934	509	526	509
query35	886	755	719	719
query36	1140	920	945	920
query37	134	71	69	69
query38	4050	3991	4051	3991
query39	1532	1482	1467	1467
query40	266	95	104	95
query41	48	48	46	46
query42	111	99	104	99
query43	501	475	470	470
query44	1237	815	807	807
query45	191	167	161	161
query46	1178	720	725	720
query47	2011	1902	1922	1902
query48	480	376	380	376
query49	943	400	417	400
query50	828	435	425	425
query51	7466	7292	7243	7243
query52	102	90	88	88
query53	267	184	180	180
query54	1291	484	478	478
query55	84	78	83	78
query56	282	259	245	245
query57	1347	1216	1216	1216
query58	236	235	239	235
query59	3139	3050	3022	3022
query60	289	262	273	262
query61	126	114	108	108
query62	844	680	719	680
query63	233	197	189	189
query64	5525	667	659	659
query65	3364	3280	3283	3280
query66	1325	301	296	296
query67	15884	15460	15554	15460
query68	5348	572	571	571
query69	431	258	262	258
query70	1157	1100	1115	1100
query71	341	258	246	246
query72	6223	4025	3977	3977
query73	764	351	367	351
query74	10589	9250	8974	8974
query75	3390	2627	2666	2627
query76	3105	1034	1128	1034
query77	413	275	269	269
query78	10567	9755	9595	9595
query79	1827	594	598	594
query80	1127	420	425	420
query81	531	225	237	225
query82	902	91	89	89
query83	227	151	142	142
query84	245	75	75	75
query85	1358	299	308	299
query86	423	284	299	284
query87	4365	4216	4274	4216
query88	3691	2403	2349	2349
query89	410	297	295	295
query90	2085	184	184	184
query91	182	146	151	146
query92	67	51	49	49
query93	2413	559	546	546
query94	980	302	288	288
query95	361	252	258	252
query96	618	287	277	277
query97	3303	3117	3112	3112
query98	219	206	202	202
query99	1500	1321	1287	1287
Total cold run time: 305884 ms
Total hot run time: 196028 ms

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

ClickBench: Total hot run time: 30.06 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8aa03ba92116e14d32bbdfb1a0f06347335149c7, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.64	0.11	0.10
query5	0.51	0.50	0.52
query6	1.14	0.72	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.55	0.50	0.49
query10	0.54	0.56	0.58
query11	0.14	0.10	0.10
query12	0.13	0.11	0.10
query13	0.61	0.60	0.60
query14	0.78	0.77	0.81
query15	0.84	0.82	0.82
query16	0.38	0.40	0.40
query17	1.01	1.07	0.99
query18	0.24	0.21	0.23
query19	1.85	1.85	1.82
query20	0.01	0.02	0.01
query21	15.40	0.57	0.55
query22	2.32	2.10	1.94
query23	16.76	1.16	0.86
query24	3.35	1.28	0.85
query25	0.36	0.17	0.08
query26	0.36	0.14	0.12
query27	0.05	0.04	0.04
query28	10.15	0.55	0.47
query29	12.54	3.21	3.17
query30	0.25	0.06	0.06
query31	2.88	0.39	0.37
query32	3.23	0.46	0.45
query33	2.92	3.03	3.04
query34	17.05	4.47	4.50
query35	4.51	4.49	4.52
query36	0.66	0.47	0.49
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.09	0.02	0.02
query42	0.04	0.02	0.03
query43	0.03	0.02	0.02
Total cold run time: 104.03 s
Total hot run time: 30.06 s

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

BE UT Coverage Report

Increment line coverage 80.56% (87/108) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 41.21% (10947/26565)
Line Coverage 31.96% (93837/293648)
Region Coverage 31.11% (48388/155559)
Branch Coverage 27.55% (24789/89978)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 17, 2025

BE Regression && UT Coverage Report

Increment line coverage 80.56% (87/108) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 50.33% (13197/26220)
Line Coverage 41.01% (120262/293254)
Region Coverage 38.70% (69723/180159)
Branch Coverage 33.18% (33745/101714)

dataroaring
dataroaring approved these changes Jul 18, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 088a1ad into apache:branch-3.0 Jul 18, 2025
23 of 25 checks passed
@airborne12 airborne12 deleted the pick_52894_to_origin_branch-3.0 branch July 23, 2025 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@airborne12 @hello-stephen @doris-robot @dataroaring