[fix](cloud compaction) horizontal compaction coredump due to tablet shard_ptr not initialized #58751

HonestManXin · 2025-12-05T03:54:36Z

#0  0x000055f11470ff29 in std::__shared_ptr<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2>::get (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1673

#1  std::__shared_ptr_access<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1370

#2  std::__shared_ptr_access<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2, false, false>::operator-> (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1364

#3  doris::BaseTablet::tablet_id (this=0x0) at /home/work/doris/be/src/olap/base_tablet.h:73

#4  doris::segment_v2::SegmentWriter::finalize (this=0x7fa160bde000, segment_file_size=0x7fa564dcc668, index_size=0x7fa564dcc5b8) at /home/work/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:1044

#5  0x000055f11442578c in doris::SegmentFlusher::_flush_segment_writer (this=0x7fa160b8a420, writer=..., flush_size=flush_size@entry=0x0) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:304

#6  0x000055f114426c4e in doris::SegmentFlusher::Writer::flush (this=<optimized out>) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:376

#7  doris::SegmentCreator::flush (this=0x7fa160b8a418) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:422

#8  0x000055f1143d4785 in doris::BaseBetaRowsetWriter::flush (this=<optimized out>) at /home/work/doris/be/src/olap/rowset/beta_rowset_writer.cpp:723

#9  0x000055f11436bf47 in doris::Merger::vmerge_rowsets (tablet=..., reader_type=<optimized out>, cur_tablet_schema=..., src_rowset_readers=..., dst_rowset_writer=0x7fa160b8a000, stats_output=0x7fa1df108298) at /home/work/doris/be/src/olap/merger.cpp:159

#10 0x000055f114345595 in doris::Compaction::merge_input_rowsets (this=this@entry=0x7fa1df108210) at /home/work/doris/be/src/olap/compaction.cpp:220

#11 0x000055f1143550d2 in doris::CloudCompactionMixin::execute_compact_impl (this=this@entry=0x7fa1df108210, permits=permits@entry=6) at /home/work/doris/be/src/olap/compaction.cpp:1491

#12 0x000055f114342bc1 in doris::CloudCompactionMixin::execute_compact (this=0x7fa1df108210) at /home/work/doris/be/src/olap/compaction.cpp:1620

#13 0x000055f11a22c6d8 in doris::CloudCumulativeCompaction::execute_compact (this=0x7fa1df108210) at /home/work/doris/be/src/cloud/cloud_cumulative_compaction.cpp:203

…shard_ptr not initialized

hello-stephen · 2025-12-05T03:55:13Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

HonestManXin · 2025-12-05T03:55:59Z

run buildall

github-actions · 2025-12-05T04:05:45Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-12-05T04:05:47Z

PR approved by anyone and no changes requested.

doris-robot · 2025-12-05T05:23:48Z

TPC-H: Total hot run time: 34592 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1542d53904849df55b538fcfceb895271359fa9b, data reload: false

------ Round 1 ----------------------------------
q1	17661	5022	4931	4931
q2	2110	318	198	198
q3	10192	1351	750	750
q4	10228	937	326	326
q5	7594	2401	2271	2271
q6	189	172	136	136
q7	966	772	639	639
q8	9349	1455	1190	1190
q9	7117	5368	5371	5368
q10	6885	2217	1793	1793
q11	522	311	289	289
q12	374	360	223	223
q13	17768	3751	3046	3046
q14	241	247	212	212
q15	601	521	519	519
q16	920	848	811	811
q17	705	815	507	507
q18	7743	7161	7052	7052
q19	1101	964	604	604
q20	381	347	224	224
q21	4041	3358	2558	2558
q22	1036	1023	945	945
Total cold run time: 107724 ms
Total hot run time: 34592 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5009	4948	4960	4948
q2	325	406	329	329
q3	2195	2690	2252	2252
q4	1323	1760	1290	1290
q5	4230	4657	4599	4599
q6	219	177	131	131
q7	2030	2001	1891	1891
q8	2722	2714	2570	2570
q9	7554	7641	7440	7440
q10	3123	3266	2803	2803
q11	584	520	490	490
q12	679	765	623	623
q13	3650	3892	3299	3299
q14	303	314	289	289
q15	570	523	515	515
q16	919	963	926	926
q17	1217	1435	1432	1432
q18	8079	7727	7486	7486
q19	850	852	881	852
q20	2056	2140	1932	1932
q21	4686	4295	4103	4103
q22	1119	1021	970	970
Total cold run time: 53442 ms
Total hot run time: 51170 ms

doris-robot · 2025-12-05T05:35:00Z

TPC-DS: Total hot run time: 180130 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1542d53904849df55b538fcfceb895271359fa9b, data reload: false

query5	5431	651	505	505
query6	348	227	212	212
query7	4654	483	274	274
query8	307	246	235	235
query9	8712	2654	2675	2654
query10	587	308	279	279
query11	15232	14968	14643	14643
query12	194	122	116	116
query13	1679	472	372	372
query14	6609	3303	3092	3092
query14_1	2998	2879	2971	2879
query15	209	195	195	195
query16	7644	478	443	443
query17	1181	703	585	585
query18	2053	425	324	324
query19	216	196	164	164
query20	134	119	119	119
query21	225	142	115	115
query22	3847	3888	3924	3888
query23	16490	16293	16052	16052
query23_1	16112	16083	16018	16018
query24	7164	1652	1218	1218
query24_1	1199	1199	1201	1199
query25	586	463	416	416
query26	1254	279	169	169
query27	2914	472	312	312
query28	4423	2181	2154	2154
query29	810	555	477	477
query30	308	248	217	217
query31	810	700	618	618
query32	89	73	69	69
query33	674	336	292	292
query34	872	889	551	551
query35	788	833	731	731
query36	894	905	836	836
query37	127	96	84	84
query38	3853	3849	3803	3803
query39	744	735	706	706
query39_1	691	703	707	703
query40	225	135	120	120
query41	68	63	61	61
query42	125	105	95	95
query43	451	422	400	400
query44	1333	756	757	756
query45	195	190	188	188
query46	915	963	606	606
query47	1675	1704	1622	1622
query48	395	317	230	230
query49	761	432	362	362
query50	717	308	246	246
query51	3961	3949	3864	3864
query52	129	98	89	89
query53	239	235	180	180
query54	318	267	263	263
query55	96	82	78	78
query56	358	296	287	287
query57	1151	1142	1107	1107
query58	291	256	262	256
query59	2412	2417	2326	2326
query60	377	321	303	303
query61	159	176	188	176
query62	787	691	654	654
query63	237	183	187	183
query64	4613	1164	890	890
query65	4038	3990	3956	3956
query66	1153	458	344	344
query67	15112	14815	14819	14815
query68	2733	983	697	697
query69	473	308	264	264
query70	1111	961	986	961
query71	384	295	279	279
query72	5860	4957	5083	4957
query73	521	532	313	313
query74	8541	8551	8580	8551
query75	3007	3030	2535	2535
query76	2340	1124	741	741
query77	377	404	302	302
query78	9398	9725	8916	8916
query79	1079	837	588	588
query80	1286	562	474	474
query81	543	282	244	244
query82	211	133	109	109
query83	375	283	256	256
query84	248	121	102	102
query85	932	487	441	441
query86	409	316	290	290
query87	4043	4071	3961	3961
query88	3000	2166	2166	2166
query89	399	319	288	288
query90	1857	170	163	163
query91	169	171	157	157
query92	70	72	62	62
query93	1152	1061	689	689
query94	759	320	295	295
query95	578	389	339	339
query96	560	490	221	221
query97	2612	2676	2598	2598
query98	247	210	195	195
query99	1330	1297	1227	1227
Total cold run time: 262031 ms
Total hot run time: 180130 ms

doris-robot · 2025-12-05T05:40:04Z

ClickBench: Total hot run time: 27.29 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1542d53904849df55b538fcfceb895271359fa9b, data reload: false

query1	0.05	0.04	0.05
query2	0.10	0.05	0.05
query3	0.26	0.08	0.08
query4	1.61	0.12	0.11
query5	0.27	0.26	0.26
query6	1.18	0.65	0.64
query7	0.04	0.03	0.03
query8	0.06	0.05	0.04
query9	0.55	0.52	0.49
query10	0.56	0.56	0.56
query11	0.16	0.11	0.12
query12	0.15	0.12	0.11
query13	0.62	0.61	0.62
query14	0.98	0.99	0.98
query15	0.82	0.79	0.79
query16	0.39	0.40	0.39
query17	1.00	1.08	1.01
query18	0.24	0.21	0.21
query19	1.91	1.82	1.86
query20	0.01	0.01	0.01
query21	15.47	0.29	0.14
query22	4.95	0.05	0.05
query23	16.14	0.28	0.10
query24	0.96	0.88	0.17
query25	0.08	0.05	0.05
query26	0.14	0.13	0.13
query27	0.05	0.08	0.08
query28	4.64	1.24	1.02
query29	12.59	4.04	3.25
query30	0.29	0.14	0.12
query31	2.81	0.64	0.40
query32	3.23	0.56	0.46
query33	3.03	3.05	3.04
query34	16.72	5.20	4.55
query35	4.57	4.55	4.56
query36	0.67	0.50	0.48
query37	0.11	0.06	0.06
query38	0.08	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.09	0.03	0.03
query42	0.05	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 97.88 s
Total hot run time: 27.29 s

freemandealer

LGTM

hello-stephen · 2025-12-05T07:15:52Z

BE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	53.29% (18638/34975)
Line Coverage	39.00% (172255/441655)
Region Coverage	33.63% (133353/396532)
Branch Coverage	34.58% (57371/165918)

hello-stephen · 2025-12-05T08:17:37Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	72.16% (24750/34297)
Line Coverage	58.87% (259804/441289)
Region Coverage	53.81% (215955/401346)
Branch Coverage	55.27% (92200/166817)

…shard_ptr not initialized (#58751) ``` #0 0x000055f11470ff29 in std::__shared_ptr<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2>::get (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1673 #1 std::__shared_ptr_access<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1370 #2 std::__shared_ptr_access<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2, false, false>::operator-> (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1364 #3 doris::BaseTablet::tablet_id (this=0x0) at /home/work/doris/be/src/olap/base_tablet.h:73 #4 doris::segment_v2::SegmentWriter::finalize (this=0x7fa160bde000, segment_file_size=0x7fa564dcc668, index_size=0x7fa564dcc5b8) at /home/work/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:1044 #5 0x000055f11442578c in doris::SegmentFlusher::_flush_segment_writer (this=0x7fa160b8a420, writer=..., flush_size=flush_size@entry=0x0) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:304 #6 0x000055f114426c4e in doris::SegmentFlusher::Writer::flush (this=<optimized out>) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:376 #7 doris::SegmentCreator::flush (this=0x7fa160b8a418) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:422 #8 0x000055f1143d4785 in doris::BaseBetaRowsetWriter::flush (this=<optimized out>) at /home/work/doris/be/src/olap/rowset/beta_rowset_writer.cpp:723 #9 0x000055f11436bf47 in doris::Merger::vmerge_rowsets (tablet=..., reader_type=<optimized out>, cur_tablet_schema=..., src_rowset_readers=..., dst_rowset_writer=0x7fa160b8a000, stats_output=0x7fa1df108298) at /home/work/doris/be/src/olap/merger.cpp:159 #10 0x000055f114345595 in doris::Compaction::merge_input_rowsets (this=this@entry=0x7fa1df108210) at /home/work/doris/be/src/olap/compaction.cpp:220 #11 0x000055f1143550d2 in doris::CloudCompactionMixin::execute_compact_impl (this=this@entry=0x7fa1df108210, permits=permits@entry=6) at /home/work/doris/be/src/olap/compaction.cpp:1491 #12 0x000055f114342bc1 in doris::CloudCompactionMixin::execute_compact (this=0x7fa1df108210) at /home/work/doris/be/src/olap/compaction.cpp:1620 #13 0x000055f11a22c6d8 in doris::CloudCumulativeCompaction::execute_compact (this=0x7fa1df108210) at /home/work/doris/be/src/cloud/cloud_cumulative_compaction.cpp:203 ```

…e to tablet shard_ptr not initialized #58751 (#58764) Cherry-picked from #58751 Co-authored-by: HonestManXin <HonestManXin@gmail.com>

…e to tablet shard_ptr not initialized #58751 (#58761) Cherry-picked from #58751 Co-authored-by: HonestManXin <HonestManXin@gmail.com>

…shard_ptr not initialized (apache#58751) ``` #0 0x000055f11470ff29 in std::__shared_ptr<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2>::get (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1673 apache#1 std::__shared_ptr_access<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1370 apache#2 std::__shared_ptr_access<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2, false, false>::operator-> (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1364 apache#3 doris::BaseTablet::tablet_id (this=0x0) at /home/work/doris/be/src/olap/base_tablet.h:73 apache#4 doris::segment_v2::SegmentWriter::finalize (this=0x7fa160bde000, segment_file_size=0x7fa564dcc668, index_size=0x7fa564dcc5b8) at /home/work/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:1044 apache#5 0x000055f11442578c in doris::SegmentFlusher::_flush_segment_writer (this=0x7fa160b8a420, writer=..., flush_size=flush_size@entry=0x0) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:304 apache#6 0x000055f114426c4e in doris::SegmentFlusher::Writer::flush (this=<optimized out>) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:376 apache#7 doris::SegmentCreator::flush (this=0x7fa160b8a418) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:422 apache#8 0x000055f1143d4785 in doris::BaseBetaRowsetWriter::flush (this=<optimized out>) at /home/work/doris/be/src/olap/rowset/beta_rowset_writer.cpp:723 apache#9 0x000055f11436bf47 in doris::Merger::vmerge_rowsets (tablet=..., reader_type=<optimized out>, cur_tablet_schema=..., src_rowset_readers=..., dst_rowset_writer=0x7fa160b8a000, stats_output=0x7fa1df108298) at /home/work/doris/be/src/olap/merger.cpp:159 apache#10 0x000055f114345595 in doris::Compaction::merge_input_rowsets (this=this@entry=0x7fa1df108210) at /home/work/doris/be/src/olap/compaction.cpp:220 apache#11 0x000055f1143550d2 in doris::CloudCompactionMixin::execute_compact_impl (this=this@entry=0x7fa1df108210, permits=permits@entry=6) at /home/work/doris/be/src/olap/compaction.cpp:1491 apache#12 0x000055f114342bc1 in doris::CloudCompactionMixin::execute_compact (this=0x7fa1df108210) at /home/work/doris/be/src/olap/compaction.cpp:1620 apache#13 0x000055f11a22c6d8 in doris::CloudCumulativeCompaction::execute_compact (this=0x7fa1df108210) at /home/work/doris/be/src/cloud/cloud_cumulative_compaction.cpp:203 ```

[fix](cloud compaction) horizontal compaction coredump due to tablet …

1542d53

…shard_ptr not initialized

yiguolei approved these changes Dec 5, 2025

View reviewed changes

yiguolei added dev/4.0.x usercase Important user case type label labels Dec 5, 2025

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 5, 2025

github-actions bot added the reviewed label Dec 5, 2025

zclllyybb approved these changes Dec 5, 2025

View reviewed changes

gavinchou approved these changes Dec 5, 2025

View reviewed changes

gavinchou added the dev/3.1.x label Dec 5, 2025

freemandealer approved these changes Dec 5, 2025

View reviewed changes

yiguolei merged commit a5fda94 into apache:master Dec 5, 2025
34 of 36 checks passed

github-actions bot mentioned this pull request Dec 5, 2025

branch-3.1: [fix](cloud compaction) horizontal compaction coredump due to tablet shard_ptr not initialized #58751 #58761

Merged

github-actions bot mentioned this pull request Dec 5, 2025

branch-4.0: [fix](cloud compaction) horizontal compaction coredump due to tablet shard_ptr not initialized #58751 #58764

Merged

yiguolei pushed a commit that referenced this pull request Dec 5, 2025

branch-4.0: [fix](cloud compaction) horizontal compaction coredump du…

e8076c8

…e to tablet shard_ptr not initialized #58751 (#58764) Cherry-picked from #58751 Co-authored-by: HonestManXin <HonestManXin@gmail.com>

morrySnow pushed a commit that referenced this pull request Dec 8, 2025

branch-3.1: [fix](cloud compaction) horizontal compaction coredump du…

105a05e

…e to tablet shard_ptr not initialized #58751 (#58761) Cherry-picked from #58751 Co-authored-by: HonestManXin <HonestManXin@gmail.com>

morrySnow added dev/3.1.4-merged and removed dev/3.1.x labels Dec 8, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[fix](cloud compaction) horizontal compaction coredump due to tablet shard_ptr not initialized #58751

[fix](cloud compaction) horizontal compaction coredump due to tablet shard_ptr not initialized #58751

Uh oh!

HonestManXin commented Dec 5, 2025

Uh oh!

hello-stephen commented Dec 5, 2025

Uh oh!

HonestManXin commented Dec 5, 2025

Uh oh!

github-actions bot commented Dec 5, 2025

Uh oh!

github-actions bot commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

freemandealer left a comment

Uh oh!

hello-stephen commented Dec 5, 2025

Uh oh!

hello-stephen commented Dec 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

[fix](cloud compaction) horizontal compaction coredump due to tablet shard_ptr not initialized #58751

[fix](cloud compaction) horizontal compaction coredump due to tablet shard_ptr not initialized #58751

Uh oh!

Conversation

HonestManXin commented Dec 5, 2025

Uh oh!

hello-stephen commented Dec 5, 2025

Uh oh!

HonestManXin commented Dec 5, 2025

Uh oh!

github-actions bot commented Dec 5, 2025

Uh oh!

github-actions bot commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

doris-robot commented Dec 5, 2025

Uh oh!

freemandealer left a comment

Choose a reason for hiding this comment

Uh oh!

hello-stephen commented Dec 5, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Dec 5, 2025

BE Regression && UT Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants