Skip to content

Conversation

@freemandealer
Copy link
Contributor

@freemandealer freemandealer commented Dec 5, 2025

*** SIGSEGV unknown detail explain (@0x0) received by PID 2118050 (TID 2119672 OR 0x7f969ebf3640) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/common/signal_handler.h:421
1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
3# 0x00007F9A01EF7520 in /lib/x86_64-linux-gnu/libc.so.6
4# unsigned long moodycamel::ConcurrentQueue<std::shared_ptrdoris::io::FileBlock, moodycamel::ConcurrentQueueDefaultTraits>::ImplicitProducer::dequeue_bulk<std::shared_ptrdoris::io::FileBlock>(std::shared_ptrdoris::io::FileBlock&, unsigned long) at /mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/thirdparty/installed/include/concurrentqueue.h:2802
5# doris::io::BlockFileCache::clear_need_update_lru_blocks() at /mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache.cpp:2289
6# doris::io::BlockFileCache::clear_file_cache_directlyabi:cxx11 at /mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache.cpp:2338
7# doris::io::FileCacheFactory::clear_file_cachesabi:cxx11 at /mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache_factory.cpp:210

…locks

*** SIGSEGV unknown detail explain (@0x0) received by PID 2118050 (TID 2119672 OR 0x7f969ebf3640) from PID 0; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 3# 0x00007F9A01EF7520 in /lib/x86_64-linux-gnu/libc.so.6
 4# unsigned long moodycamel::ConcurrentQueue<std::shared_ptr<doris::io::FileBlock>, moodycamel::ConcurrentQueueDefaultTraits>::ImplicitProducer::dequeue_bulk<std::shared_ptr<doris::io::FileBlock>*>(std::shared_ptr<doris::io::FileBlock>*&, unsigned long) at /mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/thirdparty/installed/include/concurrentqueue.h:2802
 5# doris::io::BlockFileCache::clear_need_update_lru_blocks() at /mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache.cpp:2289
 6# doris::io::BlockFileCache::clear_file_cache_directly[abi:cxx11]() at /mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache.cpp:2338
 7# doris::io::FileCacheFactory::clear_file_caches[abi:cxx11](bool) at /mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache_factory.cpp:210
@Thearas
Copy link
Contributor

Thearas commented Dec 5, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@freemandealer
Copy link
Contributor Author

run buildall

@freemandealer freemandealer changed the title [fix](filecache) using invalid memory crashes clear_need_update_lru_b… [fix](filecache) using invalid memory crashes clear_need_update_lru_blocks Dec 5, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 36375 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d4a9d8aaf93b3d4d62ba2810f2fde1e8404d8f85, data reload: false

------ Round 1 ----------------------------------
q1	17665	5148	4990	4990
q2	2064	325	195	195
q3	10257	1354	780	780
q4	10214	857	333	333
q5	7510	2548	2252	2252
q6	204	185	146	146
q7	998	799	658	658
q8	9366	1481	1226	1226
q9	7039	5457	5377	5377
q10	6814	2209	1765	1765
q11	544	332	293	293
q12	342	383	240	240
q13	17798	3735	3078	3078
q14	243	252	215	215
q15	601	519	516	516
q16	910	881	814	814
q17	703	882	508	508
q18	7732	7672	7908	7672
q19	1364	1042	635	635
q20	397	376	230	230
q21	4490	4149	3440	3440
q22	1142	1122	1012	1012
Total cold run time: 108397 ms
Total hot run time: 36375 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5302	5330	5218	5218
q2	340	407	316	316
q3	2459	2845	2651	2651
q4	1447	1861	1419	1419
q5	4380	4630	4532	4532
q6	219	174	129	129
q7	2046	2005	1863	1863
q8	2697	2623	2712	2623
q9	7500	7542	7578	7542
q10	3066	3356	2856	2856
q11	608	507	493	493
q12	678	750	572	572
q13	3371	3670	3153	3153
q14	279	293	276	276
q15	540	506	490	490
q16	885	900	922	900
q17	1176	1423	1403	1403
q18	7604	7237	7174	7174
q19	908	863	869	863
q20	1951	2024	1907	1907
q21	4707	4384	4181	4181
q22	1133	1050	982	982
Total cold run time: 53296 ms
Total hot run time: 51543 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 180817 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d4a9d8aaf93b3d4d62ba2810f2fde1e8404d8f85, data reload: false

query5	5789	672	510	510
query6	342	225	221	221
query7	4668	503	277	277
query8	312	250	235	235
query9	8717	2660	2659	2659
query10	582	313	296	296
query11	15369	15347	14981	14981
query12	187	126	118	118
query13	1687	485	396	396
query14	7733	3399	3085	3085
query14_1	2973	3036	2994	2994
query15	237	206	185	185
query16	7664	498	465	465
query17	1609	769	623	623
query18	2053	442	356	356
query19	226	202	171	171
query20	136	124	122	122
query21	225	138	117	117
query22	3840	3884	3851	3851
query23	16833	16204	16077	16077
query23_1	16121	16003	16038	16003
query24	6725	1627	1229	1229
query24_1	1258	1225	1257	1225
query25	604	476	425	425
query26	1235	297	171	171
query27	2871	468	317	317
query28	4426	2168	2172	2168
query29	837	581	449	449
query30	316	240	215	215
query31	843	723	646	646
query32	83	73	75	73
query33	649	352	301	301
query34	874	902	541	541
query35	804	828	723	723
query36	905	929	817	817
query37	132	95	80	80
query38	3944	3872	3753	3753
query39	759	726	714	714
query39_1	704	701	706	701
query40	230	138	122	122
query41	71	65	65	65
query42	166	96	95	95
query43	455	417	405	405
query44	1364	769	767	767
query45	205	194	187	187
query46	923	979	615	615
query47	1683	1708	1621	1621
query48	410	336	236	236
query49	813	451	360	360
query50	713	323	245	245
query51	3879	3846	3841	3841
query52	115	101	93	93
query53	248	237	179	179
query54	339	274	239	239
query55	101	79	76	76
query56	351	299	303	299
query57	1156	1141	1112	1112
query58	299	262	251	251
query59	2296	2428	2258	2258
query60	366	322	300	300
query61	160	154	163	154
query62	774	680	621	621
query63	227	183	189	183
query64	4607	1164	893	893
query65	4065	3925	3960	3925
query66	1123	448	338	338
query67	15318	15101	14873	14873
query68	5634	969	673	673
query69	556	314	272	272
query70	1131	974	981	974
query71	439	307	278	278
query72	6014	4984	5005	4984
query73	717	585	311	311
query74	8851	8785	8646	8646
query75	3066	3071	2544	2544
query76	3187	1161	782	782
query77	508	432	312	312
query78	9411	9723	8952	8952
query79	1100	846	586	586
query80	711	555	471	471
query81	507	271	234	234
query82	223	128	106	106
query83	283	280	261	261
query84	271	115	100	100
query85	883	500	462	462
query86	335	299	291	291
query87	4076	4079	4011	4011
query88	2924	2181	2149	2149
query89	397	324	296	296
query90	1955	170	163	163
query91	195	189	163	163
query92	77	73	73	73
query93	1161	1061	696	696
query94	717	331	277	277
query95	592	404	346	346
query96	540	476	216	216
query97	2597	2651	2534	2534
query98	247	209	195	195
query99	1328	1363	1252	1252
Total cold run time: 268960 ms
Total hot run time: 180817 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.38 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d4a9d8aaf93b3d4d62ba2810f2fde1e8404d8f85, data reload: false

query1	0.05	0.04	0.05
query2	0.10	0.05	0.05
query3	0.25	0.08	0.08
query4	1.62	0.11	0.11
query5	0.26	0.28	0.26
query6	1.17	0.65	0.62
query7	0.03	0.03	0.03
query8	0.06	0.04	0.05
query9	0.58	0.51	0.49
query10	0.57	0.56	0.57
query11	0.15	0.11	0.12
query12	0.15	0.11	0.12
query13	0.65	0.61	0.60
query14	1.01	1.00	0.97
query15	0.83	0.81	0.81
query16	0.40	0.39	0.42
query17	1.04	1.10	1.07
query18	0.24	0.22	0.21
query19	1.85	1.86	1.83
query20	0.02	0.01	0.01
query21	15.44	0.28	0.13
query22	4.92	0.05	0.05
query23	15.98	0.27	0.10
query24	1.51	0.64	0.18
query25	0.11	0.11	0.06
query26	0.15	0.13	0.14
query27	0.07	0.05	0.04
query28	5.12	1.21	1.01
query29	12.61	4.00	3.18
query30	0.28	0.14	0.11
query31	2.81	0.64	0.41
query32	3.23	0.56	0.47
query33	3.00	3.11	3.16
query34	16.96	5.36	4.57
query35	4.60	4.60	4.61
query36	0.70	0.50	0.50
query37	0.11	0.06	0.06
query38	0.07	0.05	0.04
query39	0.05	0.03	0.04
query40	0.18	0.14	0.13
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 99.1 s
Total hot run time: 27.38 s

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 57.14% (4/7) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.28% (18637/34981)
Line Coverage 38.95% (171920/441402)
Region Coverage 33.58% (133095/396395)
Branch Coverage 34.52% (57253/165877)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 57.14% (4/7) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.16% (24754/34303)
Line Coverage 58.84% (259514/441036)
Region Coverage 53.74% (215616/401209)
Branch Coverage 55.28% (92187/166776)

@freemandealer
Copy link
Contributor Author

run cloud_p0

@freemandealer
Copy link
Contributor Author

run p0

@freemandealer
Copy link
Contributor Author

run nonConcurrent

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 5, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2025

PR approved by anyone and no changes requested.

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 57.14% (4/7) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.17% (24756/34303)
Line Coverage 58.85% (259541/441036)
Region Coverage 53.77% (215741/401209)
Branch Coverage 55.29% (92203/166776)

1 similar comment
@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 57.14% (4/7) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.17% (24756/34303)
Line Coverage 58.85% (259541/441036)
Region Coverage 53.77% (215741/401209)
Branch Coverage 55.29% (92203/166776)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 57.14% (4/7) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.17% (24758/34303)
Line Coverage 58.86% (259581/441036)
Region Coverage 53.76% (215708/401209)
Branch Coverage 55.30% (92224/166776)

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 57.14% (4/7) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.17% (24758/34303)
Line Coverage 58.86% (259581/441036)
Region Coverage 53.76% (215708/401209)
Branch Coverage 55.30% (92224/166776)

@hello-stephen hello-stephen merged commit 46ccf80 into apache:master Dec 8, 2025
29 of 31 checks passed
github-actions bot pushed a commit that referenced this pull request Dec 8, 2025
…locks (#58768)

*** SIGSEGV unknown detail explain (@0x0) received by PID 2118050 (TID
2119672 OR 0x7f969ebf3640) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/common/signal_handler.h:421
1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0]
in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
2# JVM_handle_linux_signal in
/usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 3# 0x00007F9A01EF7520 in /lib/x86_64-linux-gnu/libc.so.6
4# unsigned long
moodycamel::ConcurrentQueue<std::shared_ptr<doris::io::FileBlock>,
moodycamel::ConcurrentQueueDefaultTraits>::ImplicitProducer::dequeue_bulk<std::shared_ptr<doris::io::FileBlock>*>(std::shared_ptr<doris::io::FileBlock>*&,
unsigned long) at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/thirdparty/installed/include/concurrentqueue.h:2802
5# doris::io::BlockFileCache::clear_need_update_lru_blocks() at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache.cpp:2289
6# doris::io::BlockFileCache::clear_file_cache_directly[abi:cxx11]() at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache.cpp:2338
7# doris::io::FileCacheFactory::clear_file_caches[abi:cxx11](bool) at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache_factory.cpp:210
github-actions bot pushed a commit that referenced this pull request Dec 8, 2025
…locks (#58768)

*** SIGSEGV unknown detail explain (@0x0) received by PID 2118050 (TID
2119672 OR 0x7f969ebf3640) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/common/signal_handler.h:421
1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0]
in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
2# JVM_handle_linux_signal in
/usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 3# 0x00007F9A01EF7520 in /lib/x86_64-linux-gnu/libc.so.6
4# unsigned long
moodycamel::ConcurrentQueue<std::shared_ptr<doris::io::FileBlock>,
moodycamel::ConcurrentQueueDefaultTraits>::ImplicitProducer::dequeue_bulk<std::shared_ptr<doris::io::FileBlock>*>(std::shared_ptr<doris::io::FileBlock>*&,
unsigned long) at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/thirdparty/installed/include/concurrentqueue.h:2802
5# doris::io::BlockFileCache::clear_need_update_lru_blocks() at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache.cpp:2289
6# doris::io::BlockFileCache::clear_file_cache_directly[abi:cxx11]() at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache.cpp:2338
7# doris::io::FileCacheFactory::clear_file_caches[abi:cxx11](bool) at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache_factory.cpp:210
morrySnow pushed a commit that referenced this pull request Dec 9, 2025
…update_lru_blocks #58768 (#58819)

Cherry-picked from #58768

Co-authored-by: zhengyu <zhangzhengyu@selectdb.com>
yiguolei pushed a commit that referenced this pull request Dec 9, 2025
…update_lru_blocks #58768 (#58820)

Cherry-picked from #58768

Co-authored-by: zhengyu <zhangzhengyu@selectdb.com>
nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025
…locks (apache#58768)

*** SIGSEGV unknown detail explain (@0x0) received by PID 2118050 (TID
2119672 OR 0x7f969ebf3640) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/common/signal_handler.h:421
1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0]
in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
2# JVM_handle_linux_signal in
/usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 3# 0x00007F9A01EF7520 in /lib/x86_64-linux-gnu/libc.so.6
4# unsigned long
moodycamel::ConcurrentQueue<std::shared_ptr<doris::io::FileBlock>,
moodycamel::ConcurrentQueueDefaultTraits>::ImplicitProducer::dequeue_bulk<std::shared_ptr<doris::io::FileBlock>*>(std::shared_ptr<doris::io::FileBlock>*&,
unsigned long) at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/thirdparty/installed/include/concurrentqueue.h:2802
5# doris::io::BlockFileCache::clear_need_update_lru_blocks() at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache.cpp:2289
6# doris::io::BlockFileCache::clear_file_cache_directly[abi:cxx11]() at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache.cpp:2338
7# doris::io::FileCacheFactory::clear_file_caches[abi:cxx11](bool) at
/mnt/disk3/pipeline/repo/selectdb-core_branch-selectdb-doris-3.1/selectdb-core/be/src/io/cache/block_file_cache_factory.cpp:210
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.4-merged dev/4.0.2-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants