[fix](schema-change) Prevent coredump when reading non-overlapping segments from a single rowset during heavy schema change #57191

Hastyshell · 2025-10-21T08:27:36Z

What problem does this PR solve?

Problem Summary:

*** Query id: 0-0 ***
*** is nereids: 0 ***
*** tablet id: 1751140494462 ***
*** Aborted at 1751208065 (unix time) try "date -d @1751208065" if you are using GNU date ***
*** Current BE git commitID: 34d8342994 ***
*** SIGSEGV invalid permissions for mapped object (@0x556ef4dfc000) received by PID 4975 (TID 5587 OR 0x7f71da3c4640) from PID 18446744073522888704; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so
 3# 0x00007F731FC1F520 in /lib/x86_64-linux-gnu/libc.so.6
 4# doris::vectorized::VMergeIteratorContext::copy_rows(doris::vectorized::Block*, bool) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vgeneric_iterators.cpp:148
 5# doris::Status doris::vectorized::VMergeIterator::_next_batch<doris::vectorized::Block>(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vgeneric_iterators.h:242
 6# doris::vectorized::VMergeIterator::next_batch(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vgeneric_iterators.h:201
 7# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/beta_rowset_reader.cpp:378
 8# doris::VSchemaChangeDirectly::_inner_process(std::shared_ptr<doris::RowsetReader>, doris::RowsetWriter*, std::shared_ptr<doris::BaseTablet>, std::shared_ptr<doris::TabletSchema>, std::shared_ptr<doris::TabletSchema>) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/schema_change.cpp:523
 9# doris::SchemaChange::process(std::shared_ptr<doris::RowsetReader>, doris::RowsetWriter*, std::shared_ptr<doris::BaseTablet>, std::shared_ptr<doris::BaseTablet>, std::shared_ptr<doris::TabletSchema>, std::shared_ptr<doris::TabletSchema>) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/schema_change.h:120
10# doris::SchemaChangeJob::_convert_historical_rowsets(doris::SchemaChangeParams const&, long*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/schema_change.cpp:1179
11# doris::SchemaChangeJob::_do_process_alter_tablet(doris::TAlterTabletReqV2 const&) in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be
12# doris::SchemaChangeJob::process_alter_tablet(doris::TAlterTabletReqV2 const&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/schema_change.cpp:759
13# doris::alter_tablet_callback(doris::StorageEngine&, doris::TAgentTaskRequest const&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/agent/task_worker_pool.cpp:1990
14# std::_Function_handler<void (), doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest const&)::$_0::operator()<doris::TAgentTaskRequest const&>(doris::TAgentTaskRequest const&) const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
15# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:609
16# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:499
17# start_thread at ./nptl/pthread_create.c:442
18# 0x00007F731FD03850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83

Schema of input block from base tablet should merge dropped column.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Hastyshell · 2025-10-21T08:27:46Z

run buildall

Thearas · 2025-10-21T08:27:50Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

doris-robot · 2025-10-21T09:00:21Z

TPC-DS: Total hot run time: 190939 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 46e25b5f8f877dffb1fb6dd08029fac5ec04dd79, data reload: false

query1	1097	410	406	406
query2	6566	1707	1727	1707
query3	6760	236	224	224
query4	26206	23798	23084	23084
query5	4885	647	500	500
query6	362	274	247	247
query7	4661	519	309	309
query8	322	308	273	273
query9	8742	2646	2641	2641
query10	515	360	289	289
query11	15486	15119	14970	14970
query12	193	129	120	120
query13	1701	598	447	447
query14	11765	9329	9540	9329
query15	266	200	185	185
query16	8035	763	578	578
query17	1897	847	750	750
query18	2097	456	364	364
query19	424	250	201	201
query20	150	144	135	135
query21	221	162	126	126
query22	4771	4716	4527	4527
query23	35298	34072	33919	33919
query24	8543	2490	2539	2490
query25	585	562	478	478
query26	893	305	172	172
query27	2703	528	381	381
query28	4350	2266	2248	2248
query29	764	829	536	536
query30	315	237	207	207
query31	936	874	796	796
query32	87	74	71	71
query33	592	400	357	357
query34	859	868	541	541
query35	797	840	754	754
query36	952	1000	862	862
query37	128	116	95	95
query38	3560	3525	3450	3450
query39	1503	1510	1414	1414
query40	222	131	121	121
query41	65	59	63	59
query42	122	119	112	112
query43	500	502	479	479
query44	1288	755	754	754
query45	186	181	176	176
query46	906	1016	661	661
query47	1744	1760	1714	1714
query48	399	441	326	326
query49	739	552	410	410
query50	693	700	418	418
query51	3927	4001	3833	3833
query52	111	109	104	104
query53	249	276	200	200
query54	623	612	543	543
query55	92	86	83	83
query56	352	325	315	315
query57	1183	1177	1108	1108
query58	302	285	284	284
query59	2565	2701	2495	2495
query60	351	358	344	344
query61	161	155	145	145
query62	826	746	673	673
query63	237	211	200	200
query64	3666	1155	838	838
query65	4084	3995	3944	3944
query66	898	439	333	333
query67	15341	15309	14998	14998
query68	9521	991	629	629
query69	502	330	287	287
query70	1421	1308	1339	1308
query71	509	360	341	341
query72	6046	4849	4935	4849
query73	735	607	366	366
query74	9131	9116	9165	9116
query75	4486	3356	2879	2879
query76	4198	1178	754	754
query77	1029	437	324	324
query78	9519	9898	8919	8919
query79	2115	863	619	619
query80	739	591	570	570
query81	498	274	230	230
query82	244	157	136	136
query83	300	278	252	252
query84	307	133	93	93
query85	858	476	428	428
query86	343	321	294	294
query87	3735	3750	3656	3656
query88	3006	2303	2282	2282
query89	409	343	296	296
query90	2175	229	224	224
query91	164	168	135	135
query92	83	69	65	65
query93	2180	1033	653	653
query94	693	423	336	336
query95	412	332	322	322
query96	515	578	295	295
query97	2990	2964	2905	2905
query98	249	216	216	216
query99	1472	1390	1307	1307
Total cold run time: 282403 ms
Total hot run time: 190939 ms

doris-robot · 2025-10-21T09:05:36Z

ClickBench: Total hot run time: 28.9 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 46e25b5f8f877dffb1fb6dd08029fac5ec04dd79, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.06	0.06
query3	0.25	0.09	0.09
query4	1.61	0.12	0.12
query5	0.28	0.27	0.26
query6	1.20	0.66	0.68
query7	0.03	0.02	0.03
query8	0.06	0.04	0.05
query9	0.63	0.54	0.52
query10	0.62	0.59	0.58
query11	0.17	0.12	0.12
query12	0.17	0.13	0.12
query13	0.63	0.63	0.64
query14	1.05	1.03	1.05
query15	0.88	0.87	0.89
query16	0.42	0.41	0.40
query17	1.06	1.09	1.06
query18	0.23	0.21	0.21
query19	2.02	1.92	1.91
query20	0.02	0.02	0.02
query21	15.41	0.21	0.14
query22	4.95	0.08	0.05
query23	15.63	0.29	0.10
query24	2.99	0.65	0.46
query25	0.07	0.06	0.07
query26	0.16	0.14	0.13
query27	0.08	0.06	0.06
query28	4.77	1.18	0.95
query29	12.58	3.94	3.27
query30	0.27	0.15	0.14
query31	2.84	0.61	0.40
query32	3.26	0.57	0.47
query33	3.15	3.11	3.08
query34	16.43	5.59	4.92
query35	5.04	5.00	5.05
query36	0.72	0.55	0.52
query37	0.11	0.07	0.08
query38	0.08	0.04	0.05
query39	0.04	0.03	0.03
query40	0.20	0.16	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 100.44 s
Total hot run time: 28.9 s

hello-stephen · 2025-10-21T09:40:20Z

BE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.60% (17897/34026)
Line Coverage	37.84% (162338/429021)
Region Coverage	32.24% (123719/383764)
Branch Coverage	33.66% (54237/161125)

…egments from a single rowset during heavy schema change

Hastyshell · 2025-10-28T04:46:07Z

run buildall

doris-robot · 2025-10-28T05:20:20Z

ClickBench: Total hot run time: 27.73 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5b24333a6edf6d0916fccc4ba7ac986a2909b6eb, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.05
query3	0.26	0.08	0.08
query4	1.62	0.12	0.12
query5	0.28	0.26	0.25
query6	1.18	0.66	0.64
query7	0.04	0.03	0.03
query8	0.05	0.04	0.04
query9	0.61	0.53	0.51
query10	0.57	0.59	0.57
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.63	0.61	0.60
query14	1.00	1.00	1.00
query15	0.84	0.83	0.88
query16	0.42	0.39	0.41
query17	0.99	1.03	1.03
query18	0.22	0.21	0.20
query19	1.89	1.83	1.85
query20	0.01	0.01	0.02
query21	15.46	0.22	0.14
query22	4.91	0.07	0.05
query23	15.67	0.27	0.10
query24	2.50	0.55	0.67
query25	0.08	0.08	0.06
query26	0.14	0.13	0.12
query27	0.08	0.06	0.05
query28	4.82	1.17	0.94
query29	12.60	3.93	3.30
query30	0.28	0.14	0.13
query31	2.83	0.60	0.38
query32	3.24	0.56	0.46
query33	3.06	3.03	3.05
query34	15.74	5.15	4.57
query35	4.54	4.55	4.56
query36	0.68	0.51	0.48
query37	0.11	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.04
query40	0.17	0.15	0.14
query41	0.09	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 98.26 s
Total hot run time: 27.73 s

hello-stephen · 2025-10-28T06:00:45Z

BE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.76% (18055/34221)
Line Coverage	37.99% (163678/430865)
Region Coverage	32.36% (124724/385456)
Branch Coverage	33.73% (54546/161713)

hello-stephen · 2025-10-28T07:28:23Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.35% (23920/33526)
Line Coverage	57.77% (248813/430666)
Region Coverage	53.06% (207030/390171)
Branch Coverage	54.62% (88754/162486)

github-actions · 2025-10-29T08:04:50Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-10-29T08:04:53Z

PR approved by anyone and no changes requested.

airborne12

LGTM

…gments from a single rowset during heavy schema change (#57191) ### What problem does this PR solve? Problem Summary: ``` *** Query id: 0-0 *** *** is nereids: 0 *** *** tablet id: 1751140494462 *** *** Aborted at 1751208065 (unix time) try "date -d @1751208065" if you are using GNU date *** *** Current BE git commitID: 34d8342 *** *** SIGSEGV invalid permissions for mapped object (@0x556ef4dfc000) received by PID 4975 (TID 5587 OR 0x7f71da3c4640) from PID 18446744073522888704; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/signal_handler.h:421 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 2# JVM_handle_linux_signal in /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so 3# 0x00007F731FC1F520 in /lib/x86_64-linux-gnu/libc.so.6 4# doris::vectorized::VMergeIteratorContext::copy_rows(doris::vectorized::Block*, bool) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vgeneric_iterators.cpp:148 5# doris::Status doris::vectorized::VMergeIterator::_next_batch<doris::vectorized::Block>(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vgeneric_iterators.h:242 6# doris::vectorized::VMergeIterator::next_batch(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vgeneric_iterators.h:201 7# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/beta_rowset_reader.cpp:378 8# doris::VSchemaChangeDirectly::_inner_process(std::shared_ptr<doris::RowsetReader>, doris::RowsetWriter*, std::shared_ptr<doris::BaseTablet>, std::shared_ptr<doris::TabletSchema>, std::shared_ptr<doris::TabletSchema>) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/schema_change.cpp:523 9# doris::SchemaChange::process(std::shared_ptr<doris::RowsetReader>, doris::RowsetWriter*, std::shared_ptr<doris::BaseTablet>, std::shared_ptr<doris::BaseTablet>, std::shared_ptr<doris::TabletSchema>, std::shared_ptr<doris::TabletSchema>) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/schema_change.h:120 10# doris::SchemaChangeJob::_convert_historical_rowsets(doris::SchemaChangeParams const&, long*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/schema_change.cpp:1179 11# doris::SchemaChangeJob::_do_process_alter_tablet(doris::TAlterTabletReqV2 const&) in /mnt/hdd01/PERFORMANCE_ENV/be/lib/doris_be 12# doris::SchemaChangeJob::process_alter_tablet(doris::TAlterTabletReqV2 const&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/schema_change.cpp:759 13# doris::alter_tablet_callback(doris::StorageEngine&, doris::TAgentTaskRequest const&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/agent/task_worker_pool.cpp:1990 14# std::_Function_handler<void (), doris::TaskWorkerPool::submit_task(doris::TAgentTaskRequest const&)::$_0::operator()<doris::TAgentTaskRequest const&>(doris::TAgentTaskRequest const&) const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291 15# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:609 16# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:499 17# start_thread at ./nptl/pthread_create.c:442 18# 0x00007F731FD03850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83 ``` Schema of input block from base tablet should merge dropped column.

…erlapping segments from a single rowset during heavy schema change #57191 (#57602) Cherry-picked from #57191 Co-authored-by: Siyang Tang <tangsiyang@selectdb.com>

Hastyshell force-pushed the fix-delete-with-sc branch from 46e25b5 to 60b2aaf Compare October 21, 2025 09:35

Hastyshell force-pushed the fix-delete-with-sc branch from 60b2aaf to 943bedf Compare October 23, 2025 09:25

[fix](schema-change): prevent coredump when reading non-overlapping s…

5b24333

…egments from a single rowset during heavy schema change

Hastyshell force-pushed the fix-delete-with-sc branch from 943bedf to 5b24333 Compare October 23, 2025 09:55

Hastyshell marked this pull request as ready for review October 28, 2025 04:46

gavinchou approved these changes Oct 29, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 29, 2025

github-actions bot added the reviewed label Oct 29, 2025

luwei16 approved these changes Oct 30, 2025

View reviewed changes

airborne12 approved these changes Nov 1, 2025

View reviewed changes

airborne12 merged commit 2742eac into apache:master Nov 1, 2025
30 of 32 checks passed

airborne12 added the dev/4.0.x label Nov 1, 2025

github-actions bot mentioned this pull request Nov 1, 2025

branch-4.0: [fix](schema-change) Prevent coredump when reading non-overlapping segments from a single rowset during heavy schema change #57191 #57602

Merged

yiguolei added dev/4.0.2-merged and removed dev/4.0.x labels Nov 6, 2025

yiguolei mentioned this pull request Dec 2, 2025

4.0.2 Release Notes #58605

Open

[fix](schema-change) Prevent coredump when reading non-overlapping segments from a single rowset during heavy schema change #57191

[fix](schema-change) Prevent coredump when reading non-overlapping segments from a single rowset during heavy schema change #57191

Uh oh!

Conversation

Hastyshell commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Hastyshell commented Oct 21, 2025

Uh oh!

Thearas commented Oct 21, 2025

Uh oh!

doris-robot commented Oct 21, 2025

Uh oh!

doris-robot commented Oct 21, 2025

Uh oh!

hello-stephen commented Oct 21, 2025

BE UT Coverage Report

Uh oh!

Hastyshell commented Oct 28, 2025

Uh oh!

doris-robot commented Oct 28, 2025

Uh oh!

hello-stephen commented Oct 28, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Oct 28, 2025

BE Regression && UT Coverage Report

Uh oh!

github-actions bot commented Oct 29, 2025

Uh oh!

github-actions bot commented Oct 29, 2025

Uh oh!

airborne12 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Hastyshell commented Oct 21, 2025 •

edited

Loading