[fix](reader) Fix tracing file reader nullptr issue in push handler. #55346

kaka11chen · 2025-08-27T05:42:44Z

What problem does this PR solve?

Related PR: #53729

Problem Summary:

runtime error: member access within null pointer of type 'FileReaderStats' in TracingFileReader

/home/zcp/repo_center/doris_branch-3.1/doris/be/src/io/fs/tracing_file_reader.h:34:9: runtime error: member access within null pointer of type 'FileReaderStats'
    #0 0x55c028481ace in doris::io::TracingFileReader::read_at_impl(unsigned long, doris::Slice, unsigned long*, doris::io::IOContext const*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/io/fs/tracing_file_reader.h:34:9
    #1 0x55c00922a1f7 in doris::io::FileReader::read_at(unsigned long, doris::Slice, unsigned long*, doris::io::IOContext const*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/io/fs/file_reader.cpp:34:17
    #2 0x55c028ae8a18 in doris::vectorized::parse_thrift_footer(std::shared_ptr<doris::io::FileReader>, doris::vectorized::FileMetaData**, unsigned long*, doris::io::IOContext*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/vec/exec/format/parquet/parquet_thrift_util.h:44:5
    #3 0x55c028ae8a18 in doris::vectorized::ParquetReader::_open_file() /home/zcp/repo_center/doris_branch-3.1/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:234:21
    #4 0x55c028aeb69c in doris::vectorized::ParquetReader::init_reader(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::variant<doris::ColumnValueRange<(doris::PrimitiveType)3>, doris::ColumnValueRange<(doris::PrimitiveType)4>, doris::ColumnValueRange<(doris::PrimitiveType)5>, doris::ColumnValueRange<(doris::PrimitiveType)6>, doris::ColumnValueRange<(doris::PrimitiveType)7>, doris::ColumnValueRange<(doris::PrimitiveType)36>, doris::ColumnValueRange<(doris::PrimitiveType)37>, doris::ColumnValueRange<(doris::PrimitiveType)15>, doris::ColumnValueRange<(doris::PrimitiveType)10>, doris::ColumnValueRange<(doris::PrimitiveType)23>, doris::ColumnValueRange<(doris::PrimitiveType)11>, doris::ColumnValueRange<(doris::PrimitiveType)25>, doris::ColumnValueRange<(doris::PrimitiveType)12>, doris::ColumnValueRange<(doris::PrimitiveType)26>, doris::ColumnValueRange<(doris::PrimitiveType)20>, doris::ColumnValueRange<(doris::PrimitiveType)2>, doris::ColumnValueRange<(doris::PrimitiveType)19>, doris::ColumnValueRange<(doris::PrimitiveType)28>, doris::ColumnValueRange<(doris::PrimitiveType)29>, doris::ColumnValueRange<(doris::PrimitiveType)30>, doris::ColumnValueRange<(doris::PrimitiveType)35>>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const, std::variant<doris::ColumnValueRange<(doris::PrimitiveType)3>, doris::ColumnValueRange<(doris::PrimitiveType)4>, doris::ColumnValueRange<(doris::PrimitiveType)5>, doris::ColumnValueRange<(doris::PrimitiveType)6>, doris::ColumnValueRange<(doris::PrimitiveType)7>, doris::ColumnValueRange<(doris::PrimitiveType)36>, doris::ColumnValueRange<(doris::PrimitiveType)37>, doris::ColumnValueRange<(doris::PrimitiveType)15>, doris::ColumnValueRange<(doris::PrimitiveType)10>, doris::ColumnValueRange<(doris::PrimitiveType)23>, doris::ColumnValueRange<(doris::PrimitiveType)11>, doris::ColumnValueRange<(doris::PrimitiveType)25>, doris::ColumnValueRange<(doris::PrimitiveType)12>, doris::ColumnValueRange<(doris::PrimitiveType)26>, doris::ColumnValueRange<(doris::PrimitiveType)20>, doris::ColumnValueRange<(doris::PrimitiveType)2>, doris::ColumnValueRange<(doris::PrimitiveType)19>, doris::ColumnValueRange<(doris::PrimitiveType)28>, doris::ColumnValueRange<(doris::PrimitiveType)29>, doris::ColumnValueRange<(doris::PrimitiveType)30>, doris::ColumnValueRange<(doris::PrimitiveType)35>>>>> const*, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext>>> const&, doris::TupleDescriptor const*, doris::RowDescriptor const*, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, int, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const, int>>> const*, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext>>> const*, std::unordered_map<int, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext>>>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std::shared_ptr<doris::vectorized::VExprContext>>>>>> const*, std::shared_ptr<doris::vectorized::TableSchemaChangeHelper::Node>, bool) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:318:5
    #5 0x55c00cceda85 in doris::PushBrokerReader::_get_next_reader() /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/push_handler.cpp:671:39
    #6 0x55c00cce72df in doris::PushBrokerReader::next(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/push_handler.cpp:434:9
    #7 0x55c00cce27b3 in doris::PushHandler::_convert_v2(std::shared_ptr<doris::Tablet>, std::shared_ptr<doris::Rowset>*, std::shared_ptr<doris::TabletSchema>, doris::PushType) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/push_handler.cpp:293:31
    #8 0x55c00ccdd92d in doris::PushHandler::_do_streaming_ingestion(std::shared_ptr<doris::Tablet>, doris::TPushReq const&, doris::PushType, std::vector<doris::TTabletInfo, std::allocator<doris::TTabletInfo>>*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/push_handler.cpp:199:11
    #9 0x55c00ccd96f3 in doris::PushHandler::process_streaming_ingestion(std::shared_ptr<doris::Tablet>, doris::TPushReq const&, doris::PushType, std::vector<doris::TTabletInfo, std::allocator<doris::TTabletInfo>>*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/push_handler.cpp:94:11
    #10 0x55c00cccff9c in doris::EngineBatchLoadTask::_push(doris::TPushReq const&, std::vector<doris::TTabletInfo, std::allocator<doris::TTabletInfo>>*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/task/engine_batch_load_task.cpp:293:28
    #11 0x55c00cccbbc3 in doris::EngineBatchLoadTask::_process() /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/task/engine_batch_load_task.cpp:249:18
    #12 0x55c00ccc6851 in doris::EngineBatchLoadTask::execute() /home/zcp/repo_center/doris_branch-3.1/doris/be/src/olap/task/engine_batch_load_task.cpp:83:22
    #13 0x55c0090293eb in doris::push_callback(doris::StorageEngine&, doris::TAgentTaskRequest const&) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/agent/task_worker_pool.cpp:1880:31
    #14 0x55c008ff6554 in doris::PriorTaskWorkerPool::normal_loop() /home/zcp/repo_center/doris_branch-3.1/doris/be/src/agent/task_worker_pool.cpp:687:9
    #15 0x55c00db29237 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/util/thread.cpp:498:5
    #16 0x7fa0651f2ac2 in start_thread nptl/pthread_create.c:442:8
    #17 0x7fa06528484f  misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Release note

Fix tracing file reader nullptr issue in push handler, releated PR: #53729.

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-08-27T05:42:50Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

kaka11chen · 2025-08-27T05:42:54Z

run buildall

doris-robot · 2025-08-27T06:02:26Z

TPC-H: Total hot run time: 34298 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 92ad0d570ed9ed1d58c0c461d4407f18e79d42eb, data reload: false

------ Round 1 ----------------------------------
q1	17597	5245	5086	5086
q2	2020	328	217	217
q3	10240	1270	728	728
q4	10237	1056	530	530
q5	7574	2404	2321	2321
q6	182	169	143	143
q7	943	754	618	618
q8	9354	1328	1145	1145
q9	6879	5105	5116	5105
q10	6922	2387	1978	1978
q11	480	310	281	281
q12	358	365	227	227
q13	17746	3659	3115	3115
q14	235	246	232	232
q15	534	497	483	483
q16	444	442	376	376
q17	615	870	366	366
q18	7367	7115	7169	7115
q19	1098	969	559	559
q20	343	350	227	227
q21	3879	3176	2432	2432
q22	1075	1018	1014	1014
Total cold run time: 106122 ms
Total hot run time: 34298 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5148	5137	5129	5129
q2	264	327	231	231
q3	2154	2676	2317	2317
q4	1389	1768	1367	1367
q5	4258	4422	4657	4422
q6	223	175	136	136
q7	2054	2011	1816	1816
q8	2747	2705	2598	2598
q9	7706	7322	7292	7292
q10	3357	3308	2822	2822
q11	589	540	509	509
q12	736	813	668	668
q13	3543	4010	3336	3336
q14	286	290	305	290
q15	542	490	479	479
q16	461	521	460	460
q17	1193	1543	1432	1432
q18	8111	7644	7611	7611
q19	936	835	870	835
q20	2030	2150	1956	1956
q21	5073	4722	4493	4493
q22	1097	1037	1033	1033
Total cold run time: 53897 ms
Total hot run time: 51232 ms

doris-robot · 2025-08-27T06:14:08Z

TPC-DS: Total hot run time: 187618 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 92ad0d570ed9ed1d58c0c461d4407f18e79d42eb, data reload: false

query1	1057	476	412	412
query2	6555	1852	1815	1815
query3	6762	232	227	227
query4	26096	23344	23181	23181
query5	4346	668	523	523
query6	339	249	230	230
query7	4638	556	313	313
query8	295	264	254	254
query9	8618	2909	2909	2909
query10	480	359	298	298
query11	15525	15133	14808	14808
query12	175	123	122	122
query13	1677	604	446	446
query14	9251	5954	6041	5954
query15	211	198	195	195
query16	7670	684	471	471
query17	1221	747	642	642
query18	2042	438	334	334
query19	206	202	182	182
query20	142	124	123	123
query21	218	142	118	118
query22	4159	4161	4031	4031
query23	34073	32997	33059	32997
query24	8176	2427	2420	2420
query25	595	521	442	442
query26	1242	308	172	172
query27	2704	542	370	370
query28	4365	2290	2269	2269
query29	797	599	496	496
query30	292	235	209	209
query31	919	782	718	718
query32	126	79	81	79
query33	590	403	372	372
query34	827	860	541	541
query35	840	863	764	764
query36	990	1041	925	925
query37	134	119	95	95
query38	4082	4050	4005	4005
query39	1517	1443	1432	1432
query40	230	138	133	133
query41	76	67	67	67
query42	132	117	122	117
query43	549	515	472	472
query44	1400	879	868	868
query45	182	179	171	171
query46	887	1043	668	668
query47	1781	1822	1755	1755
query48	404	448	344	344
query49	767	542	447	447
query50	686	701	409	409
query51	4215	4180	4085	4085
query52	126	127	115	115
query53	255	288	216	216
query54	683	630	569	569
query55	97	92	91	91
query56	361	354	369	354
query57	1208	1200	1143	1143
query58	308	297	302	297
query59	2657	2742	2591	2591
query60	382	381	376	376
query61	194	189	187	187
query62	826	733	665	665
query63	243	212	253	212
query64	4466	1138	825	825
query65	4377	4237	4263	4237
query66	1141	435	358	358
query67	15360	15466	15266	15266
query68	9430	962	602	602
query69	485	333	311	311
query70	1269	1120	1140	1120
query71	493	365	329	329
query72	5637	4927	4994	4927
query73	745	630	363	363
query74	8965	9174	8913	8913
query75	4323	3195	2631	2631
query76	3890	1165	766	766
query77	986	429	337	337
query78	9555	9817	8848	8848
query79	2407	865	611	611
query80	710	606	540	540
query81	479	262	243	243
query82	249	152	118	118
query83	309	269	251	251
query84	312	119	94	94
query85	915	458	425	425
query86	354	341	308	308
query87	4329	4281	4212	4212
query88	2886	2256	2270	2256
query89	433	334	295	295
query90	2059	240	246	240
query91	164	166	142	142
query92	97	78	72	72
query93	1855	963	642	642
query94	684	429	330	330
query95	422	338	338	338
query96	494	619	282	282
query97	2663	2760	2629	2629
query98	250	233	224	224
query99	1456	1456	1287	1287
Total cold run time: 277250 ms
Total hot run time: 187618 ms

doris-robot · 2025-08-27T06:19:48Z

ClickBench: Total hot run time: 33.1 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 92ad0d570ed9ed1d58c0c461d4407f18e79d42eb, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.06	0.06
query3	0.27	0.09	0.08
query4	1.61	0.12	0.12
query5	0.45	0.41	0.43
query6	1.18	0.65	0.68
query7	0.03	0.03	0.03
query8	0.06	0.05	0.05
query9	0.61	0.54	0.51
query10	0.59	0.59	0.56
query11	0.16	0.11	0.12
query12	0.16	0.13	0.12
query13	0.64	0.63	0.63
query14	0.86	0.82	0.85
query15	0.88	0.87	0.86
query16	0.40	0.40	0.38
query17	1.06	1.07	1.05
query18	0.22	0.20	0.20
query19	2.01	1.80	1.80
query20	0.02	0.01	0.01
query21	15.41	0.98	0.58
query22	0.79	1.17	0.70
query23	14.89	1.38	0.62
query24	6.64	1.28	1.00
query25	0.50	0.18	0.12
query26	0.66	0.15	0.13
query27	0.07	0.06	0.06
query28	9.71	0.96	0.44
query29	12.60	4.00	3.27
query30	3.08	3.06	3.03
query31	2.83	0.58	0.40
query32	3.25	0.56	0.48
query33	3.07	3.14	3.07
query34	16.15	5.47	4.90
query35	4.96	4.96	4.93
query36	0.70	0.52	0.50
query37	0.10	0.07	0.08
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.19	0.16	0.14
query41	0.09	0.03	0.04
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 107.22 s
Total hot run time: 33.1 s

github-actions · 2025-08-27T06:36:58Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-08-27T06:36:59Z

PR approved by anyone and no changes requested.

hello-stephen · 2025-08-27T06:50:41Z

BE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	51.78% (17148/33115)
Line Coverage	37.26% (156247/419355)
Region Coverage	31.95% (119100/372787)
Branch Coverage	33.24% (52342/157462)

hello-stephen · 2025-08-27T08:19:26Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	70.67% (22991/32533)
Line Coverage	56.93% (238671/419245)
Region Coverage	52.33% (197989/378333)
Branch Coverage	54.02% (85605/158460)

…sh handler. #55346 (#55355) Cherry-pick #55346

hello-stephen · 2025-08-28T03:48:59Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	70.67% (22991/32533)
Line Coverage	56.93% (238671/419245)
Region Coverage	52.33% (197989/378333)
Branch Coverage	54.02% (85605/158460)

[fix](reader) Fix tracing file reader nullptr issue in push handler.

92ad0d5

morningman added the dev/3.1.x label Aug 27, 2025

morningman approved these changes Aug 27, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 27, 2025

github-actions bot added the reviewed label Aug 27, 2025

kaka11chen mentioned this pull request Aug 27, 2025

branch-3.1: [fix](reader) Fix tracing file reader nullptr issue in push handler. #55346 #55355

Merged

CalvinKirs approved these changes Aug 28, 2025

View reviewed changes

suxiaogang223 approved these changes Aug 28, 2025

View reviewed changes

hubgeter approved these changes Aug 28, 2025

View reviewed changes

morrySnow pushed a commit that referenced this pull request Aug 28, 2025

branch-3.1: [fix](reader) Fix tracing file reader nullptr issue in pu…

4e4a836

…sh handler. #55346 (#55355) Cherry-pick #55346

morrySnow added dev/3.1.0-merged and removed dev/3.1.x labels Aug 28, 2025

morningman merged commit 9b5f144 into apache:master Sep 1, 2025
32 of 35 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[fix](reader) Fix tracing file reader nullptr issue in push handler. #55346

[fix](reader) Fix tracing file reader nullptr issue in push handler. #55346

Uh oh!

kaka11chen commented Aug 27, 2025

Uh oh!

hello-stephen commented Aug 27, 2025

Uh oh!

kaka11chen commented Aug 27, 2025

Uh oh!

doris-robot commented Aug 27, 2025

Uh oh!

doris-robot commented Aug 27, 2025

Uh oh!

doris-robot commented Aug 27, 2025

Uh oh!

github-actions bot commented Aug 27, 2025

Uh oh!

github-actions bot commented Aug 27, 2025

Uh oh!

hello-stephen commented Aug 27, 2025

Uh oh!

hello-stephen commented Aug 27, 2025

Uh oh!

hello-stephen commented Aug 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

[fix](reader) Fix tracing file reader nullptr issue in push handler. #55346

[fix](reader) Fix tracing file reader nullptr issue in push handler. #55346

Uh oh!

Conversation

kaka11chen commented Aug 27, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Aug 27, 2025

Uh oh!

kaka11chen commented Aug 27, 2025

Uh oh!

doris-robot commented Aug 27, 2025

Uh oh!

doris-robot commented Aug 27, 2025

Uh oh!

doris-robot commented Aug 27, 2025

Uh oh!

github-actions bot commented Aug 27, 2025

Uh oh!

github-actions bot commented Aug 27, 2025

Uh oh!

hello-stephen commented Aug 27, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Aug 27, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Aug 28, 2025

BE Regression && UT Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants