Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

branch-3.0: [Fix](multi-catalog) Fix column mutate() crash replace it by assume_mutable(). #46151 #46197

Merged
merged 1 commit into from
Jan 1, 2025

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #46151

…utable(). (#46151)

### What problem does this PR solve?

Problem Summary:

```
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
Core was generated by `/mnt/doris/be/lib/doris_be'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000000000 in ?? ()
[Current thread is 1 (LWP 3923404)]
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x000055f44f97dda7 in COW<doris::vectorized::IColumn>::release_ref (this=0x7f6bf7d07cc8) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:99
#2  COW<doris::vectorized::IColumn>::intrusive_ptr<doris::vectorized::IColumn>::~intrusive_ptr (this=0x7f6b792f9670) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:133
#3  doris::vectorized::ScalarColumnReader::_read_nested_column (this=this@entry=0x7f6be31f8900, doris_column=..., type=
    std::shared_ptr<const doris::vectorized::IDataType> (use count 1, weak count 0) = {...}, select_vector=..., batch_size=<optimized out>, batch_size@entry=4064, read_rows=0x7f6b792f9ad8, 
    eof=0x7f6b792f9af0, is_dict_filter=<optimized out>, align_rows=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:447
#4  0x000055f44f97e1fc in doris::vectorized::ScalarColumnReader::read_column_data (this=0x7f6be31f8900, doris_column=..., 
    type=std::shared_ptr<const doris::vectorized::IDataType> (use count 1, weak count 0) = {...}, select_vector=..., batch_size=4064, read_rows=0x7f6b792f9ad8, eof=0x7f6b792f9af0, 
    is_dict_filter=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:504
#5  0x000055f44f97ecbe in doris::vectorized::ArrayColumnReader::read_column_data (this=0x7f69a922ce00, doris_column=..., type=..., select_vector=..., batch_size=4064, read_rows=0x7f6b792f9ad8, 
    eof=0x7f6b792f9af0, is_dict_filter=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:616
#6  0x000055f44f975460 in doris::vectorized::RowGroupReader::_read_column_data (this=this@entry=0x7f6cf83dd180, block=block@entry=0x7f6bbcc66938, columns=..., batch_size=4064, 
    read_rows=read_rows@entry=0x7f6b792f9ee0, batch_eof=0x7f6cf83d71f0, select_vector=...) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:426
#7  0x000055f44f972682 in doris::vectorized::RowGroupReader::next_batch (this=0x7f6cf83dd180, block=0x7f6bbcc66938, batch_size=140099571418880, read_rows=0x7f6b792f9ee0, batch_eof=0x7f6f06724610)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:321
#8  0x000055f44f9141f8 in doris::vectorized::ParquetReader::get_next_block (this=0x7f6cf83d7000, block=0x7f6bbcc66938, read_rows=0x7f6b792f9ee0, eof=0x7f6bbcc66f88)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:576
#9  0x000055f450aba6f4 in doris::vectorized::IcebergTableReader::get_next_block (this=0x7f6bb5611180, block=0x7f6bbcc66938, read_rows=0x7f6b792f9ee0, eof=0x7f6bbcc66f88)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/table/iceberg_reader.cpp:138
#10 0x000055f450aaa351 in doris::vectorized::VFileScanner::_get_block_wrapped (this=0x7f6bbcc66800, state=<optimized out>, block=0x7f6bbcc66938, eof=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vfile_scanner.cpp:359
#11 0x000055f450aa9ecc in doris::vectorized::VFileScanner::_get_block_impl (this=0x0, state=0xffffffffffffa770, block=0x8c3de0, eof=0x7f6b79300700)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vfile_scanner.cpp:301
#12 0x000055f450b41e9c in doris::vectorized::VScanner::get_block (this=this@entry=0x7f6bbcc66800, state=state@entry=0x7f6f06724000, block=block@entry=0x7f6bbcc66938, eof=eof@entry=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vscanner.cpp:133
#13 0x000055f450b41977 in doris::vectorized::VScanner::get_block_after_projects (this=0x7f6bbcc66800, state=0x7f6f06724000, block=0x7f6cf8394b80, eos=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vscanner.cpp:96
#14 0x000055f450a941ff in doris::vectorized::ScannerScheduler::_scanner_scan (ctx=std::shared_ptr<doris::vectorized::ScannerContext> (use count 10, weak count 1) = {...}, 
    scan_task=std::shared_ptr<doris::vectorized::ScanTask> (use count 2, weak count 0) = {...}) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:289
#15 0x000055f450a94b73 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const::{lambda()#2}::operator()() const (this=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:180
#16 doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const (this=0x7f70ccc56ee0) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:179
#17 std::__invoke_impl<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__f=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#18 std::__invoke_r<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__fn=...)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#19 std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#20 0x000055f44ca858c8 in doris::ThreadPool::dispatch_thread (this=0x7f70ba259200) at /home/zcp/repo_center/doris_enterprise/doris/be/src/util/threadpool.cpp:543
#21 0x000055f44ca7ad91 in std::function<void ()>::operator()() const (this=0x7f6bf7d07cc0)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#22 doris::Thread::supervise_thread (arg=0x7f6fa185a020) at /home/zcp/repo_center/doris_enterprise/doris/be/src/util/thread.cpp:498
#23 0x00007f70ebf78e25 in ?? ()
#24 0x0000000000000000 in ?? ()
```
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Dec 31, 2024
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40738 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6a131a80e2f21e55e6acc7a4336500e6b3c94ed3, data reload: false

------ Round 1 ----------------------------------
q1	17579	7401	7315	7315
q2	2051	177	183	177
q3	10574	1086	1179	1086
q4	10462	759	737	737
q5	7750	2830	2850	2830
q6	236	151	144	144
q7	970	616	615	615
q8	9365	1913	2055	1913
q9	6578	6392	6415	6392
q10	7028	2266	2295	2266
q11	471	268	261	261
q12	410	209	217	209
q13	17790	3011	3004	3004
q14	245	208	212	208
q15	579	530	539	530
q16	690	614	612	612
q17	959	569	526	526
q18	7242	6645	6607	6607
q19	1438	1130	975	975
q20	484	204	215	204
q21	4010	3151	3222	3151
q22	1119	993	976	976
Total cold run time: 108030 ms
Total hot run time: 40738 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7232	7226	7244	7226
q2	325	240	232	232
q3	2907	2952	2961	2952
q4	2053	1860	1805	1805
q5	5699	5765	5742	5742
q6	224	137	138	137
q7	2227	1807	1857	1807
q8	3336	3579	3478	3478
q9	8939	8895	8886	8886
q10	3619	3563	3545	3545
q11	610	500	497	497
q12	805	617	624	617
q13	8770	3176	3182	3176
q14	317	285	281	281
q15	581	533	519	519
q16	709	686	690	686
q17	1864	1623	1592	1592
q18	8340	7813	7721	7721
q19	1691	1553	1578	1553
q20	2152	1869	1858	1858
q21	5703	5494	5569	5494
q22	1137	1101	1032	1032
Total cold run time: 69240 ms
Total hot run time: 60836 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198853 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6a131a80e2f21e55e6acc7a4336500e6b3c94ed3, data reload: false

query1	1253	949	905	905
query2	6244	2039	2056	2039
query3	10907	4334	4334	4334
query4	66560	29443	23684	23684
query5	4957	460	446	446
query6	397	175	179	175
query7	5626	303	312	303
query8	311	235	236	235
query9	8708	2701	2693	2693
query10	465	269	253	253
query11	17267	15152	15916	15152
query12	156	102	105	102
query13	1497	471	430	430
query14	10767	7747	7622	7622
query15	202	182	183	182
query16	7146	526	482	482
query17	1096	603	594	594
query18	2012	330	339	330
query19	214	167	157	157
query20	116	114	112	112
query21	60	44	46	44
query22	4809	4721	4599	4599
query23	34764	34482	34486	34482
query24	6101	2923	3009	2923
query25	525	431	412	412
query26	680	176	172	172
query27	1697	309	311	309
query28	4441	2516	2517	2516
query29	725	455	428	428
query30	246	168	163	163
query31	1018	822	843	822
query32	61	52	55	52
query33	434	290	288	288
query34	921	509	520	509
query35	853	773	744	744
query36	1082	954	979	954
query37	119	73	70	70
query38	4073	4074	4043	4043
query39	1545	1475	1458	1458
query40	137	84	79	79
query41	48	45	47	45
query42	109	95	100	95
query43	526	516	502	502
query44	1245	837	840	837
query45	207	170	171	170
query46	1157	744	772	744
query47	2018	1987	1986	1986
query48	491	385	414	385
query49	731	376	375	375
query50	824	430	434	430
query51	7318	7256	7326	7256
query52	100	86	81	81
query53	258	184	194	184
query54	546	453	444	444
query55	77	77	76	76
query56	248	230	244	230
query57	1242	1133	1090	1090
query58	222	210	204	204
query59	3101	3055	2991	2991
query60	282	260	243	243
query61	110	110	108	108
query62	783	670	690	670
query63	206	197	187	187
query64	1404	672	673	672
query65	3261	3192	3226	3192
query66	725	315	303	303
query67	15950	15667	15779	15667
query68	3849	596	580	580
query69	455	262	271	262
query70	1188	1063	1157	1063
query71	352	270	246	246
query72	6461	4021	4136	4021
query73	786	345	351	345
query74	9941	8871	9002	8871
query75	3347	2637	2665	2637
query76	1843	1062	1121	1062
query77	506	283	277	277
query78	10773	9650	9572	9572
query79	1568	604	604	604
query80	903	435	441	435
query81	516	242	242	242
query82	1186	131	113	113
query83	227	144	140	140
query84	280	84	71	71
query85	899	310	292	292
query86	353	298	287	287
query87	4405	4254	4357	4254
query88	3807	2414	2367	2367
query89	412	292	286	286
query90	2013	188	181	181
query91	178	147	148	147
query92	67	47	49	47
query93	1864	550	555	550
query94	784	292	287	287
query95	356	259	254	254
query96	612	279	283	279
query97	3330	3207	3121	3121
query98	214	199	204	199
query99	1605	1302	1297	1297
Total cold run time: 318461 ms
Total hot run time: 198853 ms

@morningman morningman merged commit 5c68b77 into branch-3.0 Jan 1, 2025
19 of 21 checks passed
@github-actions github-actions bot deleted the auto-pick-46151-branch-3.0 branch January 1, 2025 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants