Skip to content

[env](ubsan) Avoid some errors reported by UBSan when compiling with O0 #54672

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
zclllyybb merged 1 commit into from
Aug 15, 2025
Merged

[env](ubsan) Avoid some errors reported by UBSan when compiling with O0 #54672

zclllyybb merged 1 commit into from
Aug 15, 2025

Conversation

@Mryange
Copy link
Contributor

@Mryange Mryange commented Aug 13, 2025 •
edited
Loading

What problem does this PR solve?

Because on the pipeline we use O1 for compilation, but some UBSan errors only appear locally when compiling with O0, for example:

0x7ba502eb3a01: note: pointer points here
 02 00 00  00 30 52 8c 02 95 7b 00  00 03 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00
              ^ 
    #0 0x55cbe9f9acd2 in doris::Slice::compare(doris::Slice const&) const /mnt/disk8/yanxuecheng/tmp-doris/be/src/util/slice.h
    #1 0x55cbede3405c in doris::FieldTypeTraits<(doris::FieldType)13>::cmp(void const*, void const*) /mnt/disk8/yanxuecheng/tmp-doris/be/src/olap/types.h:1347:25
    #2 0x55cbe9e98dae in doris::ScalarTypeInfo::cmp(void const*, void const*) const /mnt/disk8/yanxuecheng/tmp-doris/be/src/olap/types.h:104:74
    #3 0x55cbeee73cf0 in int doris::Field::compare_cell<doris::RowCursorCell, doris::RowCursorCell>(doris::RowCursorCell const&, doris::RowCursorCell const&) const /mnt/disk8/yanxuecheng/tmp-doris/be/src/olap/field.h:120:41
    #4 0x55cbeee6a030 in doris::compare_row_key(doris::RowCursor const&, doris::RowCursor const&) /mnt/disk8/yanxuecheng/tmp-doris/be/src/olap/tablet_reader.h:75:47
    #5 0x55cbeee43237 in doris::TabletReader::_capture_rs_readers(doris::TabletReader::ReaderParams const&) /mnt/disk8/yanxuecheng/tmp-doris/be/src/olap/tablet_reader.cpp:188:17
    #6 0x55cc1d5f7985 in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/olap/block_reader.cpp:106:16
    #7 0x55cc1d5fd004 in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/olap/block_reader.cpp:226:19
    #8 0x55cc15e09f53 in doris::vectorized::OlapScanner::open(doris::RuntimeState*) /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/exec/scan/olap_scanner.cpp:260:32
    #9 0x55cc15d7d89d in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/exec/scan/scanner_scheduler.cpp:182:5
    #10 0x55cc15d88ce6 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/exec/scan/scanner_scheduler.cpp:96:17
    #11 0x55cc15d884b1 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/exec/scan/scanner_scheduler.cpp:95:27
    #12 0x55cc15d8829c in bool std::__invoke_impl<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&) /mnt/disk6/common/ldb_toolchain_taipan/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14
    #13 0x55cc15d8819c in std::enable_if<is_invocable_r_v<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&) /mnt/disk6/common/ldb_toolchain_taipan/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9
    #14 0x55cc15d87c94 in std::_Function_handler<bool (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /mnt/disk6/common/ldb_toolchain_taipan/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9
    #15 0x55cbe9632efd in std::function<bool ()>::operator()() const /mnt/disk6/common/ldb_toolchain_taipan/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9
    #16 0x55cc15d87314 in doris::vectorized::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l>>) /mnt/disk8/yanxuecheng/tmp-doris/be/src/vec/exec/scan/scanner_scheduler.cpp:407:25

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Aug 13, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange
Copy link
Contributor Author

Mryange commented Aug 13, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 13, 2025

TPC-H: Total hot run time: 34060 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 687a6d8aa4d19f06072f37f6085a77e72acb33a9, data reload: false

------ Round 1 ----------------------------------
q1	17569	5158	5072	5072
q2	1919	286	197	197
q3	10311	1270	749	749
q4	10216	991	526	526
q5	7483	2477	2332	2332
q6	179	162	128	128
q7	894	740	620	620
q8	9299	1330	1157	1157
q9	7032	5096	5153	5096
q10	6908	2414	1951	1951
q11	471	294	258	258
q12	346	340	215	215
q13	17760	3632	3023	3023
q14	227	237	213	213
q15	546	489	482	482
q16	420	429	372	372
q17	594	864	367	367
q18	7822	7258	7172	7172
q19	1085	960	555	555
q20	334	349	228	228
q21	4062	3230	2360	2360
q22	1088	1032	987	987
Total cold run time: 106565 ms
Total hot run time: 34060 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5102	5319	5072	5072
q2	247	320	217	217
q3	2179	2674	2374	2374
q4	1395	1737	1332	1332
q5	4207	4105	4107	4105
q6	210	168	122	122
q7	1871	1809	1688	1688
q8	2521	2462	2445	2445
q9	6965	6835	6803	6803
q10	2953	3162	2722	2722
q11	551	514	474	474
q12	669	725	611	611
q13	3245	3626	3047	3047
q14	272	281	255	255
q15	514	461	463	461
q16	444	455	420	420
q17	1131	1512	1335	1335
q18	7365	7070	7026	7026
q19	786	787	811	787
q20	1911	1970	1820	1820
q21	4691	4386	4468	4386
q22	1106	1051	1028	1028
Total cold run time: 50335 ms
Total hot run time: 48530 ms

@doris-robot
Copy link

doris-robot commented Aug 13, 2025

TPC-DS: Total hot run time: 185056 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 687a6d8aa4d19f06072f37f6085a77e72acb33a9, data reload: false

query1	989	375	390	375
query2	6522	1725	1721	1721
query3	6752	229	227	227
query4	25993	23635	23142	23142
query5	4326	601	479	479
query6	287	207	215	207
query7	4633	516	284	284
query8	266	231	216	216
query9	8608	2863	2833	2833
query10	465	316	287	287
query11	15681	15026	14740	14740
query12	157	115	113	113
query13	1657	538	417	417
query14	8406	5606	5618	5606
query15	192	181	163	163
query16	7125	671	492	492
query17	920	694	576	576
query18	2007	394	299	299
query19	184	184	160	160
query20	128	128	116	116
query21	216	121	107	107
query22	4259	4328	4416	4328
query23	34103	33339	33370	33339
query24	8111	2334	2388	2334
query25	518	462	400	400
query26	1238	270	162	162
query27	2740	498	357	357
query28	4367	2248	2235	2235
query29	786	559	434	434
query30	292	222	185	185
query31	850	768	711	711
query32	83	71	73	71
query33	541	396	333	333
query34	773	841	505	505
query35	778	809	781	781
query36	1006	1037	896	896
query37	112	105	82	82
query38	4054	3926	3998	3926
query39	1517	1441	1407	1407
query40	224	122	110	110
query41	58	57	53	53
query42	119	107	115	107
query43	511	503	488	488
query44	1311	840	840	840
query45	168	176	168	168
query46	862	1000	618	618
query47	1834	1834	1757	1757
query48	369	425	307	307
query49	729	500	382	382
query50	641	683	417	417
query51	4134	4131	4094	4094
query52	115	111	101	101
query53	232	269	185	185
query54	592	580	520	520
query55	90	87	84	84
query56	312	302	305	302
query57	1197	1202	1128	1128
query58	284	263	266	263
query59	2644	2717	2655	2655
query60	358	357	331	331
query61	157	146	152	146
query62	822	704	679	679
query63	228	199	191	191
query64	4442	1003	687	687
query65	4257	4221	4170	4170
query66	1164	410	337	337
query67	15613	15230	15135	15135
query68	6282	905	561	561
query69	497	318	280	280
query70	1185	1149	1096	1096
query71	405	321	304	304
query72	5650	4920	4946	4920
query73	728	688	354	354
query74	8924	8878	8728	8728
query75	3138	3029	2637	2637
query76	3155	1119	724	724
query77	480	419	324	324
query78	9644	9734	8852	8852
query79	988	836	604	604
query80	604	545	472	472
query81	488	254	219	219
query82	416	135	108	108
query83	257	243	233	233
query84	297	94	81	81
query85	762	367	335	335
query86	335	307	280	280
query87	4238	4185	4160	4160
query88	2781	2201	2272	2201
query89	384	311	275	275
query90	1760	220	211	211
query91	139	137	113	113
query92	73	64	64	64
query93	1130	981	634	634
query94	670	396	304	304
query95	384	308	301	301
query96	477	595	273	273
query97	2636	2631	2621	2621
query98	226	215	203	203
query99	1329	1392	1283	1283
Total cold run time: 265571 ms
Total hot run time: 185056 ms

@doris-robot
Copy link

doris-robot commented Aug 13, 2025

ClickBench: Total hot run time: 32.09 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 687a6d8aa4d19f06072f37f6085a77e72acb33a9, data reload: false

query1	0.05	0.04	0.03
query2	0.09	0.05	0.04
query3	0.24	0.08	0.07
query4	1.62	0.11	0.11
query5	0.41	0.41	0.40
query6	1.16	0.64	0.64
query7	0.03	0.02	0.02
query8	0.06	0.04	0.04
query9	0.60	0.52	0.52
query10	0.57	0.57	0.57
query11	0.15	0.11	0.11
query12	0.15	0.12	0.11
query13	0.63	0.61	0.61
query14	0.79	0.82	0.84
query15	0.89	0.85	0.86
query16	0.38	0.40	0.38
query17	1.10	1.04	1.04
query18	0.21	0.20	0.20
query19	1.94	1.81	1.87
query20	0.02	0.01	0.01
query21	15.40	0.86	0.54
query22	0.76	1.28	0.78
query23	14.71	1.36	0.62
query24	7.18	0.78	0.33
query25	0.47	0.30	0.07
query26	0.64	0.16	0.12
query27	0.05	0.06	0.05
query28	9.58	0.97	0.42
query29	12.55	3.92	3.25
query30	3.04	2.98	2.91
query31	2.82	0.61	0.39
query32	3.22	0.54	0.47
query33	3.13	3.24	3.08
query34	16.01	5.49	4.95
query35	4.91	4.91	4.97
query36	0.71	0.52	0.49
query37	0.09	0.07	0.09
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.18	0.13	0.14
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.78 s
Total hot run time: 32.09 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 13, 2025

BE UT Coverage Report

Increment line coverage 100.00% (6/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 59.04% (16627/28164)
Line Coverage 47.93% (150818/314677)
Region Coverage 36.77% (113031/307401)
Branch Coverage 39.69% (50130/126305)

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 13, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (6/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.68% (22576/27641)
Line Coverage 74.30% (233808/314684)
Region Coverage 61.56% (194316/315662)
Branch Coverage 65.65% (83932/127851)

zclllyybb
zclllyybb approved these changes Aug 14, 2025
View reviewed changes
Copy link
Contributor

@zclllyybb zclllyybb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 14, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Aug 14, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 14, 2025

PR approved by anyone and no changes requested.

@zclllyybb zclllyybb merged commit 4379e8f into apache:master Aug 15, 2025
32 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

@zclllyybb zclllyybb zclllyybb approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Mryange @Thearas @doris-robot @hello-stephen @BiteTheDDDDt @zclllyybb