Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ASAN] heap-buffer-overflow in SiStripClusters2ApproxClusters::produce #42131

Closed
iarspider opened this issue Jun 29, 2023 · 9 comments · Fixed by #42486
Closed

[ASAN] heap-buffer-overflow in SiStripClusters2ApproxClusters::produce #42131

iarspider opened this issue Jun 29, 2023 · 9 comments · Fixed by #42486
Labels
alca-approved pending-signatures reconstruction-pending trk

Comments

@iarspider
Copy link
Contributor 

=================================================================
==3373081==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x14c35b0b67ff at pc 0x14c375779a06 bp 0x14c36d3fd760 sp 0x14c36d3fd758
READ of size 1 at 0x14c35b0b67ff thread T2
    #0 0x14c375779a05 in SiStripClusters2ApproxClusters::produce(edm::Event&, edm::EventSetup const&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02791/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_ASAN_X_2023-06-28-2300/lib/el8_amd64_gcc11/pluginRecoLocalTrackerSiStripClusterizerPlugins.so+0x12ba05)
    #1 0x14c3a9bae417 in edm::stream::EDProducerAdaptorBase::doEvent(edm::EventTransitionInfo const&, edm::ActivityRegistry*, edm::ModuleCallingContext const*) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02791/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_ASAN_X_2023-06-28-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so+0x7c2417)
    #2 0x14c3a9b209a0 in edm::WorkerT<edm::stream::EDProducerAdaptorBase>::implDo(edm::EventTransitionInfo const&, edm::ModuleCallingContext const*) [clone .localalias] [clone .lto_priv.0] (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02791/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_ASAN_X_2023-06-28-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so+0x7349a0)
    #3 0x14c3a98d2e2c in std::__exception_ptr::exception_ptr edm::Worker::runModuleAfterAsyncPrefetch<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >(std::__exception_ptr::exception_ptr, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::Context const*) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02791/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_ASAN_X_2023-06-28-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so+0x4e6e2c)
    #4 0x14c3a98f4a1a in edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >::execute() (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02791/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_ASAN_X_2023-06-28-2300/lib/el8_amd64_gcc11/libFWCoreFramework.so+0x508a1a)
    #5 0x14c3aa87b6cb in tbb::detail::d1::function_task<edm::WaitingTaskList::announce()::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02791/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_ASAN_X_2023-06-28-2300/lib/el8_amd64_gcc11/libFWCoreConcurrency.so+0x96cb)
    #6 0x14c3a766c96e in tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /data/cmsbld/jenkins/workspace/jenkins-test-bootstrap/toolconf/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-b1d97183c007769a478c02aa8504b8cb/tbb-v2021.9.0/src/tbb/task_dispatcher.h:322
    #7 0x14c3a766c96e in tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /data/cmsbld/jenkins/workspace/jenkins-test-bootstrap/toolconf/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-b1d97183c007769a478c02aa8504b8cb/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458
    #8 0x14c3a766c96e in tbb::detail::r1::arena::process(tbb::detail::r1::thread_data&) /data/cmsbld/jenkins/workspace/jenkins-test-bootstrap/toolconf/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-b1d97183c007769a478c02aa8504b8cb/tbb-v2021.9.0/src/tbb/arena.cpp:137
    #9 0x14c3a766c96e in tbb::detail::r1::market::process(rml::job&) /data/cmsbld/jenkins/workspace/jenkins-test-bootstrap/toolconf/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-b1d97183c007769a478c02aa8504b8cb/tbb-v2021.9.0/src/tbb/market.cpp:599
    #10 0x14c3a766ebb4 in tbb::detail::r1::rml::private_worker::run() /data/cmsbld/jenkins/workspace/jenkins-test-bootstrap/toolconf/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-b1d97183c007769a478c02aa8504b8cb/tbb-v2021.9.0/src/tbb/private_server.cpp:271
    #11 0x14c3a766ebb4 in tbb::detail::r1::rml::private_worker::thread_routine(void*) /data/cmsbld/jenkins/workspace/jenkins-test-bootstrap/toolconf/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-b1d97183c007769a478c02aa8504b8cb/tbb-v2021.9.0/src/tbb/private_server.cpp:221
    #12 0x14c3a67c71c9 in start_thread (/lib64/libpthread.so.0+0x81c9)
    #13 0x14c3a6433e72 in __clone (/lib64/libc.so.6+0x39e72)

0x14c35b0b67ff is located 1 bytes to the left of 10166112-byte region [0x14c35b0b6800,0x14c35ba68760)
allocated by thread T0 here:
    #0 0x14c3a9ef5f57 in operator new(unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cpp:99
    #1 0x14c3815253db in std::vector<unsigned char, std::allocator<unsigned char> >::reserve(unsigned long) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02791/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_ASAN_X_2023-06-28-2300/lib/el8_amd64_gcc11/libCondFormatsCommon.so+0x803db)
    #2 0x14c381560515 in boost::archive::detail::iserializer<eos::portable_iarchive, std::vector<unsigned char, std::allocator<unsigned char> > >::load_object_data(boost::archive::detail::basic_iarchive&, void*, unsigned int) const [clone .localalias] [clone .lto_priv.0] (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02791/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_ASAN_X_2023-06-28-2300/lib/el8_amd64_gcc11/libCondFormatsCommon.so+0xbb515)
    #3 0x14c38132de3f in boost::archive::detail::basic_iarchive::load_object(void*, boost::archive::detail::basic_iserializer const&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02791/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_ASAN_X_2023-06-28-2300/external/el8_amd64_gcc11/lib/libboost_serialization.so.1.80.0+0x17e3f)

Thread T2 created by T0 here:
    #0 0x14c3a9e9c706 in __interceptor_pthread_create ../../../../libsanitizer/asan/asan_interceptors.cpp:216
    #1 0x14c3a766e241 in tbb::detail::r1::rml::internal::thread_monitor::launch(void* (*)(void*), void*, unsigned long) /data/cmsbld/jenkins/workspace/jenkins-test-bootstrap/toolconf/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-b1d97183c007769a478c02aa8504b8cb/tbb-v2021.9.0/src/tbb/rml_thread_monitor.h:208
    #2 0x14c3a766e241 in tbb::detail::r1::rml::private_worker::wake_or_launch() /data/cmsbld/jenkins/workspace/jenkins-test-bootstrap/toolconf/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-b1d97183c007769a478c02aa8504b8cb/tbb-v2021.9.0/src/tbb/private_server.cpp:305
    #3 0x14c3a766e241 in tbb::detail::r1::rml::private_server::wake_some(int) /data/cmsbld/jenkins/workspace/jenkins-test-bootstrap/toolconf/BUILD/el8_amd64_gcc11/external/tbb/v2021.9.0-b1d97183c007769a478c02aa8504b8cb/tbb-v2021.9.0/src/tbb/private_server.cpp:412

SUMMARY: AddressSanitizer: heap-buffer-overflow (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02791/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_ASAN_X_2023-06-28-2300/lib/el8_amd64_gcc11/pluginRecoLocalTrackerSiStripClusterizerPlugins.so+0x12ba05) in SiStripClusters2ApproxClusters::produce(edm::Event&, edm::EventSetup const&)
Shadow bytes around the buggy address:
  0x0298eb60eca0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0298eb60ecb0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0298eb60ecc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0298eb60ecd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0298eb60ece0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0298eb60ecf0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa[fa]
  0x0298eb60ed00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0298eb60ed10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0298eb60ed20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0298eb60ed30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0298eb60ed40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==3373081==ABORTING

Full log: link
@cmsbuild
Copy link
Contributor

A new Issue was created by @iarspider .

@Dr15Jones, @perrotta, @dpiparo, @rappoccio, @makortel, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

@iarspider
Copy link
Contributor Author

assign reconstruction

@cmsbuild
Copy link
Contributor

New categories assigned: reconstruction

@mandrenguyen,@clacaputo you have been requested to review this Pull request/Issue and eventually sign? Thanks

@mmusich
Copy link
Contributor

mmusich commented Jun 30, 2023 •
edited
Loading

Both this and the other companion issue #42162 are caused because there is a condition (SiStripNoises) which is read off its domain from this call:

cmssw/RecoTracker/PixelLowPtUtilities/interface/SlidingPeakFinder.h

Line 72 in fe01c0e

cut_ = std::min<float>(seedCutMIPs * mip, seedCutSN * noiseObj_->getNoise(firstStrip + 1, noises_));

when it looks for the noise of strip n. 768 for Detid 369120277.
I suppose this was triggered by #41815 (tagging also @Ksavva1021) .
I am not sure why the payload stored in the tag SiStripNoise_v2_prompt for run 326417 (which is the run used for wf 140.58) is missing this particular strip.

@mmusich mmusich mentioned this issue Jun 30, 2023
Closed
@aandvalenzuela
Copy link
Contributor

RelVal 140.58 is still failing with the same issue in the latest IBs. Full log.

@makortel
Copy link
Contributor

makortel commented Aug 1, 2023

assign alca

FYI @cms-sw/trk-dpg-l2

(since the issue is likely the same as in #42162)

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 1, 2023

New categories assigned: alca

@perrotta,@consuegs,@francescobrivio,@saumyaphor4252,@tvami you have been requested to review this Pull request/Issue and eventually sign? Thanks

@mmusich
Copy link
Contributor

mmusich commented Aug 7, 2023

type trk

@cmsbuild cmsbuild added the trk label Aug 7, 2023
@mmusich mmusich mentioned this issue Aug 7, 2023
Merged
@tvami
Copy link
Contributor

tvami commented Aug 9, 2023

+alca

@mmusich mmusich mentioned this issue Aug 9, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
alca-approved pending-signatures reconstruction-pending trk
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

SiStripClusters2ApproxClusters: run peakFilter only if cluster is usable mmusich/cmssw
6 participants
@makortel @iarspider @cmsbuild @tvami @mmusich @aandvalenzuela