Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(query): fix cannot obtain stack trace when segmentation fault #16566

Merged
merged 14 commits into from
Oct 10, 2024

Conversation

zhang2014
Copy link
Member

@zhang2014 zhang2014 commented Oct 8, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

fix(query): fix cannot obtain stack trace when segmentation fault

after this pr, macos, arm, x86 crash backtrace dump work fine.

############################### Crash fault info ###############################
PID: 22013
TID: 69
Version: v1.2.643-nightly-3284f56536(rust-1.81.0-nightly-2024-10-09T06:46:01.499999403Z)
Timestamp(UTC): 2024-10-09 07:01:16.940339208 UTC
Timestamp(Local): 2024-10-09 07:01:16.940377035 +00:00
QueryId: "5e838225-4db4-44c4-bdde-cc963ea670c1"
Signal 11 (SIGSEGV), si_code 1 (Unknown), Address 0x1f823a191b03f

Backtrace:
    0: backtrace::backtrace::libunwind::trace
             at /home/ubuntu/.cargo/git/checkouts/backtrace-rs-fb1f822361417489/72265be/src/backtrace/libunwind.rs:116:5
      backtrace::backtrace::trace_unsynchronized
             at /home/ubuntu/.cargo/git/checkouts/backtrace-rs-fb1f822361417489/72265be/src/backtrace/mod.rs:66:5
      backtrace::backtrace::trace
             at /home/ubuntu/.cargo/git/checkouts/backtrace-rs-fb1f822361417489/72265be/src/backtrace/mod.rs:53:14
   1: databend_common_tracing::panic_hook::captures_frames
             at /home/ubuntu/datafuse/src/common/tracing/src/panic_hook.rs:66:5
      databend_common_tracing::crash_hook::CrashHandler::recv_signal
             at /home/ubuntu/datafuse/src/common/tracing/src/crash_hook.rs:82:9
      databend_common_tracing::crash_hook::signal_handler
             at /home/ubuntu/datafuse/src/common/tracing/src/crash_hook.rs:274:9
   2: <unknown>
   3: __memcmpeq
             at ./string/../sysdeps/aarch64/memcmp.S:121
   4: <u8 as core::slice::cmp::SliceOrd>::compare
             at /rustc/cf2df68d1f5e56803c97d91e2b1a9f1c9923c533/library/core/src/slice/cmp.rs:199:34

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@zhang2014 zhang2014 added the ci-cloud Build docker image for cloud test label Oct 8, 2024
@github-actions github-actions bot added the pr-bugfix this PR patches a bug in codebase label Oct 8, 2024
Copy link
Contributor

github-actions bot commented Oct 8, 2024

Docker Image for PR

  • tag: pr-16566-691a314-1728377247

note: this image tag is only available for internal use,
please check the internal doc for more details.

@zhang2014 zhang2014 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Oct 8, 2024
Copy link
Contributor

github-actions bot commented Oct 8, 2024

Docker Image for PR

  • tag: pr-16566-57a91d5-1728379429

note: this image tag is only available for internal use,
please check the internal doc for more details.

@zhang2014 zhang2014 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Oct 9, 2024
Copy link
Contributor

github-actions bot commented Oct 9, 2024

Docker Image for PR

  • tag: pr-16566-e3b6afb-1728459373

note: this image tag is only available for internal use,
please check the internal doc for more details.

@zhang2014 zhang2014 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Oct 9, 2024
Copy link
Contributor

github-actions bot commented Oct 9, 2024

Docker Image for PR

  • tag: pr-16566-609ad01-1728470419

note: this image tag is only available for internal use,
please check the internal doc for more details.

@zhang2014 zhang2014 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Oct 10, 2024
@zhang2014
Copy link
Member Author

comment --compress-debug-sections=zlib to ensure output crash backtrace. binary size: 607M -> 1.4G

@zhang2014 zhang2014 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Oct 10, 2024
@zhang2014 zhang2014 removed the ci-cloud Build docker image for cloud test label Oct 10, 2024
@zhang2014 zhang2014 added the ci-cloud Build docker image for cloud test label Oct 10, 2024
Copy link
Contributor

Docker Image for PR

  • tag: pr-16566-6cab64a-1728562820

note: this image tag is only available for internal use,
please check the internal doc for more details.

@zhang2014 zhang2014 marked this pull request as ready for review October 10, 2024 12:28
@zhang2014 zhang2014 requested a review from BohuTANG October 10, 2024 12:28
@sundy-li
Copy link
Member

works now

Backtrace:
    0: backtrace::backtrace::libunwind::trace
             at /home/sundy/.cargo/git/checkouts/backtrace-rs-fb1f822361417489/72265be/src/backtrace/libunwind.rs:116:5
      backtrace::backtrace::trace_unsynchronized
             at /home/sundy/.cargo/git/checkouts/backtrace-rs-fb1f822361417489/72265be/src/backtrace/mod.rs:66:5
      backtrace::backtrace::trace
             at /home/sundy/.cargo/git/checkouts/backtrace-rs-fb1f822361417489/72265be/src/backtrace/mod.rs:53:14
   1: databend_common_tracing::panic_hook::captures_frames
             at /data/work/databend/src/common/tracing/src/panic_hook.rs:66:5
      databend_common_tracing::crash_hook::CrashHandler::recv_signal
             at /data/work/databend/src/common/tracing/src/crash_hook.rs:82:9
      databend_common_tracing::crash_hook::signal_handler
             at /data/work/databend/src/common/tracing/src/crash_hook.rs:246:9
   2: <unknown>
   3: <unknown>
   4: <u8 as core::slice::cmp::SliceOrd>::compare
             at /rustc/cf2df68d1f5e56803c97d91e2b1a9f1c9923c533/library/core/src/slice/cmp.rs:199:34
      core::slice::cmp::<impl core::cmp::Ord for [T]>::cmp
             at /rustc/cf2df68d1f5e56803c97d91e2b1a9f1c9923c533/library/core/src/slice/cmp.rs:31:9
      core::str::traits::<impl core::cmp::Ord for str>::cmp
             at /rustc/cf2df68d1f5e56803c97d91e2b1a9f1c9923c533/library/core/src/str/traits.rs:23:25
      <databend_common_expression::types::string::StringType as databend_common_expression::types::ValueType>::compare
             at /data/work/databend/src/query/expression/src/types/string.rs:174:9
      <databend_common_expression::kernels::sort_compare::SortCompare as databend_common_expression::utils::visitor::ValueVisitor>::visit_typed_column::{{closure}}
             at /data/work/databend/src/query/expression/src/kernels/sort_compare.rs:284:20
      databend_common_expression::kernels::sort_compare::SortCompare::generic_sort::{{closure}}
             at /data/work/databend/src/query/expression/src/kernels/sort_compare.rs:80:25
      core::slice::<impl [T]>::select_nth_unstable_by::{{closure}}
             at /rustc/cf2df68d1f5e56803c97d91e2b1a9f1c9923c533/library/core/src/slice/mod.rs:3147:70

@zhang2014 zhang2014 enabled auto-merge October 10, 2024 13:29
@BohuTANG BohuTANG disabled auto-merge October 10, 2024 14:11
@BohuTANG BohuTANG merged commit 131d2d8 into databendlabs:main Oct 10, 2024
70 of 71 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-cloud Build docker image for cloud test pr-bugfix this PR patches a bug in codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants