-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[Debug](distribute) Check bucket hash table before quit #53661
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
run buildall |
TPC-H: Total hot run time: 34255 ms |
TPC-DS: Total hot run time: 190128 ms |
ClickBench: Total hot run time: 33.39 s |
FE UT Coverage ReportIncrement line coverage |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
16db9d2 to
c88b5f6
Compare
|
run buildall |
TPC-H: Total hot run time: 34070 ms |
TPC-DS: Total hot run time: 187689 ms |
ClickBench: Total hot run time: 32.5 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
c88b5f6 to
9209ca7
Compare
|
run buildall |
TPC-H: Total hot run time: 33863 ms |
TPC-DS: Total hot run time: 187451 ms |
ClickBench: Total hot run time: 32.1 s |
FE UT Coverage ReportIncrement line coverage |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
0c44bed to
47b6f18
Compare
|
run buildall |
TPC-H: Total hot run time: 34190 ms |
d38d205 to
160a09c
Compare
|
run buildall |
TPC-H: Total hot run time: 34352 ms |
TPC-DS: Total hot run time: 162618 ms |
ClickBench: Total hot run time: 33.06 s |
FE UT Coverage ReportIncrement line coverage |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
dataroaring
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
We found a bug that rows may be distributed to wrong tablet when loading to bucket hash table in rare situations and it's hard to find the root cause currently. For better debugging, this PR 1. Add a function `crc32_internal` to mimic the hashing method when distributed data in sink node for bucket hash table 2. check that if all remaining bucket hash tables' data are correct after all tests finished.
What problem does this PR solve?
We found a bug that rows may be distributed to wrong tablet when loading to bucket hash table in rare situations
and it's hard to find the root cause currently(has been solved in #54024, #54072).For better debugging, this PR
crc32_internalto mimic the hashing method when distributed data in sink node for bucket hash tableRelease note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)