-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[opt](exec)lazy deserialize pblock in VDataStreamRecvr::SenderQueue #44378
Conversation
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
run buildall |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clang-tidy made some suggestions
@@ -18,6 +18,7 @@ | |||
#pragma once | |||
|
|||
#include <gen_cpp/Types_types.h> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
warning: 'gen_cpp/Types_types.h' file not found [clang-diagnostic-error]
#include <gen_cpp/Types_types.h>
^
TeamCity be ut coverage result: |
fb54dcd
to
6c96e8d
Compare
run buildall |
TeamCity be ut coverage result: |
6c96e8d
to
291194a
Compare
run buildall |
TeamCity be ut coverage result: |
291194a
to
22bb009
Compare
run buildall |
_recvr->_parent->memory_used_counter()->update(-(int64_t)block_byte_size); | ||
std::lock_guard<std::mutex> l(_lock); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rethink the logic
run buildall |
TeamCity be ut coverage result: |
run p0 |
1 similar comment
run p0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
What problem does this PR solve?
Previously, for a
pblock
(serialized block), the block would be deserialized immediatelyafter receiving the RPC request and then placed into the
data_queue
.This approach caused significant time consumption during RPC processing due to the
deserialization process, impacting overall performance.
The new approach defers deserialization until
getBlock
is called. This has the following advantages:and reducing contention on memory resources.
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)