-
Notifications
You must be signed in to change notification settings - Fork 605
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test(encoding): bench key/value encoding #5165
Conversation
Codecov Report
@@ Coverage Diff @@
## main #5165 +/- ##
==========================================
- Coverage 74.30% 74.29% -0.01%
==========================================
Files 924 924
Lines 144308 144259 -49
==========================================
- Hits 107225 107184 -41
+ Misses 37083 37075 -8
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
The memcomparable encoding should always do more things than value encoding, so I believe there's something wrong with value encoding for list/struct. Every scalar is first encoded to a protobuf message, then serialized to bytes. 😇 Note that @wzzzzd has fixed the memcomparable in #4599. risingwave/src/common/src/util/value_encoding/mod.rs Lines 89 to 94 in 9af05d1
cc @neverchanje |
Would you please also bench the ser/de for integer and decimals? Though there should be no much difference. |
I have benched every type for |
I think so. |
I guess we used protobuf in the first place just for simplicity, right? |
I think I will add some env variables to filter test cases and enable encode/decode bench. |
List/struct deser is also slower |
8a6ea15
to
8d5c892
Compare
This is kind of strange. 🤔 |
Doesn't |
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
I've also tested a lot and found that the bad performance of deserializing integer key is not from
So I guess that the overhead is indeed from |
Bench results on my MacBook:
|
I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.
What's changed and what's your intention?
As title. From the benchmark result, value encoding is better at encoding strings, while key encoding is better at encoding list or struct.
Checklist
./risedev check
(or alias,./risedev c
)Documentation
If your pull request contains user-facing changes, please specify the types of the changes, and create a release note. Otherwise, please feel free to remove this section.
Types of user-facing changes
Please keep the types that apply to your changes, and remove those that do not apply.
Release note
Please create a release note for your changes. In the release note, focus on the impact on users, and mention the environment or conditions where the impact may occur.
Refer to a related PR or issue link (optional)