Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix trie serialize #1388

Merged
merged 15 commits into from
Nov 9, 2022
Merged

Fix trie serialize #1388

merged 15 commits into from
Nov 9, 2022

Conversation

turuslan
Copy link
Contributor

@turuslan turuslan commented Nov 2, 2022

Referenced issues

Description of the Change

  • Don't store merkleValue which are not hashed.
  • Fix trie serialize encoded children nodes multiple times.
  • Deduplicate trie serialize and codec.
  • Fix (and simplify) prefix checks.

Benefits

  • Reduce storage usage.
  • Reduce trie serialize time.
  • Fix TopperTrieBatchImpl::clearPrefix

Possible Drawbacks

Signed-off-by: turuslan <turuslan.devbox@gmail.com>
Signed-off-by: turuslan <turuslan.devbox@gmail.com>
Signed-off-by: turuslan <turuslan.devbox@gmail.com>
Signed-off-by: turuslan <turuslan.devbox@gmail.com>
@codecov
Copy link

codecov bot commented Nov 2, 2022

Codecov Report

Merging #1388 (1751a12) into master (eab73e2) will decrease coverage by 0.02%.
The diff coverage is 20.58%.

@@            Coverage Diff             @@
##           master    #1388      +/-   ##
==========================================
- Coverage   24.87%   24.84%   -0.03%     
==========================================
  Files         629      629              
  Lines       23808    23794      -14     
  Branches    12328    12314      -14     
==========================================
- Hits         5922     5912      -10     
- Misses      12598    12609      +11     
+ Partials     5288     5273      -15     
Impacted Files Coverage Δ
.../service/child_state/impl/child_state_api_impl.cpp 20.73% <0.00%> (-1.37%) ⬇️
core/api/service/state/impl/state_api_impl.cpp 29.03% <0.00%> (-0.34%) ⬇️
core/network/impl/state_protocol_observer_impl.cpp 16.45% <0.00%> (+0.60%) ⬆️
core/storage/trie/impl/topper_trie_batch_impl.cpp 18.66% <0.00%> (+0.48%) ⬆️
core/storage/trie/serialization/polkadot_codec.hpp 100.00% <ø> (ø)
...torage/trie/serialization/trie_serializer_impl.hpp 100.00% <ø> (ø)
core/utils/kagome_db_editor.cpp 0.00% <0.00%> (ø)
core/storage/trie/codec.hpp 40.00% <25.00%> (-60.00%) ⬇️
core/storage/trie/serialization/polkadot_codec.cpp 34.56% <28.57%> (-0.54%) ⬇️
...torage/trie/serialization/trie_serializer_impl.cpp 51.06% <33.33%> (+1.06%) ⬆️
... and 5 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Signed-off-by: turuslan <turuslan.devbox@gmail.com>
Signed-off-by: turuslan <turuslan.devbox@gmail.com>
Signed-off-by: turuslan <turuslan.devbox@gmail.com>
Signed-off-by: turuslan <turuslan.devbox@gmail.com>
Signed-off-by: turuslan <turuslan.devbox@gmail.com>
@turuslan turuslan requested review from xDimon and Harrm November 3, 2022 07:36
Signed-off-by: turuslan <turuslan.devbox@gmail.com>
core/storage/trie/codec.hpp Outdated Show resolved Hide resolved
@@ -118,7 +88,13 @@ namespace kagome::storage::trie {
if (db_key.empty() or db_key == getEmptyRootHash()) {
return nullptr;
}
OUTCOME_TRY(enc, backend_->load(db_key));
Buffer enc;
if (codec_->isMerkleHash(db_key)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it's better to comment that a db key is actually the merkle value of the node, or better wrap Buffer in some MerkleValue type to ensure this contract on type level.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear from this comment why it is this way. I think it's better clarify it, or it may become a source of confusion.

Signed-off-by: turuslan <turuslan.devbox@gmail.com>
@turuslan turuslan requested a review from Harrm November 7, 2022 11:45
@turuslan turuslan enabled auto-merge (squash) November 9, 2022 05:49
@turuslan turuslan merged commit ed531b0 into master Nov 9, 2022
@turuslan turuslan deleted the refactor/trie branch November 9, 2022 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants