Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: avoid using RocksDB keys estimation #496

Merged
merged 25 commits into from
Nov 1, 2022

Conversation

luislhl
Copy link
Collaborator

@luislhl luislhl commented Oct 10, 2022

This was created from the discussion #490 (comment)

Acceptance Criteria

We should prefer to use the InfoIndex when possible to get a count of the number of vertices, instead of relying on specific logic for each kind of storage to count the number of vertices.

Specifically, we notices that the specific logic we had to count the number of vertices in RocksDB was not accurate. See the link above for more info.

This change highlighted a bug we had in the InfoIndex count of txs/blocks when a split-brain happens. It was counting the txs/blocks of the winner chain twice when the nodes connected again and the winner was chosen. We had to fix this bug in this PR as well.

@luislhl luislhl self-assigned this Oct 10, 2022
@luislhl luislhl force-pushed the chore/sync-time-monitoring branch from 38aa5b3 to 58899d6 Compare October 10, 2022 11:58
@luislhl luislhl force-pushed the chore/avoid-rocksdb-key-estimation branch from 599c77c to c436d15 Compare October 10, 2022 12:16
tests/tx/test_tx_storage.py Outdated Show resolved Hide resolved
hathor/transaction/storage/cache_storage.py Outdated Show resolved Hide resolved
hathor/transaction/storage/rocksdb_storage.py Outdated Show resolved Hide resolved
hathor/transaction/storage/rocksdb_storage.py Outdated Show resolved Hide resolved
@luislhl luislhl force-pushed the chore/sync-time-monitoring branch from 58899d6 to cb6ba8f Compare October 14, 2022 18:13
Base automatically changed from chore/sync-time-monitoring to dev October 17, 2022 13:26
@luislhl luislhl changed the title chore: avoid using RocksDB key estimation chore: avoid using RocksDB keys estimation Oct 17, 2022
@luislhl luislhl force-pushed the chore/avoid-rocksdb-key-estimation branch from c436d15 to 5283917 Compare October 17, 2022 17:09
@codecov
Copy link

codecov bot commented Oct 18, 2022

Codecov Report

Merging #496 (dd3826e) into dev (6bfc2f9) will decrease coverage by 0.12%.
The diff coverage is 81.33%.

@@            Coverage Diff             @@
##              dev     #496      +/-   ##
==========================================
- Coverage   83.69%   83.57%   -0.13%     
==========================================
  Files         184      187       +3     
  Lines       16452    16829     +377     
  Branches     2520     2590      +70     
==========================================
+ Hits        13770    14065     +295     
- Misses       2230     2300      +70     
- Partials      452      464      +12     
Impacted Files Coverage Δ
hathor/transaction/storage/binary_storage.py 90.25% <43.75%> (-7.59%) ⬇️
hathor/transaction/storage/compact_storage.py 89.93% <43.75%> (-6.60%) ⬇️
hathor/transaction/storage/cache_storage.py 91.97% <57.14%> (-2.73%) ⬇️
hathor/transaction/storage/transaction_storage.py 89.79% <63.00%> (-5.33%) ⬇️
hathor/transaction/storage/rocksdb_storage.py 94.73% <68.18%> (-5.27%) ⬇️
hathor/event/storage/rocksdb_storage.py 83.63% <74.28%> (-16.37%) ⬇️
...tion/storage/migrations/add_min_height_metadata.py 80.00% <80.00%> (ø)
hathor/transaction/storage/migrations/__init__.py 82.14% <82.14%> (ø)
hathor/indexes/mempool_tips_index.py 93.57% <88.46%> (-2.17%) ⬇️
hathor/event/storage/memory_storage.py 93.10% <90.00%> (-6.90%) ⬇️
... and 26 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@luislhl luislhl force-pushed the chore/avoid-rocksdb-key-estimation branch from 4001833 to 9bb01d2 Compare October 18, 2022 15:52
@luislhl luislhl marked this pull request as ready for review October 18, 2022 15:52
@luislhl luislhl force-pushed the chore/avoid-rocksdb-key-estimation branch from 9bb01d2 to 3e76498 Compare October 18, 2022 21:57
hathor/transaction/storage/rocksdb_storage.py Outdated Show resolved Hide resolved
@luislhl luislhl force-pushed the chore/avoid-rocksdb-key-estimation branch from 3e76498 to 1df4287 Compare October 28, 2022 14:12
@luislhl luislhl force-pushed the chore/avoid-rocksdb-key-estimation branch from 1df4287 to 9de646f Compare October 31, 2022 18:41
@luislhl luislhl requested review from msbrogli and jansegre November 1, 2022 18:54
@luislhl luislhl merged commit face9bb into dev Nov 1, 2022
@luislhl luislhl deleted the chore/avoid-rocksdb-key-estimation branch November 1, 2022 20:56
@jansegre jansegre mentioned this pull request Nov 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants