-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Blockstore: track when all primary-index data has been purged #33668
Conversation
…_status_with_counter
3c4c052
to
7285bc4
Compare
Codecov Report
@@ Coverage Diff @@
## master #33668 +/- ##
=========================================
- Coverage 81.8% 81.8% -0.1%
=========================================
Files 806 806
Lines 217541 217593 +52
=========================================
+ Hits 178045 178054 +9
- Misses 39496 39539 +43 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logic all looks good, just the one nit.
Incidentally, I was originally planning to add to this PR logic to clear the signature-only TransactionMemos in the CompactionFilter when all the primary-index data has been purged. But since the filters are built before the Blockstore element is created, it's a little tricky. Will approach that in a follow-up.
Sounds good!
Co-authored-by: steviez <stevecz@umich.edu>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Problem
#33419 changes the keys of special-column entries; it also adds a fallback so that nodes with mixed data can read both key versions, and checks the old version when the new-version key cannot be found. However, once a node has purged all of its old-format keys, there is no reason to spend these additional db read attempts
Summary of Changes
Track the highest_primary_index_slot in memory
Set this value to None when all the primary-index-key data has been purged
Use this value to optimize various Blockstore read methods, as well as
purge_special_columns_exact()
Incidentally, I was originally planning to add to this PR logic to clear the signature-only TransactionMemos in the CompactionFilter when all the primary-index data has been purged. But since the filters are built before the Blockstore element is created, it's a little tricky. Will approach that in a follow-up.