Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blockstore: track when all primary-index data has been purged #33668

Merged
merged 12 commits into from
Oct 12, 2023

Conversation

CriesofCarrots
Copy link
Contributor

Problem

#33419 changes the keys of special-column entries; it also adds a fallback so that nodes with mixed data can read both key versions, and checks the old version when the new-version key cannot be found. However, once a node has purged all of its old-format keys, there is no reason to spend these additional db read attempts

Summary of Changes

Track the highest_primary_index_slot in memory
Set this value to None when all the primary-index-key data has been purged
Use this value to optimize various Blockstore read methods, as well as purge_special_columns_exact()

Incidentally, I was originally planning to add to this PR logic to clear the signature-only TransactionMemos in the CompactionFilter when all the primary-index data has been purged. But since the filters are built before the Blockstore element is created, it's a little tricky. Will approach that in a follow-up.

@codecov
Copy link

codecov bot commented Oct 12, 2023

Codecov Report

Merging #33668 (620edcc) into master (21b2fce) will decrease coverage by 0.1%.
Report is 1 commits behind head on master.
The diff coverage is 79.6%.

@@            Coverage Diff            @@
##           master   #33668     +/-   ##
=========================================
- Coverage    81.8%    81.8%   -0.1%     
=========================================
  Files         806      806             
  Lines      217541   217593     +52     
=========================================
+ Hits       178045   178054      +9     
- Misses      39496    39539     +43     

Copy link
Contributor

@steviez steviez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic all looks good, just the one nit.

Incidentally, I was originally planning to add to this PR logic to clear the signature-only TransactionMemos in the CompactionFilter when all the primary-index data has been purged. But since the filters are built before the Blockstore element is created, it's a little tricky. Will approach that in a follow-up.

Sounds good!

ledger/src/blockstore/blockstore_purge.rs Outdated Show resolved Hide resolved
Co-authored-by: steviez <stevecz@umich.edu>
Copy link
Contributor

@steviez steviez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@CriesofCarrots CriesofCarrots added the automerge Merge this Pull Request automatically once CI passes label Oct 12, 2023
@mergify mergify bot merged commit d286c00 into solana-labs:master Oct 12, 2023
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge Merge this Pull Request automatically once CI passes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants