Skip to content
This repository has been archived by the owner on Jan 22, 2025. It is now read-only.

Handle cleaning of zero-lamport accounts w.r.t. Incremental Snapshots #18870

Merged

Conversation

brooksprumo
Copy link
Contributor

@brooksprumo brooksprumo commented Jul 23, 2021

Problem

Zero-lamport accounts are improperly cleaned up in the context of incremental snapshots.

Summary of Changes

  • Add Option<Slot> parameter to AccountsDb::clean_accounts() for the last_full_snapshot_slot. If Some, then purges_zero_lamports will be checked against this slot. Any purges for slots higher than the last full snapshot slot will be filtered out (i.e. _not_purged).
  • Add test to AccountsDb for cleaning zero-lamport accounts with the last_full_snapshot_slot parameter
  • Add test to snapshot_utils for cleaning zero-lamport accounts and doing incremental snapshots
  • Update all uses of clean_accounts(). Most all callers pass None now.

Fixes #18825

@codecov
Copy link

codecov bot commented Jul 24, 2021

Codecov Report

Merging #18870 (b208298) into master (f302774) will increase coverage by 0.0%.
The diff coverage is 97.2%.

@@           Coverage Diff            @@
##           master   #18870    +/-   ##
========================================
  Coverage    82.8%    82.9%            
========================================
  Files         453      453            
  Lines      128741   128967   +226     
========================================
+ Hits       106705   106977   +272     
+ Misses      22036    21990    -46     

@brooksprumo brooksprumo force-pushed the iss-v3-zero-lamport-accounts branch from d882d4a to 886697b Compare July 27, 2021 16:36
@brooksprumo
Copy link
Contributor Author

@carllin @lijunwangs @ryoqun This PR is now ready for review, but it is built on top of PR #18565. So this PR currently contains extra changes that will go away once the other PR is merged in.

Feel free to review these changes, which are mostly in AccountsDb::clean_accounts(). If you need to pick just one, the other PR is a better use of time. Once that PR is in, and this one gets rebased, I'll flip it to Ready For Review and add you all as reviewers.

Thanks in advance!

@brooksprumo brooksprumo force-pushed the iss-v3-zero-lamport-accounts branch from 886697b to e2a0ad2 Compare July 30, 2021 01:12
@brooksprumo brooksprumo marked this pull request as ready for review July 30, 2021 02:51
runtime/src/accounts_db.rs Outdated Show resolved Hide resolved
@brooksprumo
Copy link
Contributor Author

Moving this back to Draft as I iterate on it.

@brooksprumo brooksprumo marked this pull request as draft August 5, 2021 18:47
@brooksprumo brooksprumo force-pushed the iss-v3-zero-lamport-accounts branch 6 times, most recently from 2db8884 to 0572bc0 Compare August 9, 2021 19:48
@brooksprumo brooksprumo marked this pull request as ready for review August 9, 2021 23:00
@brooksprumo brooksprumo requested a review from carllin August 9, 2021 23:01
@brooksprumo
Copy link
Contributor Author

@carllin OK, PR should be good-to-review again! I've attempted to put the correct tests/checks in place, but I may've missed some. Hopefully this is moving in the right direction though!

runtime/src/accounts_db.rs Outdated Show resolved Hide resolved
Comment on lines 1785 to 1802
assert!(
last_full_snapshot_slot.is_some() || self.zero_lamport_accounts_to_purge_later.is_empty(),
"if snapshots are disabled, then zero_lamport_accounts_to_purge_later should always be empty"
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

runtime/src/accounts_db.rs Outdated Show resolved Hide resolved
runtime/src/accounts_db.rs Outdated Show resolved Hide resolved
Comment on lines 2013 to 2014
self.zero_lamport_accounts_to_purge_later
.insert((*slot, *pubkey));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like the crux of it 😃

Handle cleaning zero-lamport accounts in slots higher than the last full
snapshot slot.  This is part of the Incremental Snapshot work.

Fixes solana-labs#18825
@brooksprumo brooksprumo force-pushed the iss-v3-zero-lamport-accounts branch from e68dbb8 to 3af2c62 Compare August 10, 2021 16:40
@brooksprumo brooksprumo force-pushed the iss-v3-zero-lamport-accounts branch from 76c0527 to 450be98 Compare August 10, 2021 18:39
@brooksprumo brooksprumo requested a review from carllin August 11, 2021 21:03
Comment on lines +12290 to +12296
let do_test = |test_params: TestParameters| {
let account_info = AccountInfo {
store_id: 42,
offset: 123,
stored_size: 234,
lamports: 0,
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, good abstraction

@brooksprumo brooksprumo merged commit 5fb6b34 into solana-labs:master Aug 12, 2021
@brooksprumo brooksprumo deleted the iss-v3-zero-lamport-accounts branch August 12, 2021 20:56
steviez pushed a commit to steviez/solana that referenced this pull request Aug 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Incremental Snapshots] Handle cleaning zero-lamport accounts in slots higher than the last full snapshot slot
3 participants