Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pruner): account history #4000

Merged
merged 20 commits into from
Aug 2, 2023
Merged

feat(pruner): account history #4000

merged 20 commits into from
Aug 2, 2023

Conversation

shekhirin
Copy link
Collaborator

@shekhirin shekhirin commented Jul 31, 2023

Resolves #3692

Prunes:

  1. AccountChangeSet table: all block number keys (and their subkeys which are account addresses) which need to be pruned
  2. AccountHistory table:
    1. If the shard has highest_block_number less than or equal to the target block number for pruning, delete the shard completely.
    2. If the shard has highest_block_number greater than the target block number for pruning, filter block numbers inside the shard which are less than the target block number for pruning.

@codecov
Copy link

codecov bot commented Jul 31, 2023

Codecov Report

Merging #4000 (cdf2a8b) into main (9510a5c) will increase coverage by 0.03%.
The diff coverage is 90.64%.

Impacted file tree graph

Files Changed Coverage Δ
crates/prune/src/pruner.rs 84.59% <85.21%> (+0.18%) ⬆️
crates/stages/src/test_utils/test_db.rs 84.69% <93.93%> (+0.52%) ⬆️
crates/interfaces/src/test_utils/generators.rs 95.48% <100.00%> (+0.05%) ⬆️
crates/stages/src/stages/index_account_history.rs 96.24% <100.00%> (ø)
crates/stages/src/stages/index_storage_history.rs 96.12% <100.00%> (ø)
crates/stages/src/stages/merkle.rs 83.06% <100.00%> (ø)
...torage/provider/src/providers/database/provider.rs 78.61% <100.00%> (+0.21%) ⬆️

... and 10 files with indirect coverage changes

Flag Coverage Δ
integration-tests 16.49% <0.00%> (-0.05%) ⬇️
unit-tests 64.12% <90.64%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
reth binary 25.84% <ø> (ø)
blockchain tree 83.04% <ø> (ø)
pipeline 90.10% <94.87%> (+<0.01%) ⬆️
storage (db) 74.33% <100.00%> (+0.03%) ⬆️
trie 94.70% <ø> (ø)
txpool 47.36% <ø> (ø)
networking 77.42% <ø> (-0.02%) ⬇️
rpc 58.27% <ø> (-0.01%) ⬇️
consensus 63.50% <ø> (ø)
revm 32.71% <ø> (ø)
payload builder 6.58% <ø> (ø)
primitives 88.00% <100.00%> (+<0.01%) ⬆️

@shekhirin shekhirin marked this pull request as ready for review July 31, 2023 15:57
@shekhirin shekhirin requested review from joshieDo and mattsse and removed request for Rjected, rakita, gakonst, mattsse and rkrasiuk July 31, 2023 15:57
@shekhirin shekhirin force-pushed the alexey/pruner-account-history branch 2 times, most recently from b9a7784 to 13131a4 Compare July 31, 2023 16:55
@shekhirin shekhirin marked this pull request as draft August 1, 2023 07:37
@shekhirin shekhirin marked this pull request as ready for review August 1, 2023 14:30
@shekhirin shekhirin added C-enhancement New feature or request A-pruning Related to pruning or full node labels Aug 1, 2023
)?;

let mut cursor = provider.tx_ref().cursor_write::<tables::AccountHistory>()?;
while let Some(result) = cursor.next()? {
Copy link
Collaborator

@joshieDo joshieDo Aug 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we just add a short summary comment on what we're doing on this loop?

a lot of if-else conditions.... having a short summary before diving in is helpful imo

edit: something like the PR summary actually

@shekhirin shekhirin force-pushed the alexey/pruner-account-history branch from 670667f to f7c0d81 Compare August 2, 2023 13:13
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm,

love the tests

@shekhirin shekhirin added this pull request to the merge queue Aug 2, 2023
Merged via the queue into main with commit 88f83fc Aug 2, 2023
@shekhirin shekhirin deleted the alexey/pruner-account-history branch August 2, 2023 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-pruning Related to pruning or full node C-enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Respect Account History prune part in the pruner
3 participants