Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(book): [prune] config section #4328

Merged
merged 1 commit into from
Aug 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions bin/reth/src/args/pruning_args.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
use clap::Args;
use reth_config::config::PruneConfig;
use reth_primitives::{
ChainSpec, ContractLogsPruneConfig, PruneMode, PruneModes, MINIMUM_PRUNING_DISTANCE,
ChainSpec, PruneMode, PruneModes, ReceiptsLogPruneConfig, MINIMUM_PRUNING_DISTANCE,
};
use std::sync::Arc;

Expand Down Expand Up @@ -35,7 +35,7 @@ impl PruningArgs {
.map(|contract| PruneMode::Before(contract.block)),
account_history: Some(PruneMode::Distance(MINIMUM_PRUNING_DISTANCE)),
storage_history: Some(PruneMode::Distance(MINIMUM_PRUNING_DISTANCE)),
contract_logs_filter: ContractLogsPruneConfig(
receipts_log_filter: ReceiptsLogPruneConfig(
_chain_spec
.deposit_contract
.as_ref()
Expand Down
53 changes: 53 additions & 0 deletions book/run/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ The configuration file contains the following sections:
- [`reputation_weights`](#reputation_weights)
- [`backoff_durations`](#backoff_durations)
- [`[sessions]`](#the-sessions-section)
- [`[prune]`](#the-prune-section)

## The `[stages]` section

Expand Down Expand Up @@ -330,4 +331,56 @@ secs = 120
nanos = 0
```

## The `[prune]` section

The prune section configures the pruning configuration.

You can configure the pruning of different parts of the data independently of others.
For any unspecified parts, the default setting is no pruning.

### Default config

No pruning, run as archive node.

### Example of the custom pruning configuration

This configuration will:
- Run pruning every 5 blocks
- Continuously prune all transaction senders, account history and storage history before the block `head-128`, i.e. keep the data for the last 129 blocks
- Prune all receipts before the block 1920000, i.e. keep receipts from the block 1920000

```toml
[prune]
# Minimum pruning interval measured in blocks
block_interval = 5

[prune.parts]
# Sender Recovery pruning configuration
sender_recovery = { distance = 128 } # Prune all transaction senders before the block `head-128`, i.e. keep transaction senders for the last 129 blocks

# Transaction Lookup pruning configuration
transaction_lookup = "full" # Prune all TxNumber => TxHash mappings

# Receipts pruning configuration. This setting overrides `receipts_log_filter`.
receipts = { before = 1920000 } # Prune all receipts from transactions before the block 1920000, i.e. keep receipts from the block 1920000

# Account History pruning configuration
account_history = { distance = 128 } # Prune all historical account states before the block `head-128`

# Storage History pruning configuration
storage_history = { distance = 128 } # Prune all historical storage states before the block `head-128`
```

We can also prune receipts more granular, using the logs filtering:
```toml
# Receipts pruning configuration by retaining only those receipts that contain logs emitted
# by the specified addresses, discarding all others. This setting is overridden by `receipts`.
[prune.parts.receipts_log_filter]
# Prune all receipts, leaving only those which:
# - Contain logs from address `0x7ea2be2df7ba6e54b1a9c70676f668455e329d29`, starting from the block 17000000
# - Contain logs from address `0xdac17f958d2ee523a2206206994597c13d831ec7` in the last 1001 blocks
"0x7ea2be2df7ba6e54b1a9c70676f668455e329d29" = { before = 17000000 }
"0xdac17f958d2ee523a2206206994597c13d831ec7" = { distance = 1000 }
```

[TOML]: https://toml.io/
2 changes: 1 addition & 1 deletion crates/primitives/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ pub use net::{
};
pub use peer::{PeerId, WithPeerId};
pub use prune::{
ContractLogsPruneConfig, PruneCheckpoint, PruneMode, PruneModes, PrunePart, PrunePartError,
PruneCheckpoint, PruneMode, PruneModes, PrunePart, PrunePartError, ReceiptsLogPruneConfig,
MINIMUM_PRUNING_DISTANCE,
};
pub use receipt::{Receipt, ReceiptWithBloom, ReceiptWithBloomRef};
Expand Down
4 changes: 2 additions & 2 deletions crates/primitives/src/prune/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ pub use target::{PruneModes, MINIMUM_PRUNING_DISTANCE};

/// Configuration for pruning receipts not associated with logs emitted by the specified contracts.
#[derive(Debug, Clone, PartialEq, Eq, Default, Serialize, Deserialize)]
pub struct ContractLogsPruneConfig(pub BTreeMap<Address, PruneMode>);
pub struct ReceiptsLogPruneConfig(pub BTreeMap<Address, PruneMode>);

impl ContractLogsPruneConfig {
impl ReceiptsLogPruneConfig {
/// Checks if the configuration is empty
pub fn is_empty(&self) -> bool {
self.0.is_empty()
Expand Down
14 changes: 7 additions & 7 deletions crates/primitives/src/prune/target.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::{
prune::PrunePartError, serde_helper::deserialize_opt_prune_mode_with_min_blocks, BlockNumber,
ContractLogsPruneConfig, PruneMode, PrunePart,
PruneMode, PrunePart, ReceiptsLogPruneConfig,
};
use paste::paste;
use serde::{Deserialize, Serialize};
Expand All @@ -23,8 +23,8 @@ pub struct PruneModes {
/// Transaction Lookup pruning configuration.
#[serde(skip_serializing_if = "Option::is_none")]
pub transaction_lookup: Option<PruneMode>,
/// Configuration for pruning of receipts. This setting overrides
/// `PruneModes::contract_logs_filter` and offers improved performance.
/// Receipts pruning configuration. This setting overrides `receipts_log_filter`
/// and offers improved performance.
#[serde(
skip_serializing_if = "Option::is_none",
deserialize_with = "deserialize_opt_prune_mode_with_min_blocks::<64, _>"
Expand All @@ -42,12 +42,12 @@ pub struct PruneModes {
deserialize_with = "deserialize_opt_prune_mode_with_min_blocks::<64, _>"
)]
pub storage_history: Option<PruneMode>,
/// Retains only those receipts that contain logs emitted by the specified addresses,
/// discarding all others. Note that this setting is overridden by `PruneModes::receipts`.
/// Receipts pruning configuration by retaining only those receipts that contain logs emitted
/// by the specified addresses, discarding others. This setting is overridden by `receipts`.
///
/// The [`BlockNumber`] represents the starting block from which point onwards the receipts are
/// preserved.
pub contract_logs_filter: ContractLogsPruneConfig,
pub receipts_log_filter: ReceiptsLogPruneConfig,
}

macro_rules! impl_prune_parts {
Expand Down Expand Up @@ -90,7 +90,7 @@ macro_rules! impl_prune_parts {
$(
$part: Some(PruneMode::Full),
)+
contract_logs_filter: Default::default()
receipts_log_filter: Default::default()
}
}

Expand Down
6 changes: 3 additions & 3 deletions crates/prune/src/pruner.rs
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ impl<DB: Database> Pruner<DB> {
.record(part_start.elapsed())
}

if !self.modes.contract_logs_filter.is_empty() {
if !self.modes.receipts_log_filter.is_empty() {
let part_start = Instant::now();
self.prune_receipts_by_logs(&provider, tip_block_number)?;
self.metrics
Expand Down Expand Up @@ -305,7 +305,7 @@ impl<DB: Database> Pruner<DB> {
.map(|checkpoint| checkpoint.block_number);

let address_filter =
self.modes.contract_logs_filter.group_by_block(tip_block_number, pruned)?;
self.modes.receipts_log_filter.group_by_block(tip_block_number, pruned)?;

// Splits all transactions in different block ranges. Each block range will have its own
// filter address list and will check it while going through the table
Expand Down Expand Up @@ -411,7 +411,7 @@ impl<DB: Database> Pruner<DB> {
// one using `get_next_tx_num_range_from_checkpoint`.
let checkpoint_block = self
.modes
.contract_logs_filter
.receipts_log_filter
.lowest_block_with_distance(tip_block_number, pruned)?
.unwrap_or(to_block);

Expand Down
2 changes: 1 addition & 1 deletion crates/storage/provider/src/post_state/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -661,7 +661,7 @@ impl PostState {

let contract_log_pruner = self
.prune_modes
.contract_logs_filter
.receipts_log_filter
.group_by_block(tip, None)
.map_err(|e| Error::Custom(e.to_string()))?;

Expand Down
Loading