Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(bin): db stats --summary to display totals for snapshots #6584

Merged
merged 2 commits into from
Feb 13, 2024
Merged

feat(bin): db stats --summary to display totals for snapshots #6584

merged 2 commits into from
Feb 13, 2024

Conversation

shekhirin
Copy link
Collaborator

@shekhirin shekhirin commented Feb 13, 2024 •
edited
Loading

With 500k blocks per file, mainnet has 19 million / 500k = 38 files per segment, and we have 3 segments. Too many lines in the output just to get the idea of storage stats for e.g. in watch.

This PR adds a --summary argument that shows summarized numbers per segment.

➜  reth (feat/static-files) ./target/debug/reth db stats --chain holesky --summary                                                                  ✭
| Segment      | Block Range | Transaction Range | Shape        | Data Size | Index Size | Offsets Size | Config Size | Total Size |
|--------------|-------------|-------------------|--------------|-----------|------------|--------------|-------------|------------|
| Headers      | 0..=834214  | N/A               | 3 x 834215   | 451.9 MiB | 0 B        | 19.1 MiB     | 108 B       | 471 MiB    |
| Transactions | 0..=850110  | 0..=26189416      | 1 x 26189417 | 6.1 GiB   | 0 B        | 199.8 MiB    | 140 B       | 6.3 GiB    |
| Receipts     | 0..=850110  | 0..=26189416      | 1 x 26189417 | 913.5 MiB | 0 B        | 199.8 MiB    | 140 B       | 1.1 GiB    |
| ------------ | ----------- | ----------------- | ------------ | --------- | ---------- | ------------ | ----------- | ---------- |
| Total        |             |                   |              | 7.4 GiB   | 0 B        | 418.7 MiB    | 388 B       | 7.8 GiB    |


| Table Name        | # Entries | Branch Pages | Leaf Pages | Overflow Pages | Total Size |
|-------------------|-----------|--------------|------------|----------------|------------|
| AccountChangeSet  | 22848833  | 30988        | 233306     | 0              | 1 GiB      |
| AccountHistory    | 15796642  | 13149        | 915204     | 2568           | 3.6 GiB    |
| AccountsTrie      | 1346150   | 415          | 89748      | 0              | 352.2 MiB  |
| BlockBodyIndices  | 850111    | 23           | 4931       | 0              | 19.4 MiB   |
| BlockOmmers       | 0         | 0            | 0          | 0              | 0 B        |
| BlockWithdrawals  | 841398    | 538          | 120150     | 0              | 471.4 MiB  |
| Bytecodes         | 10886     | 19           | 1014       | 23383          | 95.4 MiB   |
| CanonicalHeaders  | 849545    | 163          | 20024      | 0              | 78.9 MiB   |
| HashedAccount     | 15793454  | 4885         | 299445     | 0              | 1.2 GiB    |
| HashedStorage     | 10099254  | 4374         | 188538     | 0              | 753.6 MiB  |
| HeaderNumbers     | 850111    | 270          | 15299      | 0              | 60.8 MiB   |
| HeaderTD          | 849545    | 20           | 4186       | 0              | 16.4 MiB   |
| Headers           | 849545    | 1795         | 206807     | 0              | 814.9 MiB  |
| PlainAccountState | 15793454  | 2503         | 234574     | 0              | 926.1 MiB  |
| PlainStorageState | 10099254  | 4169         | 185959     | 0              | 742.7 MiB  |
| PruneCheckpoints  | 1         | 0            | 1          | 0              | 4 KiB      |
| Receipts          | 25775366  | 1144         | 255656     | 99069          | 1.4 GiB    |
| StorageChangeSet  | 13072606  | 28481        | 192555     | 0              | 863.4 MiB  |
| StorageHistory    | 10340037  | 20559        | 714852     | 736            | 2.8 GiB    |
| StoragesTrie      | 973113    | 6118         | 65747      | 0              | 280.7 MiB  |
| SyncStage         | 14        | 0            | 1          | 0              | 4 KiB      |
| SyncStageProgress | 1         | 0            | 1          | 0              | 4 KiB      |
| TransactionBlock  | 685732    | 21           | 4396       | 0              | 17.3 MiB   |
| Transactions      | 0         | 0            | 0          | 0              | 0 B        |
| TxHashNumber      | 26189417  | 8161         | 485665     | 0              | 1.9 GiB    |
| TxSenders         | 26024907  | 1088         | 243227     | 0              | 954.4 MiB  |
| ----------------- | --------- | ------------ | ---------- | -------------- | ---------- |
| Tables            |           |              |            |                | 18.1 GiB   |
| Freelist          | 1974764   |              |            |                | 7.5 GiB    |

➜  reth (feat/static-files) ./target/debug/reth db stats --chain holesky --summary --only-total-size                                                ✭
| Segment      | Block Range | Transaction Range | Shape        | Size    |
|--------------|-------------|-------------------|--------------|---------|
| Headers      | 0..=834214  | N/A               | 3 x 834215   | 471 MiB |
| Transactions | 0..=850110  | 0..=26189416      | 1 x 26189417 | 6.3 GiB |
| Receipts     | 0..=850110  | 0..=26189416      | 1 x 26189417 | 1.1 GiB |
| ------------ | ----------- | ----------------- | ------------ | ------- |
| Total        |             |                   |              | 7.8 GiB |


| Table Name        | # Entries | Branch Pages | Leaf Pages | Overflow Pages | Total Size |
|-------------------|-----------|--------------|------------|----------------|------------|
| AccountChangeSet  | 22848833  | 30988        | 233306     | 0              | 1 GiB      |
| AccountHistory    | 15796642  | 13149        | 915204     | 2568           | 3.6 GiB    |
| AccountsTrie      | 1346150   | 415          | 89748      | 0              | 352.2 MiB  |
| BlockBodyIndices  | 850111    | 23           | 4931       | 0              | 19.4 MiB   |
| BlockOmmers       | 0         | 0            | 0          | 0              | 0 B        |
| BlockWithdrawals  | 841398    | 538          | 120150     | 0              | 471.4 MiB  |
| Bytecodes         | 10886     | 19           | 1014       | 23383          | 95.4 MiB   |
| CanonicalHeaders  | 849545    | 163          | 20024      | 0              | 78.9 MiB   |
| HashedAccount     | 15793454  | 4885         | 299445     | 0              | 1.2 GiB    |
| HashedStorage     | 10099254  | 4374         | 188538     | 0              | 753.6 MiB  |
| HeaderNumbers     | 850111    | 270          | 15299      | 0              | 60.8 MiB   |
| HeaderTD          | 849545    | 20           | 4186       | 0              | 16.4 MiB   |
| Headers           | 849545    | 1795         | 206807     | 0              | 814.9 MiB  |
| PlainAccountState | 15793454  | 2503         | 234574     | 0              | 926.1 MiB  |
| PlainStorageState | 10099254  | 4169         | 185959     | 0              | 742.7 MiB  |
| PruneCheckpoints  | 1         | 0            | 1          | 0              | 4 KiB      |
| Receipts          | 25775366  | 1144         | 255656     | 99069          | 1.4 GiB    |
| StorageChangeSet  | 13072606  | 28481        | 192555     | 0              | 863.4 MiB  |
| StorageHistory    | 10340037  | 20559        | 714852     | 736            | 2.8 GiB    |
| StoragesTrie      | 973113    | 6118         | 65747      | 0              | 280.7 MiB  |
| SyncStage         | 14        | 0            | 1          | 0              | 4 KiB      |
| SyncStageProgress | 1         | 0            | 1          | 0              | 4 KiB      |
| TransactionBlock  | 685732    | 21           | 4396       | 0              | 17.3 MiB   |
| Transactions      | 0         | 0            | 0          | 0              | 0 B        |
| TxHashNumber      | 26189417  | 8161         | 485665     | 0              | 1.9 GiB    |
| TxSenders         | 26024907  | 1088         | 243227     | 0              | 954.4 MiB  |
| ----------------- | --------- | ------------ | ---------- | -------------- | ---------- |
| Tables            |           |              |            |                | 18.1 GiB   |
| Freelist          | 1974764   |              |            |                | 7.5 GiB    |

@shekhirin shekhirin added C-enhancement New feature or request A-cli Related to the reth CLI A-static-files Related to static files labels Feb 13, 2024
@shekhirin shekhirin marked this pull request as ready for review February 13, 2024 16:05
@shekhirin shekhirin requested a review from onbjerg as a code owner February 13, 2024 16:05
onbjerg
onbjerg approved these changes Feb 13, 2024
View reviewed changes
Copy link
Collaborator

@onbjerg onbjerg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, can we rename Shape to Shape (cols x rows) or something? just so its more clear

@shekhirin shekhirin merged commit 71f0793 into feat/static-files Feb 13, 2024
23 of 24 checks passed
@shekhirin shekhirin deleted the alexey/db-stats-summary branch February 13, 2024 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@onbjerg onbjerg onbjerg approved these changes

@mattsse mattsse Awaiting requested review from mattsse

@joshieDo joshieDo Awaiting requested review from joshieDo

Assignees
No one assigned
Labels
A-cli Related to the reth CLI A-static-files Related to static files C-enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@shekhirin @onbjerg