triedb/pathdb: introduce file-based state journal #32060
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jsvisa
commented
Jun 19, 2025
jsvisa
commented
Jun 19, 2025
jsvisa
force-pushed
the
journal-into-file
branch
from
2ca455d to
2d222b2
Compare
2 tasks
jsvisa
force-pushed
the
journal-into-file
branch
from
2d222b2 to
9abf10a
Compare
Contributor
|
The idea is good, but the path should not be configurable. We should just resolve the path in the datadir and store the file there unconditionally. |
jsvisa
force-pushed
the
journal-into-file
branch
from
9abf10a to
516c1b3
Compare
Member
|
This pull request is backward-compatible:
The journal directory will be $DATADIR/triedb, with |
Member
|
logs: |
* all: load/dump trie disklayer into file Signed-off-by: jsvisa <delweng@gmail.com> * cmd: set cache.trie.journal=trie-disklayer.rlp as default Signed-off-by: jsvisa <delweng@gmail.com> * journal: directly write into file Signed-off-by: jsvisa <delweng@gmail.com> --------- Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
rjl493456442
force-pushed
the
journal-into-file
branch
from
c84ef8a to
bf0972b
Compare
rjl493456442
previously approved these changes
Jul 1, 2025
rjl493456442
changed the title
rjl493456442
approved these changes
Jul 15, 2025
howjmay
pushed a commit
to iotaledger/go-ethereum
that referenced
this pull request
Aug 27, 2025
Introduce file-based state journal in path database, fixing the Pebble restriction when the journal size exceeds 4GB. --------- Signed-off-by: jsvisa <delweng@gmail.com> Co-authored-by: Gary Rong <garyrong0905@gmail.com>
rjl493456442
pushed a commit
that referenced
this pull request
Sep 5, 2025
As in #32060 we introduced the file based journal path, for the other sub command(eg: snapshot, db), we should also pass the directory to the triedb, else the subcommand(eg: `geth snapshot`) failed to run: ```bash geth snapshot verify-state --datadir /geth-data ... INFO [09-02|02:12:29.493] Allocated cache and file handles database=/geth-data/geth/chaindata cache=512.00MiB handles=524,288 INFO [09-02|02:12:32.746] Opened ancient database database=/geth-data/geth/chaindata/ancient/chain readonly=true INFO [09-02|02:12:32.746] Opened Era store datadir=/geth-data/geth/chaindata/ancient/chain/era INFO [09-02|02:12:32.758] State scheme set to already existing scheme=path INFO [09-02|02:12:32.760] Load database journal from disk INFO [09-02|02:12:32.764] Failed to load journal, discard it err="journal not found" INFO [09-02|02:12:32.789] Opened ancient database database=/geth-data/geth/chaindata/ancient/state readonly=true INFO [09-02|02:12:32.790] Initialized path database readonly=true triecache=0.00B statecache=0.00B buffer=0.00B history="entire chain" ERROR[09-02|02:12:32.791] Failed to verify state root=c5458d..4cc785 err="unknown layer: c5458d476da0136a67ef24a93b909aa5c29efa5c5b885dbd1fbaed4e784cc785" ```
gballet
pushed a commit
to gballet/go-ethereum
that referenced
this pull request
Sep 11, 2025
Introduce file-based state journal in path database, fixing the Pebble restriction when the journal size exceeds 4GB. --------- Signed-off-by: jsvisa <delweng@gmail.com> Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Sahil-4555
pushed a commit
to Sahil-4555/go-ethereum
that referenced
this pull request
Oct 12, 2025
As in ethereum#32060 we introduced the file based journal path, for the other sub command(eg: snapshot, db), we should also pass the directory to the triedb, else the subcommand(eg: `geth snapshot`) failed to run: ```bash geth snapshot verify-state --datadir /geth-data ... INFO [09-02|02:12:29.493] Allocated cache and file handles database=/geth-data/geth/chaindata cache=512.00MiB handles=524,288 INFO [09-02|02:12:32.746] Opened ancient database database=/geth-data/geth/chaindata/ancient/chain readonly=true INFO [09-02|02:12:32.746] Opened Era store datadir=/geth-data/geth/chaindata/ancient/chain/era INFO [09-02|02:12:32.758] State scheme set to already existing scheme=path INFO [09-02|02:12:32.760] Load database journal from disk INFO [09-02|02:12:32.764] Failed to load journal, discard it err="journal not found" INFO [09-02|02:12:32.789] Opened ancient database database=/geth-data/geth/chaindata/ancient/state readonly=true INFO [09-02|02:12:32.790] Initialized path database readonly=true triecache=0.00B statecache=0.00B buffer=0.00B history="entire chain" ERROR[09-02|02:12:32.791] Failed to verify state root=c5458d..4cc785 err="unknown layer: c5458d476da0136a67ef24a93b909aa5c29efa5c5b885dbd1fbaed4e784cc785" ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
allow write trie journal into a local file instead of disk, used when the trie jounral is >= 4GB, ref gballet#546 (comment)