Freezer/Ancient DB: Improving Hardware Costs and DB Recoverability

[heueristik]

Geth supports a freezer DB:

Older segments of the chain are moved out of the LevelDB database and into a freezer database. Nodes rarely need to access these files so IO speed is less important and the bulk of the chain data can be stored on a cheaper HDD. Once blocks pass some threshold age (90,000 blocks behind the head by default) the block and receipt data is flattened and saved as a raw binary blob of data along with an index entry file used for identification.

Older segments of the chain are moved out of the LevelDB database and into a freezer database. Nodes rarely need to access these files so IO speed is less important and the bulk of the chain data can be stored on a cheaper HDD. Once blocks pass some threshold age (90,000 blocks behind the head by default) the block and receipt data is flattened and saved as a raw binary blob of data along with an index entry file used for identification.

also on archive nodes (ethereum/go-ethereum#20647).

I am currently running a staking node and I would like to run a reth archive node to experiment with indexing and TrueBlocks (which also supports reth by now TrueBlocks/trueblocks-core#3009).

If this was possible, setting up an archive node becomes much cheaper (since an 8 or 4 TB HDD costs a fraction of an equivalent SSD). Another advantage could be that nodes can more easily recover from Hot DB database corruption.

I am not sure on the performance breakdown this would entail for read operations and how this fits with your storage architecture.

[onbjerg]

This is being worked on: #6444 :)

[heueristik]

Amazing! Can't wait