StateTransitionData disk usage on mainnet #10578
Assignees
Labels
A-stateless-validation
Area: stateless validation
Comments
52 tasks
github-merge-queue bot
pushed a commit
that referenced
this issue
Feb 13, 2024
…10599) Currently `StateTransitionData` is a part of regular chain GC process. Unfortunately it still reaches 300GB on mainnet, see #10578. This PR introduces more aggressive garbage collection for `StateTransitionData`. If a chunk is present in a final block then we can safely clean up entries for that shard which correspond to blocks with lower height. Note that we still keep `StateTransitionData` as part of epoch-based GC just in case. Testing: * unit testing * running mainnet rpc node with shadow validation and monitoring data on disk size via `near_rocksdb_live_sst_files_size` and making sure that we still have required entries via `near_shadow_chunk_validation_failed_total`. <img width="750" alt="Screenshot 2024-02-13 at 10 32 15" src="https://github.com/near/nearcore/assets/3171838/e60fea9c-750f-4195-8b8b-63f9a8512c57">
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently we save
StateTransitionDataon disk (consists mostly of partial state needed to create state witness) if we are chunk producer for this or next epoch.I've been running a mainnet node with shadow chunk validation enabled, so it generated
StateTransitionDatafor all shards.It resulted in ~5GB per hour disk usage growth. That will stop at some point because of GC, total usage will be at least 200GB.
We'd better avoid this and clean up
StateTransitionDatamore eagerly.One way to fix this is to remove
StateTransitionDatafor relevant shards (for which chunks are present in block) for a range of previous blocks when a block is finalised.The text was updated successfully, but these errors were encountered: