Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rocksdb: configurable MaxOpenFiles #272

Open
JayT106 opened this issue Jul 15, 2022 · 0 comments
Open

rocksdb: configurable MaxOpenFiles #272

JayT106 opened this issue Jul 15, 2022 · 0 comments

Comments

@JayT106
Copy link

JayT106 commented Jul 15, 2022

#218 made a change for setup MaxOpenFiles to 4096, which makes the RAM usage could easily go over 64GB when our application doing a snapshot or the genesis state export that requires the DB iterating. We tried to reduce it to 1024 and it turns out we can run it within 30GB.

The setup here

opts.OptimizeLevelStyleCompaction(512 * 1024 * 1024)
made each sst file target size equal to 64MB(if we look at how rocksdb decided the file size), therefore I am not sure to budget 2MB for each opened file is base on which file size?

We should consider the node's capability and decide which number is proper for running the rocksdb instead of a fixed number.

ref:
https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant