-
Notifications
You must be signed in to change notification settings - Fork 504
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support to use row cache of RocksDB #407
Conversation
Meta data is mostly a point-query scenario, subkey is often a range query. Can we use only row cache in meta and only block cache in subkey? |
I don't find a solution now, row cache is a global cache. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just a minor problem about config comments.
Currently, from my some tests, i don't find row cache has obvious profit. Now i convert it to draft, maybe i have wrong understanding, if anyone have new ideas, happy to listen your thoughts |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
And in theory, row_cache is more useful for hots keys scenario since we don't need search on index and data block. But this mechanism is not always useful and also use more memory, so we disable it by default, you can enable it if you want to test or use it. |
RocksDB row cache may be useful for hots keys scenario, but it is not always useful and cost much more memory, so it is disabled by default, you can test and then decide to enable it or not in your scenario.
RocksDB row cache may be useful for hots keys scenario, but it is not always useful and cost much more memory, so it is disabled by default, you can test and then decide to enable it or not in your scenario.
For kvrocks point lookups, row cache is more useful than block cache, so we can enlarge row cache size and lessen meta/subkey block cache size.
If there are some hot keys that have the same prefix, they may be in the same block cache, so competition will be more serious, but if we use row cache, these keys may distribute into different cache shard since in row cache, distribute them by its hash value.
"4G block cache" VS "2G row cache and 2G block cache"
Test by benchmark (-t get -d 1024 -r 10000000 -n 10000000 --threads 8 -c 128)
in random mode, there is no obvious difference.