Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some rocksdb new feature #395

Merged
merged 9 commits into from
Nov 21, 2021

Conversation

caipengbo
Copy link
Contributor

@caipengbo caipengbo commented Nov 17, 2021

#209 proposed some optimization points for RocksDB, and I tried to apply them to Kvrocks through some research.

Here are my optimizations:

  1. Dynamic resize block and SST introduced by dynamic resize block and sst #120 and Support the auto-resize-block-and-sst config directive #289 to deal with the problem of large index/filter blocks.

    This mechanism resize target_file_size_base and block_size by calculating the average key-value size:

    https://github.com/KvrocksLabs/kvrocks/blob/782c106d4513eb79a94431f0750a626c26b1fdd2/src/server.cc#L1050

    It uses the compressed total size to calculate the average key-value, which can easily be less than 128, resulting in target_file_size_base always be 16. When there is a large amount of data, a large number of SST files are generated. Too many files may cause problems.

    Therefore, I use Partitioned Index/Filter to deal with large Index/Filter blocks and remove autoResizeBlockAndSST() . Through some test, I found that when our files were large(128M), we did not generate large index/filter blocks to increase the long tail latency.

  2. Add Rocksdb's new feature integrated BlobDB(Since 6.18.0) for large value scenarios. Although the integrated BlobDB has not been announced for production yet, I think it can be added for users to test.

src/config.cc Outdated Show resolved Hide resolved
src/server.cc Show resolved Hide resolved
src/storage.cc Outdated Show resolved Hide resolved
src/storage.cc Outdated Show resolved Hide resolved
src/storage.cc Outdated Show resolved Hide resolved
src/config.cc Outdated Show resolved Hide resolved
@git-hulk
Copy link
Member

Really good feature, some users also look forward to the blob store. #366

@caipengbo
Copy link
Contributor Author

By the way, I will solve #384

src/config.cc Outdated Show resolved Hide resolved
src/config.cc Outdated Show resolved Hide resolved
git-hulk
git-hulk previously approved these changes Nov 18, 2021
@git-hulk git-hulk self-requested a review November 18, 2021 11:19
src/config.cc Outdated Show resolved Hide resolved
git-hulk
git-hulk previously approved these changes Nov 19, 2021
Copy link
Member

@ShooterIT ShooterIT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just minor comments

src/server.cc Show resolved Hide resolved
src/storage.cc Show resolved Hide resolved
@ShooterIT
Copy link
Member

Please also do some surveys on memtable_prefix_bloom_size_ratio and memtable_whole_key_filtering.
i think for metadata CF, there are most point lookup, it is a good idea to enable memtable filter, for subkey CF, it may not, sine users may execute lrange hgetall zrangeXX commands that use range lookup.

@caipengbo
Copy link
Contributor Author

Please also do some surveys on memtable_prefix_bloom_size_ratio and memtable_whole_key_filtering.
i think for metadata CF, there are most point lookup, it is a good idea to enable memtable filter, for subkey CF, it may not, sine users may execute lrange hgetall zrangeXX commands that use range lookup.

Yes, I haven't noticed this optimization yet, I will investigate it later.

@ShooterIT ShooterIT linked an issue Nov 19, 2021 that may be closed by this pull request
@ShooterIT
Copy link
Member

Summary of this PR

  • Support blob db (key-value separation) which requires RocksDB version greater than 6.18
  • Enable partitioned index/filter for SST and hash index for data block
  • Enable whole key filter for memtable
  • Remove dynamic resize block and SST
  • Only apply compression to SST which level is more than 2

@caipengbo do i miss something?

@caipengbo
Copy link
Contributor Author

Summary of this PR

  • Support blob db (key-value separation) which requires RocksDB version greater than 6.18
  • Enable partitioned index/filter for SST and hash index for data block
  • Enable whole key filter for memtable
  • Remove dynamic resize block and SST
  • Only apply compression to SST which level is more than 2

@ShooterIT It's a complete summary!

@ShooterIT ShooterIT merged commit fe7efb5 into apache:unstable Nov 21, 2021
ShooterIT pushed a commit to ShooterIT/kvrocks that referenced this pull request Dec 7, 2021
- Support blob db (key-value separation) which requires RocksDB version greater than 6.18
- Enable partitioned index/filter for SST and hash index for data block
- Enable whole key filter for memtable
- Remove dynamic resize block and SST
- Only apply compression to SST which level is more than 2
@ShooterIT ShooterIT mentioned this pull request Dec 7, 2021
ShooterIT pushed a commit that referenced this pull request Dec 7, 2021
- Support blob db (key-value separation) which requires RocksDB version greater than 6.18
- Enable partitioned index/filter for SST and hash index for data block
- Enable whole key filter for memtable
- Remove dynamic resize block and SST
- Only apply compression to SST which level is more than 2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants