Release 2.0.6 #468
Merged
Release 2.0.6 #468
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, we didn't rewrite the config when the namespace was updated, which may cause the namespace loss after restarting without manual executing the config rewrite command. So we fix issue by rewriting the config here to make users happy.
…ne (apache#429) For example, we have below rename-commands in the config file: ``` rename-command KEYS KEYS_NEW rename-command GET GET_NEW rename-command SET SET_NEW ``` then `rename-command KEYS KEYS_NEW` would be overwriten to `rename-command SET SET_NEW` since we didn't skip the `rename-command` correctly in config rewrite.
…he#431) Co-authored-by: Myth <caipengbo@outlook.com>
Currently, the command bgsave was totally async behavior, which always return "OK" and users have no way to know whether bgsave was ok or not. So we export below informations: * last_bgsave_time, the last timestamp that bgsave command was executed * last_bgsave_status, whether the bgsave was ok or not * bgsave_in_progress, whether the bgsave was in-progress * last_bgsave_time_sec, the bgsave command elasped seconds
…e#434) The reply of "zrevrangebyscore zset_key min_score max_score" just include one member with max_score when the zset_key has multi members with the same max_score.
RocksDB iterator seek slows down dramatically when there are many large deleted rows. This slow down persists until a compaction occurs. To set read_options.iterate_upper_bound as tight as possible is recommended, so that internal iterating can stop sooner, rather than walking over the whole range of tombstones. Similarly, setting read_options.iterate_lower_bound for seekForPrev is recommended too. Relevant RocksDB issue: facebook/rocksdb#5265
git-hulk
approved these changes
Jan 28, 2022
Currently, we can't execute INFO command during restoring database on kvrocks, but redis can. Kvrocks returns error messages directly, it is not easy for some SDK/proxy/tools to check loading or other status, so this commit implements.
- Allow to disable the jemlloc on CMake - Allow to disable the jemalloc on Makefile - Add the build without jemalloc to the github actions - Daily schedule the disable jemalloc compiling GitHub actions
…sync (apache#446) If enabling slave-empty-db-before-fullsync, replica will core dump since we operate against the opened RocksDB, and renaming a non-exist directory never succeed. Introduced in apache#233
In Kvrocks, bitmap and string are two completely different types. Although setbit, getbit, bitcount, and bitpos operations on the string type are supported. @smartlee once mentioned that when using setbit on the string type, master-slave replication will have a tremendous amount of write amplification because Kvrocks replication does not use command propagation like Redis. In addition, using bitmap type conserves storage space. So we recommend using type bitmap. But to make it easy to use, we want to get the whole string after setbit operations, so we implement this interface. We covert bitmap to string when executing GET command and return string reply, otherwise, we provide one configuration item max-bitmap-to-string-mb which uses to limit the max size of bitmap to string transformation(MB). Why we need to swap the bit when converting bitmap type to string? If you setbit bit 0 1, the value is stored as 0x01 in Kvocks but 0x80 in Redis. So swapping bits is to get the same return value as Redis.
Add tcl tests about redis-cli to make sure that redis-cli can work well with kvrocks.
Kvrocks may leak dozen kilobytes per full replication
CLUSTERX SETSLOT $slot_id NODE $node_id $new_version - Description assign the slot to the node if new version is current version+1. - Parameters $slot_id: the slot which we want to assign $node_id: the node which we want to assign the slot into NODE: keep the same with redis CLUSTER SETSLOT command, maybe we want to expand this command in the future $new_version: the new version MUST be +1 of current version so that kvrocks could execute this command, this is to guarantee this change is based on a special cluster topology - Purpose users can use this command to set slot distribution slot migration needs this command to update slot distribution New version constraint The reason why the new version MUST be +1 of current version is that, the command changes topology based on specific topology (also means specific version), we must guarantee current topology is exactly expected, otherwise this update may make topology corrupt, so base topology version is very important. This is different with CLUSTERX SETNODES commands because it uses new version topology to cover current version, it allows kvrocks nodes lost some topology updates since of network failure, it is state instead of operation.
A new command CLUSTERX MIGRATE is used for migrate slot data, slot-based migration process mainly includes the following stages: migrating existing data and migrating incremental data. Command format: CLUSTERX MIGRATE $slot $dst_nodeid - $slot is the slot which is to migrate - $dst_nodeid is the node id of destination server in the cluster. We also introduce an internal command CLUSTER IMPORT for importing the migrating slot data into destination server. Migration status are shown into the output of CLUSTER INFO command. After migration slot, you also should use CLUSTERX SETSLOT command to change cluster slot distribution. For more details, please see apache#412 and apache#430
|
@ColinChamber @shangxiaoxiong Since the RocksDB multi get API which is used in #442 has dispute, i don't merge it into 2.0 branch, but i also don't revert it before we have a clear conclusion, so it still is in unstable branch, please you know that. |
git-hulk
approved these changes
Jan 28, 2022
git-hulk
approved these changes
Jan 28, 2022
Merged
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Version 2.0.6
New features
Biggest feature of this version, authentically makes kvrocks scalable,
user can use CLUSTERX MIGRATE command to migrate slot and use CLUSTERX
SETSLOT to change cluster slot distribution after finishing migration.
Improvements
Bug fixes
blob_garbage_collection_age_cutoffwould be always zero #425)SetCurrentConnection#440)Maybe leak dozen kilobytes per full replication