PSYNC based on Unique Replication Sequence ID #538
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ShooterIT
added
release notes
major decision
git-hulk
reviewed
May 5, 2022
git-hulk
reviewed
May 5, 2022
git-hulk
reviewed
May 5, 2022
|
@ShooterIT Need to add the License Header for new files |
git-hulk
approved these changes
May 23, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
Currently, master only checks sequence number when replica asks for PSYNC, that is not enough since they may have different replication history even the replica asking sequence is in the range of the master current WAL.
Bug report in #462
PS: Master check also DB name before checking sequence number, but it is also not enough since it is setting in conf, and don't change when role is changed.
Solution
We design 'PSYNC based on Unique Replication Sequence ID', we add unique replication id for every write batch (the operation of each command on the storage engine), so the combination of replication id and sequence is unique for write batch. The master can identify whether the replica has the same replication history by checking replication id and sequence.
Like Redis, replicas can partially resynchronize with new master if old master is failed and new selected master has largest offset, and replicas can also partially resynchronize with master after restarting if the replicas' conf file has the its origin master.
Unique replication id will be changed when replicas become master.
By default, it is not enabled since this stricter check may easily lead to full synchronization. You should enable it if you want to data correctness, maybe we enable it by default in kvrocks 3.0.