NRG (2.11): Don't run catchup when behind on applies #6216
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@MauriceVanVeen let me know when we want this merged.. |
|
Has a conflict now that needs to be resolved.. |
MauriceVanVeen
force-pushed
the
maurice/stall-catchup-behind-on-applies
branch
2 times, most recently
from
a4711ee to
bb7ebfe
Compare
MauriceVanVeen
commented
Dec 9, 2024
| @@ -1654,7 +1654,7 @@ func (n *raft) State() RaftState { | |||
| func (n *raft) Progress() (index, commit, applied uint64) { | |||
| n.RLock() | |||
| defer n.RUnlock() | |||
| return n.pindex + 1, n.commit, n.applied | |||
There was a problem hiding this comment.
This was an off-by-one, but this value was not used within the code. Now we started to use its value.
n.pindex == n.commit == n.appplied means we have a log where every entry is applied.
If we have n.pindex 1 greater than n.commit that means we have one uncommitted entry. If we have n.pindex 1 greater than n.applied but equal to n.commit, that means we have a committed entry that's not applied yet.
Signed-off-by: Maurice van Veen <github@mauricevanveen.com>
MauriceVanVeen
force-pushed
the
maurice/stall-catchup-behind-on-applies
branch
from
bb7ebfe to
3392957
Compare
|
@derekcollison, this is ready now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a server restarts and is behind enough it will require to be caught up from a snapshot. If after receiving a snapshot from the leader the leader itself shuts down, the remaining server (in a R3 scenario) will become leader.
If this new leader is behind on applies it should not fulfill the catchup request. Messages that would be returned as part of the catchup might be deleted as part of the unapplied append entries. And sending these messages over to the follower would mean the follower wouldn't be able to remove them as part of the append entries if they were meant to be deleted.
Either way, the new leader is temporarily unable to fulfill the catchup request and must wait for its applies to reach the minimum required for the catchup response to be valid.
Signed-off-by: Maurice van Veen github@mauricevanveen.com