Skip to content

Commit

Permalink
raftstore: enlarge leader-transfer-max-log-lag (tikv#9592)
Browse files Browse the repository at this point in the history
Signed-off-by: gengliqi <gengliqiii@gmail.com>


### What problem does this PR solve?

Problem Summary:
The meaning of `leader-transfer-max-log-lag` has already been changed by tikv#6539.
For now, it means the difference value between the leader's last log index and the follower's last applied index.
On the basis of implementation, we get the follower's last applied index through an RPC then we get the leader's last log index.
There are many gaps between this process. 
1. the applied index in peer fsm may be smaller than the real one in its corresponding apply fsm
2. the network latency
3. some logs may be proposed after sending the query RPC to this follower

So the follower's last applied index may be much larger when we calculate the difference value.

It thus appears that the default value of `leader-transfer-max-log-lag`(10) is too small which leads to many failures of leader transfer.

I test the value of 128 and the failure is much less than before.
* I test 256 and there is almost no failure. But maybe it's too large.

Maybe we should find a more scientific approach to solve this problem later.

### What is changed and how it works?

What's Changed:
Change the default value of `leader-transfer-max-log-lag` to 128.

### Related changes

- PR to update `pingcap/docs`/`pingcap/docs-cn`:
- PR to update `pingcap/tidb-ansible`:
- Need to cherry-pick to the release branch

Tests <!-- At least one of them must be included. -->

- No code

Side effects

- No

### Release note <!-- bugfixes or new feature need a release note -->
* Change the default `leader-transfer-max-log-lag` to 128 to increase the success rate of leader transfer
  • Loading branch information
gengliqi authored Jan 29, 2021
1 parent 3a4b8bb commit a2c7aac
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion components/raftstore/src/store/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ impl Default for Config {
max_leader_missing_duration: ReadableDuration::hours(2),
abnormal_leader_missing_duration: ReadableDuration::minutes(10),
peer_stale_state_check_interval: ReadableDuration::minutes(5),
leader_transfer_max_log_lag: 10,
leader_transfer_max_log_lag: 128,
snap_apply_batch_size: ReadableSize::mb(10),
lock_cf_compact_interval: ReadableDuration::minutes(10),
lock_cf_compact_bytes_threshold: ReadableSize::mb(256),
Expand Down

0 comments on commit a2c7aac

Please sign in to comment.