Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

binlog that needed by incr sync maybe purged in some extreme cases #2795

Closed
cheniujh opened this issue Jul 11, 2024 · 0 comments
Closed

binlog that needed by incr sync maybe purged in some extreme cases #2795

cheniujh opened this issue Jul 11, 2024 · 0 comments
Assignees
Labels
☢️ Bug Something isn't working

Comments

@cheniujh
Copy link
Collaborator

cheniujh commented Jul 11, 2024

Is this a regression?

Yes

Description

说明:该问题是在pika serveless开发测试过程中发现的,但考虑pika local时发现也可能有该问题出现,只是几率会很小,case也比较极端

上下文介绍:
在主节点收到全量同步请求时会:

  • 1 提交一个异步的bgsave任务,bgsave会mark一个最新的binlog offset, 记为A,后面使用这份bgsave进行了全量同步的从节点,就会从这个A位置开始增量续传
  • 2 提交完异步任务以后修改主这一侧SlaveNode的状态为kSlaveDbSync,于是BInlogPurge线程就不会再执行清理binlog的动作(只要有SlaveNode处于这个DBSync状态,就说明正在进行全量同步,为了避免1中提到的Binlog Offset A及其往后的Binlog被清理掉,所以这期间一律不清理Binlog)
    代码如下:
    image

隐患
当Master端写入速度极快,且Binlog file num上限设置很小时,可能会出现:处于上下文中提到的步骤1和步骤2的中间空挡,异步的BinlogPurge线程将步骤1中提到的Binlog Offset A及其后的Binlog进行了删除。这就会导致Slave在进行完全量同步,要开始增量时,发现Master端对应的续传Binlog已经被purge了,这就又得再全量同步一遍。

Please provide a link to a minimal reproduction of the bug

No response

Screenshots or videos

No response

Please provide the version you discovered this bug in (check about page for version information)

No response

Anything else?

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
☢️ Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants