Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

com.alipay.sofa.jraft.storage.impl.LogManagerImpl#waitMap 设置之前是否可以去重 #842

Closed
kdStu opened this issue Jun 6, 2022 · 0 comments

Comments

@kdStu
Copy link

kdStu commented Jun 6, 2022

waitMoreEntries的时候是否可以去除重复的数据.

下面是arthas追踪waitMoreEntries时发现, 有index为42163被处理了多次,被处理多次造成的后果是会有太多重复的WaitMeta对象写入到waitMap里, 进而导致执行wakeupAllWaiter的时候,可能会由于waitMap的数量过多导致Utils.runInThread的任务执行会被拒绝掉.

method=com.alipay.sofa.jraft.core.Replicator.waitMoreEntries location=AtExit
ts=2022-06-06 11:43:17; [cost=9.37671863814507E8ms] result=@Arraylist[
@object[][
@long[42163],
],
]
method=com.alipay.sofa.jraft.core.Replicator.waitMoreEntries location=AtExit
ts=2022-06-06 11:43:17; [cost=9.37671863798092E8ms] result=@Arraylist[
@object[][
@long[42163],
],
]
method=com.alipay.sofa.jraft.core.Replicator.waitMoreEntries location=AtEnter
ts=2022-06-06 11:43:17; [cost=0.002256ms] result=@Arraylist[
@object[][
@long[42163],
],
]
method=com.alipay.sofa.jraft.core.Replicator.waitMoreEntries location=AtExit
ts=2022-06-06 11:43:17; [cost=9.37671867507219E8ms] result=@Arraylist[
@object[][
@long[42163],
],
]
method=com.alipay.sofa.jraft.core.Replicator.waitMoreEntries location=AtEnter
ts=2022-06-06 11:43:17; [cost=0.002319ms] result=@Arraylist[
@object[][
@long[42163],
],
]
method=com.alipay.sofa.jraft.core.Replicator.waitMoreEntries location=AtExit
ts=2022-06-06 11:43:17; [cost=9.37671867767824E8ms] result=@Arraylist[
@object[][
@long[42163],
],
]

目前我这里没有足够的证据证明42163这个index是否能查询出logEntry, 从逻辑上来讲, 如果为空, prepareEntry这个方法执行的时候返回fase, 则推出对应的循环, 那是不是也存在可能会走到waitMoreEntries这个方法. waitMap的key是waitId是自增的,
所以会不会导致某个时刻waitMap的长度超过线程池的长度.

所以请问, 这里有没有办法过滤掉重复的数据? 如果去除了重复的数据, 是不是不会引起提交任务被拒绝的情况出现?

目前snapshot的策略是10分钟一次, 写raft的频率是10s一次.

killme2008 added a commit that referenced this issue Jun 18, 2022
killme2008 added a commit that referenced this issue Jun 18, 2022
killme2008 added a commit that referenced this issue Jun 18, 2022
fengjiachun pushed a commit that referenced this issue Jun 20, 2022
* fix: reset waitId unexpectedly when replicator blocks on network issue and improve log in replciator #842, #838

* test: fix testMetricRemoveOnDestroy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants