(feat) Log replication thread model #168
Assignees
Milestone
Comments
fengjiachun
added a commit
that referenced
this issue
May 22, 2019
fengjiachun
added a commit
that referenced
this issue
May 23, 2019
fengjiachun
added a commit
that referenced
this issue
May 24, 2019
fengjiachun
added a commit
that referenced
this issue
May 24, 2019
killme2008
pushed a commit
that referenced
this issue
Jul 5, 2019
* (feat) add FixedThreadsExecutorGroup #168 * (feat) rename method * (feat) add MpscSingleThreadExecutor and benchmark #168 * (fix) forget to warmup producers * (fix) fix some bugs and add unit test * (fix) add more unit test * (fix) add more unit test * (fix) add more unit test * (fix) add some comments * (fix) unit test * (fix) add some comments * (fix) refactoring Utils class * (fix) refactoring Utils class * (fix) jraft.closure.threadpool.size.max update default value * (fix) fix unit test * (fix) fix unit test * (feat) refactor ThreadId and replicator (#169) * (feat) refactor ThreadId and replicator * (feat) Adds javadoc * (feat) add pooled buf allocator (#161) * (feat) add AdaptiveBufAllocator * (feat) pooled for ByteBufferCollector #158 * (feat) pooled for ByteBufferCollector #158 * (feat) pooled for ByteBufferCollector #158 * (feat) pooled for ByteBufferCollector #158 * (fix) rename method name * (fix) minor fix * (fix) add metric for recyclers (#164) * (fix) add metric for recyclers * (fix) add metric for ByteBufferCollector.capacity * (fix) code format * (fix) by review comment * feat/zero copy with replicator (#167) * (fix) zero copy with replicator * (fix) support zero copy and add benchmark * (fix) rename field * (fix) rm zero copy and unnecessary metric * (fix) by review comment * (feat) add unit test AdaptiveBufAllocatorTest * (feat) add unit test RecyclersTest * (feat) add unit test RecyclableByteBufferListTest * (feat) add unit test ByteBufferCollectorTest * Add unit tests for com.alipay.sofa.jraft.util.BytesUtil (#166) These tests were written using Diffblue Cover. * (fix) Utils.java format * (feat) add FixedThreadsExecutorGroup #168 * (feat) rename method * (feat) add MpscSingleThreadExecutor and benchmark #168 * (fix) forget to warmup producers * (fix) fix some bugs and add unit test * (fix) add more unit test * (fix) add more unit test * (fix) add more unit test * (fix) add some comments * (fix) unit test * (fix) add some comments * (fix) refactoring Utils class * (fix) refactoring Utils class * (fix) jraft.closure.threadpool.size.max update default value * (fix) fix unit test * (fix) fix unit test * (feat) add pooled buf allocator (#161) * (feat) add AdaptiveBufAllocator * (feat) pooled for ByteBufferCollector #158 * (feat) pooled for ByteBufferCollector #158 * (feat) pooled for ByteBufferCollector #158 * (feat) pooled for ByteBufferCollector #158 * (fix) rename method name * (fix) minor fix * (fix) add metric for recyclers (#164) * (fix) add metric for recyclers * (fix) add metric for ByteBufferCollector.capacity * (fix) code format * (fix) by review comment * feat/zero copy with replicator (#167) * (fix) zero copy with replicator * (fix) support zero copy and add benchmark * (fix) rename field * (fix) rm zero copy and unnecessary metric * (fix) by review comment * (feat) add unit test AdaptiveBufAllocatorTest * (feat) add unit test RecyclersTest * (feat) add unit test RecyclableByteBufferListTest * (feat) add unit test ByteBufferCollectorTest * (fix) Utils.java format * (fix) fix bad key with executor map * (fix) bad import * (fix) fix unit test * (feat) add mor benchmark * (fix) code format * (fix) code format * (fix) benchmark with jmh * (fix) benchmark with jmh * (fix) set common daemon * (fix) fix unit test * (fix) should be no radical changes, especially if they are not fully tested. * (feat) add jctools * (feat) configure the number of processors #180 (#181) * (fix) format
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Your question
在并发量大的时候，Replicator.sendEntries() 锁竞争比较激烈，有大量的 block， 见这里
阻塞的是 JRaft-RPC-Processor 线程，这对 jraft 中其他操作比如 pre-vote、vote 等均有一定的影响
Your scenes
这个现象理论上是
sendEntries与onRpcReturned线程不同导致的锁竞争，onRpcReturned是被 bolt RPC 的 executor 调用，通用的ThreadPoolExecutor无法做到绑定Thread的方式进行调度Your advice
在
Replicator中增加一个replicatorId属性（最好用一个全局自增 int），再提供一个特殊的 thread pool，可以按照类似[replicatorId % pool.length]的方式将一个 Replicator instance 固定映射到一个Thread上，这个方式相比每个 Replicator 创建一个Thread，在 multi-group 复制组很多的情况下，可以少创建很多线程，至于 pool 的 size，需要在 options 中配置Environment
java -version):uname -a):The text was updated successfully, but these errors were encountered: