Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Akka.Remote: improved write performance with DotNetty flush-batching #4106

Merged
merged 26 commits into from
Jan 21, 2020
Merged
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
d4944bc
adding DotNetty write batching support
Aaronontheweb Dec 17, 2019
360e816
DotNetty flush-batching (performance)
Aaronontheweb Dec 17, 2019
0b474e6
stash
Aaronontheweb Dec 18, 2019
a91cf3e
Added batch triggers for byte size too
Aaronontheweb Dec 18, 2019
10f41d7
use max frame size to determine flush limit
Aaronontheweb Dec 18, 2019
25cff73
rewrote ToByteBuffer method as static with inlining
Aaronontheweb Dec 18, 2019
9a96ca6
Merge branch 'dev' into dotnetty-batching
Aaronontheweb Dec 18, 2019
51b83a8
Merge branch 'dev' into dotnetty-batching
Aaronontheweb Dec 18, 2019
0794b06
made sure to push write down the pipeline before flush
Aaronontheweb Dec 18, 2019
ebe1a0d
Merge branch 'dev' into dotnetty-batching
Aaronontheweb Dec 20, 2019
eb8d4c0
Merge branch 'dev' into dotnetty-batching
Aaronontheweb Dec 20, 2019
9df4536
Merge branch 'dev' into dotnetty-batching
Aaronontheweb Dec 21, 2019
50faacd
Merge branch 'dev' into dotnetty-batching
Aaronontheweb Jan 13, 2020
a786bc4
Merge branch 'dev' into dotnetty-batching
Aaronontheweb Jan 20, 2020
eebca2d
changed semantics around how BatchWriter behaves under low traffic
Aaronontheweb Jan 20, 2020
01754f3
added comment explaining why we call WriteAsync first before flush math
Aaronontheweb Jan 20, 2020
5bbe95b
handle termination-flush internally
Aaronontheweb Jan 20, 2020
25ba123
adding some tests to the batch writer
Aaronontheweb Jan 21, 2020
61d24a9
fixed batching test
Aaronontheweb Jan 21, 2020
95c6207
fixed issues with DotNetty batching spec
Aaronontheweb Jan 21, 2020
b1f0070
don't use built-in mechanisms to determine scheduling
Aaronontheweb Jan 21, 2020
6fe16f1
debugging RemoteDeliverySpec
Aaronontheweb Jan 21, 2020
82e50a6
added BatchWriterSettings class
Aaronontheweb Jan 21, 2020
18dcce6
added HOCON configuration for tuning the batching system
Aaronontheweb Jan 21, 2020
0384346
disable batching inside problematic Akka.Remote tests
Aaronontheweb Jan 21, 2020
0944a15
fix C#7 compilation issue
Aaronontheweb Jan 21, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
don't use built-in mechanisms to determine scheduling
Aaronontheweb committed Jan 21, 2020
commit b1f0070c2fad2b2f8937daed2d8f0037ebdf25b9
8 changes: 4 additions & 4 deletions src/core/Akka.Remote/Transport/DotNetty/BatchWriter.cs
Original file line number Diff line number Diff line change
@@ -25,6 +25,8 @@ internal class BatchWriter : ChannelHandlerAdapter
internal readonly int MaxPendingMillis;
internal readonly int MaxPendingBytes;

internal bool CanSchedule { get; private set; } = true;

public BatchWriter(int maxPendingWrites = 20, int maxPendingMillis = 40, int maxPendingBytes = 128000)
{
MaxPendingWrites = maxPendingWrites;
@@ -70,6 +72,7 @@ public override Task CloseAsync(IChannelHandlerContext context)
{
// flush any pending writes first
context.Flush();
CanSchedule = false;
return base.CloseAsync(context);
}

@@ -109,11 +112,8 @@ public void Run()
_writer.Reset();
}

// channel is still open
if (_context.Channel.Open)
{
if(_writer.CanSchedule)
_context.Executor.Schedule(this, _interval); // reschedule
}
}
}
}