Skip to content

Commit

Permalink
raft: benchmark results
Browse files Browse the repository at this point in the history
```
for sha in :/^Revert :/BenchmarkRawNode :/^raft:.directly; do git checkout raft-single-voter && git checkout $(git log -n 1 '--pretty=format:%H' $sha) && f=$(git log -1 --pretty=%s | sed -E 's/[^A-Za-z0-9]+/_/g').txt && go test -run - -count 10 -bench BenchmarkRawNode -benchmem -benchtime=100000x . > $f; done; git checkout raft-single-voter
```

The two possible solutions (directly updating progress and calling
maybeCommit in `(*raft).advance` vs calling `r.Step`) are identical.  In
fact, we've gotten a tiny bit better with the `.Step` solution in terms
of not calling `firstIndex` as much, in the common case of not being a
single voter.

```
$ benchstat raft_directly_update_leader_in_advance.txt Revert_raft_directly_update_leader_in_advance_.txt
name                     old time/op        new time/op        delta
RawNode/single-voter-10         482ns ± 2%         742ns ± 1%   +54.02%  (p=0.000 n=9+9)
RawNode/two-voters-10          1.29µs ± 1%        1.31µs ± 2%    +1.70%  (p=0.000 n=9+10)

name                     old firstIndex/op  new firstIndex/op  delta
RawNode/single-voter-10          4.00 ± 0%          5.00 ± 0%   +25.00%  (p=0.000 n=10+10)
RawNode/two-voters-10            10.0 ± 0%           9.0 ± 0%   -10.00%  (p=0.000 n=10+10)

name                     old lastIndex/op   new lastIndex/op   delta
RawNode/single-voter-10          1.00 ± 0%          2.00 ± 0%  +100.00%  (p=0.000 n=10+10)
RawNode/two-voters-10            2.00 ± 0%          2.00 ± 0%      ~     (all equal)

name                     old ready/op       new ready/op       delta
RawNode/single-voter-10          1.00 ± 0%          2.00 ± 0%  +100.00%  (p=0.000 n=10+10)
RawNode/two-voters-10            2.00 ± 0%          2.00 ± 0%      ~     (all equal)

name                     old term/op        new term/op        delta
RawNode/single-voter-10          0.00 ± 0%          0.00 ± 0%      ~     (all equal)
RawNode/two-voters-10            1.00 ± 0%          1.00 ± 0%      ~     (all equal)

name                     old alloc/op       new alloc/op       delta
RawNode/single-voter-10          372B ± 0%          388B ± 0%    +4.30%  (p=0.000 n=10+10)
RawNode/two-voters-10            964B ± 0%          964B ± 0%      ~     (all equal)

name                     old allocs/op      new allocs/op      delta
RawNode/single-voter-10          4.00 ± 0%          5.00 ± 0%   +25.00%  (p=0.000 n=10+10)
RawNode/two-voters-10            7.00 ± 0%          7.00 ± 0%      ~     (all equal)
```

We then compare the `.Step` solution against the previous "status quo"
that prematurely emitted uncommitted entries for command application
below.

Importantly, we don't regress in the case of multiple peers. We actually
gain slightly in terms of `lastIndex` calls, but run a bit more code;
acceptable.

In the single-voter case, since we now need two Ready handling cycles
per op instead of one, we see additional calls to lastIndex and
firstIndex as well as slightly increased allocations. These are expected
and trade-offs we're willing to make to avoid correctness problems. Note
that the benchmark intentionally forces full processing of each
individual entries, so some of the new overhead would likely amortize on
a singleton voter seeing high throughput as multiple proposals could
share the Ready cycles.

```
$ benchstat raft_add_BenchmarkRawNode.txt Revert_raft_directly_update_leader_in_advance_.txt
name                     old time/op        new time/op        delta
RawNode/single-voter-10         482ns ± 2%         742ns ± 1%   +54.02%  (p=0.000 n=9+9)
RawNode/two-voters-10          1.29µs ± 1%        1.31µs ± 2%    +1.70%  (p=0.000 n=9+10)

name                     old firstIndex/op  new firstIndex/op  delta
RawNode/single-voter-10          4.00 ± 0%          5.00 ± 0%   +25.00%  (p=0.000 n=10+10)
RawNode/two-voters-10            10.0 ± 0%           9.0 ± 0%   -10.00%  (p=0.000 n=10+10)

name                     old lastIndex/op   new lastIndex/op   delta
RawNode/single-voter-10          1.00 ± 0%          2.00 ± 0%  +100.00%  (p=0.000 n=10+10)
RawNode/two-voters-10            2.00 ± 0%          2.00 ± 0%      ~     (all equal)

name                     old ready/op       new ready/op       delta
RawNode/single-voter-10          1.00 ± 0%          2.00 ± 0%  +100.00%  (p=0.000 n=10+10)
RawNode/two-voters-10            2.00 ± 0%          2.00 ± 0%      ~     (all equal)

name                     old term/op        new term/op        delta
RawNode/single-voter-10          0.00 ± 0%          0.00 ± 0%      ~     (all equal)
RawNode/two-voters-10            1.00 ± 0%          1.00 ± 0%      ~     (all equal)

name                     old alloc/op       new alloc/op       delta
RawNode/single-voter-10          372B ± 0%          388B ± 0%    +4.30%  (p=0.000 n=10+10)
RawNode/two-voters-10            964B ± 0%          964B ± 0%      ~     (all equal)

name                     old allocs/op      new allocs/op      delta
RawNode/single-voter-10          4.00 ± 0%          5.00 ± 0%   +25.00%  (p=0.000 n=10+10)
RawNode/two-voters-10            7.00 ± 0%          7.00 ± 0%      ~     (all equal)
```

Signed-off-by: Tobias Grieger <tobias.b.grieger@gmail.com>
  • Loading branch information
tbg committed Sep 8, 2022
1 parent 6332731 commit ba04f6c
Showing 0 changed files with 0 additions and 0 deletions.

0 comments on commit ba04f6c

Please sign in to comment.