OffsetManager: match upstream mark/next behaviour

[eapache]

Upstream requires you mark last-consumed+1 and then returns that value directly.
We were requiring you mark last-consumed and then adding one to the returned
value.

Match upstream's behaviour so that our offset tracking is interoperable.

Fixes #705.

@Kane-Sendgrid

[tcrayford]

@eapache does this mean that all consumers that use the offset manager that upgrade past this patch will drop a single message during the upgrade?

[eapache]

It depends on what you were using before. If you were sharing the offset topic with an upstream java producer consumer you were already potentially inconsistent because of the different expectations.

If you were using the previous Sarama behaviour as documented (storing the offset of the last-consumed message) then an upgrade should cause you to consume a single message twice (the previous code will consume X and mark X, then the new code will read X, consume X, and mark X+1). Your consumer should already be able to handle that safely because it can occur in other scenarios.

Note that either way you should ensure that code using the behaviour in this PR commits last-consumed + 1 instead of just last-consumed. Otherwise you will see duplicate messages consumed every time your consumer restarts.

[eapache]

I've added a more explicit warning to the changelog, but since consumers should already be able to handle duplicates (and this change will at worst cause a single duplicate message when a consumer restarts) I'm not too worried.

[eapache]

cc @dim FYI - I don't think sarama-cluster is using our offset-manager but I know you took some inspiration from it so you might have the same issue.

[tcrayford]

@eapache ok, that seems fine

[dim]

@eapache thanks for the heads up - fixed in bsm/sarama-cluster@6fdf904