Skip to content

Commit

Permalink
[FAB-4457] Add errorChan to Kafka-based consenter
Browse files Browse the repository at this point in the history
This changeset introduces a channel (`errorChan`) which closes when any
of the following conditions apply:

1. The retriable steps in Start haven't completed successfully.
2. The chain has been halted.
3. There is a consumption error from the channel's partition.

When the orderer can read from the Kafka partition without issues again,
a new, unbuffered `errorChan` is created and replaces the reference to
the previous, closed channel.

The `Errored` method -- part of the `Chain` interface that every orderer
plugin needs to satisfy -- has been modified so as to return the
`errorChan` channel.

Expected behaviors with this changeset:

1. When `errorChan` closes all Deliver requests will receive a
SERVICE_UNAVAILABLE response and will be terminated.
2. A closed `errorChan` will be recreated when:
- The Start stage completes successfully. (Note that errorChan is
closed by default when the chain object is instantiated.)
- A new message is received by the consumer in the main
`processMessagesToBlocks` loop.

There is an edge case in which the (a) the consumer has already fetched
the newest offset, (b) there are no incoming broadcast requests, and (c)
an error in the connection with the Kafka cluster triggers the closes
the `errorChan`. In this case, nothing can render the chain available
again. This is why we have the chain attempt to post a CONNECT message
(using the same retry options as in the Start stage), whenever the
`errorChan` is closed.

This changeset additionally:

1. Shifts around the unit tests in `chain_test.go` and further organizes
them into subtests so as to minimize the repetition of setup steps. This
was necessary in order to speed up the fixing of the tests during
revisions of this changeset.
2. Adds two new unit tests to exercise the Kafka consumption error path
(Ctrl+F for `ReceiveKafkaError` in `chain_test.go`).
3. Skips three unit tests (via an unconditional `t.Skip()` call) that
introduce race conditions. (Two of those will be fixed and reactivated
in a follow-up changeset.)
4. Turns the `processMessagesToBlocks` function into a method so as to
minimize the pointer references and the risk for coding errors.

This fixes FAB-4136.

Change-Id: I453412510829c70f84dc0c0375a6c53749bc23f0
Signed-off-by: Kostas Christidis <kostas@christidis.io>
  • Loading branch information
kchristidis committed Jun 11, 2017
1 parent 5dd2e33 commit 2d10ae0
Show file tree
Hide file tree
Showing 3 changed files with 1,099 additions and 827 deletions.
Loading

0 comments on commit 2d10ae0

Please sign in to comment.