Rework mechanism to achieve quorum on signatures

[jonathan-m-hamilton]

Improve the existing gathering signature scheme to include a stronger mechanism for achieving quorum.

Current State

To gather all validators’ signatures, we define two roles:

• Leader: responsible for gathering the signatures of other validators for a given data object and for electing the next leader among the list of alive validators.
• Follower: verifies that some data the leader transmitted to him is valid then sends him its signature of the data when asked. It also answers the leader’s request for heartbeat proving that it is still running.

The process of gathering signatures is quite simple. The federation uses a publish/subscribe system to allow the leader to easily broadcast its requests to all the followers without knowing them one by one. Followers subscribe to the leader’s publishing stream whose address is known and fixed for a given period.

Once the signature request is received, each follower verifies that information to sign is valid and sends back its signature to the leader. The leader collects all the signatures for a period of time then sends them to either the follower or the client who asked for the signatures.

The signatures gathering process is started when a client1 asks a validator he knows for signatures corresponding to some given data. If the validator is a leader it starts the protocol and sends back the response to the client. Otherwise, the follower asks its leader to do the job and then relies the response the leader sends him to the client.

Leader election

The leader in the set of validators changes after a pre-defined period. We’ve chosen to implement a simple protocol to elect the next leader. It’s the responsibility of the current leader to pick its successor among the set of alive followers.

The leader first broadcasts a Heartbeat request to the followers. Once received, each follower sends back a heartbeat message, including its coordinates, to inform the leader that it’s alive. The leader waits some time to get all followers answers, after which remaining ones are considered dead, then randomly picks the next leader from the set of alive ones.

NB. We are not yet handling the fact that a leader can get down before electing its successor. This can be fixed by adding a third role that would be a vice-leader to replace the leader when the later gets down.

It then broadcasts the coordinates of the new leader to inform the followers. After that all the validators including the leader either:

• Switch from leader to follower
• Switch from follower to leader
• Start following the new leader

Signatures

For signatures we are using the ECDSA secp256k1 elliptic curve and sha3 (256) to hash data to 32 bytes prior to generating its signature.

Every validator generates a pair of keys (public, private) at setup and stores them locally. The public keys are also stored in an identity system (Not yet implemented) that associates organizations’ DLTs with the set of their transfer validators public keys. This system allows any body to retrieve them any time

_Will leverage existing consensus mechanism from external software (investigating etcd and add in distributed lock for time to live. _

[petermetz]

Found out that since v3 etcd also has a dedicated election API built-in which further simplifies the process of having a leader among our application layer nodes.

One issue is that while the Go client has support for the election API, the NodeJS etcd client does not yet have this capability contributed to it. Currently exploring options regarding this.

References:
https://github.com/etcd-io/etcd/blob/master/clientv3/concurrency/election.go
https://stackoverflow.com/a/53601629/698470