Drand - A Distributed Randomness Beacon Daemon

Drand (pronounced "dee-rand") is a distributed randomness beacon daemon written in Golang. Servers running drand can be linked with each other to produce collective, publicly verifiable, unbiased, unpredictable random values at fixed intervals using bilinear pairings and threshold cryptography. Drand nodes can also serve locally-generated private randomness to clients.

drand was first developed within the DEDIS organization, and as of December 2019, is now under the drand organization.

Disclaimer

This software is considered experimental and has NOT received a third-party audit yet. Therefore, DO NOT USE it in production or for anything security critical at this point.

Goal and Overview

The need for digital randomness is paramount in multiple digital applications ([e]voting, lottery, cryptographic parameters, embedded devices bootstrapping randomness, blockchain systems etc) as well in non-digital such as statistical sampling (used for example to check results of an election), assigning court cases to random judges, random financial audits, etc. However, constructing a secure source of randomness is nothing but easy: there are countless examples of attacks where the randomness generation was the culprit (static keys, non-uniform distribution, biased output, etc). drand aims to fix that gap by providing a Randomness-as-a-Service network (similar to NTP servers for time, or Certificate Authority servers for CAs verification), providing continuous source of randomness which is:

Decentralized: drand is a software ran by a diverse set of reputable entities on the Internet and a threshold of them is needed to generate randomness, there is no central point of failure.
Publicly verifiable & unbiased: drand periodically delivers publicly verifiable and unbiased randomness. Any third party can fetch and verify the authenticity of the randomness and by that making sure it hasn't been tampered with.
And "private" as well: drand nodes can also deliver encrypted randomness to be used in a local applications, for example to seed the OS's PRNG.

Drand currently runs a first test network composed by trustworthy organizations around the globe such as Cloudflare, EPFL, University of Chile and Kudelski Security. The main website of the first launch sponsored by Cloudflare is hosted at the league of entropy site. There is an independent drand website (source in web/) showing the same network hosted in one of the participant's server: https://drand.zerobyte.io

Public Randomness

Generating public randomness is the primary functionality of drand. Public randomness is generated collectively by drand nodes and publicly available. The main challenge in generating good randomness is that no party involved in the randomness generation process should be able to predict or bias the final output. Additionally, the final result has to be third-party verifiable to make it actually useful for applications like lotteries, sharding, or parameter generation in security protocols.

A drand randomness beacon is composed of a distributed set of nodes and has two phases:

Setup: Each node first generates a long-term public/private key pair. Then all of the public keys are written to a group file together with some further metadata required to operate the beacon. After this group file has been distributed, the nodes perform a distributed key generation (DKG) protocol to create the collective public key and one private key share per server. The participants NEVER see/use the actual (distributed) private key explicitly but instead utilize their respective private key shares for the generation of public randomness.
Generation: After the setup, the nodes switch to the randomness generation mode. Any of the nodes can initiate a randomness generation round by broadcasting a message which all the other participants sign using a t-of-n threshold version of the Boneh-Lynn-Shacham (BLS) signature scheme and their respective private key shares. Once any node (or third-party observer) has gathered t partial signatures, it can reconstruct the full BLS signature (using Lagrange interpolation). The signature is then hashed using SHA-512 to ensure that there is no bias in the byte representation of the final output. This hash corresponds to the collective random value and can be verified against the collective public key.

Private Randomness

Private randomness generation is the secondary functionality of drand. Clients can request private randomness from some or all of the drand nodes which extract it locally from their entropy pools and send it back in encrypted form. This can be useful to gather randomness from different entropy sources, for example in embedded devices.

In this mode we assume that a client has a private/public key pair and encapsulates its public key towards the server's public key using the ECIES encryption scheme. After receiving a request, the drand node produces 32 random bytes locally (using Go's crypto/rand interface), encrypts them using the received public key and sends it back to the client.

Note: Assuming that clients without good local entropy sources (such as embedded devices) use this process to gather high entropy randomness to bootstrap their local PRNGs, we emphasize that the initial client key pair has to be provided by a trusted source (such as the device manufacturer). Otherwise we run into the chicken-and-egg problem of how to produce on the client's side a secure ephemeral key pair for ECIES encryption without a good (local) source of randomness.

Local demo

To run a local demo, you can simply run:

make demo

The script spins up a few drand local processes, performe resharing and other operations and will continue to print out new randomness every Xs (currently 6s). For more information, look at the demo README.

Installation

Official release

Please go use the latest drand binary in the release page.

Manual installation

Drand can be installed via Golang or Docker. By default, drand saves the configuration files such as the long-term key pair, the group file, and the collective public key in the directory $HOME/.drand/.

Via Golang

Make sure that you have a working Golang installation and that your GOPATH is set.

Then install drand via:

git clone https://github.com/drand/drand
cd drand
make install

Via Docker

The setup is explained in README_docker.md.

TLS setup: Nginx with Let's Encrypt

Running drand behind a reverse proxy is the default method of deploying drand. Such a setup greatly simplify TLS management issues (renewal of certificates, etc). We provide here the minimum setup using Nginx and certbot - make sure you have both binaries installed with the latest version; Nginx version must be at least >= 1.13.10 for gRPC compatibility.

First, add an entry in the Nginx configuration for drand:

# /etc/nginx/sites-available/default
server {
  server_name drand.nikkolasg.xyz;
  listen 443 ssl http2;
 
  location / {
    grpc_pass grpc://localhost:8080;
  }
  location /api/ {
    proxy_pass http://localhost:8080;
    proxy_set_header Host $host;
  }
}

Note: you can change

the port on which you want drand to be accessible by changing the line listen 443 ssl http2 to use any port.
the port on which the drand binary will listen locally by changing the line proxy_pass http://localhost:8080; and grpc_pass grpc://localhost:8080; to use any local port.

Run certbot to get a TLS certificate:

sudo certbot --nginx

Running drand now requires to add the following options:

drand start --tls-disable --listen 127.0.0.1:8080

The --listen flag tells drand to listen on the given address instead of the public address generated during the setup phase (see below).

Usage

This section explains in details the workflow to have a working group of drand nodes generate randomness. On a high-level, the workflow looks like this:

Setup: generation of individual long-term key pair and the group file and starting the drand daemon.
Distributed Key Generation: each drand node collectively participates in the DKG.
Randomness Generation: the randomness beacon automatically starts as soon as the DKG protocol is finished.

Setup

The setup process for a drand node consists of the following steps:

Generate the long-term key pair for each node
Each node starts their daemon
Leader starts the command as a coordinator & every participant connect to the coordinator to setup the network

Long-Term Key

To generate the long-term key pair drand_id.{secret,public} of the drand daemon, execute

drand generate-keypair <address>

where <address> is the address from which your drand daemon is reachable. The address must be reachable over a TLS connection directly or via a reverse proxy setup. In case you need non-secured channel, you can pass the --tls-disable flag.

Starting drand daemon

The daemon does not go automatically in background, so you must run it with & in your terminal, within a screen / tmux session, or with the -d option enabled for the docker commands. Once the daemon is running, the way to issue commands to the daemon is to use the control functionalities. The control client has to run on the same server as the drand daemon, so only drand administrators can issue command to their drand daemons.

There are two ways to run a drand daemon: using TLS or using plain old regular unencrypted connections. Drand by default tries to use TLS connections.

With TLS

Drand nodes attempt to communicate by default over TLS-protected connections. Therefore, you need to point your node to the TLS certificate chain and corresponding private key you wish to use via:

drand start \
    --tls-cert <fullchain.pem> \
    --tls-key <privkey.pem>

To get TLS certificates for free you can use, for example, Let's Encrypt with its official CLI tool EFF's certbot.

Without TLS

Although we do not recommend it, you can always disable TLS in drand via:

drand start --tls-disable

Run the setup phase

To setup a new network, drand uses the notion the of a coordinator that collects the public key of the participants, setups the group configuration once all keys are received and then start the distributed key generation phase. Once the DKG phase is performed, the participants can see the list of members in the group configuration file

Coordinator: The designated coordinator node must run the following command before everyone else:

drand share --leader --nodes 10 --threshold 6 --secret mysecret --period 30s

Rest of participants: Once the coordinator has run the previous command, the rest of the participants must run the following command:

drand share --connect <leaderaddress> --nodes 10 --threshold 6 --secret mysecret

The flags usage is as follow:

--leader indicates this node is a coordinator, `
--nodes indicates how many nodes do we expect to form the network
--threshold indicates the threshold the network should use, i.e. how many nodes amongst the total needs to be online for the network to be live at any point.
--period indicates the period of the randomness beacon to use. It must be valid duration as parsed by Golang's [time.ParseDuration](https://golang.org/pkg/time/#ParseDuration) method.
--secret indicates the secret that the coordinator uses to authentify the nodes that wants to participate to the network.
--connect is the host:port address of the leader. By default, drand will connect to the leader by using tls. If you are not using tls, use the --tls-disable flag.

Interactive command: The command will run as long as the DKG is not finished yet. You can quit the command, the DKG will proceed but the group file will not be written down. In that case, once the DKG is done, you get the group file by running:

drand show group --out group.toml

Secret: For participants to be included in the group, they need to have a secret string shared by all. This method is offering some basic security however drand will provide more manual checks later-on and/or different secrets for each participants. However, since the set of participants is public and consistent accross all participants after a setup, nodes can detect if there are some unwanted nodes after the setup and in that case, setup a new network again.

Custom entropy source: By default drand takes its entropy for the setup phase from the OS's entropy source (/dev/urandom on Unix systems). However, it is possible for a participant to inject their own entropy source into the creation of their secret. To do so, one must have an executable that produces random data when called and pass the name of that executable to drand:

drand share <regular options> --source <entropy-exec>

where <entropy-exec> is the path to the executable which produces the user's random data on STDOUT. As a precaution, the user's randomness is mixed by default with crypto/rand to create a random stream. In order to introduce reproducibility, the flag user-source-only can be set to impose that only the user-specified entropy source is used. Its use should be limited to testing.

drand share <group-file> --source <entropy-exec> --user-source-only

Distributed Key Generation

Once the DKG phase is done, each node has both a private share and a group file containing the distributed public key. Using the previous commands shown, the group file will be written to group.toml. That updated group file is needed by drand to securely contact drand nodes on their public interface to gather private or public randomness. A drand administrator can get the updated group file it via the following:

drand show group

It will print the group file in its regular TOML format. If you want to save it to a file, append the --out <file> flag.

Distributed Public Key: More generally, for third party implementation of randomness beacon verification, one only needs the distributed public key. If you are an administrator of a drand node, you can use the control port as the following:

drand show cokey

Otherwise, you can contact an external drand node to ask him for its current distributed public key:

drand get cokey <address>

where <group.toml> is the group file identity file of a drand node. You can use the flag --nodes <address(es)> to indicate which node you want to contact specifically (it is a white space separated list). Use the--tls-cert flag to specify the server's certificate if needed. The group toml does not need to be updated with the collective key.

NOTE: Using the last method (get cokey), a drand node can lie about the key if no out-of-band verification is performed. That information is usually best gathered from a trusted drand operator and then embedded in any applications using drand.

Randomness Generation

After a successful setup phase, drand will switch to the randomness generation mode at the genesis time specified in the group file. At that time, each node broadcasts randomness shares at regular intervals. Once a node has collected a threshold of shares in the current phase, it computes the public random value and stores it in its local instance of BoltDB.

Timings of randomness generation: At each new period, each node will try to broadcast their partial signatures for the corresponding round and try to generate a full randomness from the partial signatures. The corresponding round is the number of rounds elapsed from the genesis time. That means there is a 1-1 mapping between a given time and a drand round.

Daemon downtime & Chain Sync: Due to the threshold nature of drand, a drand network can support some numbers of nodes offline at any given point. This number is determined by the threshold: max_offline = group_len - threshold. When a drand node goes back up, it will sync rapidly with the other nodes to catch up its local chain and participate in the next upcoming drand round.

Drand network failure: If for some reason drand goes down for some time and then backs up, the new randomn beacon will be built over the last successfully generated beacon. For example, if the network goes down at round 10 (i.e. last beacon generated contained round: 10), and back up again at round 20 (i.e. field round: 20), then this new randomness contains the field previous_round:10.

Control Functionalities

Drand's local administrator interface provides further functionality, e.g., to update group details or retrieve secret information. By default, the daemon listens on 127.0.0.1:8888, but you can specify another control port when starting the daemon with:

drand start --control 1234

In that case, you need to specify the control port for each of the following commands.

Long-Term Private Key

To retrieve the long-term private key of our node, run:

drand show private

Long-Term Public Key

To retrieve the long-term public key of our node, run:

drand show public

Private Key Share

To retrieve the private key share of our node, as determined during the DKG, run the following command:

drand show share

The JSON-formatted output has the following form:

{
  "index" : 1,
  "share" : {
    "gid": 22,
    "scalar": "764f6e3eecdc4aba8b2f0119e7b2fd8c35948bf2be3f87ebb5823150c6065764"
  }
}

The "gid" simply indicates which group the data belongs to. It is present for scalar and points on the curve, even though scalars are the same on the three groups of BN256. The field is present already to be able to accommodate different curves later on.

Distributed Key

To retrieve the collective key of the drand beacon our node is involved in, run:

drand show cokey

Using Drand

A drand beacon provides several public services to clients. A drand node exposes its public services on a gRPC endpoint as well as a REST JSON endpoint, on the same port. The latter is especially useful if one wishes to retrieve randomness from a JavaScript application. Communication is protected through TLS by default. If the contacted node is using a self-signed certificate, the client can use the --tls-cert flag to specify the server's certificate.

Fetching Public Randomness

To get the latest public random value, run

drand get public --round <i> <group.toml>

where <group.toml> is the group identity file of a drand node. You can specify the round number when the public randomness has been generated. If not specified, this command returns the most recent random beacon.

The JSON-formatted output produced by drand is of the following form:

{
    "round": 2,
d8a74d4d3b3664a90409f7ec575f7211f06502001561b00e036d0fbd42d2b",
    "signature": "357562670af7e67f3534f5a5a6e01269f3f9e86a7b833591b0ec2a51faa7c11111
2a1dc1baea73926c1822bc5135469cc1c304adc6ccc942dac7c3a52977a342",
    "previous_signature": "5e59b03c65a82c9f2be39a7fd23e8e8249fd356c4fd7d146700fc428ac80ec3f7a22a1dc1baea73926c1822bc5135469cc1c304adc6ccc942dac7c3a52977a342",
    "previous_round": 1,
    "randomness": "ee9e1aeba4a946ce2ac2bd42ab04439c959d8538546ea637418394c99c522eec2
    92bbbfac2605cbfe3734e40a5d3cc762428583b243151b2a84418e376ea0af6"
}

Here Signature is the threshold BLS signature on the previous signature value Previous and the current round number. Randomness is the hash of Signature, to be used as the random value for this round. The field Round specifies the index of Randomness in the sequence of all random values produced by this drand instance. The message signed is therefore the concatenation of the round number treated as a uint64 and the previous signature. At the moment, we are only using BLS signatures on the BN256 curves and the signature is made over G1.

Fetching Private Randomness

To get a private random value, run the following:

drand get private group.toml

The JSON-formatted output produced by drand should look like the following:

{
    "Randomness": "764f6e3eecdc4aba8b2f0119e7b2fd8c35948bf2be3f87ebb5823150c6065764"
}

The command outputs a 32-byte hex-encoded random value generated from the local randomness engine of the contacted server. If the encryption is not correct, the command outputs an error instead.

Using HTTP endpoints

One may want get the distributed key or public randomness by issuing a GET to a HTTP endpoint instead of using a gRPC client. Here is a basic example on how to do so with curl.

To get the distributed key, you can use:

curl <address>/api/info/distkey

Similarly, to get the latest round of randomness from the drand beacon, you can use

curl <address>/api/public

All the REST endpoints are specified in the protobuf/drand/client.proto file.

Updating Drand Group

Drand allows for "semi-dynamic" group update with a resharing protocol that offers the following:

new nodes can join an existing group and get new shares. Note that, in fact, all nodes get new shares after running the resharing protocol.
nodes can leave their current group. It may be necessary for nodes that do not wish to operate drand anymore.
nodes can update the threshold associated with their current distributed public key.

The main advantage of this method is that the distributed public key stays the same even with new nodes coming in. That can be useful when the distributed public key is embedded inside the application using drand, and hence is difficult to update.

Setting up the coordinator: The coordinator must be a member of the current network. To run the coordinator, run the following:

drand share --leader --transition --nodes 15 --treshold 10 --secret mysecret2 --out
group2.toml

Setting up the current members for the resharing: The current members can simply run the following command:

drand share --transition --nodes 15 --threshold 10 --secret mysecret2 --out
group2.toml

Setting up the new members: The new members need the current group file to proceed. Check how to get the group file in the "Using the drand daemon" section. Then run the command:

drand share --from group.toml --nodes 15 --threshold 10 --secret mysecret2 --out
group2.toml

After the protocol is finished, each node will have the new group file written out as group2.toml. The randomness generation starts only at the specified transition time specified in the new group file.

DrandJS

To facilitate the use of drand's randomness in JavaScript-based applications, we provide DrandJS. The main method fetchAndVerify of this JavaScript library fetches from a drand node the latest random beacon generated and then verifies it against the distributed key. For more details on the procedure and instructions on how to use it, refer to the readme.

Note this library is still a proof of concept and uses a rather slow pairing based library in JavaScript.

Documentation

Here is a list of all documentation related to drand:

For a high level presentation of motivations and background, here are some public slides about drand or online video.
The client-side API documentation of of drand: link
The drand operator guide documentation: link
A basic explainer of the cryptography behind drand: link,

As well, here is a list of background readings w.r.t to the cryptography used in drand:

Pairing-based cryptography and Barreto-Naehrig curves.
Pedersen's distributed key generation protocol for the setup.
Threshold BLS signatures for the generation of public randomness.
The resharing scheme used comes from the paper from Y. Desmedt and S. Jajodia.
ECIES for the encryption of private randomness.

Note that drand was originally a DEDIS-owned project that is now spinning off on its own Github organization. For related previous work on public randomness, see DEDIS's academic paper Scalable Bias-Resistant Distributed Randomness.

What's Next?

Although being already functional, drand is still at an early development stage and there is a lot left to be done. The list of opened issues is a good place to start. On top of this, drand would benefit from higher-level enhancements such as the following:

Implement a more failure-resilient DKG protocol or an approach based on verifiable succinct computations (zk-SNARKs, etc).
Use / implement a faster pairing based library in JavaScript
implement "customizable" randomness, where input is chosen from the user (drand would be acting as a distributed threshold oPRF)
expand the network
implemented ECIES private randomness in JavaScript (?)
Add more unit tests
Reduce size of Docker
Add a systemd unit file
Support multiple drand instances within one node

Feel free to submit feature requests or, even better, pull requests ;) But please note like, this is still currently a side project! Contact me on twitter for more information about the project.

License

The drand source code is released under MIT license originated at the DEDIS lab, see the file LICENSE for the full text. All modifications brought to this repository are as well under an MIT license.

Contributors

Here's the list of people that contributed to drand:

Nicolas Gailly (@nikkolasg1)
Philipp Jovanovic (@daeinar)
Mathilde Raynal (@PizzaWhisperer)
Ludovic Barman (@Lbarman)
Gabbi Fisher (@gabbifish)
Linus Gasser (@ineiti)
Jeff Allen (@jeffallen)

Acknowledgments

Thanks to @herumi for providing support on his optimized pairing-based cryptographic library used in the first version.

Thanks to Apostol Vassilev for its interest in drand and the extensive and helpful discussions on the drand design.

Thanks to @Bren2010 and @grittygrease for providing the native Golang bn256 implementation and for their help in the design of drand and future ideas.

Finally, a special note for Bryan Ford from the DEDIS lab for letting me work on this project and helping me grow it.

Coverage

EPFL blog post
Cloudflare crypto week introduction post and the more technical post.
Kudelski Security blog post
OneZero post on the league of entropy
SlashDot post
Duo post
Liftr
(French) nextimpact

Supporting

Drand is an open source project, currently as a side project. If you believe in the project, your financial help would be very valuable. Please contact me on twitter to know more about the project and its continuation and how to fund it. More documentation on that front will arrive.

Name		Name	Last commit message	Last commit date
Latest commit History 409 Commits
.circleci		.circleci
beacon		beacon
core		core
demo		demo
deploy		deploy
dkg		dkg
docker		docker
ecies		ecies
entropy		entropy
fs		fs
key		key
log		log
net		net
protobuf		protobuf
test		test
vendor		vendor
web		web
.dockerignore		.dockerignore
.gitignore		.gitignore
.gitmodules		.gitmodules
.goreleaser.yml		.goreleaser.yml
.travis.yml		.travis.yml
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
control.go		control.go
daemon.go		daemon.go
go.mod		go.mod
go.sum		go.sum
main.go		main.go
main_test.go		main_test.go
public.go		public.go

License

lukevalenta/drand

Folders and files

Latest commit

History

Repository files navigation

Drand - A Distributed Randomness Beacon Daemon

Disclaimer

Table of Contents

Goal and Overview

Public Randomness

Private Randomness

Local demo

Installation

Official release

Manual installation

Via Golang

Via Docker

TLS setup: Nginx with Let's Encrypt

Usage

Setup

Long-Term Key

Starting drand daemon

With TLS

Without TLS

Run the setup phase

Distributed Key Generation

Randomness Generation

Control Functionalities

Long-Term Private Key

Long-Term Public Key

Private Key Share

Distributed Key

Using Drand

Fetching Public Randomness

Fetching Private Randomness

Using HTTP endpoints

Updating Drand Group

DrandJS

Documentation

What's Next?

License

Contributors

Acknowledgments

Coverage

Supporting

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages