Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add client-diversity page to /developer/docs #5218

Merged
merged 48 commits into from
Jan 29, 2022
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
d204fa9
"execution client" & "execution layer" to glossary
jmcook1186 Jan 24, 2022
9565751
Merge branch 'dev' of https://github.com/jmcook1186/ethereum-org-webs…
jmcook1186 Jan 24, 2022
753055d
"consensus client" & "consensus layer" to glossary
jmcook1186 Jan 24, 2022
2365a20
fix typo
jmcook1186 Jan 24, 2022
12401a0
Merge branch 'ethereum:dev' into dev
jmcook1186 Jan 24, 2022
449d2fc
Merge branch 'dev' into dev
minimalsm Jan 24, 2022
412c280
Merge branch 'ethereum:dev' into dev
jmcook1186 Jan 25, 2022
9654219
Update src/content/glossary/index.md
jmcook1186 Jan 25, 2022
377b087
Update src/content/glossary/index.md
jmcook1186 Jan 25, 2022
668473b
Merge branch 'ethereum:dev' into dev
jmcook1186 Jan 25, 2022
a10db44
initial commit for client-diversity page
jmcook1186 Jan 25, 2022
1f9d746
Merge branch 'ethereum:dev' into client_div_page
jmcook1186 Jan 25, 2022
4e21e75
refine "importance" section and fix typos
jmcook1186 Jan 25, 2022
753cafb
Merge branch 'dev' of https://github.com/jmcook1186/ethereum-org-webs…
jmcook1186 Jan 25, 2022
550abed
Merge branch 'client_div_page' of https://github.com/jmcook1186/ether…
jmcook1186 Jan 26, 2022
73eda64
Update src/content/glossary/index.md
samajammin Jan 26, 2022
9c60b92
+ info about slashing risk, + clientdiversity.org
jmcook1186 Jan 27, 2022
29c06aa
Merge branch 'dev' of https://github.com/jmcook1186/ethereum-org-webs…
jmcook1186 Jan 27, 2022
017bc0c
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
29f1bb9
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
83f014c
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
27ed4cd
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
f01f227
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
b555896
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
01a2bd5
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
fcd5272
Merge branch 'ethereum:dev' into client_div_page
jmcook1186 Jan 27, 2022
4b425fe
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
026d8c8
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
0589725
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
dad7932
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
faae3a9
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 27, 2022
5872bc3
update according to @minimalism comments
jmcook1186 Jan 27, 2022
b67bbd9
revert accidental commits relating to glossary PR
jmcook1186 Jan 27, 2022
5a15db0
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
725f826
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
b1cafe2
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
8e02b1f
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
ad45148
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
8bb594b
update according to @wackerow comments
jmcook1186 Jan 28, 2022
097644c
remove emoji
jmcook1186 Jan 28, 2022
918e70b
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
d68ee9f
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
9ee4521
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
0a87843
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
45e3b05
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
98475d7
Update src/content/developers/docs/nodes-and-clients/client-diversity…
jmcook1186 Jan 28, 2022
1377126
update to @minimalsm comments
jmcook1186 Jan 28, 2022
a9663c8
Apply suggestions from code review
wackerow Jan 29, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
title: Client Diversity
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved
description: A high level explanation of the importance of Ethereum client diversity.
lang: en
sidebar: true
sidebarDepth: 2
---

## Introduction {#Introduction}
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

The behaviour of an Ethereum node is controlled by the client software it runs. There are several production-level Ethereum clients, each one developed and maintained in different languages by separate teams. The clients are built to a common spec that ensures the clients seamlessly communicate with each other and have the same functionality and provide an equivalent user experience. Multiple clients exist because the development community has decided to build multiple implementations as a way to strengthen the network. However, at the moment the distribution of clients across nodes is not even enough to realise this network fortification to its full potential. Ideally, users divide roughly equally across the various clients to bring as much client diversity as possible to the network.
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

## Prerequisites {#prerequisites}

If you don't already have an understanding of what nodes and clients are, check out [Nodes and clients](/developers/docs/nodes-and-clients/). The Beacon Chain is explained [here](/src/content/eth2/beacon-chain/index.md). Execution and consensus layers are defined in the [glossary](/src/content/glossary/index.md).
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

## Current Client Diversity? {#current-client-diversity}
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

![Diagram showing snapshot of client diversity](../client-diversity.jpg)
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved
_This diagram shows snapshots of client diversity replotted using data from Ethernodes.org (execution layer) and from Michael Sproul (https://github.com/sigp/blockprint) for the consensus layer_
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

The two pie charts above show snapshots of the current client diversity for the execution and consensus layers. The execution layer is overwhelmingly dominated by [Geth](https://geth.ethereum.org/), with [Open Ethereum](https://openethereum.github.io/) a distant second, [Erigon](https://github.com/ledgerwatch/erigon) third and [Nethermind](https://nethermind.io/) fourth, with other clients comprising less than 1 % of the network. The most commonly used client on the consensus layer - [Prysm](https://prysmaticlabs.com/#projects) - is not as dominant as Geth but still represents over 60% of the network. [Lighthouse](https://lighthouse.sigmaprime.io/) and [Teku](https://consensys.net/knowledge-base/ethereum-2/teku/) make up ~20% and ~14% respectively, and other clients are rarely used.

The execution layer data were obtained from [Ethernodes](https://ethernodes.org) on 23/01/22. Data for consensus clients was obtained from [Michael Sproul](https://github.com/sigp/blockprint). Consensus client data is more difficult to obtain because the Beacon Chain clients do not always have unambiguous traces that can be used to identify them. The data was generated using a classification algorithm that sometimes confuses some of the minority clients (see [here](https://twitter.com/sproulM_/status/1440512518242197516) for more details). In the diagram above, these ambiguous classifications are treated with an either/or label (e.g. Nimbus/Teku). Nevertheless, it is clear that the majority of the network is running Prysm. The data is a snapshot over a fixed set of blocks (in this case Beacon blocks in slots 2048001 to 2164916) and Prysm's dominance has sometimes been higher, exceeding 68%. Despite only being snapshots, the values in the diagram provide a good general sense of the current state of client diversity.

## Why is client diversity important? {#client-diversity-importance}
Copy link
Contributor

@minimalsm minimalsm Jan 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could split this into subsections.

  • bugs
  • resilience to attacks
  • finalizing
  • Shared responsibility


Having many clients that are independently developed and maintained is important for the health of a decentralised network. There are several reasons why. First, a bug in one client is less threatening to the network as a whole if that client only represents a small proportion of the total client population. With relatively even distribution of nodes across many clients, the likelihood of a large fraction of them suffering from a shared issue is small and as a result the network is more robust.
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

Client diversity also offers resilience to attacks. For example, an attack that [tricks a particular client](https://twitter.com/vdWijden/status/1437712249926393858) onto a particular branch of the chain is unlikely to be successful because other clients are unlikely to be exploitable in the same way and the canonical chain remains uncorrupted. Low client diversity increases the risk associated with a hack on the dominant client.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Client diversity also offers resilience to attacks. For example, an attack that [tricks a particular client](https://twitter.com/vdWijden/status/1437712249926393858) onto a particular branch of the chain is unlikely to be successful because other clients are unlikely to be exploitable in the same way and the canonical chain remains uncorrupted. Low client diversity increases the risk associated with a hack on the dominant client.
Client diversity offers resilience to attacks. For example, an attack that [forces an individual client](https://twitter.com/vdWijden/status/1437712249926393858) onto an invalid chain fails because other clients aren't exploitable in the same way. In these cases, the canonical chain remains uncorrupted. Low client diversity increases the risk of an attack on the dominant client.


After the merge to proof-of-stake some of these risks become more acute because a critical bug in a consensus client with >33% share of all nodes could prevent the Beacon Chain from finalizing, stalling Ethereum. A client with both a 2/3 majority and a bug affecting consensus could cause the chain to [incorrectly split and then finalize](https://www.symphonious.net/2021/09/23/what-happens-if-beacon-chain-consensus-fails/), leading to a large set of validators being stuck on an incorrect chain with a slow and expensive voluntary withdrawal their only option for escape. Although these are unlikely scenarios, the risk associated with them can be mitigated by evening out the distribution of clients across the active nodes. Ideally, no consensus client would ever have more than 33% share of the total nodes.

At the same time, there is a human cost to single client dominance because it puts excess strain and responsibility on one small team of developers. The smaller the client diversity, the greater the consequences of a mistake for the developers maintaining the dominant client. Spreading this load across multiple teams is good for the health of Ethereum's network of nodes and network of people.
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

## Use a minority client {use-minority-client}

Addressing client diversity requires more than individual users to choose minority clients - it requires mining/validator pools and institutions like the major dapps and exchanges to switch clients too. However, all users can do their part in redressing the current imbalance and normalising the use of all the available Ethereum software. After the merge, all node operators will be required to run an execution client and a consensus client. Choosing combinations of the clients suggested below will help increase client diversity.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wackerow any thoughts on a stronger CTA here?

e.g.

  • ping your staking pool
  • email your CEX
  • contact your senator

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@minimalsm also added reference to Shanghai here in the "bugs" section, but happy to delete if you think it is superfluous.

jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

### Execution Clients
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

[Besu](https://www.hyperledger.org/use/besu)

[Nethermind](https://downloads.nethermind.io/)

[Erigon](https://github.com/ledgerwatch/erigon)

[CoreGeth](https://core-geth.org/)

### Consensus Clients
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

[Nimbus](https://nimbus.team/)

[Lighthouse](https://github.com/sigp/lighthouse)

[Teku](https://consensys.net/knowledge-base/ethereum-2/teku/)

and soon [Lodestar](https://github.com/ChainSafe/lodestar)
wackerow marked this conversation as resolved.
Show resolved Hide resolved

Technical users can help accelerate this process by writing more tutorials and documentation for minority clients and encouraging their node-operating peers to migrate away from the dominant clients.

## Further reading {#further-reading}

- [List of Ethereum node services](https://ethereumnodes.com/)
- [Importance of client diversity](https://our.status.im/the-importance-of-client-diversity/)
- ["Five Why's" of the client diversity problem](https://notes.ethereum.org/@afhGjrKfTKmksTOtqhB9RQ/BJGj7uh08)
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved
- [Ethereum Diversity and How to Solve For It (Youtube)](https://www.youtube.com/watch?v=1hZgCaiqwfU)

## Related topics {#related-topics}

- [Nodes and clients](/developers/docs/nodes-and-clients/)
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved
16 changes: 16 additions & 0 deletions src/content/glossary/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,14 @@ A group of at least 128 [validators](#validator) assigned to beacon and shard bl

When numerous nodes (usually most nodes on the network) all have the same blocks in their locally validated best blockchain. Not to be confused with [consensus rules](#consensus-rules).

### consensus client {consensus-client}
jmcook1186 marked this conversation as resolved.
Show resolved Hide resolved

After the merge to [proof-of-stake](#pos), consensus about the true head of the blockchain will be governed by a new set of clients running on the Beacon Chain known as "consensus clients". Consensus clients do not participate in validating transactions or executing state transitions. They request this to be done by [execution clients](#execution-client).

### consensus layer {consensus-layer}

The network of Ethereum [consensus clients](#consensus-client).

### consensus rules {#consensus-rules}

The block validation rules that full nodes follow to stay in consensus with other nodes. Not to be confused with [consensus](#consensus).
Expand Down Expand Up @@ -233,6 +241,14 @@ The ENS registry is a single central [contract](#smart-contract) that provides a

In the context of cryptography, lack of predictability or level of randomness. When generating secret information, such as [private keys](#private-key), algorithms usually rely on a source of high entropy to ensure the output is unpredictable.

### execution client {#execution-client}

After the merge to [proof-of-stake](#pos), existing Ethereum Mainnet clients will continue to host the [Ethereum Virtual Machine](#evm), validate transactions and execute state transitions but will not participate in consensus. These clients will therefore be referred to as "execution clients".

### execution layer

Ethereum's execution layer is the network of [execution clients](#execution-client).

### externally owned account (EOA) {#eoa}

An [account](#account) created by or for human users of the Ethereum network.
Expand Down