Cleanup RFCs on branch generation and leader rotation #1967

aeyakovenko · 2018-11-29T23:49:44Z

Problem

RFCs for consensus and leader rotation are incomplete and confusing.

Summary of Changes

RFCs for leader selection and branch generation. TBD: rfc for branch selection

Fixes #1924

rob-solana · 2018-11-30T20:32:16Z

rfcs/0002-branch-generation.md

+
+Validators can ignore forks at other points (e.g. from the wrong leader), or slash the leader responsible for the fork.
+
+Validators vote based on a greedy choise to maximze their reward described in [branch selection](rfcs/0008-branch_selection.md).


rob-solana · 2018-11-30T20:32:48Z

rfcs/0002-branch-generation.md

+4. Leader packages transactions into entries based off its current PoH slot.
+5. Leader transmits the entries to validator nodes (in signed blobs)
+    a. The PoH stream includes ticks; empty entries that indicate liveness of the leader and the passage of time on the network.
+    b. A leader's stream begins with the tick entries necessary complete the PoH back to the leaders most recently observed prior leader period.


period => slot everywhere?

rob-solana · 2018-11-30T20:37:07Z

rfcs/0004-leader-rotation.md

-2. Append them together.
-3. Hash the string for `N` counts via a similar process as PoH itself.
-4. The resulting hash is the random seed for `M` counts, `M` leader periods, where M > N
+1. Periodically at a specific `PoH tick count` use the tick count (simple monotonically increasing counter) as a seed to a stable psudo-random algorithm.


are we concerned that the leader schedule seeds are all pre-compute-able?

nevermind, answered below...

garious · 2018-11-30T20:37:26Z

rfcs/0002-branch-generation.md

@@ -0,0 +1,76 @@
+# Branch Generation
+
+The goal of this RFC is to define how Solana generates branches.


Solana doesn't generate branches. And if "branches" are different than "forks", you'll want to explain that here.

garious · 2018-11-30T20:39:29Z

rfcs/0002-branch-generation.md

+
+## Basic Design Idea
+
+Nodes take turns being leader and generating the PoH that encodes state changes.  The network can tolerate loss of connection to any leader by synthesizing what the leader ***would have generated*** had it been connected but not ingesting any state changes.  The complexity of forks is thereby limited to a "there/not-there" skip list of branches that may arise on leader rotation periods boundaries.  A leader can only transmit durring their predefined PoH slot.


"complexity of forks .. list of branches" huh?

on a boundary or after?

A leader can transmit at any time, but it's transactions will only be accepted if they fall within the leader's slot.

garious · 2018-11-30T20:39:42Z

rfcs/0002-branch-generation.md

+
+## Message Flow
+
+1. Transactions are ingested at the current leader.


garious · 2018-11-30T20:40:07Z

rfcs/0002-branch-generation.md

+## Message Flow
+
+1. Transactions are ingested at the current leader.
+2. Leader filters for valid transactions.


no need for "for" here

garious · 2018-11-30T20:40:34Z

rfcs/0002-branch-generation.md

+
+1. Transactions are ingested at the current leader.
+2. Leader filters for valid transactions.
+3. Leader executes valid transactions on its state.


"on its state" -> ", updating its state."

garious · 2018-11-30T20:52:16Z

rfcs/0004-leader-rotation.md

@@ -1,104 +1,29 @@
 # Leader Rotation

-The goal of this RFC is to define how leader nodes are rotated in Solana, how rotation may cause forks to arise, and how the converges
-in response.
+The goal of this RFC is to define how leader nodes are rotated in Solana.


More precisely, it's to define how fullnodes rotate in taking the leader role.

@garious why not leaders? a leader doesn't need to be a full node per se.

garious · 2018-11-30T20:52:48Z

rfcs/0004-leader-rotation.md

-2. Append them together.
-3. Hash the string for `N` counts via a similar process as PoH itself.
-4. The resulting hash is the random seed for `M` counts, `M` leader periods, where M > N
+1. Periodically at a specific `PoH tick count` use the tick count (simple monotonically increasing counter) as a seed to a stable psudo-random algorithm.


garious · 2018-11-30T20:54:17Z

rfcs/0004-leader-rotation.md

-3. Hash the string for `N` counts via a similar process as PoH itself.
-4. The resulting hash is the random seed for `M` counts, `M` leader periods, where M > N
+1. Periodically at a specific `PoH tick count` use the tick count (simple monotonically increasing counter) as a seed to a stable psudo-random algorithm.
+2. At that height, compute all the currently staked accounts and their assigned leader identities and weights.


This sentence doesn't make sense to me.

garious · 2018-11-30T20:55:18Z

rfcs/0004-leader-rotation.md

+2. At that height, compute all the currently staked accounts and their assigned leader identities and weights.
+3. Sort them by stake weight.
+4. Using the random seed select nodes weighted by stake to create a stake weighted ordering.
+5. This ordering becomes valid in `N` `PoH tick counts`.


That doesn't seem right. Seems like it ought to be something like N counts after the start of the slot.

the leader schedule needs to be available for a good while before it comes into effect to reduce forking that might arise because schedules themselves have forked

ideally, I'd want to see leader schedules available no later than 32 slots before they come into effect...

garious · 2018-11-30T20:59:02Z

rfcs/0004-leader-rotation.md

-V     |----|  |     |  |      |   |    |   |    |      vote(E(L4))
-V     | L5 |  xx   xx  xx   E(L5) xx   xx  xx   xx
-V     +----+                                           hang on to E(L4) and E(L5) for more...
+The seed that is selected is predictable but unbiasable.  There is no grinding attack to influence its outcome.  The set of **staked accounts** and their leader identities is computed over a large period, which is our approach to censorship resistance of the staking set.  If at least 1 leader in the schedule is not censoring staking transactions then over a long period of time that leader can ensure that the set of active nodes is not censored.


"our approach to censorship resistance" -> "is used to resist censorship". Also, how? Alternatively say that's out of the scope of this document.

Also, typos in the last sentence. Two missing commas and "is" should be "are".

That last sentence should either be dropped (out of scope) or expanded on a bunch.

garious · 2018-11-30T22:15:41Z

@aeyakovenko, why was this merged?

Finalize the stake pool struct for when a lockup will eventually be present.

* extract clock crate * update clock usage in solana-program * fmt * fmt after rebase * update lock file after rebase * fmt after rebase * fmt after rebase * fmt * make serde optional in solana-clock * fix description Co-authored-by: Jon C <me@jonc.dev> * fix docs link Co-authored-by: Jon C <me@jonc.dev> * fix accidental deletions from workspace members table --------- Co-authored-by: Jon C <me@jonc.dev>

rework rfcs

cd2408e

aeyakovenko force-pushed the branch_generation_rfc branch from a122fad to cd2408e Compare November 29, 2018 23:50

aeyakovenko requested a review from garious November 29, 2018 23:51

aeyakovenko changed the title ~~Rework RFCs~~ Cleanup RFCs on branch generation and leader rotation Nov 29, 2018

aeyakovenko requested a review from rob-solana November 30, 2018 19:58

rob-solana reviewed Nov 30, 2018

View reviewed changes

rob-solana approved these changes Nov 30, 2018

View reviewed changes

comments

7ef0bc9

aeyakovenko merged commit b8ebb4d into solana-labs:master Nov 30, 2018

garious reviewed Nov 30, 2018

View reviewed changes

aeyakovenko mentioned this pull request Nov 30, 2018

Fork generation RFCs #1976

Merged

vkomenda pushed a commit to vkomenda/solana that referenced this pull request Aug 29, 2021

stake-pool: Update version for crate publish (solana-labs#1967)

edf07b1

Finalize the stake pool struct for when a lockup will eventually be present.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cleanup RFCs on branch generation and leader rotation #1967

Cleanup RFCs on branch generation and leader rotation #1967

aeyakovenko commented Nov 29, 2018 •

edited

Loading

rob-solana Nov 30, 2018

rob-solana Nov 30, 2018

rob-solana Nov 30, 2018

rob-solana Nov 30, 2018

garious Nov 30, 2018

garious Nov 30, 2018

garious Nov 30, 2018

garious Nov 30, 2018

garious Nov 30, 2018

garious Nov 30, 2018

aeyakovenko Nov 30, 2018

garious Nov 30, 2018

garious Nov 30, 2018

garious Nov 30, 2018

rob-solana Nov 30, 2018

rob-solana Nov 30, 2018

garious Nov 30, 2018

garious commented Nov 30, 2018


		Validators can ignore forks at other points (e.g. from the wrong leader), or slash the leader responsible for the fork.

		Validators vote based on a greedy choise to maximze their reward described in [branch selection](rfcs/0008-branch_selection.md).

		@@ -0,0 +1,76 @@
		# Branch Generation

		The goal of this RFC is to define how Solana generates branches.


		## Basic Design Idea

		Nodes take turns being leader and generating the PoH that encodes state changes. The network can tolerate loss of connection to any leader by synthesizing what the leader *would have generated* had it been connected but not ingesting any state changes. The complexity of forks is thereby limited to a "there/not-there" skip list of branches that may arise on leader rotation periods boundaries. A leader can only transmit durring their predefined PoH slot.


		## Message Flow

		1. Transactions are ingested at the current leader.

Cleanup RFCs on branch generation and leader rotation #1967

Cleanup RFCs on branch generation and leader rotation #1967

Conversation

aeyakovenko commented Nov 29, 2018 • edited Loading

Problem

Summary of Changes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

garious commented Nov 30, 2018

aeyakovenko commented Nov 29, 2018 •

edited

Loading