Merge transition process with computed transition total difficulty

README.md

@@ -43,6 +43,7 @@ while the details are in review and may change.
  * [ethereum.org](https://ethereum.org) high-level description of the merge [here](https://ethereum.org/en/eth2/docking/)
 * Specifications:
  * [Beacon Chain changes](specs/merge/beacon-chain.md)
+ * [Merge fork](specs/merge/fork.md)
  * [Fork Choice changes](specs/merge/fork-choice.md)
  * [Validator additions](specs/merge/validator.md)
 

configs/mainnet.yaml

@@ -32,7 +32,7 @@ SHARDING_FORK_VERSION: 0x03000000
 SHARDING_FORK_EPOCH: 18446744073709551615
 
 # TBD, 2**32 is a placeholder. Merge transition approach is in active R&D.
-TRANSITION_TOTAL_DIFFICULTY: 4294967296
+MIN_ANCHOR_POW_BLOCK_DIFFICULTY: 4294967296
 
 
 # Time parameters

configs/minimal.yaml

@@ -31,7 +31,7 @@ SHARDING_FORK_VERSION: 0x03000001
 SHARDING_FORK_EPOCH: 18446744073709551615
 
 # TBD, 2**32 is a placeholder. Merge transition approach is in active R&D.
-TRANSITION_TOTAL_DIFFICULTY: 4294967296
+MIN_ANCHOR_POW_BLOCK_DIFFICULTY: 4294967296
 
 
 # Time parameters

setup.py

@@ -509,7 +509,7 @@ def sundry_functions(cls) -> str:
 
 def get_pow_block(hash: Bytes32) -> PowBlock:
  return PowBlock(block_hash=hash, is_valid=True, is_processed=True,
- total_difficulty=config.TRANSITION_TOTAL_DIFFICULTY)
+ total_difficulty=uint256(0), difficulty=uint256(0))
 
 
 def get_execution_state(execution_state_root: Bytes32) -> ExecutionState:
@@ -865,6 +865,7 @@ def finalize_options(self):
  specs/phase0/validator.md
  specs/phase0/weak-subjectivity.md
  specs/merge/beacon-chain.md
+ specs/merge/fork.md
  specs/merge/fork-choice.md
  specs/merge/validator.md
  """

specs/merge/beacon-chain.md

@@ -14,8 +14,6 @@
 - [Custom types](#custom-types)
 - [Constants](#constants)
  - [Execution](#execution)
-- [Configuration](#configuration)
- - [Transition](#transition)
 - [Containers](#containers)
  - [Extended containers](#extended-containers)
  - [`BeaconBlockBody`](#beaconblockbody)
@@ -35,6 +33,7 @@
  - [Block processing](#block-processing)
  - [Execution payload processing](#execution-payload-processing)
  - [`process_execution_payload`](#process_execution_payload)
+- [Initialize state for pure Merge testnets and test vectors](#initialize-state-for-pure-merge-testnets-and-test-vectors)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 <!-- /TOC -->
@@ -62,18 +61,6 @@ We define the following Python custom types for type hinting and readability:
 | `MAX_EXECUTION_TRANSACTIONS` | `uint64(2**14)` (= 16,384) |
 | `BYTES_PER_LOGS_BLOOM` | `uint64(2**8)` (= 256) |
 
-## Configuration
-
-Warning: this configuration is not definitive.
-
-### Transition
-
-| Name | Value |
-| - | - |
-| `MERGE_FORK_VERSION` | `Version('0x02000000')` |
-| `MERGE_FORK_EPOCH` | `Epoch(18446744073709551615)` **TBD** |
-| `TRANSITION_TOTAL_DIFFICULTY` | **TBD** |
-
 ## Containers
 
 ### Extended containers
@@ -247,3 +234,63 @@ def process_execution_payload(state: BeaconState,
  transactions_root=hash_tree_root(execution_payload.transactions),
  )
 ```
+
+## Initialize state for pure Merge testnets and test vectors
+
+This helper function is only for initializing the state for pure Merge testnets and tests.
+
+*Note*: The function `initialize_beacon_state_from_eth1` is modified: (1) using `MERGE_FORK_VERSION` as the current fork version, (2) utilizing the Merge `BeaconBlockBody` when constructing the initial `latest_block_header`, and (3) adding initial `latest_execution_payload_header`.
+
+```python
+def initialize_beacon_state_from_eth1(eth1_block_hash: Bytes32,
+ eth1_timestamp: uint64,
+ deposits: Sequence[Deposit]) -> BeaconState:
+ fork = Fork(
+ previous_version=GENESIS_FORK_VERSION,
+ current_version=MERGE_FORK_VERSION, # [Modified in Merge]
+ epoch=GENESIS_EPOCH,
+ )
+ state = BeaconState(
+ genesis_time=eth1_timestamp + GENESIS_DELAY,
+ fork=fork,
+ eth1_data=Eth1Data(block_hash=eth1_block_hash, deposit_count=uint64(len(deposits))),
+ latest_block_header=BeaconBlockHeader(body_root=hash_tree_root(BeaconBlockBody())),
+ randao_mixes=[eth1_block_hash] * EPOCHS_PER_HISTORICAL_VECTOR, # Seed RANDAO with Eth1 entropy
+ )
+
+ # Process deposits
+ leaves = list(map(lambda deposit: deposit.data, deposits))
+ for index, deposit in enumerate(deposits):
+ deposit_data_list = List[DepositData, 2**DEPOSIT_CONTRACT_TREE_DEPTH](*leaves[:index + 1])
+ state.eth1_data.deposit_root = hash_tree_root(deposit_data_list)
+ process_deposit(state, deposit)
+
+ # Process activations
+ for index, validator in enumerate(state.validators):
+ balance = state.balances[index]
+ validator.effective_balance = min(balance - balance % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE)
+ if validator.effective_balance == MAX_EFFECTIVE_BALANCE:
+ validator.activation_eligibility_epoch = GENESIS_EPOCH
+ validator.activation_epoch = GENESIS_EPOCH
+
+ # Set genesis validators root for domain separation and chain versioning
+ state.genesis_validators_root = hash_tree_root(state.validators)
+
+ # [New in Merge] Construct execution payload header
+ # Note: initialized with zero block height
+ state.latest_execution_payload_header = ExecutionPayloadHeader(
+ block_hash=eth1_block_hash,
+ parent_hash=Hash32(),
+ coinbase=Bytes20(),
+ state_root=Bytes32(),
+ number=uint64(0),
+ gas_limit=uint64(0),
+ gas_used=uint64(0),
+ timestamp=eth1_timestamp,
+ receipt_root=Bytes32(),
+ logs_bloom=ByteVector[BYTES_PER_LOGS_BLOOM](),
+ transactions_root=Root(),
+ )
+
+ return state
+```

specs/merge/fork-choice.md

@@ -12,13 +12,13 @@
  - [`ExecutionEngine`](#executionengine)
  - [`set_head`](#set_head)
  - [`finalize_block`](#finalize_block)
-- [Containers](#containers)
-  - [`PowBlock`](#powblock)
-- [Helper functions](#helper-functions)
-  - [`get_pow_block`](#get_pow_block)
-  - [`is_valid_transition_block`](#is_valid_transition_block)
+- [Helpers](#helpers)
+ - [`TransitionStore`](#transitionstore)
+ - [`PowBlock`](#powblock)
+ - [`get_pow_block`](#get_pow_block)
+ - [`is_valid_terminal_pow_block`](#is_valid_terminal_pow_block)
 - [Updated fork-choice handlers](#updated-fork-choice-handlers)
-  - [`on_block`](#on_block)
+ - [`on_block`](#on_block)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 <!-- /TOC -->
@@ -66,44 +66,52 @@ def finalize_block(self: ExecutionEngine, block_hash: Hash32) -> bool:
  ...
 ```
 
-## Containers
+## Helpers
 
-#### `PowBlock`
+### `TransitionStore`
 
 ```python
-class PowBlock(Container):
+@dataclass
+class TransitionStore(object):
+ transition_total_difficulty: uint256
+```
+
+### `PowBlock`
+
+```python
+@dataclass
+class PowBlock(object):
  block_hash: Hash32
  is_processed: boolean
  is_valid: boolean
  total_difficulty: uint256
+ difficulty: uint256
 ```
 
-## Helper functions
-
-#### `get_pow_block`
+### `get_pow_block`
 
 Let `get_pow_block(block_hash: Hash32) -> PowBlock` be the function that given the hash of the PoW block returns its data.
 
 *Note*: The `eth_getBlockByHash` JSON-RPC method does not distinguish invalid blocks from blocks that haven't been processed yet. Either extending this existing method or implementing a new one is required.
 
-#### `is_valid_transition_block`
+### `is_valid_terminal_pow_block`
 
 Used by fork-choice handler, `on_block`.
 
 ```python
-def is_valid_transition_block(block: PowBlock) -> bool:
- is_total_difficulty_reached = block.total_difficulty >= TRANSITION_TOTAL_DIFFICULTY
+def is_valid_terminal_pow_block(transition_store: TransitionStore, block: PowBlock) -> bool:
+ is_total_difficulty_reached = block.total_difficulty >= transition_store.transition_total_difficulty
  return block.is_valid and is_total_difficulty_reached
 ```
 
 ## Updated fork-choice handlers
 
-#### `on_block`
+### `on_block`
 
 *Note*: The only modification is the addition of the verification of transition block conditions.
 
 ```python
-def on_block(store: Store, signed_block: SignedBeaconBlock) -> None:
+def on_block(store: Store, signed_block: SignedBeaconBlock, transition_store: TransitionStore=None) -> None:
  block = signed_block.message
  # Parent block must be known
  assert block.parent_root in store.block_states
@@ -119,11 +127,11 @@ def on_block(store: Store, signed_block: SignedBeaconBlock) -> None:
  assert get_ancestor(store, block.parent_root, finalized_slot) == store.finalized_checkpoint.root
 
  # [New in Merge]
- if is_transition_block(pre_state, block):
+ if (transition_store is not None) and is_transition_block(pre_state, block):
  # Delay consideration of block until PoW block is processed by the PoW node
  pow_block = get_pow_block(block.body.execution_payload.parent_hash)
  assert pow_block.is_processed
- assert is_valid_transition_block(pow_block)
+ assert is_valid_terminal_pow_block(transition_store, pow_block)
 
  # Check the block is valid and compute the post-state
  state = pre_state.copy()

specs/merge/fork.md

@@ -0,0 +1,121 @@
+# Ethereum 2.0 The Merge
+
+**Notice**: This document is a work-in-progress for researchers and implementers.
+
+## Table of contents
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+
+- [Introduction](#introduction)
+- [Configuration](#configuration)
+- [Fork to Merge](#fork-to-merge)
+ - [Fork trigger](#fork-trigger)
+ - [Upgrading the state](#upgrading-the-state)
+ - [Initializing transition store](#initializing-transition-store)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## Introduction
+
+This document describes the process of the Merge upgrade.
+
+## Configuration
+
+Warning: this configuration is not definitive.
+
+| Name | Value |
+| - | - |
+| `MERGE_FORK_VERSION` | `Version('0x02000000')` |
+| `MERGE_FORK_EPOCH` | `Epoch(18446744073709551615)` **TBD** |
+| `MIN_ANCHOR_POW_BLOCK_DIFFICULTY` | **TBD** |
+| `TARGET_SECONDS_TO_MERGE` | `uint64(7 * 86400)` = (604,800) |
+
+## Fork to Merge
+
+### Fork trigger
+
+TBD. Social consensus, along with state conditions such as epoch boundary, finality, deposits, active validator count, etc. may be part of the decision process to trigger the fork. For now we assume the condition will be triggered at epoch `MERGE_FORK_EPOCH`.
+
+Since the Merge transition process relies on `Eth1Data` in the beacon state we do want to make sure that this data is fresh. This is achieved by forcing `MERGE_FORK_EPOCH` to point to eth1 voting period boundary, i.e. `MERGE_FORK_EPOCH` should satisfy the following condition `MERGE_FORK_EPOCH % EPOCHS_PER_ETH1_VOTING_PERIOD == 0`.
+
+Note that for the pure Merge networks, we don't apply `upgrade_to_merge` since it starts with Merge version logic.
+
+### Upgrading the state
+
+If `state.slot % SLOTS_PER_EPOCH == 0` and `compute_epoch_at_slot(state.slot) == MERGE_FORK_EPOCH`, an irregular state change is made to upgrade to Merge.
+
+The upgrade occurs after the completion of the inner loop of `process_slots` that sets `state.slot` equal to `MERGE_FORK_EPOCH * SLOTS_PER_EPOCH`.
+Care must be taken when transitioning through the fork boundary as implementations will need a modified [state transition function](../phase0/beacon-chain.md#beacon-chain-state-transition-function) that deviates from the Phase 0 document.
+In particular, the outer `state_transition` function defined in the Phase 0 document will not expose the precise fork slot to execute the upgrade in the presence of skipped slots at the fork boundary. Instead the logic must be within `process_slots`.
+
+```python
+def upgrade_to_merge(pre: phase0.BeaconState) -> BeaconState:
+ epoch = phase0.get_current_epoch(pre)
+ post = BeaconState(
+ # Versioning
+ genesis_time=pre.genesis_time,
+ genesis_validators_root=pre.genesis_validators_root,
+ slot=pre.slot,
+ fork=Fork(
+ previous_version=pre.fork.current_version,
+ current_version=MERGE_FORK_VERSION,
+ epoch=epoch,
+ ),
+ # History
+ latest_block_header=pre.latest_block_header,
+ block_roots=pre.block_roots,
+ state_roots=pre.state_roots,
+ historical_roots=pre.historical_roots,
+ # Eth1
+ eth1_data=pre.eth1_data,
+ eth1_data_votes=pre.eth1_data_votes,
+ eth1_deposit_index=pre.eth1_deposit_index,
+ # Registry
+ validators=pre.validators,
+ balances=pre.balances,
+ # Randomness
+ randao_mixes=pre.randao_mixes,
+ # Slashings
+ slashings=pre.slashings,
+ # Attestations
+ previous_epoch_attestations=pre.previous_epoch_attestations,
+ current_epoch_attestations=pre.current_epoch_attestations,
+ # Finality
+ justification_bits=pre.justification_bits,
+ previous_justified_checkpoint=pre.previous_justified_checkpoint,
+ current_justified_checkpoint=pre.current_justified_checkpoint,
+ finalized_checkpoint=pre.finalized_checkpoint,
+ # Execution-layer
+ latest_execution_payload_header=ExecutionPayloadHeader(),
+ )
+
+ return post
+```
+
+### Initializing transition store
+
+If `state.slot % SLOTS_PER_EPOCH == 0` and `compute_epoch_at_slot(state.slot) == MERGE_FORK_EPOCH`, a transition store is initialized to be further utilized by the transition process of the Merge.
+
+Transition store initialization occurs after the state has been modified by corresponding `upgrade_to_merge` function.
+
+```python
+def compute_transition_total_difficulty(anchor_pow_block: PowBlock) -> uint256:
+ seconds_per_voting_period = EPOCHS_PER_ETH1_VOTING_PERIOD * SLOTS_PER_EPOCH * SECONDS_PER_SLOT
+ pow_blocks_per_voting_period = seconds_per_voting_period // SECONDS_PER_ETH1_BLOCK
+ pow_blocks_to_merge = TARGET_SECONDS_TO_MERGE // SECONDS_PER_ETH1_BLOCK
+ pow_blocks_after_anchor_block = ETH1_FOLLOW_DISTANCE + pow_blocks_per_voting_period + pow_blocks_to_merge
+ anchor_difficulty = max(MIN_ANCHOR_POW_BLOCK_DIFFICULTY, anchor_pow_block.difficulty)
+
+ return anchor_pow_block.total_difficulty + anchor_difficulty * pow_blocks_after_anchor_block
+
+
+def get_transition_store(anchor_pow_block: PowBlock) -> TransitionStore:
+ transition_total_difficulty = compute_transition_total_difficulty(anchor_pow_block)
+ return TransitionStore(transition_total_difficulty=transition_total_difficulty)
+
+
+def initialize_transition_store(state: BeaconState) -> TransitionStore:
+ pow_block = get_pow_block(state.eth1_data.block_hash)
+ return get_transition_store(pow_block)
+```

specs/merge/validator.md

@@ -20,7 +20,6 @@
  - [Constructing the `BeaconBlockBody`](#constructing-the-beaconblockbody)
  - [Execution Payload](#execution-payload)
  - [`get_pow_chain_head`](#get_pow_chain_head)
- - [`produce_execution_payload`](#produce_execution_payload)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 <!-- /TOC -->
@@ -68,18 +67,15 @@ All validator responsibilities remain unchanged other than those noted below. Na
 
 Let `get_pow_chain_head() -> PowBlock` be the function that returns the head of the PoW chain. The body of the function is implementation specific.
 
-###### `produce_execution_payload`
-
-Let `produce_execution_payload(parent_hash: Hash32, timestamp: uint64) -> ExecutionPayload` be the function that produces new instance of execution payload.
-The `ExecutionEngine` protocol is used for the implementation specific part of execution payload proposals.
-
-* Set `block.body.execution_payload = get_execution_payload(state)` where:
+* Set `block.body.execution_payload = get_execution_payload(state, transition_store, execution_engine)` where:
 
 ```python
-def get_execution_payload(state: BeaconState, execution_engine: ExecutionEngine) -> ExecutionPayload:
+def get_execution_payload(state: BeaconState,
+ transition_store: TransitionStore,
+ execution_engine: ExecutionEngine) -> ExecutionPayload:
  if not is_transition_completed(state):
  pow_block = get_pow_chain_head()
- if not is_valid_transition_block(pow_block):
+ if not is_valid_terminal_pow_block(transition_store, pow_block):
  # Pre-merge, empty payload
  return ExecutionPayload()
  else: