Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merge] Implement execution_layer #2635

Merged
merged 65 commits into from
Sep 29, 2021
Merged
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
55e5b5b
Checkout serde_utils from rayonism
paulhauner Sep 21, 2021
1ce8339
Make eth1::http functions pub
paulhauner Sep 21, 2021
7433385
Add bones of execution_layer
paulhauner Sep 21, 2021
ac1cdc5
Modify decoding
paulhauner Sep 21, 2021
18dc88f
Expose Transaction, cargo fmt
paulhauner Sep 22, 2021
3d2bc6d
Add executePayload
paulhauner Sep 22, 2021
08308c0
Add all minimal spec endpoints
paulhauner Sep 22, 2021
cb5e33d
Start adding json rpc wrapper
paulhauner Sep 23, 2021
95e9407
Finish custom JSON response handler
paulhauner Sep 23, 2021
31ad323
Switch to new rpc sending method
paulhauner Sep 24, 2021
9e7b432
Add first test
paulhauner Sep 24, 2021
68e24d4
Fix camelCase
paulhauner Sep 24, 2021
74a25ce
Finish adding tests
paulhauner Sep 24, 2021
4fe318c
Begin threading execution layer into BeaconChain
paulhauner Sep 24, 2021
9c8bf49
Fix clippy lints
paulhauner Sep 25, 2021
95ef497
Fix clippy lints
paulhauner Sep 25, 2021
81a62e3
Thread execution layer into ClientBuilder
paulhauner Sep 25, 2021
f698b91
Add CLI flags
paulhauner Sep 25, 2021
203a93b
Add block processing methods to ExecutionLayer
paulhauner Sep 25, 2021
1c2b59f
Add block_on to execution_layer
paulhauner Sep 27, 2021
7091adf
Integrate execute_payload
paulhauner Sep 27, 2021
03b984a
Add extra_data field
paulhauner Sep 27, 2021
3c816a3
Begin implementing payload handle
paulhauner Sep 27, 2021
82d491c
Send consensus valid/invalid messages
paulhauner Sep 27, 2021
5098da5
Fix minor type in task_executor
paulhauner Sep 27, 2021
c329fae
Call forkchoiceUpdated
paulhauner Sep 27, 2021
f9fd6ac
Add search for TTD block
paulhauner Sep 27, 2021
6cf83db
Thread TTD into execution layer
paulhauner Sep 27, 2021
b6909b8
Allow producing block with execution payload
paulhauner Sep 27, 2021
c1b0093
Add LRU cache for execution blocks
paulhauner Sep 27, 2021
4093a06
Remove duplicate 0x on ssz_types serialization
paulhauner Sep 27, 2021
92829c7
Add tests for block getter methods
paulhauner Sep 27, 2021
5323681
Add basic block generator impl
paulhauner Sep 28, 2021
dd570f4
Add is_valid_terminal_block to EL
paulhauner Sep 28, 2021
559236d
Verify merge block in block_verification
paulhauner Sep 28, 2021
c69c3ea
Partially implement --terminal-block-hash-override
paulhauner Sep 28, 2021
1135a8f
Add terminal_block_hash to ChainSpec
paulhauner Sep 28, 2021
137a9fd
Remove Option from terminal_block_hash in EL
paulhauner Sep 28, 2021
8080c78
Revert merge changes to consensus/fork_choice
paulhauner Sep 28, 2021
ce56996
Remove commented-out code
paulhauner Sep 28, 2021
354955c
Add bones for handling RPC methods on test server
paulhauner Sep 28, 2021
20106bb
Add first ExecutionLayer tests
paulhauner Sep 28, 2021
c069244
Add testing for finding terminal block
paulhauner Sep 28, 2021
9678f77
Prevent infinite loops
paulhauner Sep 28, 2021
af6095a
Add insert_merge_block to block gen
paulhauner Sep 28, 2021
91a3cbc
Add block gen test for pos blocks
paulhauner Sep 28, 2021
c80fd81
Start adding payloads to block gen
paulhauner Sep 28, 2021
f812548
Fix clippy lints
paulhauner Sep 28, 2021
f76a1d1
Add execution payload to block gen
paulhauner Sep 29, 2021
837acc1
Add execute_payload to block_gen
paulhauner Sep 29, 2021
57f6a9b
Refactor block gen
paulhauner Sep 29, 2021
3a73cde
Add all routes to mock server
paulhauner Sep 29, 2021
6c88f19
Use Uint256 for base_fee_per_gas
paulhauner Sep 29, 2021
133bb07
Add working execution chain build
paulhauner Sep 29, 2021
d366a47
Remove unused var
paulhauner Sep 29, 2021
015c9bc
Revert "Use Uint256 for base_fee_per_gas"
paulhauner Sep 29, 2021
889eeba
Fix base_fee_for_gas Uint256
paulhauner Sep 29, 2021
591d494
Update execute payload handle
paulhauner Sep 29, 2021
118613f
Improve testing, fix bugs
paulhauner Sep 29, 2021
e5a3403
Fix default fee-recipient
paulhauner Sep 29, 2021
1f36650
Fix fee-recipient address (again)
paulhauner Sep 29, 2021
7c5b37f
Add check for terminal block, add comments, tidy
paulhauner Sep 29, 2021
b786495
Apply suggestions from code review
paulhauner Sep 29, 2021
b877b3a
Fix is_none on handle Drop
paulhauner Sep 29, 2021
5d45b37
Remove commented-out tests
paulhauner Sep 29, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -7,6 +7,7 @@ members = [
"beacon_node/client",
"beacon_node/eth1",
"beacon_node/eth2_libp2p",
"beacon_node/execution_layer",
"beacon_node/http_api",
"beacon_node/http_metrics",
"beacon_node/network",
1 change: 1 addition & 0 deletions beacon_node/beacon_chain/Cargo.toml
Original file line number Diff line number Diff line change
@@ -63,3 +63,4 @@ exit-future = "0.2.0"
slasher = { path = "../../slasher" }
eth2 = { path = "../../common/eth2" }
strum = { version = "0.21.0", features = ["derive"] }
execution_layer = { path = "../execution_layer" }
118 changes: 110 additions & 8 deletions beacon_node/beacon_chain/src/beacon_chain.rs
Original file line number Diff line number Diff line change
@@ -46,6 +46,7 @@ use crate::BeaconForkChoiceStore;
use crate::BeaconSnapshot;
use crate::{metrics, BeaconChainError};
use eth2::types::{EventKind, SseBlock, SseChainReorg, SseFinalizedCheckpoint, SseHead, SyncDuty};
use execution_layer::ExecutionLayer;
use fork_choice::ForkChoice;
use futures::channel::mpsc::Sender;
use itertools::process_results;
@@ -59,7 +60,9 @@ use slot_clock::SlotClock;
use state_processing::{
common::get_indexed_attestation,
per_block_processing,
per_block_processing::errors::AttestationValidationError,
per_block_processing::{
compute_timestamp_at_slot, errors::AttestationValidationError, is_merge_complete,
},
per_slot_processing,
state_advance::{complete_state_advance, partial_state_advance},
BlockSignatureStrategy, SigVerifiedOp,
@@ -272,6 +275,8 @@ pub struct BeaconChain<T: BeaconChainTypes> {
Mutex<ObservedOperations<AttesterSlashing<T::EthSpec>, T::EthSpec>>,
/// Provides information from the Ethereum 1 (PoW) chain.
pub eth1_chain: Option<Eth1Chain<T::Eth1Chain, T::EthSpec>>,
/// Interfaces with the execution client.
pub execution_layer: Option<ExecutionLayer>,
/// Stores a "snapshot" of the chain at the time the head-of-the-chain block was received.
pub(crate) canonical_head: TimeoutRwLock<BeaconSnapshot<T::EthSpec>>,
/// The root of the genesis block.
@@ -2385,7 +2390,7 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
let _fork_choice_block_timer =
metrics::start_timer(&metrics::FORK_CHOICE_PROCESS_BLOCK_TIMES);
fork_choice
.on_block(current_slot, &block, block_root, &state, &self.spec)
.on_block(current_slot, &block, block_root, &state)
.map_err(|e| BlockError::BeaconChainError(e.into()))?;
}

@@ -2787,12 +2792,42 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
}))
};
// Closure to fetch a sync aggregate in cases where it is required.
let get_execution_payload = || -> Result<ExecutionPayload<_>, BlockProductionError> {
// TODO: actually get the payload from eth1 node..
Ok(ExecutionPayload::default())
let get_execution_payload = |latest_execution_payload_header: &ExecutionPayloadHeader<
T::EthSpec,
>|
-> Result<ExecutionPayload<_>, BlockProductionError> {
let execution_layer = self
.execution_layer
.as_ref()
.ok_or(BlockProductionError::ExecutionLayerMissing)?;

let parent_hash;
if !is_merge_complete(&state) {
let terminal_pow_block_hash = execution_layer
.block_on(|execution_layer| execution_layer.get_terminal_pow_block_hash())
.map_err(BlockProductionError::TerminalPoWBlockLookupFailed)?;

if let Some(terminal_pow_block_hash) = terminal_pow_block_hash {
parent_hash = terminal_pow_block_hash;
} else {
return Ok(<_>::default());
}
} else {
parent_hash = latest_execution_payload_header.block_hash;
}

let timestamp =
compute_timestamp_at_slot(&state, &self.spec).map_err(BeaconStateError::from)?;
let random = *state.get_randao_mix(state.current_epoch())?;

execution_layer
.block_on(|execution_layer| {
execution_layer.get_payload(parent_hash, timestamp, random)
})
.map_err(BlockProductionError::GetPayloadFailed)
};

let inner_block = match state {
let inner_block = match &state {
BeaconState::Base(_) => BeaconBlock::Base(BeaconBlockBase {
slot,
proposer_index,
@@ -2829,9 +2864,10 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
},
})
}
BeaconState::Merge(_) => {
BeaconState::Merge(state) => {
let sync_aggregate = get_sync_aggregate()?;
let execution_payload = get_execution_payload()?;
let execution_payload =
get_execution_payload(&state.latest_execution_payload_header)?;
BeaconBlock::Merge(BeaconBlockMerge {
slot,
proposer_index,
@@ -3034,6 +3070,14 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
.start_slot(T::EthSpec::slots_per_epoch());
let head_proposer_index = new_head.beacon_block.message().proposer_index();

// Used later for the execution engine.
let new_head_execution_block_hash = new_head
.beacon_block
.message()
.body()
.execution_payload()
.map(|ep| ep.block_hash);

drop(lag_timer);

// Update the snapshot that stores the head of the chain at the time it received the
@@ -3171,9 +3215,67 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
}
}

// If this is a post-merge block, update the execution layer.
if let Some(new_head_execution_block_hash) = new_head_execution_block_hash {
let execution_layer = self
.execution_layer
.clone()
.ok_or(Error::ExecutionLayerMissing)?;
let store = self.store.clone();
let log = self.log.clone();

// Spawn the update task, without waiting for it to complete.
execution_layer.spawn(
move |execution_layer| async move {
if let Err(e) = Self::update_execution_engine_forkchoice(
execution_layer,
store,
new_finalized_checkpoint.root,
new_head_execution_block_hash,
)
.await
{
error!(
log,
"Failed to update execution head";
"error" => ?e
);
}
},
"update_execution_engine_forkchoice",
)
}

Ok(())
}

pub async fn update_execution_engine_forkchoice(
execution_layer: ExecutionLayer,
store: BeaconStore<T>,
finalized_beacon_block_root: Hash256,
head_execution_block_hash: Hash256,
) -> Result<(), Error> {
// Loading the finalized block from the store is not ideal. Perhaps it would be better to
// store it on fork-choice so we can do a lookup without hitting the database.
//
// See: https://github.com/sigp/lighthouse/pull/2627#issuecomment-927537245
let finalized_block = store
.get_block(&finalized_beacon_block_root)?
.ok_or(Error::MissingBeaconBlock(finalized_beacon_block_root))?;

let finalized_execution_block_hash = finalized_block
.message()
.body()
.execution_payload()
.map(|ep| ep.block_hash)
.unwrap_or_else(Hash256::zero);

execution_layer
.forkchoice_updated(head_execution_block_hash, finalized_execution_block_hash)
.await
.map_err(Error::ExecutionForkChoiceUpdateFailed)
}

/// This function takes a configured weak subjectivity `Checkpoint` and the latest finalized `Checkpoint`.
/// If the weak subjectivity checkpoint and finalized checkpoint share the same epoch, we compare
/// roots. If we the weak subjectivity checkpoint is from an older epoch, we iterate back through
Loading