-
Notifications
You must be signed in to change notification settings - Fork 142
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feature: add e2hs mode to portal-bridge #1735
Conversation
846dd80
to
8d6a1be
Compare
8d6a1be
to
ca527b2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good. I think it is fine if we iterate on this like you explained in the PR's description.
bin/portal-bridge/src/bridge/e2hs.rs
Outdated
fn validate_block_tuple(&self, block_tuple: &BlockTuple) -> anyhow::Result<()> { | ||
self.header_validator | ||
.validate_header_with_proof(&block_tuple.header_with_proof.header_with_proof)?; | ||
let receipts = &block_tuple.receipts.receipts; | ||
let receipts_root = receipts.root(); | ||
ensure!( | ||
receipts_root | ||
== block_tuple | ||
.header_with_proof | ||
.header_with_proof | ||
.header | ||
.receipts_root, | ||
"Receipts root mismatch" | ||
); | ||
let body = &block_tuple.body.body; | ||
body.validate_against_header(&block_tuple.header_with_proof.header_with_proof.header)?; | ||
Ok(()) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fn validate_block_tuple(&self, block_tuple: &BlockTuple) -> anyhow::Result<()> { | |
self.header_validator | |
.validate_header_with_proof(&block_tuple.header_with_proof.header_with_proof)?; | |
let receipts = &block_tuple.receipts.receipts; | |
let receipts_root = receipts.root(); | |
ensure!( | |
receipts_root | |
== block_tuple | |
.header_with_proof | |
.header_with_proof | |
.header | |
.receipts_root, | |
"Receipts root mismatch" | |
); | |
let body = &block_tuple.body.body; | |
body.validate_against_header(&block_tuple.header_with_proof.header_with_proof.header)?; | |
Ok(()) | |
} | |
fn validate_block_tuple(&self, block_tuple: &BlockTuple) -> anyhow::Result<()> { | |
let header_with_proof = &block_tuple.header_with_proof.header_with_proof; | |
self.header_validator | |
.validate_header_with_proof(header_with_proof)?; | |
let body = &block_tuple.body.body; | |
body.validate_against_header(&header_with_proof.header)?; | |
let receipts = &block_tuple.receipts.receipts; | |
ensure!( | |
receipts.root() | |
== header_with_proof | |
.header | |
.receipts_root, | |
"Receipts root mismatch" | |
); | |
Ok(()) | |
} |
I think ordering it like this would be a bit more readable. Also I don't think we need the extra variable receipts_root
Idk up to you
bin/portal-bridge/src/bridge/e2hs.rs
Outdated
block_stats: Arc<Mutex<HistoryBlockStats>>, | ||
metrics: BridgeMetricsReporter, | ||
) -> anyhow::Result<()> { | ||
let header_number = block_tuple |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let header_number = block_tuple | |
let block_number = block_tuple |
I think this is more clear
bin/portal-bridge/src/bridge/e2hs.rs
Outdated
|
||
info!("Serving block tuple for block #{header_number}"); | ||
|
||
let header_hash = block_tuple |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let header_hash = block_tuple | |
let block_hash = block_tuple |
I think this is more clear
let block_number = block_tuple | ||
.header_with_proof | ||
.header_with_proof | ||
.header | ||
.number; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are doing this 3 times with 3 different variable names
- block_number
- number
- header_number
Could we only do it once, then just pass it as a variable and use block_number
as I think that is the standard terminology.
bin/portal-bridge/src/bridge/e2hs.rs
Outdated
match block_range { | ||
Some(_) => { | ||
info!("Gossiping block range: {block_range:?} from epoch {epoch_index}",); | ||
} | ||
None => { | ||
info!("Gossiping entire epoch {epoch_index}"); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
match block_range { | |
Some(_) => { | |
info!("Gossiping block range: {block_range:?} from epoch {epoch_index}",); | |
} | |
None => { | |
info!("Gossiping entire epoch {epoch_index}"); | |
} | |
} | |
match block_range { | |
Some(_) => info!("Gossiping block range: {block_range:?} from epoch {epoch_index}"), | |
None => info!("Gossiping entire epoch {epoch_index}"), | |
} |
Perhaps we could condense it now like this?
bin/portal-bridge/src/bridge/e2hs.rs
Outdated
error!( | ||
"Failed to gossip history content key: {:?} - {:?}", | ||
hwp_by_number_content_key, err | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
error!( | |
"Failed to gossip history content key: {:?} - {:?}", | |
hwp_by_number_content_key, err | |
); | |
error!("Failed to gossip history content key: {hwp_by_number_content_key:?} - {err:?}"); |
bin/portal-bridge/src/bridge/e2hs.rs
Outdated
error!( | ||
"Failed to gossip history content key: {:?} - {:?}", | ||
body_content_key, err | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
error!( | |
"Failed to gossip history content key: {:?} - {:?}", | |
body_content_key, err | |
); | |
error!("Failed to gossip history content key: {body_content_key:?} - {err:?}"); |
bin/portal-bridge/src/bridge/e2hs.rs
Outdated
error!( | ||
"Failed to gossip history content key: {:?} - {:?}", | ||
receipts_content_key, err | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
error!( | |
"Failed to gossip history content key: {:?} - {:?}", | |
receipts_content_key, err | |
); | |
error!("Failed to gossip history content key: {receipts_content_key:?} - {err:?}"); |
bin/portal-bridge/src/bridge/e2hs.rs
Outdated
if parts.len() != 2 { | ||
return Err(anyhow!("Invalid block range format")); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if parts.len() != 2 { | |
return Err(anyhow!("Invalid block range format")); | |
} | |
ensure!(parts.len() == 2, "Invalid block range format"); |
bin/portal-bridge/src/types/range.rs
Outdated
/// HashMap mapping epoch indexes to either None or Some((start_block, end_block)) | ||
pub fn block_range_to_epochs(start_block: u64, end_block: u64) -> HashMap<u64, Option<(u64, u64)>> { | ||
let mut epoch_map = HashMap::new(); | ||
const EPOCH_SIZE: u64 = 8192; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const EPOCH_SIZE: u64 = 8192; |
Isn't this a pretty common constant couldn't we import this from somewhere else?
7aa8d89
to
1c0b1f3
Compare
What was wrong?
With the introduction of the new e2hs file format, we need a bridge mode to support these files.
This is a very basic bridge mode, that won't really work unless you have all necessary
e2hs
files in a localtest-e2hs
dir, inside the root level of the trin dir. But the intention is that this will be updated to use a centralized server that hosts alle2hs
files, once they are generated and made available.The existing
BridgeMode
type has turned into a bit of a frankenstein, as a result of us trying to push too much into a single cli arg. After this new bridge mode matures, I expect a fair amount of legacy bridge code to be deleted, that's why I decided to create new, isolated cli args for thee2hs
mode to use, rather than try and squeeze them into the oldBridgeMode
type.There's another trade-off to consider. I opted to require the user to provide the entire block range, rather than using a live el provider to detect the head of the chain. I'm not married to this, but I think it's easier if a user just wants to casually run the bridge, without an infura / pandaops account.
But both of these decisions are fairly trivial, and can be changed as new use-cases for the bridge are implemented.
Usage
"--mode e2hs --e2hs-range 1000-10000 --e2hs-randomize"
: randomize the order in which epochs from block range are gossipedHow was it fixed?
e2hs
mode to portal-bridgeTo-Do