-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(stages): remove static files stage, run before loop #6763
feat(stages): remove static files stage, run before loop #6763
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
great
Pipeline::builder().build( | ||
provider_factory.clone(), | ||
StaticFileProducer::new( | ||
provider_factory.clone(), | ||
provider_factory.static_file_provider(), | ||
PruneModes::default(), | ||
), | ||
), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would like to improve the builder pattern here such that we don't have 3 mentions of provider factory in the call -- not a blocker
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's a bit of an unreal example because of debug cmd. in practice, it looks like this
reth/crates/node-builder/src/builder.rs
Lines 359 to 364 in 0a6e95c
// configure static_file_producer | |
let static_file_producer = reth_static_file::StaticFileProducer::new( | |
provider_factory.clone(), | |
provider_factory.static_file_provider(), | |
config.prune_config()?.unwrap_or_default().segments, | |
); |
reth/crates/node-core/src/node_config.rs
Line 890 in a1df62c
.build(provider_factory, static_file_producer); |
but I agree that we can pass only provider_factory
and prune_modes
inside the StaticFileProducer::new
…emove-static-files-stage
StaticFile
stage doesn't fit well into the pipeline abstraction, because the expectation is thatHeaders
executes first, converting the target block hash into the target block number, and all consequent stages execute on top ofHeaders
stage checkpoint.Instead, we run the
StaticFileProducer
before main pipeline loop to ensure that all data are copied from MDBX to static files, so the pipeline stages likeHeaders
,Bodies
andExecution
could write directly to them.h/t @rkrasiuk for the idea