Added chunk impl for parallel sender recovery #5622
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolving issue #5189
Creating new pull request, I was not able to run the mainnet SenderRecovery stage because of my device specs. Instead, I ran optimized tests using cargo test --package reth-primitives --release to get more reliable measurement as pointed out by @rakita. I compared the performance before/after (without/with chunks) for recovering 100,000 transactions.
The test:
Additional crates used in mod tests:
use std::{fs::File, io::Write, time::Instant};
use rayon::prelude::*;
The results:
Chunk implementation takes 20-30% less time than the current implementation in all runs. It would be great if anyone else could run it and compare the performance. Since they are all optimized test runs, I think the estimates are reliable.