-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(engine): introduce sync implementation of StateRootTask #12378
Conversation
use super::{StateRootConfig, StateRootHandle, StateRootResult}; | ||
use reth_trie::updates::TrieUpdates; | ||
use revm_primitives::{EvmState, B256}; | ||
use std::sync::mpsc::{self, Receiver, RecvError}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
worth checking if we get perf improvements by using crossbeam channels
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry for the previous message, i was benching the wrong thing (the channel for sending the final result instead of the channel for sending each state update).
ok, we have the right bench here 7239538 to compare crossbeam_channel::unbounded
and std::sync::mpsc::channel()
when sending EvmState
(just sending items, not taking into account how they are processed), these are the results:
state_stream_channels/std_channel/1
time: [49.237 µs 50.471 µs 51.552 µs]
state_stream_channels/crossbeam_channel/1
time: [47.803 µs 50.518 µs 55.016 µs]
state_stream_channels/std_channel/10
time: [286.21 µs 293.42 µs 297.39 µs]
Found 4 outliers among 10 measurements (40.00%)
1 (10.00%) low severe
1 (10.00%) low mild
2 (20.00%) high mild
state_stream_channels/crossbeam_channel/10
time: [285.84 µs 295.41 µs 306.75 µs]
state_stream_channels/std_channel/100
time: [1.0524 ms 1.0585 ms 1.0670 ms]
Found 1 outliers among 10 measurements (10.00%)
1 (10.00%) high mild
state_stream_channels/crossbeam_channel/100
time: [1.0317 ms 1.0411 ms 1.0510 ms]
so, mostly the same performance for both implementations, slightly better crossbeam for big state changes (100 accounts), but this is not going to be very frequent.
given these results I think it would be better to keep std::syn::mpsc::channel
, one dep less, wdyt?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, then this doesn't really matter.
crossbeam is already part of the dep tree so we can use it or not
rayon::spawn(move || { | ||
debug!(target: "engine::tree", "Starting sync state root task"); | ||
let result = self.run(); | ||
let _ = tx.send(result); | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think rayon is a bit problematic here because this does a lot of IO.
since we're always using this, we could use a dedicated background thread for this that loops/recv StateRootSyncTask
s
similar to the existing persistence task
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
indeed, this is what we had initially 891edaa recovered, also switched from mpsc::channel
to mpsc::sync_channel(1)
d16d1c3
to
d0f48d5
Compare
d0f48d5
to
fdb55e9
Compare
fdb55e9
to
5d2048f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Towards #12053
Replaces async implementation with sync given that
EngineApiTreeHandler
is sync and runs in its own thread without setting up ny tokio runtime. We can always run this same implementation in an async context using spwan_blocking + send.It also adds a bench to compare
std::sync::mpsc::channel
withcrossbeam_channel::unbounded
for receiving state updates.