-
-
Notifications
You must be signed in to change notification settings - Fork 343
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(beacon-node): network worker new space adjustment #5829
Conversation
Performance Report✔️ no performance regression detected Full benchmark results
|
packages/beacon-node/src/network/core/networkCoreWorkerHandler.ts
Outdated
Show resolved
Hide resolved
Metrics looks promising! This PR does not add metrics, but changes a fundamental parameter, can you split:
|
49fc485
to
ed0b59c
Compare
This was updated to just have the single line change. I rebased this onto unstable from a discussion with @wemeetagain but there is a memory leak on unstable so its hard to see how this is actually performing at 128mb. I will move to ready for review once we get the leak sorted and can see how this performs with @tuyennhv update with limited subscriptions. Dashboard changes were moved to PR #5858 |
…/lodestar into mkeil/network-worker-new-space
Value was set from research on #2115. You can find the resolution to that issue here There is a 3x1 ratio of semi space to young generation space in the code but in practice it seems like setting the semi space to the average value of scavenge collection seemed optimal. In previous empirical testing it seemed to also be the optimal value for the worker young generation setting despite the ratio. I am not certain of the reason for this but its worth further investigation. |
packages/beacon-node/src/network/core/networkCoreWorkerHandler.ts
Outdated
Show resolved
Hide resolved
🎉 This PR is included in v1.11.0 🎉 |
Motivation
While going through the suggestions that Ben gave us I found that there was a very large amount of new space scavenge collection. I updated the dashboards and realized it was over 200mb and 3% of the worker thread time spent on scavenge. There was a lot of page-faults as well which lends itself to a bunch of data being moved during GC. There was also not very much mark-and-sweep GC which means the objects were most likely getting moved only once before getting un-ref'd. This realization that most of the objects are very short lived, but likely high in volume to trigger the scavenge. So in essence causing them to get moved back and forth between the
To
andFrom
spaces a single time.I played around with the values of new space to get around this. I bumped the new space way up above the GC volume. Now the objects do not live long enough to need to be moved in the new space. The HandleScope and new space management system are quite efficient so there is little to no performance degradation from leaving the objects in new space once they are no longer ref'd. They just either sit at the back of the line or get written over when the HandleScope pointers shift.
Description
unstable
GC volume)[X] - Figure out method to determine how large is too large
[X] - Verify that new space is not too large
Helps with #5604
** Metrics **
Need to let the node stabilize for a few more hours so traffic is back up to levels on
unstable
for even comparison but things are looking pretty good so far.