Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/rawdb: add accessors with reusable key buffers #30770

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

holiman
Copy link
Contributor

@holiman holiman commented Nov 20, 2024

This came from an alloc dump on a running sync (mingeth).
Screenshot 2024-11-20 at 09-55-10 tinygeth alloc_objects

Quite a lot of bytes are allocated temporarily when we write to db (batches), in order to construct the keys. These allocs are very shortlived, since the write-operations are not allowed to hold on to the keys, but instead copies it to some interrnal buffers.

This PR changes some of the most heavy mass-write-locations to reuse a key buffer. The locations are for example writing accounts received over snapsync, storage received over snapsync and merging in-memory snapshot-changes to disk.

Not sure how large effect this has. I suspect not a whole lot, but I'll run a benchmark sync and see if it anything is discernable.

Not high prio

@rjl493456442
Copy link
Member

Can you measure the reduction in memory allocations achieved by applying this change? Please evaluate its impact during both snap sync and full sync.

This pull request introduces optimizations to reuse the buffer for keying. These changes are only meaningful if they result in a obvious reduction in memory allocations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants