Increase stateCache size to fix headState missing in checkpoint sync start #4534
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Performance Report✔️ no performance regression detected Full benchmark results
|
twoeths
reviewed
Sep 12, 2022
| @@ -6,7 +6,7 @@ import {IMetrics} from "../../metrics/index.js"; | |||
| import {MapTracker} from "./mapMetrics.js"; | |||
| import {stateInternalCachePopulated} from "./stateContextCheckpointsCache.js"; | |||
|
|
|||
| const MAX_STATES = 3 * 32; | |||
| const MAX_STATES = 6 * 32; | |||
There was a problem hiding this comment.
we need to investigate the impact of this with regards to the heap memory (which lead to gc percentage metrics). In the past when chain was not finalized for a while I saw the heap memory spiked at the same time.
dapplion
suggested changes
Sep 12, 2022
👍 let me see how we can handle this particular case in a better way |
|
fixed in #4562 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In checkpoint sync start, the stateCache of 3 * 32 caches seem to be not sufficient since generally we start from finalized which is almost 3 epochs behind
So before bn can sync blocks and generate more states nearer to head, the getHeadState functions seem to be running out of the states.
This PR does a quick-fix for expanding the stateCache's cache to 6 epochs (to give margin on 3 epochs) so that bns can stay behind 3 epochs + while still trying to find/sync peers.
There could be a better solution, but this is a hotfix to the current issue users might run into.
(see the notifier crossing 96+ skipped slots here since last finalized checkpoint sync comforably without error)
User issue also seems to have resolved.
Closes #4523